Computational Mafia

[Bicephalous Bob]

the main issue with trying to automate anything on mafiascum is the lack of an API. (it's in development?) It's FeelsBadMan to work with <document> response types, which means an addition layer of reading/parsing XML/HTML content which makes the whole process more time-consuming and error prone

It would be much much better if you could do something like query "[b]API[/b]/viewtopic.php?f=5&t=78755" and get back JSON instead

Omg so much this.

Like that’s the reason my VoteCount scrubber is slower than I would like.

Getting all that html and crunching it for hundreds of pages is a pita.

The average game should be around 15 light-weight pages if you use &view=print&ppp=200, of which only the last three pages are probably relevant for a vc

A twenty-line artoo.js game scraper: https://bitbucket.org/bopjesvla/thesis/ ... ew-default

[Bicephalous Bob]

oh i remember somehow loading 200 post pages long ago but forgot how i did that

why'd you put the scraper in js, btw?

I generally like scraping from a web browser because I already know the DOM API. I'd recommend puppeteer, not artoo.js, for anything more serious though.