Modbot to Help You Run Games: Now With A Votecounter Demo!

[Flubbernugget]

Forgive me if this is already in the tutorial because the json doesn't render well on my phone.

Can mods make sandbox forum threads to figure out how to use Modbot?

[Flubbernugget]

This is already looking pretty cool!
I don't have any knowledge about how to automate operations to the forum, such as posting and whatnot, but the bit I've been working on is intended to be able to store a "game state" which can be modified by passing events to it, like "Player A votes for Player B" and stuff like that. I also have a thing for automatically generating vote counts, the player list, the list of events in the first/second post, and the alive/dead/modkilled lists. I'm in the middle of rewriting stuff so it's not really in a suitable state to use at the moment but I would be happy to share what I have when I'm done in like a week or so. You might be able to scavenge some useful code out of it for the project.

Psyche did a really good job at abastracting away the automation of posting to/reading from the thread.

You should just need to use something like:

Code: Select all

mydonbot = Modbot.Donbot("some username", "some password", "thread url")

print(mydonbot.getPosts()) 

mydonbot.makePost("hello world!") 

Just be careful when you're running the makePost that you're in the thread you really want to be in.

I would also try to print() any strings you pass to makePost() before actually using the latter method. Especially if you want to put it in a loop.

[GreenLiquid]

For testing, maybe you could use a PT? Can the DonBot access PTs visible to your account when logging in under it?

[Psyche]

Right, the donbot module I've started makes it so you don't have to know how to automate operations on the forum in order to do it - you just import the module and start using its API. A detailed guide on how to do that is here: https://github.com/MafiaScum-Unofficial ... demo.ipynb

The module is definitely incomplete (I already want to change some things about getPosts() and probably need to add an editPost() function), but the tutorial covers everything donbot currently does. Please let me know if it's not straightforward to read!

***

I haven't set up any perfect testing environment anywhere for donbot. There's a test post thread in the Help! subforum, this thread is pretty friendly, and I've already tested a lot of these functions myself, but if you want a clean thread to test your own inventions at, you're on your own at the moment.

***

more on what's coming soon:

The votecounter I developed was written a long time ago and wasn't written especially well, so I'm rewriting it to some extent. It's elaborate in the bad sense of the word, and I don't remember how some parts of it work.

But with so many votecounters already going around, what should be of most interest to a lot of people here is the votecounter tester. The gist of it is that it tests how reliably your votecount can correctly figure out who got lynched D1 across a range of 300 Mini Normal games drawn from this archive here: viewtopic.php?f=53&t=29549 .

an initial idea of I expect the final product to work is like this:
1) You pass to the tester a function that

takes

a player slot list and a post dictionary variable (much like an element of what is output from Donbot's getPosts() function) as parameters and

returns

a list comprising the sequence of slots from the input slot list it believe were voted for in the input post.
2) the tester applies the function to each successive post in a game, maintaining a simple votecount as it iterates. Once the votecount shows that a slot is being voted by a majority of players (or once every post in the thread has been cycled through), that slot is treated as the Day's lynch.
3) if who your votecounter says is the Day's lynch matches what the archive says is the Day's lynch, then your votecounter has succeeded for that thread. Otherwise, it has failed.
4) the votecount tester returns the success rate of your votecounter as well as information about how your votecounter processed each thread, including:
- whether the votecounter succeeded or failed for that thread
- what the votecounter's final votecount was
- the sequence of votes that the votecounter detected in that thread until it found a lynch, including the post# of each vote and who voted for whom
That should be enough to help you understand where your votecounter succeeds and fails so you can make changes.

The set of games that'll be looped through and its associated archive have been thoroughly cleaned and processed already to only include games where someone got lynched D1 and you can infer who got lynched D1 by counting votes this way. Obviously, to discern if a no lynched happened, or if someone's a doublevoter, or if someone died N0, you need more just than a votecounter to figure out how D1 ended (eg something that understands mod posts).

but hopefully i'll wait until my workweek is over before moving forward on this

[yessiree]

The other last warning I would recommend is DDOS defense

I have a troll that bombs the shit out of mine to the point where I had to make a DB for it :/ just can’t find said troll

I don't think this would be an issue for this project. Psyche is not hosting the service anywhere. We are running a local instance to interact with the site.

...If anything, the site should worry about getting DDOS'd with this tool lol

[Flubbernugget]

There's a throttling constant in the code, so anyone that would try to flood the site would have to have some coding background, at which point it would be easy enough for them to do so with their own code anyway.

I also don't think there's http calls in the code that aren't linked to a username and password, making Modbot based DDOS extremely transparent on the attacker's end.

[Flubbernugget]

I've looked at the code and hope to actually be able to read the tutorials and run it by tonight

[Psyche]

there are a few things wrong with that description
it's very easy to set the throttling constant to 0 using the API (ie without changing any code in the module), and you don't need a username/password to do things on the site that don't require a username/password (ie collect posts from a thread in a forum that's not hidden)

but i think i will change it so that a username/password *is* required,
I believe anyone trying to make my code work will have to understand it pretty well to take out that requirement once I add it, so I'll go for that.

and said throttling constant isn't so much for the benefit of the site as it is for our code: the site won't *let* you make posts or send pms faster than once every 1-2 seconds. So the site has some good defenses

and the code is very simple, designwise; i think that if you really wanted to stage a serious DDOS attack or something like that you'd want to approach the problem in a totally different way i think

i am indeed worried about the possibility of this codebase being abused, but at the same time i think we're at a point where the prospective harm is outweighed by the benefits

anyway another thing i've been thinking of adding is a version of getPosts() that's not well-suited for bot work but is a lot more powerful/faster for collecting large numbers of posts
it's what i used when i scraped the thread data needed to test my votecounter, for example
but in light of this convo, probably unnecessary
lmk whenever you want it, though, and include what you'll use it for because i should be suspicious yes

[yessiree]

yeah i didnt mean in the sense that someone was gonna take your work and do some serious damage to the server, but it's always good to have that consideration for what

can

go wrong

like just bombarding the server with GET requests is enough to strain it (I hope there's server side caching), but ive noticed that sometimes the server will simply terminate your session if you send too many requests in a short time (not sure if intentional or not)

on semi-related note, it looks like you're working with document response types and just parsing the HTML. I hope we get a more streamlined API once the upgrade happens (fingers crossed)

[Flubbernugget]

Similar to lxml, I don't think requests comes with python by default, and may need a pip install command as well.

Is this expected to be compatible with both python 2 and 3?

[Flubbernugget]

On Radar

Auto-prodder

Taking claim to this.

[Flubbernugget]

Okay. I think the best way to implement an auto-prodder is to parse the activity overview page.

My initial thoughts to accomplish this would be to add a 'getActivity' method to the Donbot class, which would return each player's username, and the time since their last post.

An external utility (Prodbot???) would then check the results of getActivity, and use Donbot's sendPM method to prod all the slackers.

[Psyche]

ok, i'll make that a priority then

noting here too that i should modify sendPM() to make it possible to send a pm to more than one user at a time

[Flubbernugget]

First attempt at adding the activity parser to donbot was a partial success. I had everything worked out until I realized the V/LA column was being parsed as a column of null strings.

I tried following the nature of how donbot works, which involved XPath, something I had never heard of until I saw this code. My XPath pattern was long and ugly; I need to find the balance between a simple pattern match and a clean algorithm to format the data it gives me.

[Psyche]

activity overview page has a few quirks
first, the timestamps of the firstpost and lastpost columns don't include a year, making it sort of tricky to convert these into datetime objects in a way that is semantically sound
second, the elapsedtime column doesn't include minutes, so you can't take the time of your request and subtract elapsedtime to figure out when lastpost was (let alone firstpost)
you can use info from one column to inform parsing of info in the other, but further work seems necessary to reliably find the correct year matching firstpost

could easily leave this problem to the client and just output the text in the table without trying to parse them as datetime objects, but i believe it's worth the extra work

[Psyche]

this xpath: "//table//table//div" should pick out each row of the activity overview table, including column name
then for row in page.xpath(activityoverviewpath)[1:] or whatever, you can assign row.xpath(".//text()") to rowtext or something.

rowtext[5] will have the associated user's username
rowtext[8] will have the firstpost information
10, lastpost
12, sincelast
15, total number of posts

by the way, chrome's inspect panel has a lot of useful stuff for finding xpaths
you can rightclick an element to find its associated xpath, but what i usually do is ctrl+f in the panel, type in the xpath i think is associated with the element i want, and trial-and-error from there

***

note to make it possible to get posts by user in addition to post number range

[Psyche]

viewtopic.php?f=114&t=70764&activity_overview=1

yah there's no way to tell just from this page whether my first post in this thread was in february 2018 or february 2017
in a game thread, a client might be able to infer that the thread does not span more than one february (games don't usually last ~12 months), but we don't want our bot to only be accurate for these users
consequently, we'll have to just return the string variable associated with each element of the activity overview row instead of converting entries into datetime variables
the client will have to make the call themself about how to interpret it

***

ah ive misread the thread
sorry i already coded getActivityOverview
i'll be more careful to step off in the future;
just didn't catch your "claim" post and read your posts as you asking for getActivityOverview() to be added to donbot so you could make prodbot

***

btw step 2 on from here explains how to add stuff to the repo through the browsr: https://guides.github.com/activities/hello-world/
will have to add ur github account to the list of collaborators though

[MathBlade]

On Radar

Auto-prodder

Taking claim to this.

Mine does all but send the PM except a day start but I am fixing

[Psyche]

urge to just do it is strong but i am stronger
and there are other things to do

[Psyche]

ok let's convert votecounterdev.ipynb into votecounter.ipynb
since im about to change a lot of code i think i'll do the tester first
i'd like to see some of you other guys with working votecounters try it out!

***

EDIT: seems along with the votecounter tester, i'm also creating a VoteCount class that initializes with a list of the slots associated with a game, straightforwardly takes in vote changes (with a line like `votecount.update(voter, voted)`), and outputs stuff about its state

[Psyche]

mightve been kind of silly to do the votecountertester first since i don't have anything to put into it yet to make sure the code isn't buggy
still, i'm about to push an initial version of it with the votecounter class i mentioned earlier
wouldn't advise trying to use it or anything

[Kison]

Two thumbs up from me.

Let me know if you need anything. For example, I see what you said about activity overview. We can totally change that around so, for example, the timestamps match whatever format you specify in your profile.

[Psyche]

it should be the full timestamp, like what heads every post

[Kison]

The format of the timestamps displayed on posts is set in your profile: Edit board preferences -> My date format. You can set it to whatever format you want.

Activity overview ignores this setting. Ideally it would match that format as well.

[Psyche]

so i decided to temporarily pivot to the problem of generating a client-side thing for all this,
because the way you code the stuff we've talked about so far can constrain our options a bit
i'd rather make changes to how i do things based on how we'd *like* the client-side tool to work before committing to any constraints
after all, you can't just stick any ol' python script into a html file and expect everything to work well (I've spent some time confirming this)

there are some things i really want about the client-side tool:
- i want the user interface to be an html/css/js/etc-based thing you handle in your browser so we don't have to invent a unique GUI for it and do other clunky things
- i want functionality to not depend on some external server that someone has to maintain or even pay for
- i want the code used to generate the information provided for the client-side tool to be the same as i want the code used to generate the information provided for developers

To expand on that third point with an example: I don't want to be in a situation where I'm maintaining "votecounter.js" for users who want to build a votecount pagetopping bot chrome extension for game moderators here on MS and also maintaining "votecounter.py" for users who aren't doing any client-side work and want to quickly scrape voting patterns across hundreds of threads for their machine learning project or something. That would double our workload!

unfortunately, the most favored way to interface html/css/js/etc-based pages with python is specifically to have these pages make requests to external servers that execute python code and return information, and javascript doesn't have the data processing capabilities that python does

after a bit of research, i think the solution here is to give clients an application that starts a private, local server (like a pyinstaller-packaged flask app)
that delivers our html/css/js/etc-based user interface in their browser at http://localhost:.../
and talks w/ the interface to execute whatever python functions the clients need for their goals

it gets us everything we want, but with the downside that it's a file clients must download and start
rather than a web page or extension they can directly navigate to in their browser

another upside also comes up with this approach too: the option to make the server non-local if a user desires
beyond opening the possibility of maintaining an persistent server for all clients to use (even though i don't want to do that),
this would make it possible for game moderators to control their modbot instance from any web-connected device as long as they have a computer online hosting it

sry im a bit sleepdeprived now but it seems like a neat idea that also leaves open doors for other ones down the line