Modbot to Help You Run Games: Now With A Votecounter Demo!

This forum is for discussion related to the game.
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #175 (ISO) » Mon May 28, 2018 12:50 pm

Post by Psyche »

Oh, one more option. We can talk to Kison and he'll probably let us build Modbot right into the site.

That might be optimal UX wise, but I dislike it for two reasons:
- There will be institutional resistance to anything that actually changes the site, even if Kison's not the one providing said resistance, and that'll slow us down
- I've always wanted the stuff we're coding to have broad functionality that developers with different purposes besides automating game moderation might use, and I feel like . But honestly, this is a problem with the "translate to javascript" option, too.
- I honestly don't see it requiring anymore work than writing a chrome extension, which would achieve a similar effect without having to interact with the skittles. This is probably the biggest reason not to do it.

Oh, and davesaz is reminding me of a similar option, where we do bite the bullet and get some server space for our python implementation, perhaps even using some of MS's. It...is probably doable. That's already been sort of done with the yearly valentine's day quiz and the anonymous questions service that's in the speakeasy; we could justify using server space for our project, too, since its goal is explicitly to serve game moderation on the site.
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
davesaz
davesaz
He
Survivor
User avatar
User avatar
davesaz
He
Survivor
Survivor
Posts: 12556
Joined: August 24, 2014
Pronoun: He
Location: Socially distant

Post Post #176 (ISO) » Mon May 28, 2018 12:53 pm

Post by davesaz »

There is active interest in modernizing the site, the things you're doing here would be synergistic with that effort. :cool:
Wanna play Minecraft with your ms friends? Check out the minecraft thread, or the channel on discord
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #177 (ISO) » Mon May 28, 2018 1:11 pm

Post by Psyche »

I kinda regret putting the google colab thing at the bottom of the last page because it's my secret favorite.
Probably in part because it would take the least work of all these options.

in fact, i think i'll try to write a version of my pagetopping bot based on the google colab approach

i've confirmed that the maximum life of a colab VM is 12 hours
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #178 (ISO) » Mon May 28, 2018 9:43 pm

Post by Psyche »

EDIT: <there was a link here>

So I didn't have the flexibility I expected, as Javascript is a pretty...restrictive language, to say the least. I think we'll have more freedom with a chrome extension, but I'm not very sure. Think of this as a sort of brainstorm (and also at least a superior demo of my votecounter than what I've been putting out via Flask)

Anyway, to try out the VotecounterDemo here, you just have to click Connect at the top right of the screen (it'll make you log into your Google account if you aren't logged in already), fill out the forms, and run each cell. Instructions are thorough all along the way.
Last edited by Psyche on Tue May 29, 2018 9:39 pm, edited 1 time in total.
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #179 (ISO) » Tue May 29, 2018 3:09 am

Post by yessiree »

What roadblocks did you run into with JS? I've already written a vote counter in vanilla JS that can be ran in the browser console; I can show it to you if you'd like. I'm sure if I turned it into a proper web app and made use of all the libraries out there, the entire code base can be converted to JS pretty easily.
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #180 (ISO) » Tue May 29, 2018 5:09 am

Post by Psyche »

The scraping
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #181 (ISO) » Tue May 29, 2018 7:31 am

Post by yessiree »

like HTML/DOM parsing? that's literally what JS is made for though. There might be semantic differences but I don't think you will have too much trouble with it since you already have experiences with tools like lxml and scrapy

but I like this google colab thing, it is also exactly what we're looking for, so I think this is the way to go?
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #182 (ISO) » Tue May 29, 2018 7:35 am

Post by Psyche »

No, it seems that if your GET request is like on a web page, modern browsers will block it, preventing you from grabbing content.
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #183 (ISO) » Tue May 29, 2018 7:38 am

Post by yessiree »

User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #184 (ISO) » Tue May 29, 2018 7:46 am

Post by yessiree »

or maybe are you running into an CORS issue? yea you are running into an CORS issue.

I think you are requesting a resource from a domain that does not have the 'Access-Control-Allow-Origin' header set (which is the case for any mafiascum.net pages), then you can only do so while on the same domain
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #185 (ISO) » Tue May 29, 2018 7:47 am

Post by Psyche »

do you think you could demo that for me real quick in one of these colab notebooks? you can make a cell render html by making the first line %%html

here's a link to a notebook we can share:
https://drive.google.com/file/d/1tTGWLo ... sp=sharing

if you just edit the javascript cell to scrape the MS homepage and display it for me, I think you might understand my troubles atm

PEDIT: ok yeah you get it
I believe that a Chrome Extension can get around the CORS issue. Is that right?
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #186 (ISO) » Tue May 29, 2018 7:51 am

Post by yessiree »

unlike our python implementation, where we log in before sending any requests, and the 'request' package automatically attaches the session token for every requests we send afterwards

on the other hand, i dont think vanilla javascript has automatic session management. so you'd have to use a library for that, Axios.js for example, but I think if you followed the same model it will work
User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #187 (ISO) » Tue May 29, 2018 8:08 am

Post by yessiree »

In post 185, Psyche wrote:I believe that a Chrome Extension can get around the CORS issue. Is that right?
yes, provided that the user is logged in AND on mafiascum.net when using it

trying running this code snippet in your browser console, you should be able to see the document in the console

Code: Select all

var test = function() {
    var req = new XMLHttpRequest();

    req.open('GET', 'https://forum.mafiascum.net/index.php', true);
    req.responseType = 'document';

    req.onload = function() {
        console.log(req.response);
    }

    req.send();
}()
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #188 (ISO) » Tue May 29, 2018 8:13 am

Post by Psyche »

ugh i would really hate if this is the way to go but i do think we should try building a demo of this to see how well it works
translating my votecounter will be the hardest part but i do think it's doable, i do think it's doable

(by the way, i've come up with some ways to make it run faster than i haven't implemented yet)
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #189 (ISO) » Tue May 29, 2018 8:18 am

Post by yessiree »

but then again, if we use google colab to run ipython notebooks we dont need any of this

I'm completely new to this but I think python-javascript resource sharing is possible, so you can do stuff like: getting web resources with python, then letting javascript handle any html parsing work, storing the result texts, then accessing those results from the python side again, etc...

pedit: ok, it'll take a lot of work though, i dont know how well the packages you used would translate to js (well the regex part should be fine, but not sure about the other ones)
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #190 (ISO) » Tue May 29, 2018 8:20 am

Post by Psyche »

mehh
like i said before, the ipython notebook thing is really easy

the only challenge is handling the limited lifetime of individual runtimes
i believe chrome extensions already exist that can automate reconnecting when one dies
an alternative is just alerting users (we have a variety of options for this, including emailing, texting, sending the moderator a pm, in-browser notification and so on)
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #191 (ISO) » Tue May 29, 2018 8:24 am

Post by Psyche »

i was disappointed i couldn't make a user-friendly form for the votecount parameters stuff like i did for login information
at the same time, though, it may be that a text editting thing is better for that sort of processing
i really like how easily the notebook enables reasonably savvy game moderators to customize their pipeline though
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #192 (ISO) » Tue May 29, 2018 9:02 am

Post by Psyche »

I think ultimately I am not prepared to spend time converting this into javascript just for a marginally better UX. I really want to transition to other projects. I think we should either rely on google colab as a client side tool or try harder to make the flask approach easier on the eyes.

The google colab approach, even in it’s current state, provides a better user experience (especially since it saves input for later use), frees us up to focus on features rather than interface, and gives creative freedom to mods that would always be missing in a .exe file. Even its most glaring issues (kernel longevity and setup time) can be resolved by having users install Jupyter to their systems and connecting to that local runtime instead of starting a cloud server.

Furthermore, google colab a new product that’s improving in functionality at an amazing pace. I believe soon I’ll have even more control over functionality and ui than I do now. Instead of worrying so much, we can just piggyback off of smarter people.
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #193 (ISO) » Tue May 29, 2018 9:13 am

Post by yessiree »

yea i agree just go with google colab
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #194 (ISO) » Tue May 29, 2018 9:48 pm

Post by Psyche »

Ok, I'm going to focus in the short term on improving the votecounter demo notebook so that it's faster and more worthwhile.
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #195 (ISO) » Tue May 29, 2018 10:44 pm

Post by Psyche »

Anyway, I have some ideas about how we can really commit to this idea of using a computational notebook to moderate games.

The idea is to make these notebooks have a sort of dual functionality: they are a set of computational tools on the one hand, and a living document/record/representation of the gamestate on the other. The idea is that when you start working on a new game to moderate, the notebook begins as the place where you centralize information about your setup, your roles, and everything - with a gamut of click-to-run computational tools to make it easier to record that information.

Later, when you need your setup reviewed or are co-designing it, etc, you can either link reviewers directly to the notebook (which support commenting like typical google docs), or we can make accessible another specific click-to-run tool for pming or posting setup information to the relevant place.

Similarly, we can make tools to quickly generate Game OPs and send out Role PMs, and similarly output similar setup information at the end of a game.

And of course, in between we automate basic modding activities like posting votecounts, collecting and handling night actions, updating the OP, and so on, storing a record of key events such that it's easy to see what's happened from reading the notebook on the one hand, or easy to transmit to a thread or specific user on the other.

Indeed, when the game is over, the notebook should be a complete record of all the activities and events that went into the game, and can be duly converted into a MafiaWiki entry or whatever.

By embracing what iPython notebooks are about instead of just using them because we don't have any better option, we can totally blur the line between documenting your game and running it.
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #196 (ISO) » Tue May 29, 2018 11:01 pm

Post by Psyche »

biggest challenge for this vision is that i need a way to programmatically edit the notebook itself as a result of running a code cell
that way, form information for further cells can be pre-populated, and old cells post-populated

this would support basic stuff like not always saving a mod's user/password after they input it and similarly automatically toggling that you don't want this cell to send Role PMs to everyone (for example) the next time it's run.
this stuff is important for making notebooks more shareable (something I explicitly warn users not to do a lot with this demo), as well as for giving mods the option of just clicking "Run All" to make stuff happen instead of manually running a setup cell and manually running the specific tool cell they want, etc.
it also means that people can input setup information and stuff into formatted Markdown cells instead of putting them inside Python string variables and thereby making users interact with the code

if we can get that, then i'd be confident that a notebook approach has a good balance of being feature-full and offering a comfortable learning curve for mods who don't code

***

I believe all this is doable by using the Google Drive API to access the guts of the notebook file and changing that; it should then update in realtime. The downside to this is that users will probably have to do additional work to give their notebooks this kind of access to their Google Drives (their virtual server is separate from that, and we'll probably want users to have the option to not connect to it anyway). We want to reduce that workload as much as possible, while also protecting notebooks from being insecure access points to users' Google accounts.

***

Ok, this turned out to be easy thanks to how amazing Google Colab is.

Code: Select all

from google.colab import auth
auth.authenticate_user()


is all that's necessary to authorize the notebook to access the user's google account - as long as you're connected to the google virtual server associated with the account. This is an amazingly simple way to get access without making users store important information in the notebook or jump through other hoops. It doesn't seem to work if you're connected to a local runtime (though that might change if I figure out how to install the package associated with "google.colab"?)

For people using their local runtimes, it looks like the leading alternative is Pydrive. With Pydrive, we can basically have users sign into google drive whenever they start running the notebook. But the result of this is that they basically have to sign in twice: once to open the notebook in Google Colab, and again to use the notebook. Seems annoying. There are some other alternative processes that Pydrive supports, but none of them seem as convenient as the above.
EDIT: Hm, actually it seems that Pydrive doesn't work the right way for local runtimes, either.

Finally, people can set up an API key and then place it into their notebooks for authorization. I think that's terrible for user experience.
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #197 (ISO) » Tue May 29, 2018 11:57 pm

Post by Psyche »

ok, so where necessary we don't have to put the API key in the notebook
if users are using their local runtime they can just store it wherever they want, and if they're using the cloud - well there should never be a reason to do this
it's not as nice, but it's nice enough assuming users using the local runtime will be comparatively savvy

for now ill design the notebook with cloud users in mind
next step is to confirm that i can edit the notebook in realtime like i'm hoping
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
Psyche
Psyche
he/they
Survivor
User avatar
User avatar
Psyche
he/they
Survivor
Survivor
Posts: 10069
Joined: April 28, 2011
Pronoun: he/they

Post Post #198 (ISO) » Wed May 30, 2018 2:16 am

Post by Psyche »

So, it turns out that it's impossible to programmatically update notebook cells like I wanted. Even though I can change the associated notebook file stored in a user's Drive, the currently open notebook doesn't register the changes and overwrites them soon afterward.

But in the process, I found support in Google Colab for cross-communication between python and javascript output, something I'd had trouble finding beforehand. I still need to explore just how much control we get for that, but that's really big! It means we have almost total control over user experience instead of having to rely on Python/Markdown code cells and the simple Form feature that Google colab offers for specifying individual variables. The possibilities are endless, and I'm gonna spend a bit of time exploring them.
youtube playlist extracter | donbot | game scraper | vca | setupsim | strategist | llm
User avatar
yessiree
yessiree
he
Mafia Scum
User avatar
User avatar
yessiree
he
Mafia Scum
Mafia Scum
Posts: 4361
Joined: June 6, 2013
Pronoun: he

Post Post #199 (ISO) » Wed May 30, 2018 4:48 am

Post by yessiree »

I wouldn't get too excited; you only have access to the form output's iframe with javascript, i dont think you can do anything to the form itself
Post Reply

Return to “Mafia Discussion”