Mafiascum.net

Oh, one more option. We can talk to Kison and he'll probably let us build Modbot right into the site.

That might be optimal UX wise, but I dislike it for two reasons:
- There will be institutional resistance to anything that actually changes the site, even if Kison's not the one providing said resistance, and that'll slow us down
- I've always wanted the stuff we're coding to have broad functionality that developers with different purposes besides automating game moderation might use, and I feel like . But honestly, this is a problem with the "translate to javascript" option, too.
- I honestly don't see it requiring anymore work than writing a chrome extension, which would achieve a similar effect without having to interact with the skittles. This is probably the biggest reason not to do it.

Oh, and davesaz is reminding me of a similar option, where we do bite the bullet and get some server space for our python implementation, perhaps even using some of MS's. It...is probably doable. That's already been sort of done with the yearly valentine's day quiz and the anonymous questions service that's in the speakeasy; we could justify using server space for our project, too, since its goal is explicitly to serve game moderation on the site.

There is active interest in modernizing the site, the things you're doing here would be synergistic with that effort.

I kinda regret putting the google colab thing at the bottom of the last page because it's my secret favorite.
Probably in part because it would take the least work of all these options.

in fact, i think i'll try to write a version of my pagetopping bot based on the google colab approach

i've confirmed that the maximum life of a colab VM is 12 hours

EDIT: <there was a link here>

So I didn't have the flexibility I expected, as Javascript is a pretty...restrictive language, to say the least. I think we'll have more freedom with a chrome extension, but I'm not very sure. Think of this as a sort of brainstorm (and also at least a superior demo of my votecounter than what I've been putting out via Flask)

Anyway, to try out the VotecounterDemo here, you just have to click Connect at the top right of the screen (it'll make you log into your Google account if you aren't logged in already), fill out the forms, and run each cell. Instructions are thorough all along the way.

What roadblocks did you run into with JS? I've already written a vote counter in vanilla JS that can be ran in the browser console; I can show it to you if you'd like. I'm sure if I turned it into a proper web app and made use of all the libraries out there, the entire code base can be converted to JS pretty easily.

The scraping

like HTML/DOM parsing? that's literally what JS is made for though. There might be semantic differences but I don't think you will have too much trouble with it since you already have experiences with tools like lxml and scrapy

but I like this google colab thing, it is also exactly what we're looking for, so I think this is the way to go?

No, it seems that if your GET request is like on a web page, modern browsers will block it, preventing you from grabbing content.

try XMLHttpRequest
https://developer.mozilla.org/en-US/doc ... ttpRequest

or maybe are you running into an CORS issue? yea you are running into an CORS issue.

I think you are requesting a resource from a domain that does not have the 'Access-Control-Allow-Origin' header set (which is the case for any mafiascum.net pages), then you can only do so while on the same domain

do you think you could demo that for me real quick in one of these colab notebooks? you can make a cell render html by making the first line %%html

here's a link to a notebook we can share:
https://drive.google.com/file/d/1tTGWLo ... sp=sharing

if you just edit the javascript cell to scrape the MS homepage and display it for me, I think you might understand my troubles atm

PEDIT: ok yeah you get it
I believe that a Chrome Extension can get around the CORS issue. Is that right?

unlike our python implementation, where we log in before sending any requests, and the 'request' package automatically attaches the session token for every requests we send afterwards

on the other hand, i dont think vanilla javascript has automatic session management. so you'd have to use a library for that, Axios.js for example, but I think if you followed the same model it will work

In post 185, Psyche wrote:I believe that a Chrome Extension can get around the CORS issue. Is that right?

yes, provided that the user is logged in AND on mafiascum.net when using it

trying running this code snippet in your browser console, you should be able to see the document in the console

ugh i would really hate if this is the way to go but i do think we should try building a demo of this to see how well it works
translating my votecounter will be the hardest part but i do think it's doable, i do think it's doable

(by the way, i've come up with some ways to make it run faster than i haven't implemented yet)

but then again, if we use google colab to run ipython notebooks we dont need any of this

I'm completely new to this but I think python-javascript resource sharing is possible, so you can do stuff like: getting web resources with python, then letting javascript handle any html parsing work, storing the result texts, then accessing those results from the python side again, etc...

pedit: ok, it'll take a lot of work though, i dont know how well the packages you used would translate to js (well the regex part should be fine, but not sure about the other ones)

mehh
like i said before, the ipython notebook thing is really easy

the only challenge is handling the limited lifetime of individual runtimes
i believe chrome extensions already exist that can automate reconnecting when one dies
an alternative is just alerting users (we have a variety of options for this, including emailing, texting, sending the moderator a pm, in-browser notification and so on)

i was disappointed i couldn't make a user-friendly form for the votecount parameters stuff like i did for login information
at the same time, though, it may be that a text editting thing is better for that sort of processing
i really like how easily the notebook enables reasonably savvy game moderators to customize their pipeline though

I think ultimately I am not prepared to spend time converting this into javascript just for a marginally better UX. I really want to transition to other projects. I think we should either rely on google colab as a client side tool or try harder to make the flask approach easier on the eyes.

The google colab approach, even in it’s current state, provides a better user experience (especially since it saves input for later use), frees us up to focus on features rather than interface, and gives creative freedom to mods that would always be missing in a .exe file. Even its most glaring issues (kernel longevity and setup time) can be resolved by having users install Jupyter to their systems and connecting to that local runtime instead of starting a cloud server.

Furthermore, google colab a new product that’s improving in functionality at an amazing pace. I believe soon I’ll have even more control over functionality and ui than I do now. Instead of worrying so much, we can just piggyback off of smarter people.

yea i agree just go with google colab

Ok, I'm going to focus in the short term on improving the votecounter demo notebook so that it's faster and more worthwhile.

Anyway, I have some ideas about how we can really commit to this idea of using a computational notebook to moderate games.

The idea is to make these notebooks have a sort of dual functionality: they are a set of computational tools on the one hand, and a living document/record/representation of the gamestate on the other. The idea is that when you start working on a new game to moderate, the notebook begins as the place where you centralize information about your setup, your roles, and everything - with a gamut of click-to-run computational tools to make it easier to record that information.

Later, when you need your setup reviewed or are co-designing it, etc, you can either link reviewers directly to the notebook (which support commenting like typical google docs), or we can make accessible another specific click-to-run tool for pming or posting setup information to the relevant place.

Similarly, we can make tools to quickly generate Game OPs and send out Role PMs, and similarly output similar setup information at the end of a game.

And of course, in between we automate basic modding activities like posting votecounts, collecting and handling night actions, updating the OP, and so on, storing a record of key events such that it's easy to see what's happened from reading the notebook on the one hand, or easy to transmit to a thread or specific user on the other.

Indeed, when the game is over, the notebook should be a complete record of all the activities and events that went into the game, and can be duly converted into a MafiaWiki entry or whatever.

By embracing what iPython notebooks are about instead of just using them because we don't have any better option, we can totally blur the line between documenting your game and running it.

biggest challenge for this vision is that i need a way to programmatically edit the notebook itself as a result of running a code cell
that way, form information for further cells can be pre-populated, and old cells post-populated

this would support basic stuff like not always saving a mod's user/password after they input it and similarly automatically toggling that you don't want this cell to send Role PMs to everyone (for example) the next time it's run.
this stuff is important for making notebooks more shareable (something I explicitly warn users not to do a lot with this demo), as well as for giving mods the option of just clicking "Run All" to make stuff happen instead of manually running a setup cell and manually running the specific tool cell they want, etc.
it also means that people can input setup information and stuff into formatted Markdown cells instead of putting them inside Python string variables and thereby making users interact with the code

if we can get that, then i'd be confident that a notebook approach has a good balance of being feature-full and offering a comfortable learning curve for mods who don't code

***

I believe all this is doable by using the Google Drive API to access the guts of the notebook file and changing that; it should then update in realtime. The downside to this is that users will probably have to do additional work to give their notebooks this kind of access to their Google Drives (their virtual server is separate from that, and we'll probably want users to have the option to not connect to it anyway). We want to reduce that workload as much as possible, while also protecting notebooks from being insecure access points to users' Google accounts.

***

Ok, this turned out to be easy thanks to how amazing Google Colab is.



is all that's necessary to authorize the notebook to access the user's google account - as long as you're connected to the google virtual server associated with the account. This is an amazingly simple way to get access without making users store important information in the notebook or jump through other hoops. It doesn't seem to work if you're connected to a local runtime (though that might change if I figure out how to install the package associated with "google.colab"?)

For people using their local runtimes, it looks like the leading alternative is Pydrive. With Pydrive, we can basically have users sign into google drive whenever they start running the notebook. But the result of this is that they basically have to sign in twice: once to open the notebook in Google Colab, and again to use the notebook. Seems annoying. There are some other alternative processes that Pydrive supports, but none of them seem as convenient as the above.
EDIT: Hm, actually it seems that Pydrive doesn't work the right way for local runtimes, either.

Finally, people can set up an API key and then place it into their notebooks for authorization. I think that's terrible for user experience.

ok, so where necessary we don't have to put the API key in the notebook
if users are using their local runtime they can just store it wherever they want, and if they're using the cloud - well there should never be a reason to do this
it's not as nice, but it's nice enough assuming users using the local runtime will be comparatively savvy

for now ill design the notebook with cloud users in mind
next step is to confirm that i can edit the notebook in realtime like i'm hoping

So, it turns out that it's impossible to programmatically update notebook cells like I wanted. Even though I can change the associated notebook file stored in a user's Drive, the currently open notebook doesn't register the changes and overwrites them soon afterward.

But in the process, I found support in Google Colab for cross-communication between python and javascript output, something I'd had trouble finding beforehand. I still need to explore just how much control we get for that, but that's really big! It means we have almost total control over user experience instead of having to rely on Python/Markdown code cells and the simple Form feature that Google colab offers for specifying individual variables. The possibilities are endless, and I'm gonna spend a bit of time exploring them.

I wouldn't get too excited; you only have access to the form output's iframe with javascript, i dont think you can do anything to the form itself