Beta testing mafia games?

[Sméagol]

Has this been done before?

Are mods interested in this?
Are players interested in this?

I have ideas about this, but I want some initial responses first.
In short: if you have a complicated set-up, you run a trial for say, 3 days, simulating a complete day- and night cycle, in order to get results which are better than random, to check if there are unwanted effects you may have overlooked. Beta testers are obviously encouraged to break your game.

I don't have a need for it currently, but can imagine wanting to do beta tests in the future for some of the ideas I have.

[Sméagol]

Obviously, I'm not talking about mountainous set-ups..

[Sméagol]

That's indeed one of the potential problems. Open or semi-opens are fine, but with closed set-ups you'll need a dedicated beta test team that doesn't have any interest in playing the game for real. I have no idea how large the playerbase on this site currently is, I get the feeling it's dwindling? That said, since people are not expected to play with maximum effort, and probably just one complete day- and nightcycle (depending on your needs), it's possible you may tap in the group of lurkers, ex-players, or other people who are on this site for some reason other than playing mafia games. This includes me for example.. I'm not sure how likely I am to join a game soon, maybe when the Blitz queue is back up. But Beta testing sounds like fun, I'm definitely up for that.

[Sméagol]

[..]
you can still write programs for more complicated setups, it's just the more roles where there are grey areas with regards to decisions, the more complex it gets

e.g. cops, ICs, etc are relatively easy to account for (assuming optimal play and factoring in an average of X claims a day based on the size of the game where X is some number you'd need to figure out by grinding sample data), non-compulsary vigs on the other hand are an example of a role that is not so easy to account for... and whether or not things like day-talk, neighbourhoods, etc have much of an effect on win cons and (if so) how much of an effect also isn't so easy to account for

Problems:
1] Automatic simulations don't take into account psychology.
2] The whole point is to check for things you may have overlooked.. A script you made will obviously miss the things you have overlooked. A reviewer can help with this, but if it's a complicated role madness game, it's very possible you both overlook things.
3] A script won't try to break your game, which is exactly what you'd be asking of beta testers.
4] Plus this:

this is something i keep saying i'll write a program for one day but never end up having the time for (or wanting to spend the time i have doing it). i imagine i could write one for normal setups without too much difficulty

I'm a lazy motherfucker. If you build a script with the logic for ability interactions.. Then add a forum and voting capabilities, and you have an automated game. I actually came pretty far with a forum with voting capabilities, but never finished it. But that aside. Coding a script that takes into account ability interactions, and adds some rudimentary AI, is a lot harder than asking players to have a quick go at your set-up.

[..]]

Yes, this was I meant. Also, checking out the "EV Project" page on the wiki may be a good starting point too. I just don't see how the beta testing would help too much.

I did check it on the wiki, didn't see anything useful for my potential future needs.

Would probably be interesting to beta test the beta testing. Think I'll just go do it when I have a set-up that could really make use of it, and see if there's any interest in it by then.

[Sméagol]

Improv skills is no doubt another useful skill for a mod to have.. I've had some unforseen situations of my own off-site.