In post 11, hitogoroshi wrote:Hoopla, when doing this analysis, did you get much of a sense of what proportion of the game played "last scum PR alive can use action/kill", and/or how many scum wins were achieved with one living PR scum?
When I was in the normal review group, I used to make mods select beforehand which option they'd go with, but I don't know if this is standardized now or what. Perhaps someone currently in the group can tell you about the frequency of these.
In post 11, hitogoroshi wrote:I think I'm pretty infamous for not liking mafia goons and I'm curious what makes my setups tend to skew neutral to slightly town-sided despite usually having full PR scum teams. I think it might be because I don't give the last scum double actions when they're the last alive - and if that rule is something that helps full-PR scumteams be balanced, could be something to look in to.
Perhaps as a well-established mod you have a higher ratio of good players which decreases your chances of a VI-ridden town and making it harder for scum, or perhaps it is just random luck over a small sample size. Your games could be scum-sided for all you know, but a couple of results or closeness of games makes it seem otherwise.
But this is the thing - many people seem to think this way. They think if a game was close or the town won, it must have been balanced. There isn't really a way to tell if a game was balanced or not based on the result. You have to make inferences from large collections of data to establish patterns across setups with similar design elements. Most people's instincts for setup design are scumsided - we know this because across 3:9's winrates were 36% and in 3:10's (a much friendly base number to work with), winrates are only 40%. People are instinctively giving scum more power and town less because that is what 'feels' right.
The problem is, because each game has a lot of variation in roles and our sample size still isn't huge yet, there is only so much data you can collect (and it has to be broad), so when designing setups you still have to make a lot of inferences and instinct calls.
~~
The main things I've learnt are...
- Blocking roles for scum are detrimental to town winrates.
I've already covered that a bit, but in 3:9's with a scum-RB, mafia won 69% and in 3:10's, scum win 66%. I think the reason this is the most effective scumrole, is because it gives scum twice the chances of a neutralizing a town PR at night (either hitting their kill or block). An advantage PR's have in normal games is when they're outed, they can draw the NK, taking attention away from obv-townies or other PR's. But with a RB, scum can blanket the town at night far more effectively.
- Vigs were the most powerful role in 3:9's for town, but in 3:10's they're borderline anti-town.
In 3:9's, setups with a vig were an admirable 44% winrate for town (well above the average of 36%), while setups without a vig were a mere 28%. This is primarily because the vig in 3:9's were getting their first shot for free (ie; they don't cost the town a mislynch for shooting). The majority of shots vigs took in 3:9's were in even numbers, whereas the majority of shots vigs take in 3:10's are in odd numbers and come at the expense of a town mislynch.
The main purpose of running 3:10's instead of 3:9's was to start town in odd numbers, which gives them an extra mislynch with better odds for the rest of the game's lynches. To me it's no surprise that vigs in 3:10's have a
worse
town winrate than 3:9 games with vigs. 3:10 games with vigs only have a 39% winrate, and it's only 35% in games with 1-shot-vigs (a role that eliminates the chances of getting back in odds with another shot, something a full-vig or multi-shot vig can).
Personally, I think 1-shot vigs are an extremely anti-town role for 3:10 games, as they will almost certainly be making a -EV shot and roles like doctors/roleblockers etc. simply don't save a kill often enough to balance this out. I probably wouldn't use this role at all in a 3:10 game without something like a Weak Doc, 2+ of JK/Doc/RB/1-shot-BP or possibly another 1-shot-vig. The games we're making are already unbalanced and I think 1-shot-vig games are some of the worst. They offer the same amount of town-aligned kills in the game, but at worse lynching odds for every day you're in evens.
- The strength of the Cop has gone up with the additional dayphase 3:10's have brought
In 3:9's, Cop winrates were fractionally below average (35%, as opposed to 36% overall), whereas in 3:10's Cop winrates are 47% (compared to 41% overall). I think it can conclusively be said, the value of the Cop has increased given the length of the game has increased. It gives the Cop a higher chance of survival deeper into the game, something that Cops were having difficulty with in 3:9's. IIRC, only 40% of Cops lived to Day 3 in 3:9 games, which is predominately where people will claim Cop, so many Cops would die/be outed by D3 without actually helping the town at all (and that doesn't even factor in things like your investigation targets dying or being blocked etc). It would also sometimes be mylo on D3 in 3:9's which makes Cop claims less trustworthy/untestable without putting the game on the line.
To me, it's clear that an extra day phase is extremely helpful for Cops and this is reflected in the numbers. Another role winrate I checked out were masons, who had a 44% winrate, again, a bit better than average.
I think for the most part, we're collectively not good enough scumhunters to win near 50% without some confirmed information being in the game to aid process-of-elimination and increase lynch odds on some days. Masons and Cops are the two best roles for narrowing down the pool and trapping scum, and to me, it makes sense that these roles win more often than vigs as more lynches at improved odds (+ the occasionally guilty) is far better than the vig speeding up the game and making odds worse.
~~
In post 8, implosion wrote:Thanks for doing this again, Hoopla. I enjoyed looking at the stats last time as well.
I agree with you in general. I also have felt for a while that people tend to call setups more townsided than they are. I'm curious what you think about weak PRs in terms of balancing (since your guidelines never suggest any), and where you draw the cutoff in your guidelines for what a weak, middling or strong town PR is. There has definitely been a trend towards both having generally weaker town PRs in mini normals, and weakening those that have been used (most notably, it seems like the trio of gunsmith + x-shot-vig + mafia doctor, three roles which all make sense together which I've seen in at least one game and I think has been in others, makes the stereotypically pretty strong gunsmith somewhat weak - in fact just looking at some recent completed games the second game I looked at had this trio as well).
It sort of feels like a couple of weak town PRs should be about as strong as a single stronger one, but if there are too many then it sort of becomes a stoofer's-third-law grayzone, particularly depending on what the site meta around mini normal setup design is like.
I agree with stoofer's law, especially if the roles aren't confirming or overly powerful type roles that can puzzle the game out. Most of the time overly PR heavy games in mini normals have a heap of backup doc, even-night tracker, one-night-rb type roles that will generally not receive/do any anything meaningful all game, but give an illusion of power. Because these roles can be on either side too, them claiming doesn't often clear them or attract the NK or do any of the cool things other full PR's can do.
For the most part, I think the simplest way to produce a balanced game is to include a Cop or Masons but they're a very blunt tool in terms of creating balance as you always have confirmed players in the game, and understandably some mods don't like using this role. The challenge is to create a balanced game when these roles aren't present, and I think it's necessary that the town has multiple full-power roles, like Watcher, Tracker, Jailkeeper, Gunsmith etc, with scum getting no more than a 1-shot RB. The dinky little weak PR's that can on either team like 1-shot Doc, Even-Night Role Cop etc, should be like garnish on the finished setup to make it a bit unpredictable, and not really counted when tallying up the power you've given to town. But I'll talk about that more when I do this:
In post 9, BBmolla wrote:Hoopla can you give me an example of a couple setups you think are balanced?
I noted a handful of setups in that I noticed that looked the least balanced for town when I was looking through all the games. Theoretically, the weakest looking setups in data set of 40% town wins should be the setups we should be trying to eliminate, so I will show you some of those when I have time. Then I'll analyse a few others that I think are good.
Trying to see what you're aiming for.
When I was going through the games, I noted a handful of the really bad ones. I'll make a post soon analysing a few different normal setups that have run and show you some that I think are good.