Mini Normal Stats Update

callforjudgement · Post Post #50 (ISO) » Fri Jul 08, 2016 10:15 am

In post 48, mith wrote:2 wins out of 7 is not anywhere close to a statistically significant result on the "actual" balance (never mind that only 5 are actually listed on the page, and one of those was 11p).

Right, I was trying to point out that they probably don't tell the full story (and I know they don't include every game). So I'm agreed with you there.

BBmolla · Post Post #51 (ISO) » Fri Jul 08, 2016 2:35 pm

What's EV for 2:11, 3:11, 4:11, 5:11 etc.? Just curious

zMuffinMan · Post Post #52 (ISO) » Fri Jul 08, 2016 3:06 pm

11:4 is 87.8% scum win, 11:2 is 60.5% scum win

i threw away the code i wrote for an odds calculator, but even numbers would generally be expected to perform worse than if you removed a vt (worse odds of hitting scum and the same amount of lynches to do it). so 11:3 is slightly worse than 10:3 (<20% without doing exact calculations)

dont think i ever bothered calculating past 4 scum in a vanilla because break-even on 4 scum was something like 80+ town?

Antihero · Post Post #53 (ISO) » Fri Jul 08, 2016 3:18 pm

In post 52, zMuffinMan wrote:dont think i ever bothered calculating past 4 scum in a vanilla because break-even on 4 scum was something like 80+ town?

sounds like a good time

someone run that

zMuffinMan · Post Post #54 (ISO) » Fri Jul 08, 2016 3:29 pm

actually im wrong woulda been closer to 60 i think? but still

Hoopla · Post Post #55 (ISO) » Fri Jul 08, 2016 7:24 pm

Alright, looking at some individual setups, I'll show you some that I think are unbalanced first. The easiest ones to pick off are those with just two town PR's, such as:

Spoiler: Mini 1445

Spoiler: Mini 1517

There were a couple more with just two PR's, but I think we all get the point that this isn't enough, so moving on. The biggest issues for games with three town PR's is giving scum significant power (RB or multiple powers), giving the town roles like 1-shot vig where they're likely get stuck in evens for a significant part of the game, or diluting the three roles town get with a lot of modifiers. Here's a couple games with those issues:

Spoiler: Mini 1422

Spoiler: Mini 1653

Here's a couple more I think are unbalanced for other reasons:

Spoiler: Mini 1514

Spoiler: Mini 1304

~~

Anyway, here are a variety of good setups that I think were fair.

Spoiler: Mini 1387

Spoiler: Mini 1462

Spoiler: Mini 1509

Spoiler: Mini 1516

Spoiler: Mini 1540

Spoiler: Mini 1699

Spoiler: Mini 1719

~~

The problem when assessing balance on a macro level like this, you kinda get the assumption that all the games are balanced 60/40 to scum, but this isn't true. There are plenty of games in Little Italy that will be balanced or even potentially town-sided (though, these ones are rare and I haven't seen any glaring examples of this), which means there are a bunch of setups that we're running that are even more scumsided than the current winrates. These are the setups we should be trying to eliminate. I don't think we need to obsessively pore over every setup and get it perfect, but there are some clear ways we can improve setups without making dramatic overhauls or spending ages on each setup. ANYWAY...

My checklist for balancing 3:10's:

1) If designing a game with just 3 town PR's, make sure they're strong and synergize well together.

Example of strong roles: Cops, Masons, Vigs, Watchers, Trackers, X-shot BP, Jailkeeper, Doctor, Innocent Child, etc. Stuff like Role Cops, Neighbourisers can be strong situationally, but most of the time they're not.

2) Don't dilute your town PR's with X-shot, Odd/Even-Night modifiers.

If you want to use these modifiers, include a fourth or fifth PR for town.

3) If your game only has 3 town PR's, consider an all goon team or something like an Encryptor.

Scum don't need to have power in these low power games. Encryptor is a good role to not negate town's small power pool. If you really want to give scum something, make it 1-Shot or give town a fourth PR.

4) Don't use a mafia RB without giving the town four decent PR's.

This role simply gives mafia too much agency in shutting down towns' power and it's reflected in the stats: mafia win two thirds of the time they're given a RB, and many of these games are in 3 town PR games. Make sure the town has a good amount of power to counter this - perhaps even consider roles like Masons or 1-Shot BP's for town that are immune to the RB.

5) When adding killing roles to the game, think about how likely it is town will get stuck in even numbers.

The main purpose of moving to 3:10 games was to give town an extra mislynch. When you use a role like 1-Shot or Even-Night Vig, most of the time this player is going to put the town in even numbers again giving the town worse lynching odds during the day at the cost of a mislynch. Vigs are a fun and cool role to use, but either make them unlimited or give the town two of something like BP, Jailkeeper, Doc, Weak Doc, Commuter etc. Do not design a 3 town PR game with one of them being 1-Shot Vig. It isn't strong enough and often will make -EV shots. A 35% town winrate in 1-Shot Vig games is a testament to this fact.

6) Don't think every strong town PR needs a scum role to negate it.

I see a lot of games with 2-3 scum PR's that seem to be there as a direct counter to town power. Most town PR's aren't successful often enough to justify giving scum something to subvert them. Again, consider the 1-Shot Modifier for scum if you really want to give them a Ninja/Strongman/Roleblocker.

7) If you want to have a game with less town power, that's OK...

...but run a setup of 2:11, 2:1:10 or a 3:10 with a Traitor. Again, in 3:10's, three good town PR's with no/limited scum power should be the absolute minimum you're aiming for. If you're ever in doubt, add a fourth town PR.

zMuffinMan · Post Post #56 (ISO) » Fri Jul 08, 2016 9:00 pm

those are decent guidelines

callforjudgement · Post Post #57 (ISO) » Fri Jul 08, 2016 9:35 pm

Now I'm really curious as to whether town win rates went up after Encryptor became explicitly normal. Giving scum power when there's no reason to is a pretty common mistake, but Encryptors serve as a nice relief valve for the designs of newbie mods (it's one of the most commonly seen roles in setups designed by first-time mods). Note that it's actually negative utility compared to just giving scum daytalk.

On the subject of scum power roles, by the way, it's worth noting that you can give scum power roles that don't make their team stronger. In one of my Normals I gave scum a Tracker, and the main purpose of this was to try to steer them into fixing potential odd/even issues of the setup for me; given that many of the town's power roles didn't target, it didn't meaningfully help them hurt the Town (it might have improved their fakeclaims slightly but that's about it). Likewise, it's possible to give scum a power role that doesn't actually do anything in the setup, to mislead them about which of their players is the most valuable.

I definitely agree with the odd/even points made here. When I'm reviewing a setup, I count the power of a 1-shot vig as being based

entirely

on the ability for town to get back to odds (and the power of a 2+-shot vig is based on the chances of it hitting twice and/or the town saving a player). This is counteracted by the risk of killing a townie who could otherwise save themself by claiming, and so a 1-shot vig can often be negative utility. (The main upside for the role is that the vig can at least normally confirm themself by claiming the kill.) I should mention that saving roles like Doctor have exactly the same issue as killing roles like Vigilante (actually, I'd be willing to consider Bodyguard

more

useful for town than Doctor in some setups, because town may have to no-lynch anyway after a save and there's less ambiguity in the night results with a Bodyguard than with a Doctor). That said, this isn't a mistake I see mods commonly make nowadays; people are fairly aware of it. So it may be that analysing old setups doesn't make much sense to learn about current balance issues. (I'm much more interested in whether normals are being misbalanced right now, than if they've historically been messed up. What are the stats for Mini Normals since the Micro queue was established? That'd allow for a fairer 10:3 to 7:2 comparison.)

Incidentally, I was actually in one of the games you mentioned above (Mini 1304), and agree with you that it was highly scumsided. (That said, I actually figured out the scum the morning after the night I got nightkilled, which basically just shows that scum made a good nightkill choice. They were trying to be too cute, though, and could have won easily with a simpler strategy.)

Meanwhile, I disagree with the approach of assessing balance by counting roles. As an extreme case, 3×Cops + 7×VT versus 3×Goon is highly townsided even if you allow for the town potentially not believing the setup, whereas 3×Doctor + 7×VT versus 3×Goon is probably somewhat scumsided in practice even if you allow for the possibility of the Doctors identifying each other (e.g. at massclaim) and setting up a protection loop. My preferred method of assessing balance in a setup that I don't have a similar setup to compare it with is to count the number of players that are likely to get their alignment confirmed overNight over the course of the game (my current estimate for this when balancing a 10:3 is "slightly more than 3"), plus the number of players likely to be mostly confirmed via what makes sense in the setup at massclaim (ideally around 1).

An example: Looking at Mini 1445, we have Cop, JK, 8×VT vs. Tracker, 2×Goon; the Tracker effectively counts as a delayed vanillaise for the purpose of balance (because scum will kill a suspected power role the night after), so we can give the power roles a 11/13 chance of being alive at the end of N1, 8/13 at the end of N2, 5/13 at the end of N3. Let's call this 80%, 60%, 40% for the purpose of approximation (as the numbers aren't perfectly accurate anyway), so we can assume that on average the Cop and JK each get a little under 2 shots off (slightly less for the Cop due to the chance of an accidental block, slightly more for the JK as their role can save themself from dying as it confirms a scum). The JK is almost blank in this setup; if they stop a kill early and aim at unlikely kill targets they may guess that their target is scum, but unlikely kill targets rarely perform kills, and thus the JK's only useful late (when they probably won't be alive). Any positive effect they have is therefore mostly a rounding error. I'll happily allocate 1 and a half or even two confirmations to the Cop (it's a powerful role), and if I were being generous I'd add one massclaim confirmation to the town total (not two as scum can cause confusion by claiming power, with only two town power roles). We're still somewhat short, though. You could add something like an Innocent Child to this game and it'd be much closer to balanced (especially if they were revealed at game start to give some amount of JK WIFOM that'll likely work mostly in town's favour). In this case, my method comes out with much the same conclusion as Hoopla's. If the town roles were different, though (say Cop+Watcher, which are both very powerful and also synergise well with each other), I'd be willing to let the game run with just the two power roles, at least from the point of view of win:loss balance (and the setup's actually relatively low-swing for a two-power-role setup, with the main swing risk being the Watcher being shot or forced to claim early).

Hoopla · Post Post #58 (ISO) » Fri Jul 08, 2016 10:43 pm

In post 57, callforjudgement wrote:So it may be that analysing old setups doesn't make much sense to learn about current balance issues. (I'm much more interested in whether normals are being misbalanced right now, than if they've historically been messed up. What are the stats for Mini Normals since the Micro queue was established? That'd allow for a fairer 10:3 to 7:2 comparison.)

When I was in change of the normal group, there was a period of time where I was reviewing and signing off on almost every setup that was proposed. I didn't really have the time or drive for that to last longterm, but the winrates were close at around the time period I stepped down: 29 town wins/32 mafia wins. As for time frame, I think that update was shortly after the micro queue became a thing?

So, if anything games were being balanced decently in the first year or two of 3:10's and have been getting worse since then.

In post 57, callforjudgement wrote: Meanwhile, I disagree with the approach of assessing balance by counting roles. As an extreme case, 3×Cops + 7×VT versus 3×Goon is highly townsided even if you allow for the town potentially not believing the setup, whereas 3×Doctor + 7×VT versus 3×Goon is probably somewhat scumsided in practice even if you allow for the possibility of the Doctors identifying each other (e.g. at massclaim) and setting up a protection loop. My preferred method of assessing balance in a setup that I don't have a similar setup to compare it with is to count the number of players that are likely to get their alignment confirmed overNight over the course of the game (my current estimate for this when balancing a 10:3 is "slightly more than 3"), plus the number of players likely to be mostly confirmed via what makes sense in the setup at massclaim (ideally around 1).

An example: Looking at Mini 1445, we have Cop, JK, 8×VT vs. Tracker, 2×Goon; the Tracker effectively counts as a delayed vanillaise for the purpose of balance (because scum will kill a suspected power role the night after), so we can give the power roles a 11/13 chance of being alive at the end of N1, 8/13 at the end of N2, 5/13 at the end of N3. Let's call this 80%, 60%, 40% for the purpose of approximation (as the numbers aren't perfectly accurate anyway), so we can assume that on average the Cop and JK each get a little under 2 shots off (slightly less for the Cop due to the chance of an accidental block, slightly more for the JK as their role can save themself from dying as it confirms a scum). The JK is almost blank in this setup; if they stop a kill early and aim at unlikely kill targets they may guess that their target is scum, but unlikely kill targets rarely perform kills, and thus the JK's only useful late (when they probably won't be alive). Any positive effect they have is therefore mostly a rounding error. I'll happily allocate 1 and a half or even two confirmations to the Cop (it's a powerful role), and if I were being generous I'd add one massclaim confirmation to the town total (not two as scum can cause confusion by claiming power, with only two town power roles). We're still somewhat short, though. You could add something like an Innocent Child to this game and it'd be much closer to balanced (especially if they were revealed at game start to give some amount of JK WIFOM that'll likely work mostly in town's favour). In this case, my method comes out with much the same conclusion as Hoopla's. If the town roles were different, though (say Cop+Watcher, which are both very powerful and also synergise well with each other), I'd be willing to let the game run with just the two power roles, at least from the point of view of win:loss balance (and the setup's actually relatively low-swing for a two-power-role setup, with the main swing risk being the Watcher being shot or forced to claim early).

In an ideal world, I'd love everyone to think about setup review in such a deep way, but many mods/reviewers don't and it's unrealistic to expect the majority of setups to be thought about in any meaningful way at all, so establishing simple soundbites like "town need 3+ strong PR's" or "don't give scum an RB with just 3 town roles" etc. seems like the best way to influence people's instincts and get games that aren't being reviewed thoroughly (or well) to adhere to some basic rules that intrinsically do a good job to nudge the game into balanced territory no matter what else they do.

I believe it's possible to design good balanced setups that don't adhere to everything in my guidelines, but it's harder, requires more thought and like a lot things on this site dunning-kruger might be at play.

mhsmith0 · Post Post #59 (ISO) » Fri Jul 08, 2016 11:32 pm

What are some examples of town sided normal games that have happened? My gut was that mini normal 1775 was town sided, and I remember complaints on mini normal 1782 (though I disagreed there). Any that stand out from those who know a lot on those things?

mykonian · Post Post #60 (ISO) » Sat Jul 09, 2016 3:15 am

ETL's game was passed as unbalanced by the review group, it was a normality only review. Since then, N has removed that option, it was very rarely used. If it was used however, odds were decent the setup was odd in some way. I think nowadays we'd catch that one.

mhsmith0 · Post Post #61 (ISO) » Sat Jul 09, 2016 3:26 am

In post 53, Antihero wrote:
In post 52, zMuffinMan wrote:dont think i ever bothered calculating past 4 scum in a vanilla because break-even on 4 scum was something like 80+ town?
sounds like a good time

someone run that

/pre-in

ISO · Post Post #62 (ISO) » Sat Jul 09, 2016 4:27 am

In post 59, mhsmith0 wrote:What are some examples of town sided normal games that have happened? My gut was that mini normal 1775 was town sided, and I remember complaints on mini normal 1782 (though I disagreed there). Any that stand out from those who know a lot on those things?

After looking through this thread, my setup prob. ended up scumsided after all, but I think it was fairly close to balance (perhaps the ninja should've been 1-shot instead of Odd night...).

mhsmith0 · Post Post #63 (ISO) » Sat Jul 09, 2016 4:30 am

Iirc your neighborhoods were no day talk right? That makes it super hard to do anything useful w those IMO.

ISO · Post Post #64 (ISO) » Sat Jul 09, 2016 4:35 am

Yeah. I also didn't expect the Follower to get killed N1.

Make the ninja 1-Shot and possibly give daytalk to the neighbors probably would've balanced it.

callforjudgement · Post Post #65 (ISO) » Sat Jul 09, 2016 4:42 am

In post 59, mhsmith0 wrote:What are some examples of town sided normal games that have happened? My gut was that mini normal 1775 was town sided, and I remember complaints on mini normal 1782 (though I disagreed there). Any that stand out from those who know a lot on those things?

1782 doesn't look obviously townsided to me (although I haven't analysed it in detail), although I'd have been very worried that it's excessively swingy (with most of the town roles getting better as scum's numbers dwindle, an early scum lynch would be very hard to come back from; and the JK result that gives town more information also leaves more townies alive). In the actual game, it looked somewhat like town snowballed after blocking a kill, which would have left the game feeling townsided. It'd have quite possibly felt equally scumsided if more town PRs had died early. Swingy games have an unfortunate habit of leaving someone upset pretty much whatever happens.

ISO · Post Post #66 (ISO) » Sat Jul 09, 2016 4:44 am

Yep, that's exactly what hapened. Part of the problem was scum didn't have too many good fakeclaims when ran up. Swing was a big part of it though.

mhsmith0 · Post Post #67 (ISO) » Sat Jul 09, 2016 4:53 am

In post 66, Ircher wrote:Yep, that's exactly what hapened. Part of the problem was scum didn't have too many good fakeclaims when ran up. Swing was a big part of it though.

Luck ran both ways though. Wolves shot prs n1 AND n3 while killing players they wanted dead vs pr hunting; that kind of luck is abnormal IMO. I still get chuckles from the "I bet they just role copped smith" bit after my death. But I think the result spoke to town playing better than wolves, when one mafia death snowballed through the rest of the game.

Antihero · Post Post #68 (ISO) » Sat Jul 09, 2016 5:43 am

In post 60, mykonian wrote:ETL's game was passed as unbalanced by the review group, it was a normality only review. Since then, N has removed that option, it was very rarely used. If it was used however, odds were decent the setup was odd in some way. I think nowadays we'd catch that one.

ETL...?

you mean TSQ's 4 faction thing?

Antihero · Post Post #69 (ISO) » Sat Jul 09, 2016 5:51 am

In post 59, mhsmith0 wrote:What are some examples of town sided normal games that have happened? My gut was that mini normal 1775 was town sided, and I remember complaints on mini normal 1782 (though I disagreed there). Any that stand out from those who know a lot on those things?

1775:
2 Mafia Goons
Mafia Encryptor

Town JOAT with 1 shot of each: Watcher, Commuter, Cop, BG
Town JK
Town Universal Backup
7 VT

no, not town sided

Antihero · Post Post #70 (ISO) » Sat Jul 09, 2016 5:54 am

ircher's game that you brought up: http://forum.mafiascum.net/viewtopic.php?f=2&t=65928

scum: odd night ninja, encryptor, even night jk

town: voyeur, follower, odd night jk, even night roleblocker, 6 vt

also not townsided

ISO · Post Post #71 (ISO) » Sat Jul 09, 2016 6:38 am

The investigatives were neighbors; PAY ATTN!

ISO · Post Post #72 (ISO) » Sat Jul 09, 2016 6:39 am

Lik e I said, the ninja pught to be 1-Shot and the neighbors should've had daytalk.... Maybe for as long as the mafia encryptor lived.

Hoopla · Post Post #73 (ISO) » Sat Jul 09, 2016 9:39 am

In post 65, callforjudgement wrote:Swingy games have an unfortunate habit of leaving someone upset pretty much whatever happens.

This is a good point.

Most post-game balance complaints usually happen after a scum or town sweep, and naturally, these extremes happen more in swingy games where many results are possible. People are naturally results-oriented and tend to make judgments based on what happened, even though that is just one of many possibilities that could have occurred.

Hoopla · Post Post #74 (ISO) » Sat Jul 09, 2016 9:55 am

In post 59, mhsmith0 wrote:What are some examples of town sided normal games that have happened? My gut was that mini normal 1775 was town sided, and I remember complaints on mini normal 1782 (though I disagreed there). Any that stand out from those who know a lot on those things?

There were no setups that I saw that stood out to me as obviously town-sided. It is rare for town to get more than four PR's in mini normals, and when they do, they tend to feature a lot of weak roles likes neighbours, backup whatevers, 1-Shot-whatevers that don't really do anything and often can't be confirmed, and/or scum getting heaps of power.

An underrated component of what makes a town PR powerful is that when massclaim happens, town can roughly assess how much power they should have and can occasionally confirm (or semi-confirm) a player or two based on balance. Hypothetically, if a mod gives a town more town power than usual (say, 5 strong PR's), it doesn't necessarily equate with a greater town advantage as towns will often lynch these PR's on the belief they can't all be true claims. This is essiantly Stoofer's Law on PR's, but in general I find it much more difficult to create a town-sided game than a scum-sided one.