The Newbie Matrix6 stats thread (complete)

This forum is for discussion related to the game.
Toomai
Goon
 
User avatar
Joined: January 12, 2013
Location: Ontario
Pronoun: He

Post Post #0  (ISO)  » Wed Mar 05, 2014 12:59 pm

Percentages may not add up to exactly 100% due to rounding and/or draws.
Spoiler: General/subsetup stats
SetupWinsLossesWinrateGames
1 (JK)403553.3%75
2 (RB, Cop, Doc)453854.2%83
3 (BP, Trk)344841.5%82
A (JK, RB, BP)314839.2%79
B (Cop)334542.3%78
C (Doc, Trk)344244.7%76
Total21725645.9%473

Image

Spoiler: Result stats
Average player types per game: 5.5 Newbies, 2.5 SEs, 1.0 ICs
PlayerWin %Total slots
Town Newbies46.1%2029
Scum Newbies52.8%566
Total Newbies47.6%2595
Town SEs45.3%924
Scum SEs57.0%265
Total SEs47.9%1189
Town ICs45.8%358
Scum ICs53.9%115
Total ICs47.8%473
All town45.9%3311
All scum54.1%946
All players47.7%4257

Image

Spoiler: Replacement stats
Notes: If a slot is replaced twice, that counts as one replaced slot and two replaced players. Therefore, the player rate cannot be any lower than the slot rate. A slot rate of 100% means that all 9 slots were replaced at least once. A player rate of 100% means that 9 players (from any slots) were replaced (so it can exceed 100%). I don't know which type of rate is better to use, though using both is useful (it tells us whether all the replacements came from one slot or were distributed amongst multiple slots). (Reminder: The Matrix6 setup has 9 players, hence why 9 appears in the previous.)

On average, games replace 2.9 slots with a standard deviation of 1.63, and 3.5 players with a standard deviation of 2.21.
PlayerSlot replacement ratePlayer replacement rate
Town Newbies38.9%46.8%
Scum Newbies43.5%56.3%
Total Newbies39.9%48.9%
Town SEs24.3%27.0%
Scum SEs26.7%29.3%
Total SEs24.8%27.5%
Town ICs10.3%11.1%
Scum ICs16.4%16.4%
Total ICs11.8%12.4%
All town31.7%37.4%
All scum35.5%43.9%
All players32.6%38.9%

Image
Image
Image

Spoiler: Power role stats
The rate of each role living to endgame is as follows:
RoleSetupsTown winScum winTotal
Jailkeeper1 and A48% (71 games)12% (83 games)29% (154 games)
Cop2 and B51% (78 games)7% (83 games)29% (161 games)
Doctor2 and C51% (79 games)19% (80 games)35% (159 games)
Bulletproof3 and A86% (65 games)68% (96 games)75% (161 games)
Tracker3 and C49% (68 games)11% (90 games)27% (158 games)
Roleblocker2 and A0% (by definition)88% (86 games)47% (162 games)

Interesting notes (ask for more):
  • In setup 2, in scum wins, 8% of Cops get lynched Day 1, and 39% get killed Night 1, leaving only 53% alive for Day 2 (38 games)
  • In setup 2, in scum wins, 18% of Doctors get killed Night 1, and 21% get killed Night 2 (38 games)
  • In setup 2, in scum wins, the Roleblocker is alive at the end 92% of the time (38 games)
  • In setup 3, 35% of Trackers are killed Night 2 in town wins, while 42% are killed N2 in scum wins (82 games)
  • In setup 3, in scum wins, only 25% of Trackers are alive by Day 3, and 10% survive to endgame (48 games)
  • In setup B, in town wins, 24% of Cops get killed Night 1, and only 48% survive to endgame (33 games)
  • In setup B, in scum wins, 11% of Cops get lynched Day 1, 42% of Cops get killed Night 1, and only 16% of Cops are alive by Day 3 (45 games)
  • In setup C, in scum wins, 26% of Doctors get killed Night 1 while another 50% die Night 2, leaving 17% alive for Day 3 (42 games)

Spoiler: Day 1 stats
  • Town-scum-NL rate: 368-94-10 (random lynching would give 367-105-0; the difference is [0.19%]-[-2.31%]-[+2.12%])
  • If town is lynched D1, they go 137-231 (37.2%)
  • If scum is lynched D1, they go 17-77 (18.1%)
  • If there is no lynch D1, town goes 2-8 (20.0%)

Spoiler: Lynch accuracy stats
In the following table, the first three columns are the raw percentages, while the last two are the difference between that and random (positive = more, negative = less). So when town does better than random, it has a negative in lynching town and a positive in lynching scum (random never no-lynches, so the difference is the data).
Note that, due to the way in which I store the stats, if a game has a no-lynch followed by a no-kill (two chances to lynch with the same ratio), the first one is ignored and the second one counts. As a result there's probably a slightly higher number of no-lynches than is being reported.
RatioTownScumNo lynchTownScum
7:2 (474 samples)78.7%19.8%1.5%+0.9%-2.4%
7:1 (30 samples)53.3%46.7%0.0%-34.2%+34.2%
6:2 (83 samples)55.4%38.6%6.0%-19.6%+13.6%
6:1 (76 samples)57.9%40.8%1.3%-27.8%+26.5%
5:2 (308 samples)64.6%33.1%2.3%-6.8%+4.5%
5:1 (68 samples)52.9%45.6%1.5%-30.4%+28.9%
4:2 (78 samples)38.5%29.5%32.1%-28.2%-3.8%
4:1 (123 samples)57.7%41.5%0.8%-22.3%+21.5%
3:2 (194 samples)59.8%38.7%1.5%-0.2%-1.3%
3:1 (77 samples)53.2%32.5%14.3%-21.8%+7.5%
2:1 (132 samples)53.8%44.7%1.5%-12.9%+11.4%

Image
Image

Spoiler: Game length stats
  • No games have ended with Day 1 lynch (only possible via modkills)
  • No games have ended with Night 1 kill (only possible via modkills)
  • Games that end with Day 2 lynch take 20.0 days with standard deviation of 5.43 (40 games)
  • No games have ended with Night 2 kill (only possible via modkills)
  • Games that end with Day 3 lynch take 34.6 days with standard deviation of 9.99 (196 games)
  • Games that end with Night 3 kill take 39.5 days with standard deviation of 9.10 (10 games)
  • Games that end with Day 4 lynch take 44.5 days with standard deviation of 12.86 (180 games)
  • Games that end with Night 4 kill take 46.7 days with standard deviation of 8.16 (4 games)
  • Games that end with Day 5 lynch take 49.9 days with standard deviation of 14.49 (39 games)
  • One game has ended with Night 5 kill; it took 62.5 days
  • Games that end with Day 6 lynch take 56.6 days with standard deviation of 10.88 (4 games)
  • No games have ended with Night 6 kill
  • One game has ended with Day 7 lynch; it took 84.0 days
  • Overall, games take 39.0 days with standard deviation of 13.92; the vast majority of games end with a Day 3 or Day 4 lynch (79.16%), with Day 2 and Day 5 lynches being tied for second-most common (16.63%)

Spoiler: Replacement vs. length stats
  • When 0 slots are replaced all game, games take 27.9 days with standard deviation of 11.86 (24 games)
  • When 1 slot is replaced all game, games take 34.3 days with standard deviation of 12.81 (76 games)
  • When 2 slots are replaced all game, games take 35.5 days with standard deviation of 12.23 (102 games)
  • When 3 slots are replaced all game, games take 38.4 days with standard deviation of 13.26 (103 games)
  • When 4 slots are replaced all game, games take 44.2 days with standard deviation of 14.43 (84 games)
  • When 5 slots are replaced all game, games take 44.2 days with standard deviation of 12.34 (58 games)
  • When 6 slots are replaced all game, games take 47.6 days with standard deviation of 12.35 (20 games)
  • When 7 slots are replaced all game, games take 50.1 days with standard deviation of 14.57 (7 games)
  • One game replaced 8 slots; it took 71.5 days
  • No games have had 9 slots replaced
  • When 0 players are replaced all game, games take 27.9 days with standard deviation of 11.86 (24 games)
  • When 1 player is replaced all game, games take 33.8 days with standard deviation of 12.87 (66 games)
  • When 2 players are replaced all game, games take 34.9 days with standard deviation of 12.62 (85 games)
  • When 3 players are replaced all game, games take 37.0 days with standard deviation of 11.83 (89 games)
  • When 4 players are replaced all game, games take 41.0 days with standard deviation of 14.41 (78 games)
  • When 5 players are replaced all game, games take 42.7 days with standard deviation of 12.99 (48 games)
  • When 6 players are replaced all game, games take 46.3 days with standard deviation of 12.54 (37 games)
  • When 7 players are replaced all game, games take 45.3 days with standard deviation of 11.13 (22 games)
  • When 8 players are replaced all game, games take 54.4 days with standard deviation of 9.85 (14 games)
  • When 9 players are replaced all game, games take 56.5 days with standard deviation of 16.49 (4 games)
  • When 10 players are replaced all game, games take 50.9 days with standard deviation of 14.19 (6 games)
  • When 11 players are replaced all game, games take 51.1 days with standard deviation of 28.88 (2 games)
  • No games have had more than 11 players replaced

Spoiler: Replacement vs. winrate stats
For slots:
  • When 0 slots are replaced all game, town goes 13-11 (54.2%)
  • When 1 slot is replaced all game, town goes 40-36 (52.6%)
  • When 2 slots are replaced all game, town goes 48-54 (47.1%)
  • When 3 slots are replaced all game, town goes 51-52 (49.5%)
  • When 4 slots are replaced all game, town goes 41-43 (48.8%)
  • When 5 slots are replaced all game, town goes 21-36 (36.8%)
  • When 6 slots are replaced all game, town goes 3-16 (15.8%)
  • When 7 slots are replaced all game, town goes 0-7 (0.0%)
  • When 8 slots are replaced all game, town goes 0-1 (0.0%)
  • No games have had 9 slots replaced
Image
For players:
  • When 0 players are replaced all game, town goes 13-11 (54.2%)
  • When 1 player is replaced all game, town goes 35-31 (53.0%)
  • When 2 players are replaced all game, town goes 39-46 (45.9%)
  • When 3 players are replaced all game, town goes 43-46 (48.3%)
  • When 4 players are replaced all game, town goes 34-44 (43.6%)
  • When 5 players are replaced all game, town goes 22-26 (45.8%)
  • When 6 players are replaced all game, town goes 17-20 (45.9%)
  • When 7 players are replaced all game, town goes 7-14 (33.3%)
  • When 8 players are replaced all game, town goes 4-9 (30.8%)
  • When 9 players are replaced all game, town goes 1-3 (25.0%)
  • When 10 players are replaced all game, town goes 2-4 (33.3%)
  • When 11 players are replaced all game, town goes 0-2 (0.0%)
  • No games have had more than 11 players replaced
Image

Spoiler: Replacement vs. role stats
RoleSlotsSlots replacedPlayers replaced
Vanilla Townie252833.0%39.2%
Jailkeeper15631.4%33.3%
Cop16120.5%24.2%
Doctor15933.3%39.6%
1-Shot Bulletproof16330.1%35.6%
Tracker15822.8%25.9%
Mafia Goon78634.7%44.4%
Mafia Roleblocker16439.0%41.5%
The following is a z-score graph. 0 means "is exactly average". 1 means "is 1 standard deviation above average"; -1 means "is 1 standard deviation below average". In other words, high bars mean "more replaces", while low mean "less replaces".
Image

Spoiler: Perfect win stats
  • Out of 217 town wins, 40 wins (18.4%) involved only scum lynches.
    • 26 wins (12.0%) were semi-perfect (no town lynches).
    • 14 wins (6.5%) were perfect (no town deaths).
  • Out of 256 scum wins, 146 wins (57.0%) were perfect (no scum deaths).


Notes
  • For any stat that counts wins and losses, draws are ignored.
  • The replacement stats recorded here are likely lower than in reality, as replacements are not recorded if the player never confirmed, or the player was force-replaced due to a mod error or something similar that was not the replaced player's fault. If a player posts in any way (or has been acknowledged by the mod as having confirmed/read their role PM), or gets themselves force-replaced (such as getting banned), replacing them counts.
  • Slot-based stats might be a bit off due to cases where slots "level up" via replacement (e.g. an experienced player replaces into a newbie slot).
  • Players are assumed to have won regardless of modkills (unnecessary complication).
  • Only notable and requested PR stats are listed. Ask for one to get it added.

For more comprehensive graphs, please see the original spreadsheet (.xlsx, 3.57MB).

If you'd like to know something that isn't here, I'll see what I can do.
Last edited by Toomai on Sun Feb 04, 2018 11:48 am, edited 40 times in total.
This should be required reading for...everyone for anything, really.

Damon_Gant
Goon
 
User avatar
Joined: January 22, 2010
Pronoun: He

Post Post #1  (ISO)  » Wed Mar 05, 2014 6:33 pm

Oh I am a sucker for stats! I'm going to have to play around with your spreadsheet - I think there is a lot to be learned from stat analysis, both about the successfulness of the Matrix6 setup but also about mafia strategy in general.

Some interesting things I note from this at a glance are:
- The win rates overall are looking about right, but whether the individual setups are balanced remains to be seen.
- Experience does not seem to play a huge role with regards to win rates. It just means you're less likely to need to be replaced.
- Day 1 is crucial
Swimming, anyone?

Call me Gant

Cheery Dog
Kayak
 
User avatar
Joined: July 01, 2012
Location: OMG BALL!

Post Post #2  (ISO)  » Wed Mar 05, 2014 8:23 pm

I'm confused how it was possible for games to have ended on night 3.

I thought modkills, but if a town is modkilled, then the game should be going into night.
It'll all end in tears, it probably already has.

Toomai
Goon
 
User avatar
Joined: January 12, 2013
Location: Ontario
Pronoun: He

Post Post #3  (ISO)  » Wed Mar 05, 2014 9:07 pm

In post 2, Cheery Dog wrote:I'm confused how it was possible for games to have ended on night 3.
Example would be 1376. No lynch occured in D3 LyLo, so the N3 kill made the game 2:2 for the win. The JK was still alive so it was still possible for town to pull it out, which I guess is why the game wasn't ended on the spot.
This should be required reading for...everyone for anything, really.

Mr. Flay
Metatron
 
User avatar
Joined: March 12, 2004
Location: Gormenghast
Pronoun: He

Post Post #4  (ISO)  » Wed Mar 05, 2014 10:52 pm

^
Retired as of October 2014.

callforjudgement
Microprocessor
 
User avatar
Joined: September 01, 2011

Post Post #5  (ISO)  » Mon Mar 10, 2014 5:06 pm

I guess the big question here is "why do people no-lynch in lylo?". (EDIT: I read the game, the answer is "mass V/LA near deadline".)
scum · scam · seam · team · term · tern · torn · town

~Jordan`
Goon
 
User avatar
Joined: August 31, 2009
Pronoun: He

Post Post #6  (ISO)  » Tue Mar 11, 2014 1:40 am

presenting data and then talking about it afterward seems like a good plan
kick logic to the curb and do the impossible!

Mr. Flay
Metatron
 
User avatar
Joined: March 12, 2004
Location: Gormenghast
Pronoun: He

Post Post #7  (ISO)  » Tue Mar 11, 2014 7:15 am

In post 5, callforjudgement wrote:I guess the big question here is "why do people no-lynch in lylo?". (EDIT: I read the game, the answer is "mass V/LA near deadline".)

Decision paralysis? For scum the result of No Lynch is almost as good as a mislynch...
Retired as of October 2014.

TierShift
Jack of All Trades
 
User avatar
Joined: November 05, 2013
Pronoun: He

Post Post #8  (ISO)  » Tue Mar 11, 2014 10:15 am

Okay, I've been running some chi-square tests.
Scumteams aren't significant, alas. Setups, however, are.

Setup 1 is significantly better for town than setup C (p~0.02) and setup 3 (p~0.01)
the rest of the tests have come up as not statistically significant (at p<0.05).

This means that a JK is better for town than a tracker+doc or tracker+1-shot BP.
Does this need balancing? I'm leaning towards no, as newbie games are more about learning the game and about the most used roles than about absolute balance, but if it is something that needs to be done, tracker should be replaced by a slightly stronger investigative role, such as (x-shot) watcher. It's possibly worth discussing.

Cogito Ergo Sum
YARR!
 
User avatar
Joined: October 29, 2005
Location: Nottingham
Pronoun: He

Post Post #9  (ISO)  » Tue Mar 11, 2014 11:23 am

Does that take into account that you're cherry-picking the set-ups that have the most extreme win-rates? The Jailkeeper set-up's win-rate specifically seems weirdly high. I also doubt Doc+Tracker is really that bad, with Town having 2 perfectly serviceable PRs.

Also, Watchers don't seem fun in Newbies (just consider an IC-newbie scum team; who do you think is going to make the kill?) and Watcher-Doctor specifically is very much undesirable.
Scumchat is awesome. Yarr!

~"Multiple exclamation marks are a sure sign of a diseased mind."~

Eden
Townie
 
User avatar
Joined: February 16, 2014
Location: Baton Rouge, LA

Post Post #10  (ISO)  » Tue Mar 11, 2014 11:25 am

I don't think so. You're getting significant p values, sure, but the number of attempts of each setup aren't significant (setup 1 has 18 iterations, setup 3 has 22, C has 15). I'd revisit this question once every setup has been tried 30 times and see if you're still getting significant differences
“Many men go fishing all of their lives without knowing that it is not fish they are after.” - Henry David Thoreau

TierShift
Jack of All Trades
 
User avatar
Joined: November 05, 2013
Pronoun: He

Post Post #11  (ISO)  » Tue Mar 11, 2014 12:57 pm

Hmm...apparently my knowledge of statistical tests is not large enough.

It wouldn't suprise me at all to find significant results at 30 games and even more at 100 games each.

Eden
Townie
 
User avatar
Joined: February 16, 2014
Location: Baton Rouge, LA

Post Post #12  (ISO)  » Tue Mar 11, 2014 2:58 pm

There's a hundred total games, not a hundred games with setup 1, C or 3. Like Toomai said in the OP, the results of each individual permutation aren't significant yet (they're all in italics which indicates statistically insignificant results).

Like I said, give it a few months and we'll probably pass the 30-game threshold for individual setups and be able to compare from there.
“Many men go fishing all of their lives without knowing that it is not fish they are after.” - Henry David Thoreau

TierShift
Jack of All Trades
 
User avatar
Joined: November 05, 2013
Pronoun: He

Post Post #13  (ISO)  » Tue Mar 11, 2014 3:00 pm

I'm not dumb. When we get to a 100 games each there's probably gonna be a lot of statistical significant differences out there, is what I'm saying.

Eden
Townie
 
User avatar
Joined: February 16, 2014
Location: Baton Rouge, LA

Post Post #14  (ISO)  » Tue Mar 11, 2014 3:59 pm

Maybe? That's why I said I'd suggest revisit once we get more results. Like I said, the initial differences in these results aren't significant so we can't really extrapolate anything. In particular I'd expect the jailkeeper setup with such a high win % to regress toward whatever the eventual mean is going to be. Beyond that I don't think anyone can really guess.
“Many men go fishing all of their lives without knowing that it is not fish they are after.” - Henry David Thoreau

GreyICE
Fifty Shades
 
User avatar
Joined: December 15, 2010
Pronoun: He

Post Post #15  (ISO)  » Fri Mar 14, 2014 9:20 am

As I recall the only thing we concluded is that newbie scum are significantly more likely to replace out.
That which is done out of love always takes place beyond good and evil

Official Visigoth

Mr. Flay
Metatron
 
User avatar
Joined: March 12, 2004
Location: Gormenghast
Pronoun: He

Post Post #16  (ISO)  » Fri Mar 14, 2014 1:24 pm

All I remember is that newbies, period, are more likely to replace out. What data are you recalling?
Retired as of October 2014.

GreyICE
Fifty Shades
 
User avatar
Joined: December 15, 2010
Pronoun: He

Post Post #17  (ISO)  » Fri Mar 14, 2014 2:28 pm

Go look at the data at the top, Flay:

Town newbies: 28%-33%-39% (412 newbies)
Scum newbies: 25%-20%-54% (114 newbies)

That jumps from 39% to 54% based on whether they're town or scum.
That which is done out of love always takes place beyond good and evil

Official Visigoth

~Jordan`
Goon
 
User avatar
Joined: August 31, 2009
Pronoun: He

Post Post #18  (ISO)  » Fri Mar 14, 2014 3:19 pm

Coooool.
kick logic to the curb and do the impossible!

Quilford
Jack of All Trades
 
User avatar
Joined: March 11, 2011

Post Post #19  (ISO)  » Tue Apr 22, 2014 6:45 pm

Just bumping this so that everyone is aware of nuu data

Zachrulez
Jack of All Trades
 
User avatar
Joined: December 05, 2008
Location: Minnesota
Pronoun: He

Post Post #20  (ISO)  » Wed Apr 23, 2014 9:48 am

In post 9, Cogito Ergo Sum wrote:Does that take into account that you're cherry-picking the set-ups that have the most extreme win-rates? The Jailkeeper set-up's win-rate specifically seems weirdly high. I also doubt Doc+Tracker is really that bad, with Town having 2 perfectly serviceable PRs.

Also, Watchers don't seem fun in Newbies (just consider an IC-newbie scum team; who do you think is going to make the kill?) and Watcher-Doctor specifically is very much undesirable.


The ability to stop kills is probably quite a bit more powerful than we give it credit for in a 9p setup when the role has two chances to stop the kill. (Hitting the kill target or the source.) Notice the winrate when paired with a roleblocker is much lower.

It'll probably take a bit bigger sample size to solidify that thought process though.

GreyICE
Fifty Shades
 
User avatar
Joined: December 15, 2010
Pronoun: He

Post Post #21  (ISO)  » Wed Apr 23, 2014 9:57 am

In post 12, Eden wrote:There's a hundred total games, not a hundred games with setup 1, C or 3. Like Toomai said in the OP, the results of each individual permutation aren't significant yet (they're all in italics which indicates statistically insignificant results).

Like I said, give it a few months and we'll probably pass the 30-game threshold for individual setups and be able to compare from there.

Statistical significance doesn't quite work like that. Something is not magically insignificant at 29 games, and significant at 30. If a setup was 15-0, for instance, you wouldn't need to sit around waiting for 30 to see if it was a fluke.

That being said, they do appear to be close enough to be a fluke, but I do think you can draw some interesting observations.

P.S. Statistical significance must be the most misunderstood concept in human history.
That which is done out of love always takes place beyond good and evil

Official Visigoth

Burning_Earth
Mafia Scum
 
User avatar
Joined: January 26, 2014
Pronoun: He

Post Post #22  (ISO)  » Fri Apr 25, 2014 1:08 am

Yeah.

Majiffy
Go with the Flow
 
User avatar
Joined: November 23, 2011
Location: Buffalo, NY
Pronoun: He

Post Post #23  (ISO)  » Fri Apr 25, 2014 1:23 pm

Lol'd at >50% scum newbie replace-rate.

Inb4 replacement PLs become a thing sitewide for a short while.
Only playing in games at personal moderator and/or 50%+ playerlist request.

GTKAS Jiffy! (Part 1) || GTKAS Jerfy (The Sequel!) || How To Win Every Game At Mafiascum (The Flowchart) || In case anyone was unsure...
Svenskt Stål (23:38) majiffy, worst mod on ms? we talk to a surviving victim of his game

Mr. Flay
Metatron
 
User avatar
Joined: March 12, 2004
Location: Gormenghast
Pronoun: He

Post Post #24  (ISO)  » Fri Apr 25, 2014 1:50 pm

Yeah, but "on average, games replace 3.0 slots with a standard deviation of 1.72". How will you pick the right replacement to PL?

I know, I know, rhetorical....
Retired as of October 2014.

Next
[ + ]

Return to Mafia Discussion