Newbie Game Player Ratings

This forum is for discussion related to the game.
User avatar
Raskolnikov
Raskolnikov
Jack of All Trades
User avatar
User avatar
Raskolnikov
Jack of All Trades
Jack of All Trades
Posts: 6395
Joined: November 15, 2015

Post Post #25 (ISO) » Thu Jun 22, 2017 2:27 pm

Post by Raskolnikov »

do you think you'll do mini normals at one point?
deranged and incoherent
?
User avatar
mhsmith0
mhsmith0
Balancing Act
User avatar
User avatar
mhsmith0
Balancing Act
Balancing Act
Posts: 10830
Joined: March 7, 2016
Location: Phoenix, AZ

Post Post #26 (ISO) » Thu Jun 22, 2017 2:30 pm

Post by mhsmith0 »

It's possible, but I think the big issue is that the sample size is much lower (not all mini normals are 10/3, and they are slower to fill than newbie games) So I could do a separate model there, but I strongly suspect there'd be very few players who have enough data even to hit the 8x town threshold, which means that I'd guess it'd be a lot less interesting.

Also setup meta varies from time to time, and whereas a lot of data exists to demonstrate that matrix6 is pretty blaanced across setups (all are 40-60% town win rate, and few would basically only rand the most favorable or most unfavorable setups), I strongly suspect that there have been a number of mini normal setups that are simply unbalanced, in either direction, which means there's more rand luck in the results.
Show
http://wiki.mafiascum.net/index.php?title=Mhsmith0
Conq: you, sir, are great at being town.
BATMAN: Only jugg was the only one we didn’t scum read at least not me
Quick: There is little to no chance this slot is Power-Wolfing.
SR: I want to give him a day
Life is simply unfair, don't you think?
User avatar
Raskolnikov
Raskolnikov
Jack of All Trades
User avatar
User avatar
Raskolnikov
Jack of All Trades
Jack of All Trades
Posts: 6395
Joined: November 15, 2015

Post Post #27 (ISO) » Thu Jun 22, 2017 2:47 pm

Post by Raskolnikov »

ahh I see

mmm, aren't most of the people in any given newbie unlikely to reach that threshold anyways before they siteflake/move onto other queues? whereas most people in each mini seem to hang around. just speculation though idk
can't argue against the balance concern though (esp all those games i lost of course), and I guess realeo already sorta did it
deranged and incoherent
?
User avatar
Ircher
Ircher
He / Him / His
What A Grand Idea
User avatar
User avatar
Ircher
He / Him / His
What A Grand Idea
What A Grand Idea
Posts: 15170
Joined: November 9, 2015
Pronoun: He / Him / His
Location: CST/CDT

Post Post #28 (ISO) » Fri Jun 23, 2017 4:34 am

Post by Ircher »

Yeh, 8 games is EXTREMELY high.

I understand it's done for statistical accuracy, but perhaps 5 would be a better threshold?
Links: User Page | GTKAS
Do you have questions, ideas, or feedback for the Scummies? Please pm me!
Hosting: The Grand Neighborhood [In Signups: 8/9 ONE MORE!]
User avatar
mhsmith0
mhsmith0
Balancing Act
User avatar
User avatar
mhsmith0
Balancing Act
Balancing Act
Posts: 10830
Joined: March 7, 2016
Location: Phoenix, AZ

Post Post #29 (ISO) » Fri Jul 21, 2017 9:18 am

Post by mhsmith0 »

new update, v1.009:

Update for 1792, 1797, 1799, 1800, 1801, 1802, 1803, 1804, 1805:

Spoiler: v1.009 results
Log Rating Record
Player Town Scum Combined Town Scum Combined
RadiantCowbells 0.36 -1.29 1.65 21-20 8-1 29-21
GuyInFreezer 1.65 1.65 12-3 1-1 13-4
Thor665 0.38 -0.68 1.06 12-13 6-3 18-16
fferyllt 1.00 1.00 13-4 4-1 17-5
Micc 0.99 0.99 6-3 2-1 8-4
Malakittens 0.73 0.73 17-11 3-3 20-14
Creature 0.70 0.70 5-3 3-0 8-3
T S O 0.62 0.62 7-5 2-2 9-7
Xayzeck 0.52 0.52 7-4 1-4 8-8
Loopdan 0.47 0.47 8-6 2-0 10-6
LicketyQuickety 0.46 0.46 5-3 1-2 6-5
Huntress 0.37 0.37 8-7 1-4 9-11
JaeReed 0.34 0.34 5-5 0-0 5-5
Guyett 0.28 0.28 7-7 2-2 9-9
Bulbazak 0.24 0.24 5-4 2-2 7-6
singersigner 0.24 0.24 6-5 1-0 7-5
Raskolnikov 0.22 0.22 5-6 1-2 6-8
PenguinPower 0.21 0.21 5-5 3-1 8-6
House 0.19 0.19 6-6 2-0 8-6
jmo16mla 0.09 0.09 5-5 0-0 5-5
Hopkirk 0.02 0.02 5-4 2-0 7-4
Nobody Special 0.02 0.02 7-7 1-2 8-9
Wisdom 0.00 0.00 3-4 6-4 9-8
hayatoBL -0.01 -0.01 5-6 3-1 8-7
innocentvillager -0.05 -0.05 5-7 4-1 9-8
BlueBloodedToffee 0.08 0.16 -0.08 11-16 6-5 17-21
goodmorning -0.20 -0.10 -0.10 15-22 7-5 22-27
ThinkBig -0.13 -0.13 3-6 3-0 6-6
TheIrishPope -0.14 -0.14 4-5 1-3 5-8
Nachomamma8 0.56 0.73 -0.17 17-12 5-12 22-24
tojam2 -0.17 -0.17 3-5 0-1 3-6
Jake from State Farm -0.19 -0.19 4-8 1-3 5-11
Sakura Hana -0.24 -0.24 5-6 3-3 8-9
JasonWazza -0.27 -0.27 4-5 3-3 7-8
theslimer3 -0.28 -0.28 4-5 0-0 4-5
Accountant -0.28 -0.28 6-12 4-2 10-14
Dierfire -0.28 -0.28 6-10 2-2 8-12
RachMarie -0.30 -0.30 11-18 2-2 13-20
copper223 -0.33 -0.33 3-5 2-0 5-5
Cabd -0.41 -0.41 7-8 0-0 7-8
Ms Marangal -0.42 -0.42 3-6 1-4 4-10
Drixx 0.48 0.95 -0.48 12-12 3-6 15-18
NicCage -0.48 -0.48 2-6 0-0 2-6
notscience -0.36 0.14 -0.50 9-16 5-4 14-20
PhantomCobalt -0.50 -0.50 5-8 1-1 6-9
enomis -0.56 -0.56 2-6 0-0 2-6
Alexcellent -0.64 -0.64 3-5 0-1 3-6
Not_Mafia -0.39 0.28 -0.67 6-10 3-5 9-15
MarioManiac4 -0.74 -0.74 3-9 2-3 5-12
Flubbernugget -0.77 -0.77 3-8 0-0 3-8
Titus -1.03 -1.03 3-13 0-2 3-15
GuiltyLion -1.23 -1.23 1-8 4-1 5-9
Alisae -1.59 -1.59 1-8 1-1 2-9


intercept: -0.291 (42.8% town win odds given entirely "other" players)
combined error: 265.52 (average 0.612)


Spoiler: updates and corrections list
Added newbies 1792, 1797, 1799, 1800, 1801, 1802, 1803, 1804, 1805

town!ThinkBig, town!LicketyQuickety, scum!Drixx now have 8 results and are added to the table

I also made a correction to a few older games. 1556, 1557, 1562, 1565, 1566, were town wins not scum wins. The issue was that https://wiki.mafiascum.net/index.php?ti ... _1500-1749 showed a handful of games as "Current Update" isntead of "Outcome" and that screwed up my lookup formula for results. I checked against Toomai's spreadsheet and every result now matches.


Interesting changes and other notes:
Not massively (Drixx's scum rating dropped his overall rating a lot, but I think that was fairly predictable from looking at the record pre-update). One interesting thing I've started to play with is potentially breaking games up into which setup was randed; doing so shifts some player ratings around a bit, but few of them really moved that much. Row 2 (cop/doc/rb) is by far the easiest town rating game (duh), but it was interesting that A (jk/bp/rb), 3 (tracker/bp) AND C (tracker/doc) were all rated as being much harder than the other setups. I hadn't really expected setup C to be much closer to A and 3, but that's what early analysis suggests.

Next up on v1.010 will be town!Draynth in all likelihood. There are others who may hop into the 8+ results categories, but no one who necessarily seems imminent (unless I've missed something of course).
Last edited by mhsmith0 on Fri Jul 21, 2017 9:43 am, edited 2 times in total.
Show
http://wiki.mafiascum.net/index.php?title=Mhsmith0
Conq: you, sir, are great at being town.
BATMAN: Only jugg was the only one we didn’t scum read at least not me
Quick: There is little to no chance this slot is Power-Wolfing.
SR: I want to give him a day
Life is simply unfair, don't you think?
User avatar
mhsmith0
mhsmith0
Balancing Act
User avatar
User avatar
mhsmith0
Balancing Act
Balancing Act
Posts: 10830
Joined: March 7, 2016
Location: Phoenix, AZ

Post Post #30 (ISO) » Fri Jul 21, 2017 9:24 am

Post by mhsmith0 »

More specifics on the setup variations:

1 (jk): -0.139, equivalent to town win probability of 46.5% given an all "other" group (actual town win rate 52.2%)
2 (cop/doc/rb): +0.215, equivalent to town win probability of 55.4% given an all "other" group (actual town win rate 54.7%)
3 (tracker/bp): -0.544, equivalent to town win probability of 36.7% given an all "other" group (actual town win rate 39.2%)
A (jk/bp/rb): -0.474, equivalent to town win probability of 38.4% given an all "other" group (actual town win rate 40.0%)
B (cop): -0.175, equivalent to town win probability of 45.6% given an all "other" group (actual town win rate 45.6%)
C (tracker/doc): -0.440, equivalent to town win probability of 39.2% given an all "other" group (actual town win rate 43.1%)
Show
http://wiki.mafiascum.net/index.php?title=Mhsmith0
Conq: you, sir, are great at being town.
BATMAN: Only jugg was the only one we didn’t scum read at least not me
Quick: There is little to no chance this slot is Power-Wolfing.
SR: I want to give him a day
Life is simply unfair, don't you think?
User avatar
mhsmith0
mhsmith0
Balancing Act
User avatar
User avatar
mhsmith0
Balancing Act
Balancing Act
Posts: 10830
Joined: March 7, 2016
Location: Phoenix, AZ

Post Post #31 (ISO) » Thu Sep 28, 2017 11:50 am

Post by mhsmith0 »

new update, v1.010:

Update for 1806, 1807, 1809-1820:

Spoiler: v1.010 results
Log Rating Record
Player Town Scum Combined Town Scum Combined
RadiantCowbells 0.34 -1.30 1.64 22-24 8-1 30-25
GuyInFreezer 1.63 1.63 12-3 1-2 13-5
Thor665 0.37 -0.69 1.06 12-13 6-3 18-16
fferyllt 1.03 1.03 13-4 4-1 17-5
Micc 0.98 0.98 6-3 2-1 8-4
LicketyQuickety 0.88 0.88 6-3 1-2 7-5
Malakittens 0.70 0.70 17-11 4-3 21-14
T S O 0.60 0.60 7-5 2-2 9-7
Loopdan 0.55 0.55 8-6 2-0 10-6
Xayzeck 0.52 0.52 7-4 1-4 8-8
Huntress 0.42 0.42 8-7 1-4 9-11
JaeReed 0.41 0.41 6-6 0-0 6-6
PenguinPower 0.37 0.37 5-5 5-1 10-6
Fykus 0.36 0.36 4-5 0-0 4-5
Draynth 0.35 0.35 5-5 2-1 7-6
Creature 0.30 0.30 4-4 3-0 7-4
Guyett 0.27 0.27 7-7 2-2 9-9
Bulbazak 0.26 0.26 5-4 2-2 7-6
Raskolnikov 0.25 0.25 5-6 1-2 6-8
singersigner 0.25 0.25 6-5 2-0 8-5
mhsmith0 0.23 0.23 4-4 1-0 5-4
House 0.21 0.21 6-6 2-0 8-6
jmo16mla 0.13 0.13 5-5 0-0 5-5
Hopkirk 0.09 0.09 5-4 2-0 7-4
Nobody Special 0.04 0.04 7-7 1-2 8-9
Wisdom -0.03 0.03 3-4 6-4 9-8
hayatoBL 0.02 0.02 5-6 3-1 8-7
tojam2 -0.08 -0.08 3-5 0-1 3-6
BlueBloodedToffee 0.09 0.17 -0.09 11-16 6-5 17-21
innocentvillager -0.10 -0.10 5-8 4-2 9-10
goodmorning -0.18 -0.05 -0.13 15-22 7-5 22-27
Nachomamma8 0.57 0.72 -0.15 17-12 5-12 22-24
TheIrishPope -0.16 -0.16 4-5 1-3 5-8
Jake from State Farm -0.19 -0.19 4-8 1-3 5-11
Cabd -0.21 -0.21 8-8 0-0 8-8
RachMarie -0.22 -0.22 11-18 2-2 13-20
Sakura Hana -0.24 -0.24 5-6 3-3 8-9
JasonWazza -0.26 -0.26 4-5 3-3 7-8
Dierfire -0.27 -0.27 6-10 2-2 8-12
Accountant -0.27 -0.27 6-12 4-2 10-14
copper223 -0.33 -0.33 3-5 2-0 5-5
theslimer3 -0.37 -0.37 4-6 0-0 4-6
Ms Marangal -0.41 -0.41 3-6 1-4 4-10
ThinkBig -0.41 -0.41 3-8 3-0 6-8
nancy -0.44 -0.44 2-6 1-0 3-6
NicCage -0.46 -0.46 2-6 0-0 2-6
PhantomCobalt -0.47 -0.47 5-8 1-1 6-9
notscience -0.34 0.16 -0.50 9-16 5-4 14-20
Drixx 0.47 0.96 -0.50 12-12 3-6 15-18
enomis -0.52 -0.52 2-6 0-0 2-6
Alexcellent -0.56 -0.56 3-5 0-1 3-6
Not_Mafia -0.40 0.26 -0.66 6-10 3-5 9-15
Flubbernugget -0.74 -0.74 3-8 0-0 3-8
MarioManiac4 -0.77 -0.77 3-10 2-3 5-13
WhemeStar -0.83 -0.83 3-9 1-0 4-9
Titus -1.01 -1.01 3-13 0-2 3-15
GuiltyLion -1.03 -1.03 2-9 4-1 6-10
Alisae -1.46 -1.46 1-8 1-1 2-9


intercept: -0.331 (41.8% town win odds given entirely "other" players)
combined error: 276.77 (average 0.610)


Spoiler: updates and corrections list
Added newbies 1806, 1807, 1809-1820

town!Draynth, town!Fykus, town!WhemeStar, town!mhsmith0, town!nancy now have 8 results and are added to the table; town!creature was for some reason double-counted in 1743; this is now corrected).


Interesting changes and other notes:
town!LicketyQuickety's rating shot up on a relatively small amount of data (a win in 1813). Eyeballing the data, it looks like the other main impact was that 1803 punished his rating a lot less, mainly due to Wheme coming in with a relatively low score.

Fykus's high score given his 4-5 record was a bit surprising, but this seems to be LARGELY driven by the 1787 win (third lowest rated town to win a newbie game) getting an outsize + rating.

Newbie
1720
1820 is actually a pretty decent chance to talk about the ratings and the impacts of "upset" type results. Prior to that game, Draynth and Creature were both rated at about 0.60, which meant that they both took a fairly substantial hit for that single game. Prior to the game result, town was estimated at about a 71% win chance (Draynth and Creature had enough data to be rated, the other 7 slots were all "other"). After the game, it went down to 58%. Especially for players with low sample sizes, the model inherently revises ratings to eliminate "weird" results. In that case, the weird result would be a fairly highly rated town losing (before the game, it would have been about tied for the #5 top rated town to lose; after the ratings revised, they went down to about the #25 rated town to lose, still a notable result but not quite as major an outcome).

Also, since I'm a high ego guy, I'm going to talk about me :P
I'd say that the 0.24 rating is probably around fair given the methodology. I'll note that I took a fair sized hit for 1756 (town of ThinkBig/Loop/Jae was overall an unusually positive rated town; and I subbed in at 3/2 LYLO with all town PRs dead :( ), and a somewhat lesser hit for 1691 (a town loss with town!Thor), but I also had some nice positives on other games to help balance that :)

Right now, there's really no one who jumps out at me as being particularly likely to be hopping onto the 8+ list for whenever this updates next. Jingle is at 7 scum games, so he'll pop onto the list after his next newbie scum game, but it's hard to say when that might be.
Last edited by mhsmith0 on Fri Sep 29, 2017 7:36 am, edited 1 time in total.
Show
http://wiki.mafiascum.net/index.php?title=Mhsmith0
Conq: you, sir, are great at being town.
BATMAN: Only jugg was the only one we didn’t scum read at least not me
Quick: There is little to no chance this slot is Power-Wolfing.
SR: I want to give him a day
Life is simply unfair, don't you think?
Not Known 15
Not Known 15
Mafia Scum
Not Known 15
Mafia Scum
Mafia Scum
Posts: 3744
Joined: September 15, 2017

Post Post #32 (ISO) » Fri Sep 29, 2017 6:39 am

Post by Not Known 15 »

Hmm Newbie 1720 or 1820?
User avatar
mhsmith0
mhsmith0
Balancing Act
User avatar
User avatar
mhsmith0
Balancing Act
Balancing Act
Posts: 10830
Joined: March 7, 2016
Location: Phoenix, AZ

Post Post #33 (ISO) » Fri Sep 29, 2017 7:37 am

Post by mhsmith0 »

:oops:
Show
http://wiki.mafiascum.net/index.php?title=Mhsmith0
Conq: you, sir, are great at being town.
BATMAN: Only jugg was the only one we didn’t scum read at least not me
Quick: There is little to no chance this slot is Power-Wolfing.
SR: I want to give him a day
Life is simply unfair, don't you think?
User avatar
fferyllt
fferyllt
Titan of Trajectory
User avatar
User avatar
fferyllt
Titan of Trajectory
Titan of Trajectory
Posts: 20488
Joined: December 28, 2012
Location: Left Coast

Post Post #34 (ISO) » Fri Sep 29, 2017 11:29 am

Post by fferyllt »

I thought I might eventually reach 8 scum games in Matrix6. Not going to happen.
Amid the pressure of great events, a general principle gives no help.

Idic
User avatar
mhsmith0
mhsmith0
Balancing Act
User avatar
User avatar
mhsmith0
Balancing Act
Balancing Act
Posts: 10830
Joined: March 7, 2016
Location: Phoenix, AZ

Post Post #35 (ISO) » Fri Sep 29, 2017 11:33 am

Post by mhsmith0 »

Just replace into a bunch of slots you think are scum.
Show
http://wiki.mafiascum.net/index.php?title=Mhsmith0
Conq: you, sir, are great at being town.
BATMAN: Only jugg was the only one we didn’t scum read at least not me
Quick: There is little to no chance this slot is Power-Wolfing.
SR: I want to give him a day
Life is simply unfair, don't you think?
User avatar
fferyllt
fferyllt
Titan of Trajectory
User avatar
User avatar
fferyllt
Titan of Trajectory
Titan of Trajectory
Posts: 20488
Joined: December 28, 2012
Location: Left Coast

Post Post #36 (ISO) » Fri Sep 29, 2017 11:50 am

Post by fferyllt »

That's not how it works.
Amid the pressure of great events, a general principle gives no help.

Idic
User avatar
PenguinPower
PenguinPower
He/Him
.peng
User avatar
User avatar
PenguinPower
He/Him
.peng
.peng
Posts: 24535
Joined: June 15, 2016
Pronoun: He/Him

Post Post #37 (ISO) » Fri Sep 29, 2017 11:50 am

Post by PenguinPower »

I thought I was 6-1...damnit.

Who wants to take over as Listmod while I eek out a few more scum wins?
<(") | (")>
User avatar
mhsmith0
mhsmith0
Balancing Act
User avatar
User avatar
mhsmith0
Balancing Act
Balancing Act
Posts: 10830
Joined: March 7, 2016
Location: Phoenix, AZ

Post Post #38 (ISO) » Fri Sep 29, 2017 11:54 am

Post by mhsmith0 »

In post 36, fferyllt wrote:That's not how it works.
Well not with THAT attitude anyway :P
In post 37, PenguinPower wrote:I thought I was 6-1...damnit.
Who wants to take over as Listmod while I eek out a few more scum wins?
I think it's 5-1?
1730 win
1732 win
1739 loss
1740 win
1807 win
1812 win

Am I missing one?
Show
http://wiki.mafiascum.net/index.php?title=Mhsmith0
Conq: you, sir, are great at being town.
BATMAN: Only jugg was the only one we didn’t scum read at least not me
Quick: There is little to no chance this slot is Power-Wolfing.
SR: I want to give him a day
Life is simply unfair, don't you think?
User avatar
PenguinPower
PenguinPower
He/Him
.peng
User avatar
User avatar
PenguinPower
He/Him
.peng
.peng
Posts: 24535
Joined: June 15, 2016
Pronoun: He/Him

Post Post #39 (ISO) » Fri Sep 29, 2017 11:57 am

Post by PenguinPower »

No, you're right. Damnit.
<(") | (")>
User avatar
Alisae
Alisae
lolbalance
User avatar
User avatar
Alisae
lolbalance
lolbalance
Posts: 47090
Joined: October 31, 2016
Location: Cali~ (PST)

Post Post #40 (ISO) » Fri Sep 29, 2017 5:56 pm

Post by Alisae »

I should play more newbies and suffer to improve my terrible winrate in newbies
GTKAS
| here.
User avatar
Raskolnikov
Raskolnikov
Jack of All Trades
User avatar
User avatar
Raskolnikov
Jack of All Trades
Jack of All Trades
Posts: 6395
Joined: November 15, 2015

Post Post #41 (ISO) » Fri Sep 29, 2017 6:21 pm

Post by Raskolnikov »

TFW you really want to play newbies but the list is so long and there's so few newbies to move it along and you don't know what your schedule will be like ~2 months from now when the games fire and heat up :(
deranged and incoherent
?
User avatar
mhsmith0
mhsmith0
Balancing Act
User avatar
User avatar
mhsmith0
Balancing Act
Balancing Act
Posts: 10830
Joined: March 7, 2016
Location: Phoenix, AZ

Post Post #42 (ISO) » Fri Sep 29, 2017 6:34 pm

Post by mhsmith0 »

That's what replacing in is for ;)
Show
http://wiki.mafiascum.net/index.php?title=Mhsmith0
Conq: you, sir, are great at being town.
BATMAN: Only jugg was the only one we didn’t scum read at least not me
Quick: There is little to no chance this slot is Power-Wolfing.
SR: I want to give him a day
Life is simply unfair, don't you think?
User avatar
LicketyQuickety
LicketyQuickety
Survivor
User avatar
User avatar
LicketyQuickety
Survivor
Survivor
Posts: 12785
Joined: May 14, 2015
Location: Where the moon and the sea meet.

Post Post #43 (ISO) » Wed Oct 11, 2017 11:26 pm

Post by LicketyQuickety »

Cool beans.
I was anything worse than you! Anything worse than you was I!

You was doided teh aposit_tisopa het dedoid saw em.
Post Reply

Return to “Mafia Discussion”