Newbie Game Player Ratings

mhsmith0 · Post Post #0 (isolation #0) » Wed Feb 22, 2017 5:34 pm

I've decided to pull this away form what realeo is doing in his ELO thread, since they're related but not really the same.

v1.010:

	Log Rating			Record
Player	Town	Scum	Combined	Town	Scum	Combined
RadiantCowbells	0.34	-1.30	1.64	22-24	8-1	30-25
GuyInFreezer	1.63		1.63	12-3	1-2	13-5
Thor665	0.37	-0.69	1.06	12-13	6-3	18-16
fferyllt	1.03		1.03	13-4	4-1	17-5
Micc	0.98		0.98	6-3	2-1	8-4
LicketyQuickety	0.88		0.88	6-3	1-2	7-5
Malakittens	0.70		0.70	17-11	4-3	21-14
T S O	0.60		0.60	7-5	2-2	9-7
Loopdan	0.55		0.55	8-6	2-0	10-6
Xayzeck	0.52		0.52	7-4	1-4	8-8
Huntress	0.42		0.42	8-7	1-4	9-11
JaeReed	0.41		0.41	6-6	0-0	6-6
PenguinPower	0.37		0.37	5-5	5-1	10-6
Fykus	0.36		0.36	4-5	0-0	4-5
Draynth	0.35		0.35	5-5	2-1	7-6
Creature	0.30		0.30	4-4	3-0	7-4
Guyett	0.27		0.27	7-7	2-2	9-9
Bulbazak	0.26		0.26	5-4	2-2	7-6
Raskolnikov	0.25		0.25	5-6	1-2	6-8
singersigner	0.25		0.25	6-5	2-0	8-5
mhsmith0	0.23		0.23	4-4	1-0	5-4
House	0.21		0.21	6-6	2-0	8-6
jmo16mla	0.13		0.13	5-5	0-0	5-5
Hopkirk	0.09		0.09	5-4	2-0	7-4
Nobody Special	0.04		0.04	7-7	1-2	8-9
Wisdom		-0.03	0.03	3-4	6-4	9-8
hayatoBL	0.02		0.02	5-6	3-1	8-7
tojam2	-0.08		-0.08	3-5	0-1	3-6
BlueBloodedToffee	0.09	0.17	-0.09	11-16	6-5	17-21
innocentvillager	-0.10		-0.10	5-8	4-2	9-10
goodmorning	-0.18	-0.05	-0.13	15-22	7-5	22-27
Nachomamma8	0.57	0.72	-0.15	17-12	5-12	22-24
TheIrishPope	-0.16		-0.16	4-5	1-3	5-8
Jake from State Farm	-0.19		-0.19	4-8	1-3	5-11
Cabd	-0.21		-0.21	8-8	0-0	8-8
RachMarie	-0.22		-0.22	11-18	2-2	13-20
Sakura Hana	-0.24		-0.24	5-6	3-3	8-9
JasonWazza	-0.26		-0.26	4-5	3-3	7-8
Dierfire	-0.27		-0.27	6-10	2-2	8-12
Accountant	-0.27		-0.27	6-12	4-2	10-14
copper223	-0.33		-0.33	3-5	2-0	5-5
theslimer3	-0.37		-0.37	4-6	0-0	4-6
Ms Marangal	-0.41		-0.41	3-6	1-4	4-10
ThinkBig	-0.41		-0.41	3-8	3-0	6-8
nancy	-0.44		-0.44	2-6	1-0	3-6
NicCage	-0.46		-0.46	2-6	0-0	2-6
PhantomCobalt	-0.47		-0.47	5-8	1-1	6-9
notscience	-0.34	0.16	-0.50	9-16	5-4	14-20
Drixx	0.47	0.96	-0.50	12-12	3-6	15-18
enomis	-0.52		-0.52	2-6	0-0	2-6
Alexcellent	-0.56		-0.56	3-5	0-1	3-6
Not_Mafia	-0.40	0.26	-0.66	6-10	3-5	9-15
Flubbernugget	-0.74		-0.74	3-8	0-0	3-8
MarioManiac4	-0.77		-0.77	3-10	2-3	5-13
WhemeStar	-0.83		-0.83	3-9	1-0	4-9
Titus	-1.01		-1.01	3-13	0-2	3-15
GuiltyLion	-1.03		-1.03	2-9	4-1	6-10
Alisae	-1.46		-1.46	1-8	1-1	2-9

intercept: -0.331 (41.8% town win odds given entirely "other" players)
combined error: 273.77 (average 0.610) - this is the mathematical equivalent of calling every game with 70% confidence and being correct 70% of the time (in actuality, it's much messier than that, but that's a reasonable analogue)

mhsmith0 · Post Post #1 (isolation #1) » Wed Feb 22, 2017 5:34 pm

most recent version was v1.003:

Update for 1768 and 1770, plus a couple other fixes:

Spoiler: v1.003 results

Spoiler: updates and corrections list

Interesting changes and other notes:
Nothing in particular from this update tbh

mhsmith0 · Post Post #2 (isolation #2) » Wed Feb 22, 2017 5:46 pm

new update, v1.004:

Update for 1771, 1772, 1775:

Spoiler: v1.004 results

Spoiler: updates and corrections list

Interesting changes and other notes:
RC and drixx got nice bumps (about 0.06 and 0.08) for their wins in 1755, which is about the scale you can expect from a normalish result from players with that many town games already on record. Also, given RC's solid town rating and 4-1 scum record, my guess is he's on track to get to a rating around Thor's level if/when he gets up to 8 scum games on the newbie queue.
Also notable that, once again, the base intercept gets more negative; after 2/3 results came in as scum wins (and the two scum wins were all "other" players, town and scum), it's now dropped back below a base town win probability of 40%.

mhsmith0 · Post Post #3 (isolation #3) » Wed Feb 22, 2017 6:04 pm

A bit more discussion on the numbers and waht they mean:

(using v1.004 values):

A game with NONE of the individually rated players would be rated at the intercept, -0.434, probability of a town win = 39.3% (L = -0.434, exp(L) = 0.648, p(win) = exp(L) / (1+exp(L) = 39.3%, which compares to the actual town win rate of 43.3% in that set of games)

A game with any of these players would be rated at the intercept plus that player's rating (or plus multiple if multiple are in the game). As a simple example, if there are eight "other" players and thor, then the odds become:
If Thor is town: L = -0.434 + 0.478 = 0.045 -> p(town win) = 51.1% (Thor's actual town record: 12-13, 48%)
If Thor is scum: L = -0.434 + -0.393 = -0.826 -> p(town win) = 30.4% (Thor's actual scum record: 6-3, 67% which inverts to 33% town odds)

So basically, high positive numbers are good if town, high negative numbers are good if scum, and I combined them for a total rating (though most players don't have the data to really be rated for scum).

It's also worth noting that the "all others" town win percentage calculation is materially lower than the actual town win percentage. What that essentially means is that, as a whole, that town players with enough data to get rated have been good enough to increase town's win odds beyond replacement player level. It also means that scum players with enough data to get rated have NOT been good enough to decrease town's win odds beyond replacement player level (though it's notable that some successful scum players such as fferyllt, RadiantCowbells, innocentvillager, Accountant, and GuiltyLion haven't reached the 8 scum game threshold - if/when they do, the numbers may change yet again).

mhsmith0 · Post Post #5 (isolation #4) » Fri Feb 24, 2017 9:06 am

Since updates are going to happen pretty sporadically (probably looking at every other week or so given the pace, and even those probably net like 1-3 games each), I figured it might be interesting to instead take some time and talk some more about what this means and why it makes sense to use this kind of methodology (logistic regression).

Spoiler: The Very Basics, and a 50-50 coin flip

So as I'd noted previously, this kind of methodology estimates win probabilities by giving what are called "log ratings". So what does that mean?

Well, let's take a really simple example. You have a 2 sided coin, heads and tails. Let's say every time you get a heads, you mark it down as a 1, and every time you get tails, you mark it down as a 0. You flip it 10 times, 5 heads, 5 tails.

A logistic regression model would take those results, and then backsolve to get the estimated value that maximizes the associated probability of the results (this is done by minimizing an "error" function). Essentially, we look at the combined probability of the observed events given both the actual events and our assumed likelihood per flip, and create a model that maximizes what the total probability is. As a simplifying process, we take the negative of the log of the probability of each result and add those together, instead of mulitplying the calculated probabilities together (more on this later).

Let's say, for the sake of the argument, that you decided to model out a 100% heads flip probability. For each flip that was heads, your prediction was exactly right, and for each tails, your prediction was exactly wrong. Unfortunately, the combined probability of the observed events given your assumed likelihood per flip is zero, as you've observed at least one (five in fact!) theoretically impossible results (tails results given an assumption that 100% of the flips will be heads). If you take the negative logs, you see this pretty easily. The correct flips get a zero error functoin, and the wrong flips each give you an infinite error function. Not such a good result

Now let's say you make a more reasonable assumption (though still not correct). Let's say you assume it's really a 60% heads probability. Now, for each correct flip, you get a modeled result probability of 60%, and for each wrong flip, it's 40% (i.e. you said that heads was 60% and it was heads, or you said that tails was 40% and it was tails). For each pair of heads-tails flips, you get a combined probability of 60% * 40% = 24%, which is CLOSE to the correct answer (50% each), but not quite there. Incidentally, your log error would then be 0.511 for the heads flips, and 0.916 for the tails flips, for an overall average of 0.714 per flip.

That 60% estimate is NEARLY as good as the correct answer of 50-50, but not quite there. Two results (whatever they are) would get you 25% combined probability, and a log error of 0.693 per flip.

Indeed, that 0.693 per flip is the baseline of what you will want to measure up against. Any system you put into place needs to beat that error (otherwise it would default to a "well we have no clue it's just random" state) in order to get any results at all!

Spoiler: 60-40 example

Spoiler: logit for gaming results

Spoiler: converting that into a mafia / newbie game model

mhsmith0 · Post Post #6 (isolation #5) » Fri Mar 03, 2017 1:09 pm

I'm going to switch tacks a bit and talk about some of the more interesting game results. Specifically, I'm going to look at games where especially good towns were defeated by mafia, and vice-versa. Really impressive wins are the sorts of things that can strongly bump players' ratings, as either alignments; in general, if you're going to have a strong rating, you're either going to have a really good resume with a strong record and generally strong opponents, OR you're going to have some number of standout performances that really boost your ratings.

Best towns beaten by mafia (as of v1.004)

1 - 1429
Town Rating: 1.769: Nachomamma8, fferyllt, Bulbazak, 4 {OTHER} players
Mafia Rating: 0.000: 2 {OTHER} players (F-16_Fighting_Falcon, Bert)
2 - 1593
Town Rating: 1.659: GuyInFreezer, 6 {OTHER} players
Mafia Rating: 0.000: 2 {OTHER} players (Sparx555, AxleGreaser)
3 - 1404
Town Rating: 1.440: GuyInFreezer, Sakura Hana, 5 {OTHER} players
Mafia Rating: 0.000: 2 {OTHER} players (SalmonellaDreams, SXTLHGaiden)
4 - 1644
Town Rating: 1.428: Malakittens, Drixx, 5 {OTHER} players
Mafia Rating: -0.088: goodmorning, 1 {OTHER} player (Vedith)
5 - 1451
Town Rating: 1.238: goodmorning, Malakittens, Huntress, 4 {OTHER} players
Mafia Rating: 0.000: 2 {OTHER} players (Yiley, emeraldemon)

Best mafia teams beaten by town (as of v1.004)

1A - 1548
Town Rating: 0.576: Micc, 6 {OTHER} players
Mafia Rating: -0.393: Thor665, {OTHER}
1B - 1693
Town Rating: 0.664: RadiantCowbells, PhantomCobalt, Micc, 4 {OTHER} players
Mafia Rating: -0.393: Thor665, {OTHER}
1C - 1724
Town Rating: 1.270: Malakittens, JaeReed, LicketyQuickety, 4 {OTHER} players
Mafia Rating: -0.393: Thor665, {OTHER}
4A - 1447
Town Rating: 0.564: Guyett, Bulbazak, 5 {OTHER} players
Mafia Rating: -0.393: goodmorning, {OTHER}
4B - 1546
Town Rating: -0.282: BlueBloodedToffee, 6 {OTHER} players
Mafia Rating: -0.393: goodmorning, {OTHER}
(1602, 1627, 1683 were also scum losses by goodmorning and a non-rated scum player. Thor and GM are currently the only scum players rated as above-average).

mhsmith0 · Post Post #7 (isolation #6) » Mon Mar 20, 2017 2:49 pm

new update, v1.005:

Update for 1773, 1776, 1777:

Spoiler: v1.005 results

Spoiler: updates and corrections list

Interesting changes and other notes:
Two fairly "high error" results just happened in that set:
1776 - Town with RC and Drixx with five "other" players against two "other" players lost (even more notably, PenguinPower, who's currently 4-2 in newbies as town, was ALSO in that town and was defeated). This is the sort of result which, if Mewtaph and/or ChrisOrmie hang around for a while, could substantially boost their long-term ratings.
1777 - This was a game with entirely "other" players (Grey doesn't have enough town games, and Drixx doesn't have enough scum games, to be rated), and town won (I think scum have a very solid win rate in these types of games). My guess is that this in particular was what drove the intercept up a decent amount from v1.004

mhsmith0 · Post Post #9 (isolation #7) » Tue Mar 21, 2017 8:21 pm

Well fortunately you've got a large enough sample size it's not a huge swing (0.40 to 0.31 is not a major shift in rating), but yeah there's plenty of noise associated with a system that basically punishes or rewards every member of a winning or losing team equally (one easy example: when I hit eight town games, my own rating will be eating a nasty detriment for 1756 thanks to losing with positive rated townie jae and loop, despite subbing in 5p LYLO with all town power dead and correctly tunneling scum!accountant, one of the many results underlying my "you're the unluckiest player I've ever seen" description that I was given a while ago and that continues to basically be correct

)

mhsmith0 · Post Post #10 (isolation #8) » Sun Apr 02, 2017 10:03 am

new update, v1.006:

Update for 1774, 1778, 1780, 1781:

Spoiler: v1.006 results

Spoiler: updates and corrections list

Interesting changes and other notes:
scum!RC is now 5-1, and if he gets to 6-1, I'll probably give him the mor_tilt treatment (see http://www.mafiauniverse.com/forums/thr ... post948532) and add him to the table, because even if 7 games isn't quite the sample size I want to see, 6-1 as a record (if/when he hits that point) suggests that he's WAY better than the average scum, and should both be recognized as such and the players he's beaten shouldn't get the hit of seemingly losing to a generic scum (it also substantially boosts Huntress's credit for 1700 which is odd considering she was mislynched day 1 there as a PR but *shrugs*)

mhsmith0 · Post Post #13 (isolation #9) » Mon Apr 03, 2017 4:29 am

Well I mean winning a lot does help

mhsmith0 · Post Post #14 (isolation #10) » Sat May 13, 2017 6:33 am

new update, v1.007:

Update for 1779, 1782, 1783, 1784, 1785, 1786, 1788:

Spoiler: v1.007 results

Spoiler: updates and corrections list

Interesting changes and other notes:
Probably the most notable thing is that RC's scum game is now on the table, and, to probably no one's surprise, it is a very very highly rated scum game. Basically, at the top of the table it's GIF, RC, and then a pretty big gap until the next tier.

FWIW, there's also a decent stack of people who are close to getting on the table, with 6 or 7 games in a particular alignment while being on the queue often enough to probably make the list in the next couple of months.[/quote]

mhsmith0 · Post Post #20 (isolation #11) » Tue May 16, 2017 12:24 pm

yes, see
viewtopic.php?f=5&t=69237

though that's for 10v3 mini normals (which isn't all of them), and it's I think like 3 months out of date.

mhsmith0 · Post Post #22 (isolation #12) » Thu Jun 22, 2017 12:55 pm

new update, v1.008:

Update for 1787, 1789, 1790, 1791, 1793, 1794, 1795, 1796:

Spoiler: v1.008 results

	Log Rating			Record
Player	Town	Scum	Combined	Town	Scum	Combined
GuyInFreezer	1.69		1.69	12-3	1-1	13-4
RadiantCowbells	0.37	-1.13	1.50	19-19	7-1	26-20
Thor665	0.41	-0.55	0.96	12-13	6-3	18-16
Malakittens	0.88		0.88	17-10	3-3	20-13
fferyllt	0.71		0.71	12-5	3-1	15-6
T S O	0.68		0.68	7-5	2-2	9-7
Creature	0.65		0.65	5-3	3-0	8-3
Huntress	0.64		0.64	8-5	1-4	9-9
Micc	0.55		0.55	5-4	2-1	7-5
Xayzeck	0.53		0.53	7-4	1-4	8-8
Drixx	0.51		0.51	12-12	3-4	15-16
singersigner	0.47		0.47	6-5	1-0	7-5
Loopdan	0.44		0.44	8-6	2-0	10-6
JaeReed	0.35		0.35	5-5	0-0	5-5
Hopkirk	0.28		0.28	5-3	2-0	7-3
Bulbazak	0.25		0.25	5-4	2-2	7-6
Guyett	0.24		0.24	7-7	2-2	9-9
Raskolnikov	0.22		0.22	5-6	1-2	6-8
PenguinPower	0.17		0.17	5-5	3-1	8-6
theslimer3	0.05		0.05	4-4	0-0	4-4
jmo16mla	0.04		0.04	5-5	0-0	5-5
tojam2	0.02		0.02	3-5	0-1	3-6
hayatoBL	0.00		0.00	5-6	3-1	8-7
TheIrishPope	-0.01		-0.01	4-5	1-3	5-8
innocentvillager	-0.05		-0.05	5-7	4-1	9-8
Wisdom		0.06	-0.06	3-4	6-4	9-8
copper223	-0.15		-0.15	3-5	2-0	5-5
Nachomamma8	0.61	0.76	-0.16	17-12	5-12	22-24
House	-0.16		-0.16	5-7	2-0	7-7
goodmorning	-0.27	-0.10	-0.17	14-23	7-5	21-28
JasonWazza	-0.20		-0.20	4-5	3-3	7-8
Accountant	-0.23		-0.23	6-12	4-2	10-14
Nobody Special	-0.23		-0.23	6-8	1-2	7-10
Dierfire	-0.25		-0.25	6-10	2-2	8-12
Sakura Hana	-0.25		-0.25	5-6	3-3	8-9
RachMarie	-0.27		-0.27	11-18	2-2	13-20
BlueBloodedToffee	-0.08	0.20	-0.28	10-17	6-5	16-22
PhantomCobalt	-0.36		-0.36	5-8	1-1	6-9
NicCage	-0.45		-0.45	2-6	0-0	2-6
Ms Marangal	-0.46		-0.46	3-6	1-4	4-10
Cabd	-0.50		-0.50	6-8	0-0	6-8
notscience	-0.33	0.18	-0.52	9-16	5-4	14-20
enomis	-0.57		-0.57	2-6	0-0	2-6
Not_Mafia	-0.34	0.26	-0.59	6-10	3-5	9-15
Alexcellent	-0.62		-0.62	3-5	0-1	3-6
Flubbernugget	-0.78		-0.78	3-8	0-0	3-8
MarioManiac4	-0.93		-0.93	2-8	2-3	4-11
Jake from State Farm	-0.95		-0.95	2-10	1-3	3-13
Titus	-1.02		-1.02	3-13	0-2	3-15
GuiltyLion	-1.27		-1.27	1-8	4-1	5-9
Alisae	-1.55		-1.55	1-8	0-1	1-9

intercept: -0.315 (42.2% town win odds given entirely "other" players)
combined error: 258.75 (average 0.609)

Spoiler: updates and corrections list

Interesting changes and other notes:
Not massively. Having played with Alisae before, I do think there's probably some hard luck baked in there (1779 and 1789 in particular I'd say really weren't his fault when he 1v1'd scum before LYLO and town mislynched him in each), though I couldn't say exactly how much.

Next up on v1.009 will be town!ThinkBig in all likelihood. There are others who may hop into the 8+ results categories, but no one who necessarily seems imminent (unless I've missed something of course).

mhsmith0 · Post Post #24 (isolation #13) » Thu Jun 22, 2017 2:16 pm

1.58 to 1.50 isn't really what i'd consider an especially material shift, fwiw. That said...

1) "It's complicated" (there are a lot of moving parts - every player's rating changes every iteration since it's a regression methodlogy, and the intercept moves too)

2) Since the main thing was your scum rating dropped (your town rating bumped a bit), I'll talk about that. Again, it's complicated, but the main drivers on cursory glance are three games:
1655 - GuiltyLion is now rated, and pretty lowly as town, so that scum win became worth substantially less
1780 - Alisae is now rated, and pretty lowly as town, so that scum win became worth substantially less
1700 - Huntress's town rating has gone down, so that loss hurt a bit more (and since it's your one and only scum loss, it has a relatively outsize impact on your scum rating)

PS To put it into context a bit, in v1.007, a game with you as scum and all unrated players would model as:
X = -0.388 (intercept) - 1.27 (your rating) = -1.658. Town win odds then = exp(-1.658) = 19.1%
Now it's
X = -.315 - 1.13 = -1.445, town win odds = exp(-1.445) = 23.5%

So basically about a 4.5% win odds swing model to model, with a third of that being explicitly the change in intercept (i.e. what the odds of an entirely "other players" game would look like), so basically a 3% lesser impact from your scum game compared to before, and even that is counteracted by your town rating bumping up by about a third of what your scum rating dropped by.

mhsmith0 · Post Post #26 (isolation #14) » Thu Jun 22, 2017 2:30 pm

It's possible, but I think the big issue is that the sample size is much lower (not all mini normals are 10/3, and they are slower to fill than newbie games) So I could do a separate model there, but I strongly suspect there'd be very few players who have enough data even to hit the 8x town threshold, which means that I'd guess it'd be a lot less interesting.

Also setup meta varies from time to time, and whereas a lot of data exists to demonstrate that matrix6 is pretty blaanced across setups (all are 40-60% town win rate, and few would basically only rand the most favorable or most unfavorable setups), I strongly suspect that there have been a number of mini normal setups that are simply unbalanced, in either direction, which means there's more rand luck in the results.

mhsmith0 · Post Post #29 (isolation #15) » Fri Jul 21, 2017 9:18 am

new update, v1.009:

Update for 1792, 1797, 1799, 1800, 1801, 1802, 1803, 1804, 1805:

Spoiler: v1.009 results

	Log Rating			Record
Player	Town	Scum	Combined	Town	Scum	Combined
RadiantCowbells	0.36	-1.29	1.65	21-20	8-1	29-21
GuyInFreezer	1.65		1.65	12-3	1-1	13-4
Thor665	0.38	-0.68	1.06	12-13	6-3	18-16
fferyllt	1.00		1.00	13-4	4-1	17-5
Micc	0.99		0.99	6-3	2-1	8-4
Malakittens	0.73		0.73	17-11	3-3	20-14
Creature	0.70		0.70	5-3	3-0	8-3
T S O	0.62		0.62	7-5	2-2	9-7
Xayzeck	0.52		0.52	7-4	1-4	8-8
Loopdan	0.47		0.47	8-6	2-0	10-6
LicketyQuickety	0.46		0.46	5-3	1-2	6-5
Huntress	0.37		0.37	8-7	1-4	9-11
JaeReed	0.34		0.34	5-5	0-0	5-5
Guyett	0.28		0.28	7-7	2-2	9-9
Bulbazak	0.24		0.24	5-4	2-2	7-6
singersigner	0.24		0.24	6-5	1-0	7-5
Raskolnikov	0.22		0.22	5-6	1-2	6-8
PenguinPower	0.21		0.21	5-5	3-1	8-6
House	0.19		0.19	6-6	2-0	8-6
jmo16mla	0.09		0.09	5-5	0-0	5-5
Hopkirk	0.02		0.02	5-4	2-0	7-4
Nobody Special	0.02		0.02	7-7	1-2	8-9
Wisdom		0.00	0.00	3-4	6-4	9-8
hayatoBL	-0.01		-0.01	5-6	3-1	8-7
innocentvillager	-0.05		-0.05	5-7	4-1	9-8
BlueBloodedToffee	0.08	0.16	-0.08	11-16	6-5	17-21
goodmorning	-0.20	-0.10	-0.10	15-22	7-5	22-27
ThinkBig	-0.13		-0.13	3-6	3-0	6-6
TheIrishPope	-0.14		-0.14	4-5	1-3	5-8
Nachomamma8	0.56	0.73	-0.17	17-12	5-12	22-24
tojam2	-0.17		-0.17	3-5	0-1	3-6
Jake from State Farm	-0.19		-0.19	4-8	1-3	5-11
Sakura Hana	-0.24		-0.24	5-6	3-3	8-9
JasonWazza	-0.27		-0.27	4-5	3-3	7-8
theslimer3	-0.28		-0.28	4-5	0-0	4-5
Accountant	-0.28		-0.28	6-12	4-2	10-14
Dierfire	-0.28		-0.28	6-10	2-2	8-12
RachMarie	-0.30		-0.30	11-18	2-2	13-20
copper223	-0.33		-0.33	3-5	2-0	5-5
Cabd	-0.41		-0.41	7-8	0-0	7-8
Ms Marangal	-0.42		-0.42	3-6	1-4	4-10
Drixx	0.48	0.95	-0.48	12-12	3-6	15-18
NicCage	-0.48		-0.48	2-6	0-0	2-6
notscience	-0.36	0.14	-0.50	9-16	5-4	14-20
PhantomCobalt	-0.50		-0.50	5-8	1-1	6-9
enomis	-0.56		-0.56	2-6	0-0	2-6
Alexcellent	-0.64		-0.64	3-5	0-1	3-6
Not_Mafia	-0.39	0.28	-0.67	6-10	3-5	9-15
MarioManiac4	-0.74		-0.74	3-9	2-3	5-12
Flubbernugget	-0.77		-0.77	3-8	0-0	3-8
Titus	-1.03		-1.03	3-13	0-2	3-15
GuiltyLion	-1.23		-1.23	1-8	4-1	5-9
Alisae	-1.59		-1.59	1-8	1-1	2-9

intercept: -0.291 (42.8% town win odds given entirely "other" players)
combined error: 265.52 (average 0.612)

Spoiler: updates and corrections list

Interesting changes and other notes:
Not massively (Drixx's scum rating dropped his overall rating a lot, but I think that was fairly predictable from looking at the record pre-update). One interesting thing I've started to play with is potentially breaking games up into which setup was randed; doing so shifts some player ratings around a bit, but few of them really moved that much. Row 2 (cop/doc/rb) is by far the easiest town rating game (duh), but it was interesting that A (jk/bp/rb), 3 (tracker/bp) AND C (tracker/doc) were all rated as being much harder than the other setups. I hadn't really expected setup C to be much closer to A and 3, but that's what early analysis suggests.

Next up on v1.010 will be town!Draynth in all likelihood. There are others who may hop into the 8+ results categories, but no one who necessarily seems imminent (unless I've missed something of course).

mhsmith0 · Post Post #30 (isolation #16) » Fri Jul 21, 2017 9:24 am

More specifics on the setup variations:

1 (jk): -0.139, equivalent to town win probability of 46.5% given an all "other" group (actual town win rate 52.2%)
2 (cop/doc/rb): +0.215, equivalent to town win probability of 55.4% given an all "other" group (actual town win rate 54.7%)
3 (tracker/bp): -0.544, equivalent to town win probability of 36.7% given an all "other" group (actual town win rate 39.2%)
A (jk/bp/rb): -0.474, equivalent to town win probability of 38.4% given an all "other" group (actual town win rate 40.0%)
B (cop): -0.175, equivalent to town win probability of 45.6% given an all "other" group (actual town win rate 45.6%)
C (tracker/doc): -0.440, equivalent to town win probability of 39.2% given an all "other" group (actual town win rate 43.1%)

mhsmith0 · Post Post #31 (isolation #17) » Thu Sep 28, 2017 11:50 am

new update, v1.010:

Update for 1806, 1807, 1809-1820:

Spoiler: v1.010 results

	Log Rating			Record
Player	Town	Scum	Combined	Town	Scum	Combined
RadiantCowbells	0.34	-1.30	1.64	22-24	8-1	30-25
GuyInFreezer	1.63		1.63	12-3	1-2	13-5
Thor665	0.37	-0.69	1.06	12-13	6-3	18-16
fferyllt	1.03		1.03	13-4	4-1	17-5
Micc	0.98		0.98	6-3	2-1	8-4
LicketyQuickety	0.88		0.88	6-3	1-2	7-5
Malakittens	0.70		0.70	17-11	4-3	21-14
T S O	0.60		0.60	7-5	2-2	9-7
Loopdan	0.55		0.55	8-6	2-0	10-6
Xayzeck	0.52		0.52	7-4	1-4	8-8
Huntress	0.42		0.42	8-7	1-4	9-11
JaeReed	0.41		0.41	6-6	0-0	6-6
PenguinPower	0.37		0.37	5-5	5-1	10-6
Fykus	0.36		0.36	4-5	0-0	4-5
Draynth	0.35		0.35	5-5	2-1	7-6
Creature	0.30		0.30	4-4	3-0	7-4
Guyett	0.27		0.27	7-7	2-2	9-9
Bulbazak	0.26		0.26	5-4	2-2	7-6
Raskolnikov	0.25		0.25	5-6	1-2	6-8
singersigner	0.25		0.25	6-5	2-0	8-5
mhsmith0	0.23		0.23	4-4	1-0	5-4
House	0.21		0.21	6-6	2-0	8-6
jmo16mla	0.13		0.13	5-5	0-0	5-5
Hopkirk	0.09		0.09	5-4	2-0	7-4
Nobody Special	0.04		0.04	7-7	1-2	8-9
Wisdom		-0.03	0.03	3-4	6-4	9-8
hayatoBL	0.02		0.02	5-6	3-1	8-7
tojam2	-0.08		-0.08	3-5	0-1	3-6
BlueBloodedToffee	0.09	0.17	-0.09	11-16	6-5	17-21
innocentvillager	-0.10		-0.10	5-8	4-2	9-10
goodmorning	-0.18	-0.05	-0.13	15-22	7-5	22-27
Nachomamma8	0.57	0.72	-0.15	17-12	5-12	22-24
TheIrishPope	-0.16		-0.16	4-5	1-3	5-8
Jake from State Farm	-0.19		-0.19	4-8	1-3	5-11
Cabd	-0.21		-0.21	8-8	0-0	8-8
RachMarie	-0.22		-0.22	11-18	2-2	13-20
Sakura Hana	-0.24		-0.24	5-6	3-3	8-9
JasonWazza	-0.26		-0.26	4-5	3-3	7-8
Dierfire	-0.27		-0.27	6-10	2-2	8-12
Accountant	-0.27		-0.27	6-12	4-2	10-14
copper223	-0.33		-0.33	3-5	2-0	5-5
theslimer3	-0.37		-0.37	4-6	0-0	4-6
Ms Marangal	-0.41		-0.41	3-6	1-4	4-10
ThinkBig	-0.41		-0.41	3-8	3-0	6-8
nancy	-0.44		-0.44	2-6	1-0	3-6
NicCage	-0.46		-0.46	2-6	0-0	2-6
PhantomCobalt	-0.47		-0.47	5-8	1-1	6-9
notscience	-0.34	0.16	-0.50	9-16	5-4	14-20
Drixx	0.47	0.96	-0.50	12-12	3-6	15-18
enomis	-0.52		-0.52	2-6	0-0	2-6
Alexcellent	-0.56		-0.56	3-5	0-1	3-6
Not_Mafia	-0.40	0.26	-0.66	6-10	3-5	9-15
Flubbernugget	-0.74		-0.74	3-8	0-0	3-8
MarioManiac4	-0.77		-0.77	3-10	2-3	5-13
WhemeStar	-0.83		-0.83	3-9	1-0	4-9
Titus	-1.01		-1.01	3-13	0-2	3-15
GuiltyLion	-1.03		-1.03	2-9	4-1	6-10
Alisae	-1.46		-1.46	1-8	1-1	2-9

intercept: -0.331 (41.8% town win odds given entirely "other" players)
combined error: 276.77 (average 0.610)

Spoiler: updates and corrections list

Interesting changes and other notes:
town!LicketyQuickety's rating shot up on a relatively small amount of data (a win in 1813). Eyeballing the data, it looks like the other main impact was that 1803 punished his rating a lot less, mainly due to Wheme coming in with a relatively low score.

Fykus's high score given his 4-5 record was a bit surprising, but this seems to be LARGELY driven by the 1787 win (third lowest rated town to win a newbie game) getting an outsize + rating.

Newbie

1720

1820 is actually a pretty decent chance to talk about the ratings and the impacts of "upset" type results. Prior to that game, Draynth and Creature were both rated at about 0.60, which meant that they both took a fairly substantial hit for that single game. Prior to the game result, town was estimated at about a 71% win chance (Draynth and Creature had enough data to be rated, the other 7 slots were all "other"). After the game, it went down to 58%. Especially for players with low sample sizes, the model inherently revises ratings to eliminate "weird" results. In that case, the weird result would be a fairly highly rated town losing (before the game, it would have been about tied for the #5 top rated town to lose; after the ratings revised, they went down to about the #25 rated town to lose, still a notable result but not quite as major an outcome).

Also, since I'm a high ego guy, I'm going to talk about me

I'd say that the 0.24 rating is probably around fair given the methodology. I'll note that I took a fair sized hit for 1756 (town of ThinkBig/Loop/Jae was overall an unusually positive rated town; and I subbed in at 3/2 LYLO with all town PRs dead

), and a somewhat lesser hit for 1691 (a town loss with town!Thor), but I also had some nice positives on other games to help balance that

Right now, there's really no one who jumps out at me as being particularly likely to be hopping onto the 8+ list for whenever this updates next. Jingle is at 7 scum games, so he'll pop onto the list after his next newbie scum game, but it's hard to say when that might be.

mhsmith0 · Post Post #33 (isolation #18) » Fri Sep 29, 2017 7:37 am

mhsmith0 · Post Post #35 (isolation #19) » Fri Sep 29, 2017 11:33 am

Just replace into a bunch of slots you think are scum.

mhsmith0 · Post Post #38 (isolation #20) » Fri Sep 29, 2017 11:54 am

In post 36, fferyllt wrote:That's not how it works.

Well not with THAT attitude anyway

In post 37, PenguinPower wrote:I thought I was 6-1...damnit.
Who wants to take over as Listmod while I eek out a few more scum wins?

I think it's 5-1?
1730 win
1732 win
1739 loss
1740 win
1807 win
1812 win

Am I missing one?

mhsmith0 · Post Post #42 (isolation #21) » Fri Sep 29, 2017 6:34 pm

That's what replacing in is for