Project ELO (Dataset released)

[Realeo]

I do not compute the average score.

[LicketyQuickety]

I do not compute the average score.

Why not?

[Realeo]

Because they are believed not to converge yet, remember?

If you mean the average of considered player, that would come in next update. However, as you said, the curve of ELO distribution is like bell curve.

[Raskolnikov]

Do you have any plans for calculating town/scum play separately in future?

[Realeo]

I am considering that after I tabulated Open. From 2013, there are around 200 games so that bumpts the tabulized game total to around 700ish.

Although, it's not how do I calculate town/scum. It's how I calculate town/scum1/scum2/SK that becomes the problem.

[Realeo]

Because town ELO is defined as scumhunting skill and scum ELO is defined as scumavoiding skill.

In Town v Mafia v Werewolf v SK, everyone is scumhunting of the other factions. Which ELo is used for SK?

[LicketyQuickety]

Something has been on my mind Realeo. You said in a game I was in with you that I had played 55 games. At the time I only had 51 (?) games recorded. What was that all about?

[Realeo]

You said in a thread that you would only mentor-hydra if you have at least 55 games. Educated guess.

Some early Open Games are lost due to TIGERS attack =/

=/

[Realeo]

You are welcomed to give idea, especially about how to calculate elo in multi ball if the elo is split town/scum

[Raskolnikov]

town use full town rating

scum 2/3 scum rating 1/3 town rating ? use both and adjust the weighting as desired ?

I haven't played much multiball tho

[LicketyQuickety]

You said in a thread that you would only mentor-hydra if you have at least 55 games. Educated guess.

Some early Open Games are lost due to TIGERS attack =/

=/

Ah, It was based on misconception then. I said I would only want to get taught by someone who has 50 games played, not 55 iirc. I think I have 53 completed games with some ongoing and some I have subbed out of. I think you are a bit behind in the games I have played calculating my ELO if you only have 8 games on record. I have 9 newbie games alone which have been completed (which is not a lot). naturally, 8/53 completed games is not exactly a good depiction of how good a player I am. Granted I only have 21 (?) games completed on this site. So 8/21 is a low number.

[LicketyQuickety]

You are welcomed to give idea, especially about how to calculate elo in multi ball if the elo is split town/scum

I would account for win rates of SK. Just a thought, I have no idea what you are actually doing with all this math stuff. I am quite rusty when it comes to maths.

[LicketyQuickety]

You are welcomed to give idea, especially about how to calculate elo in multi ball if the elo is split town/scum

I would account for win rates of SK. Just a thought, I have no idea what you are actually doing with all this math stuff. I am quite rusty when it comes to maths.

I meant average win rates in standard games with SKs. *shrug*

[mhsmith0]

For kicks, I did a quick log regression of the newbie games (1370-1765), rating only players with at least 8 games in a particular faction (anyone without at least 8 games in at least one alignment was dropped), dropping in a dummy win and a dummy loss (for each player for each alignment) for smoothing purposes. Results:

Spoiler:

	Log Rating			Record
Player	Town	Scum	Combined	Town	Scum	Combined
GuyInFreezer	1.65		1.65	12-3	1-1	13-4
Thor665	0.53	-0.37	0.90	12-12	6-3	18-15
Malakittens	0.88		0.88	17-10	3-3	20-13
fferyllt	0.79		0.79	12-5	3-1	15-6
T S O	0.73		0.73	7-5	2-2	9-7
singersigner	0.56		0.56	6-5	1-0	7-5
Micc	0.55		0.55	5-4	2-1	7-5
Drixx	0.52		0.52	11-10	3-2	14-12
Xayzeck	0.52		0.52	6-4	1-4	7-8
Huntress	0.41		0.41	6-5	1-2	7-7
Hopkirk	0.34		0.34	5-3	2-0	7-3
Bulbazak	0.30		0.30	5-4	2-2	7-6
Guyett	0.25		0.25	7-7	2-2	9-9
RadiantCowbells	0.17		0.17	12-14	3-1	15-15
Raskolnikov	0.16		0.16	4-5	1-1	5-6
hayatoBL	0.04		0.04	5-6	3-1	8-7
Dierfire	0.02		0.02	7-10	2-2	9-12
Wisdom		0.00	0.00	3-4	6-4	9-8
goodmorning	-0.16	-0.10	-0.06	14-23	7-5	21-28
TheIrishPope	-0.09		-0.09	4-5	1-3	5-8
innocentvillager	-0.10		-0.10	5-7	3-1	8-8
House	-0.11		-0.11	5-7	2-0	7-7
JasonWazza	-0.14		-0.14	4-5	3-3	7-8
RachMarie	-0.17		-0.17	11-18	2-2	13-20
Nachomamma8	0.69	0.86	-0.17	17-12	5-12	22-24
Nobody Special	-0.20		-0.20	6-8	1-2	7-10
Accountant	-0.21		-0.21	6-11	2-1	8-12
PhantomCobalt	-0.26		-0.26	5-8	1-1	6-9
Sakura Hana	-0.29		-0.29	5-6	3-3	8-9
Not_Mafia	-0.31		-0.31	6-10	3-4	9-14
BlueBloodedToffee	-0.24	0.19	-0.43	9-17	6-5	15-22
Cabd	-0.43		-0.43	6-8	0-0	6-8
notscience	-0.29	0.17	-0.46	9-16	5-4	14-20
Ms Marangal	-0.62		-0.62	2-6	1-4	3-10
Titus	-0.66		-0.66	3-9	0-2	3-11
Flubbernugget	-0.70		-0.70	3-8	0-0	3-8
MarioManiac4	-0.87		-0.87	2-8	1-2	3-10
Jake from State Farm	-0.90		-0.90	2-10	1-3	3-13

Interpretation:
A game with NONE of these players would be rated at the intercept, -0.422, probability of a town win = 39.6% (L = -0.422, exp(L) = 0.656, p(win) = exp(L) / (1+exp(L) = 39.6%, which compares to the actual town win rate of 44.2% in that set of games)

A game with any of these players would be rated at the intercept plus that player's rating (or plus multiple if multiple are in the game). As a simple example, if there are eight "other" players and thor, then the odds become:
If Thor is town: L = -0.422 + 0.530 = 0.108 -> p(town win) = 52.7% (Thor's actual town record: 12-12, 50%)
If Thor is scum: L = -0.422 + -0.369 = -0.791 -> p(town win) = 31.2% (Thor's actual scum record: 6-3, 67% which inverts to 33% town odds)

So basically, high positive numbers are good if town, high negative numbers are good if scum, and I combined them for a total rating (though most players don't have the data to really be rated for scum).

[Realeo]

What is the train/test data split

[mhsmith0]

None, it's one big regression. There isn't NEARLY enough data to split out a validation sample so I simply didn't, instead choosing as a methodology to just chuck out data for individual/alignment pairings with too low of a sample size for me to consider credible (and 8 is somewaht arbitrary of course).

[Realeo]

Do you train per player? If you do, please be aware of overfitting.

And how do you exactly do that? Multivariate logistic regression?

I think I may as well spilt ELO from town/mafia without waiting to input Open Data. I don't think I'm capable of inputing data during college, but coding for it is definitely a go.

[mhsmith0]

Yes, multivariate logistic regression (X = indicators, 0/1, for each player; Y = 0 or 1, town loss or town win). There is a clear possibility, indeed likelihood, of overfitting. The main protection I imposed was simply cutting off data that didn't have enough of a sample size, but beyond that, I've pretty much accepted that there is substantial noise in the numbers, and that they're a loose but reasonable approximation of what they "should" be. I just don't see much of a gain from doing more complicated train/test mechanisms here; there's enough data to get a reasonable approximation of what it ought to say, and IMO a logit methodology is relatively more accurate than most others, and at least has some reasonable level of statistical support, compared to ones like the (IMO) fairly arbitrary ELO.

[Realeo]

arbitrary ELO.

Now that you mention that, I should probably fine tune the K factor.

[Realeo]

IMO a logit methodology is relatively more accurate than most others

Is the win/rate distribution even a logit? I thought the win/rate distribution is a gaussian?

[mhsmith0]

Skimming results, in general, rated players' win rates predicted by the logit (intercept + rating) are pretty close to their actual. One itneresting exception is hopkirk, who is 5-3 (63% win rate) but is predicted at a 48% win rate.

Spoiler: Looking into hopkirk's games in more detail

He has 8 games played:

Scum opponents:
Three of his 8 games had Nacho as one of the two members of the scum team (0.86 rating). All other opponents were players with under 8 scum games, rolled into "other". This suggests that his win rate was to some degree boosted by playing against someone who has struggled as scum.

Town teammates:
Of his 8 games:
1 was with nacho (0.69 rating)
1 was with notscience (-0.29 rating)
1 was with malakittens (0.88 rating)
1 was with guyett (0.25 rating)
1 was with nobody special (-0.20 rating)
1 was with jasonwazza (-0.14 rating)
1 was with Xayzeck (0.52 rating)
1 was with House (-0.11 rating)
1 was with bulba (0.30 rating)
1 was with msmarangal (-0.62 rating)
1 was with singersinger (0.56 rating)
This suggests that his win rate was to some degree bossted by overall playing with teammates that have been above-average successful as town

So in this case, it's not surprising that there was a notable gap between Hopkirk's actual town win rate and the win rate suggested by the simple model of intercept + hopkirk's own rating. Of course, with a simple size of 8 there's very probably noise in the numbers too, but this is a result that overall makes sense on further review, which is what you hope to see in seemingly surprising model outputs.

Another interesting difference is fferyllt, who has a very high town rating (expected logit win rate: 59%) but not quite as high as you might expect from a 12-5 record (71%)

Spoiler: Looking into fferyllt's games in more detail

She has 17 games played:

scum opponents:
One of her 17 games had BBT as one of the two members of the scum team (0.19 rating). All other opponents were players with under 8 scum games, rolled into "other". This suggests that her win rate not especially strongly affected by opponents (or at least not opponents who have enough data to properly evaluate).

town teammates:
Of his 17 games:
3 were with goodmorning (-0.16 rating)
2 were with nacho (0.69 rating)
4 were with malakittens (0.88 rating)
1 was with thor (0.53 rating)
2 were with nobody special (-0.20 rating)
2 were with GIF (1.65 rating)
1 was with Xayzeck (0.52 rating)
1 was with House (-0.11 rating)
1 was with bulba (0.30 rating)
1 was with theirishpope (-0.09 rating)
2 were with cabd (-0.43 rating)
1 was with singersinger (0.56 rating)

As with hopkirk, this suggests that ffery's win rate was to some degree bossted by overall playing with teammates that have been above-average successful as town, which is why her rating, while still very high, is lower than you might expect just from looking at her 12-5 record.

[zoraster]

There are 90 games yet to analyze~a combined players list of at least 1 000 players.

We haven't consider Open Games.

We won't consider Theme Games.

So you aren't going to consider what in all likelyhood amounts to a majority of the games on site?

Well thanks for the clarity on that.

I disagree.

Top 25 is top 8%.

Top 8% is quite impressive.

Feel free to disagree but throwing stats that aren't meaningful isn't very helpful.

Your 25 players listed may not even be in the real Top 100 of all players. Or Top 200. or Top 500. There is no way to know given the small sample size you are using. As I said until you get at least half the player-base included in your pool of eligible players then your Top is just listing of the best ranked players in a small strata of the site's games.

I mean I understand the fun in doing it for you.

What is your deal.

[mhsmith0]

Is the win/rate distribution even a logit? I thought the win/rate distribution is a gaussian?

Well, W/L for every given game is a binary 0 or 1 variable, with pretty explicit input variables (I've chosen to handwave away the setup here, though certainly 1/2/3/A/B/C is a valid variable affecting win rates and a more fine-tuned distribution would adjust for it). A gaussian variable, which can be looked at via normal distribution, is a continuous variable (or, for a large enough sample size, can be discrete and it's "good enough").

So you can pretty reasonably run a logistic regression based on the combined data each individual game (0 or 1 for result), with whatever independent binary variables (0 or 1 for town!GIF, 0 or 1 for town!thor, 0 or 1 for scum!thor, etc) you want to account for. https://en.wikipedia.org/wiki/Logistic_regression for the curious (I'd guess you already know that link but others might be curious from reading along).

[Realeo]

The premise of logistic regression is promising.

I have an idea. Is it possible for the algorithm to merge to identical players? Apply cluster algorithm by merging similar player with higher data.

[Realeo]

Since some player are similiar, you would have bigger W/L data to process if you merge one into one?

Plot the W/L of players of both mafia and town. If the Pythaghoras distance of 2 player is very small, we can conclude that this player has similiar skill. You can merge them, add up their W/L to create one super player.