Project ELO (Dataset released)

[Realeo]

I am sorry but what fitness is that? Can I have a name? I want to study more?

My first impression you uses entropy, but that's wrong.

[mhsmith0]

It's been a long time since I took a stats course so formal names escape me. Essentially it's a maximum likelihood calculation, maximizing the calculated likelihood (prob of outcome in trial 1 * prob in trial 2...) just using the simplification that log (p1*p2*...) = log(p1)+log(p2)..., and then just maximizing the log of the probability function instead of the probability function itself, which has the bene fit of not having to muck about with maximizing valies that are like 5x10^-100 or the like, which excel can't really process.

[Loopdan]

@loop:

I've got you as 4-3 town, 2-0 scum. My guess is three of the games are your TTTT alt?

town games:
1711
1725
1736
1739
1746
1756
1767

I was also town (as Loopdan) in 1723, 1729, and 1758.

Thanks for both of your work on this, btw. I, for one, think it is a worthwhile project and am interested in following the results.

[Realeo]

Any reason why logloss is an inferior choice? A little bit of opportunity to study here.

[mhsmith0]

@realeo: it sounds like what I did basically is logloss based on
https://www.r-bloggers.com/making-sense ... hmic-loss/
Tho I could be misreading. I freely admit my stats terminology knowledge isn't what it used to be.

[mhsmith0]

@loop:
1723 - mod misspelling (loop)
1729/1758 - weird spacing error (some mods don't seem to use regular spaces and it screws up match function)

[Realeo]

@realeo: it sounds like what I did basically is logloss based on
https://www.r-bloggers.com/making-sense ... hmic-loss/
Tho I could be misreading. I freely admit my stats terminology knowledge isn't what it used to be.

My brain!

You are right. I was confused, "Why logloss is -142."

[mhsmith0]

Adding those loopdan games you get:

Spoiler: updated table with loopdan

	Log Rating			Record
Player	Town	Scum	Combined	Town	Scum	Combined
GuyInFreezer	1.65		1.65	12-3	1-1	13-4
Thor665	0.53	-0.39	0.92	12-12	6-3	18-15
Malakittens	0.87		0.87	17-10	3-3	20-13
fferyllt	0.77		0.77	12-5	3-1	15-6
T S O	0.70		0.70	7-5	2-2	9-7
Micc	0.57		0.57	5-4	2-1	7-5
singersigner	0.56		0.56	6-5	1-0	7-5
Huntress	0.55		0.55	7-5	1-2	8-7
Xayzeck	0.48		0.48	6-4	1-4	7-8
Drixx	0.47		0.47	11-10	3-2	14-12
Loopdan	0.38		0.38	5-4	2-0	7-4
Hopkirk	0.32		0.32	5-3	2-0	7-3
Bulbazak	0.28		0.28	5-4	2-3	7-7
Guyett	0.22		0.22	7-7	2-2	9-9
Raskolnikov	0.19		0.19	4-5	1-2	5-7
RadiantCowbells	0.18		0.18	12-14	3-1	15-15
hayatoBL	0.08		0.08	5-6	3-1	8-7
TheIrishPope	0.00		0.00	4-5	1-3	5-8
Dierfire	0.00		0.00	7-10	2-2	9-12
innocentvillager	-0.05		-0.05	5-7	3-1	8-8
Wisdom		0.07	-0.07	3-4	6-4	9-8
jmo16mla	-0.09		-0.09	4-5	0-0	4-5
goodmorning	-0.17	-0.08	-0.10	14-23	7-5	21-28
JasonWazza	-0.14		-0.14	4-5	3-3	7-8
House	-0.15		-0.15	5-7	2-0	7-7
RachMarie	-0.18		-0.18	11-18	2-2	13-20
Nachomamma8	0.69	0.88	-0.19	17-12	5-12	22-24
Nobody Special	-0.20		-0.20	6-8	1-2	7-10
Accountant	-0.27		-0.27	6-11	2-1	8-12
Sakura Hana	-0.27		-0.27	5-6	3-3	8-9
PhantomCobalt	-0.29		-0.29	5-8	1-1	6-9
Not_Mafia	-0.32		-0.32	6-10	3-4	9-14
NicCage	-0.46		-0.46	2-6	0-0	2-6
notscience	-0.30	0.16	-0.46	9-16	5-4	14-20
Cabd	-0.46		-0.46	6-8	0-0	6-8
BlueBloodedToffee	-0.25	0.21	-0.47	9-17	6-5	15-22
Alexcellent	-0.50		-0.50	3-5	0-1	3-6
Ms Marangal	-0.51		-0.51	2-6	1-3	3-9
enomis	-0.52		-0.52	2-6	0-0	2-6
Titus	-0.69		-0.69	3-9	0-2	3-11
Flubbernugget	-0.75		-0.75	3-8	0-0	3-8
MarioManiac4	-0.85		-0.85	2-8	1-2	3-10
Jake from State Farm	-0.92		-0.92	2-10	1-3	3-13
theslimer3			0.00	4-4	0-0	4-4

I'm probably going to have to spend some time working on data corrections in more detail, but probably not an ASAp thing, especially since realeo seems to be working on it currently as well.

[Realeo]

Average error = -0.627.

Logloss is a positive number. I assume + 0.627?

[LicketyQuickety]

@Realeo, If you can add Open games that would be suptastic.

I just want to see how I rank when every thing is accounted for.

I prolly won't be super high on the list after the opens are done, even thought I have a 3/5 (3 out of 5) record with opens. (the last 3 have been wins so don't know if you will get to them for a while)

Also, I think the majority of my games fall under Newbie, Normal and Open (16/23)... record for those is 10/16.

[Realeo]

@LQ: Patience, my dear. Data entry is more asshole than you can imagine.

What makes Open even tougher is that unlike Newbies or Normal, nobody compiled it.

So my logloss is + 0.569, so currently ELO is better than multivariate log. regression. However, none of us did Cross Validation so overfitting is really dangerous.

So both ELO and multivariate is trained using Newbies data. We can use Normals as test data.

I will use Normals game where 1) solid-ball and 2)non-tie result as test data.

[LicketyQuickety]

@LQ: Patience, my dear. Data entry is more asshole than you can imagine.

What makes Open even tougher is that unlike Newbies or Normal, nobody compiled it.

So my logloss is + 0.569, so currently ELO is better than multivariate log. regression. However, none of us did Cross Validation so overfitting is really dangerous.

So both ELO and multivariate is trained using Newbies data. We can use Normals as test data.

I will use Normals game where 1) solid-ball and 2)non-tie result as test data.

Yikes. If no one records opens you def have your hands full, that is no joke. You got some sick determination there.

I hope you share when you get all your data needed for all Normals and Newbies.

[Realeo]

Yes, except 1 tied game and the newbie game that is just on the book.

[mhsmith0]

So my logloss is + 0.569, so currently ELO is better than multivariate log. regression. However, none of us did Cross Validation so overfitting is really dangerous.

Mildly curious what basis you're using to compare ELO to logloss, since ELO isn't a probabilistic formula. I suspect there are also material differences in how fitted your data is, since I believe you're explicitly solving for ALL players, while I'm cutting off anyone with games <8 in a given alignment. Adding in, say, all players with at least four games in a given alignment would doubtless lower my error value quite a bit, but at the cost of introducing potentially severe overfitting issues that largely can't be measured.

I also get the sense that you're combining ratings between town and scum (and sk?) for each player, and I'm skeptical that this is optimal, as there are a number of players who are MUCH better at one alignment or the other (nacho basically jumps out of the page on this basis, fwiw, but is merely the most obvious example as opposed to the only one).

Ps yes the logloss based on the given definition would be the positive of the number I gave.
Pps I may also have mire material data issues than you do, but since I don't know your feed and what corrections were made I can't really comment in any informed manner.

[Realeo]

So my logloss is + 0.569, so currently ELO is better than multivariate log. regression. However, none of us did Cross Validation so overfitting is really dangerous.

Mildly curious what basis you're using to compare ELO to logloss, since ELO isn't a probabilistic formula. I suspect there are also material differences in how fitted your data is, since I believe you're explicitly solving for ALL players, while I'm cutting off anyone with games <8 in a given alignment.

1. You can derive probability from Elo rating.

2. I did solve all, but when I use the ELO to predict the data, I cut off at >= 5 games due to the hypothesized non-converging ELO.

Adding in, say, all players with at least four games in a given alignment would doubtless lower my error value quite a bit, but at the cost of introducing potentially severe overfitting issues that largely can't be measured.

Which is why I proposed using Normals as test data.

I also get the sense that you're combining ratings between town and scum (and sk?) for each player, and I'm skeptical that this is optimal, as there are a number of players who are MUCH better at one alignment or the other (nacho basically jumps out of the page on this basis, fwiw, but is merely the most obvious example as opposed to the only one).

1. Since I do have plans for splitting it, I treat it as benchmark algorithm.

2. I trained using Newbies data only.

[Realeo]

I discovered a bug in my coding...
...and discover extra 200 possible misspelling.

[LicketyQuickety]

I discovered a bug in my coding...
...and discover extra 200 possible misspelling.

Don't you dare fucking give up this thing man.

[Realeo]

The cleaned data is now available in a dedicated repository

[LicketyQuickety]

The cleaned data is now available in a dedicated repository

What did you give to me? I mean, this power - its absolutely incredible!

[mhsmith0]

OK I've built in some of the data realeo gave, updated #s:

Spoiler: v1.001 results

	Log Rating			Record
Player	Town	Scum	Combined	Town	Scum	Combined
GuyInFreezer	1.62		1.62	12-3	1-1	13-4
Malakittens	0.84		0.84	17-10	3-3	20-13
Thor665	0.44	-0.39	0.83	12-13	6-3	18-16
fferyllt	0.76		0.76	12-5	3-1	15-6
T S O	0.70		0.70	7-5	2-2	9-7
Xayzeck	0.57		0.57	7-4	1-4	8-8
Micc	0.57		0.57	5-4	2-1	7-5
Huntress	0.57		0.57	7-5	1-3	8-8
singersigner	0.57		0.57	6-5	1-0	7-5
JaeReed	0.53		0.53	5-3	0-0	5-3
Drixx	0.49		0.49	11-10	3-2	14-12
Raskolnikov	0.42		0.42	5-5	1-2	6-7
Hopkirk	0.31		0.31	5-3	2-0	7-3
Bulbazak	0.27		0.27	5-4	2-2	7-6
Guyett	0.25		0.25	7-7	2-2	9-9
RadiantCowbells	0.13		0.13	12-15	3-1	15-16
Loopdan	0.11		0.11	5-5	2-0	7-5
theslimer3	0.08		0.08	4-4	0-0	4-4
jmo16mla	0.06		0.06	5-5	0-0	5-5
hayatoBL	0.05		0.05	5-6	3-1	8-7
TheIrishPope	0.00		0.00	4-5	1-3	5-8
innocentvillager	-0.03		-0.03	5-7	4-1	9-8
Wisdom		0.05	-0.05	3-4	6-4	9-8
copper223	-0.09		-0.09	3-5	2-0	5-5
goodmorning	-0.20	-0.08	-0.12	14-23	7-5	21-28
House	-0.15		-0.15	5-7	2-0	7-7
Nachomamma8	0.70	0.86	-0.16	17-12	5-12	22-24
JasonWazza	-0.18		-0.18	4-5	3-3	7-8
Nobody Special	-0.20		-0.20	6-8	1-2	7-10
RachMarie	-0.21		-0.21	11-18	2-2	13-20
Accountant	-0.21		-0.21	6-11	2-1	8-12
Sakura Hana	-0.22		-0.22	5-6	3-3	8-9
Dierfire	-0.23		-0.23	6-10	2-2	8-12
PhantomCobalt	-0.30		-0.30	5-8	1-1	6-9
Not_Mafia	-0.30		-0.30	6-10	3-4	9-14
BlueBloodedToffee	-0.16	0.22	-0.38	10-17	6-5	16-22
Ms Marangal	-0.41		-0.41	3-6	1-4	4-10
NicCage	-0.46		-0.46	2-6	0-0	2-6
Cabd	-0.47		-0.47	6-8	0-0	6-8
notscience	-0.30	0.17	-0.47	9-16	5-4	14-20
Alexcellent	-0.49		-0.49	3-5	0-1	3-6
enomis	-0.50		-0.50	2-6	0-0	2-6
Flubbernugget	-0.72		-0.72	3-8	0-0	3-8
Titus	-0.84		-0.84	3-11	0-2	3-13
MarioManiac4	-0.85		-0.85	2-8	1-2	3-10
Jake from State Farm	-0.90		-0.90	2-10	1-3	3-13

intercept: -0.390 (40.4% town win odds given entirely "other" players)
combined error: 247.33 (average 0.625)

Spoiler: corrections list

Added theslimer3 to log list (calculation had previously excluded him for some reason), and copper and JaeReed (now up to 8 games counted)
Corrected Newbie 1408 - Added in jmo16mla
Corrected Newbie 1504 - Added in Ms Marangal
Corrected Newbie 1531 - Added in Xayzeck
Corrected Newbie 1561 - Added in copper223
Corrected Newbie 1709 - Added in Thor665
Corrected Newbie 1717 - Added in RC, Titus, IV
Corrected Newbie 1723 - Added in Loopdan
Corrected Newbie 1734 - Added in BBT, Raskolnikov
Corrected Newbie 1741 - Added in Titus
Corrected Newbie 1750 - Added in BBT, Huntress
(and others that weren't reflected in the overall stats table in the first place)

Interesting changes:
Rask's rating went way up from adding 1734 - town!BBT had a negative modifier and everyone else was "other", so that was a meaningful positive for his rating to be associated with that win
Loop's rating went way down from adding 1723 - scum!nacho has a huge positive modifier (towns are 12-5 against nacho), so adding town!Loop to a scum win in that one really swung his rating

[Raskolnikov]

Take that loopdan!

[Loopdan]

Take that loopdan!

Clearly the methodology is flawed.

The painful part about that update is that in 1723 I replaced into a cop slot day2 that had failed to investigate N1. Then I copped a guilty on Nacho N2 but was NK'd.

I guess that's the way this works, though. I'm sure I've been carried to a win as often as I've had bad breaks.

[mhsmith0]

*remembers 1756*
*hardens heart against Loopdan*

[Loopdan]

*remembers 1756*
*hardens heart against Loopdan*

And that's not even my worst game.

[gigabyteTroubadour]

*remembers 1756*
*hardens heart against Loopdan*

And that's not even my worst game.

but for me........................... a true legend...................

((i didnt think u were that bad btw in that game just town sucked as a cohesive unit))