Mafia rating/ELO/reputation system?

[Thestatusquo]

Replacement is actively a good thing IMO. The fetishization of not replacing out is actively harmful to site meta.

It is explicitly in my rule set that people who are not enjoying or able to fully engage with the game for any reason should replace out.

[Raskolnikov]

Even if it's not truly competitive or accurate a system like this would at least be fun. Right now the only form of recognition is for really standout players in the scummies but that doesn't apply to 99% of people, especially with half the rewards being removed like the good old best town award I used to see around. But with this you can compare and banter with people casually even if neither of you are ever going to be in top echelon or goats discussion.

Although I think it'd also be interesting maybe having something measuring impact (number of games played to completion) and commendations/this player was fun to play with. Maybe throw in infamy and the option to negative rate if people could handle that too and there wouldn't be too much drama lol. Right now the closest thing to an at-a-glance of someone is someones joindate x postcount which is pretty terrible.

[davesaz]

Well, if they're not going to play then they need to be replaced of course. If it's a pattern of not wanting to play then it's an issue.

[Psyche]

Sure, but in that case we should still be focused not on the reason one is replacing but their replacement rates overall.

[Psyche]

So, there are a few really clean things we can start tracking about players to help get a sense of how much they've added to their games.

We can track...
- Replace-out rates
- Replace-in rates
- (Perhaps both above with supplementary statistics indicating how long a game thread was when the replaced in or out)
- The proportion of scum and town lynches they're on when they're town
- winrate as scum (i think winrate is much more skill-indicative for scum than town)
- proportion of games where player is mislynched

and so on

we just gotta be a bit creative, and tolerant of having many statistics to understand a player with rather than just one

The frequency of threads with this topic make me think that there's actually a lot of demand for it. I'm gonna add it to my to-do list.

[Thestatusquo]

I think the first two are not really individual game metrics but rather track service to the site in general.

Knowing that X player has replaced into 15% of their games doesn't really tell me anything about how good of a mafia player they are.

[Lang Buddha]

Replacements should get karma points and be able to make-believe buy cars with it, and everyone has to act like they're impressed when someone gets a new car. But seriously if I wasn't hauling it already I could get behind a good system that keeps track of and rewards voting for scum and victory as town and instigating mislynches and endgaming the highest amount of ppl possible as scum.

Some things it takes into consideration, and you therefore should contemplate how to do the same for your elo: not all games are sufficiently balanced, especially theme and bastard ones, so I as the rewarden only /in, replace into or host games with a balanced setup and thus make them count.
If you get nightkilled by scum they rob you of the potential to get a higher rating even though you probably played well so to compensate for this your buy-in for the game -- consisting of the number of scum -- gets cut in half for n1 rip and you only pay 2/3 for n2 rip, whereas upon getting vigkilled n1 you pay 2/3 and n2 onward doesn't change your buy-in

Once there's enough ppl to afford it I host

Ranked Games

Face off two randomly selected players by having them compete for victory in 3 simultaneous games (one's dayphase is the other two's nightphase) with the same setup and players list.
The player who is being ®anked will be scum once, while the other is town, and town two times: once while the other is scum and in an eventual tiebreaker where they're both townies, relying on bragon income to determine the better performance and thus the victor.

Every player will have to deposit a fee to participate but the amount of bragons they receive for voting scum doubles by default and quadruples for games won by their faction. However only the result of the face-off counts for the ®leage, the power ranking of which bears a purely relativist signifcance, gaining relevance exponentially with every new result.

[Gamma Emerald]

No to the ranked system Lang proposed, you can guess alignment based off this which is a no-no, an idea I was throwing around at one point had to be scrapped due to this issue.

[Lang Buddha]

No you can't. The players being ranked don't get informed of this fact until after the game bundle is over.

[Thestatusquo]

What?

You just posted it.

[Creature]

Still think nothing beats a subjective rating.

[Lang Buddha]

One randomizes the roles for all 3 games until there's a suitable pair, or maybe two. There's no reason to let a player know he is being ranked until after the game bundle concludes. If 3 games are too much 2 games will do; same principle just without the game in which they're both town. But I like a trinity better

[mhsmith0]

Well if you really wanted you could do an enforced secret alts game in that setup, but that's both fairly non-standard and somewhat breakable given that people can be somewhat obvious as to their mains when they're in alt games. More acceptable is to have something like three simultaneous games with the same setup and player list, and you judge based on that.

Something like Triple Mafia (http://www.playdiplomacy.com/forum/view ... 16&t=53610 - which, uh, didn't go very well ) but without the explicit crossover between simultaneous games.

[Lang Buddha]

No alts; they won't be able to sign up for the ranked matches since the sign up thread will be in the speakeasy, which is convenient because no alts. No bragons or eels (scum points, because slippery ergo you'd have trouble trying to lynch an eel) for alts either.

[Psyche]

I think the first two are not really individual game metrics but rather track service to the site in general.

Knowing that X player has replaced into 15% of their games doesn't really tell me anything about how good of a mafia player they are.

i agree, but i think they track more than that
the replace-out stats, especially i think, tracks whether someone's a good player in he "is it a good idea to add this person to my game" sense

either way, still worth tracking, though, right?

[Thestatusquo]

I'm pretty much against anything that would create a chilling effect on people replacing out of games when it is in their and the games best interest for them to do so.

I think tracking such a stat publicly would definitely do so.

[Psyche]

Here's what I'm thinking about a 'ranked' system.

First, I think that maybe only Normal Games should count toward the ranked system. They are, after all, the traditional "standard" game of mafia. They're pretty homogenous and (i think) are all reviewed for Normalcy and Balance. Furthermore, 1) it's easier to track statistics in these kinds of games, 2) it'll give normal games more of a distinctive purpose/verve beyond being where mods get qualified to run theme games etc.

Second, as a compromise between the view that the quality of mafia play is endlessly subjective and the view that play can be summed up with a single ELO, we can collect

many

skill-indicative statistics about players and allow comparison between players that way. The strategy here is to enable player rankings within discretized domains related to mafia - success finding and lynching scum, success avoiding mislynches, etc - without even apparently seeming to assert that any of these statistics capture all of what it means to be a good player.

What I imagine then is a table like what you see on champion.gg for LoL champions http://champion.gg/statistics/#?sortBy= ... ort=Middle where you can sort champions by their measured statistics (or just their usernames), or immediately go to a player's wiki to see all their stats in one place, and thus easily determine how successful a player has been relative to other players on the site.

That's what I think we should do, now the how. I have a votecounter mostly implemented that can help track a lot of statistics (with the help of a thoroughly organized Game Archive), though I do think we need more complete discussion of which statistics we want to track. I believe that the Archiving project me and Aristo and maybe others are about too start could include also building and maintaining the infrastructure for tracking player stats. We'll maintain the archive, extract statistics from them and threads, and finally represent them on the wiki (in a unified page on the one hand, and on many user wikis on the other).

Finally, the why - why collect and track these statistics in the first place? The idea is that these statistics might work as a discretized, broad-based counterpoint to the more holistic, superlative-oriented approach of the Scummies to recognizing user success. First, I think that they would incentivize players to play better in inarguable ways (supposing we design tracked statistics correctly). Second, they'd help us to identify players who particularly uplift or drag down games, informing a lot of decisions, like who will be in our games, whose opinions we'll listen to, et cetera. Some people will use this information poorly, but I think that's not a big deal compared to the good such a system could produce.

Incidentally, though with enough statistics that cover a diverse enough range of mafia subdomains it would be possible to summarize them all with one number so you could compare players holistically, I think we should avoid that. The summary statistic would be much less informative than any of our individual statistics, and could obscure things a lot. If anything, win rate should be enough. If we find, though, that the statistics are heavily correlated, maybe then a summary scoring of players might be worthwhile in addition to discretized stats.

[Thestatusquo]

So instead of an Elo, a statistic similar to wins over replacement like the stat heads use in baseball?

[Psyche]

I'm thinking many statistics along those lines.

[Thestatusquo]

Right but war does what you're suggesting. Takes a lot of different stats, corrects for park factors and what not, and then boils them down to one number, which is the amount of 'wins' worth of value that player has been worth when compared to a baseline player.

It's a counting stat not a rate stat, so that probably doesn't exactly port to mafia, but I think it would be a good baseline thing for you to look into. I'll grab a link when I'm off my phone and at work that explains all the things that go into it and how they boil it down to one number.

[Psyche]

I'm pretty much against anything that would create a chilling effect on people replacing out of games when it is in their and the games best interest for them to do so.

I think tracking such a stat publicly would definitely do so.

I think that maybe we can track it in a way that incentivizes things more cleanly. But this is probably beside the point of the thread and can be returned to at some later date.

[Thestatusquo]

https://www.fangraphs.com/library/misc/war/

[Psyche]

awesome. so we can use that as a framework once we have a robust enough pool of discrete statistics. it'll be the culmination of everything.

[Psyche]

So, so there are a lot of moving parts to this and I wanna be careful to do it all one at a time.

Before such a system can be produced we need to produce the following:
- A clean (eg accurate, painless to scrape through with code), consistently maintained archive of games
- A working, easily maintained base of code that can extract any of the statistics we're interested in tracking from game threads / archives
- A list of statistics we're interested in tracking (or at least a rough idea!)
- A chosen location/design to display the statistics to other players and the infrastructure to regularly place and update said stats

Manual stuff
- Choose statistics (intellectually demanding, requires discourse)
- Make/maintain archive (tedious but simple)
- Choose layout/design/location for statistics (???)

Coding stuff
- System for collecting statistics (hard part)
- System for displaying statistics (easy part)

It's so doable! But we really need to start discussing which stats we want. Can we do that here or do we need a new thread?

Good candidates for stats are...
- not too hard to collect (Can a straightforwardly coded computer program or attentive-yet-fallible archivist reliably collect this statistic?)
- clearly skill/value-indicative (Does this statistic reliably and inarguably indicate something about skill at the game / added value for one's faction?)
- stable across games/setups (more ambiguous - i just don't want situations where a player's success in tough situations isn't weighted more than success in easy situations, or on the other hand where a player's success in two situations with similar circumstances are weighted differently)

[davesaz]

I'm pretty much against anything that would create a chilling effect on people replacing out of games when it is in their and the games best interest for them to do so.

I think tracking such a stat publicly would definitely do so.

Replacing out damages games, sometimes beyond repair. If someone cares so little about mafia that they replace out 20% of the time, we're better off knowing that.