Mafia rating/ELO/reputation system?

[ironstove]

Is there any plan or desire to implement a mafia ranking system?

I think having some sort of benchmark score which starts very simple like:

+10 points for winning
-10 points for losing
-5 points for replacing out/being force-replaced for AFK
-5-15 points for being mod-killed
+2 points for being voted MVP
-1 point decay per month for no activity (So people can't get to the top of the leaderboards and make a new account in order to freeze their rank).
+1 point for full completion (Not subbing out/etc..)

And something of that nature would improve the self-accountability of players to want to put in more effort, replace out less frequently, and overall make playing mafia on MS a much better experience.

What are your guys' thoughts on that? I think it would not be too difficult to start all users at a base ELO of 1000, and do a test run just for funzies.

Maybe like once every season, we can have a Spring/Summer/Fall/Winter top scummers mafia tournament or something else of that nature? I'd love to see a championship game which could also be used as a reference for newer players on what excellent play on MS can/should look like.

[Creature]

I don't think we should make games that competitive...

[ironstove]

I don't think we should make games that competitive...

Why? I mean outside of the blatant cheating that could occur because people take it too seriously i.e. sign up multiple accounts under the same name or something.

But yea, please elaborate. I feel like this is a good idea so I'd like to understand why others might be against it so we can edge out some good discussion on the topic.

[skirt skirt]

I don't think we should make games that competitive...

It's only competitive if you treat it as so, if you choose to not care about your ranking literally nothing changes. Unless you're saying you don't normally play towards your wincon and try at least somewhat hard to win?

[RadiantCowbells]

Reserving a slot here

[Mulch]

This will have bad effects

[Realeo]

So I used to ran the ELO system here with mhsmith (before my life stop having free time) and one of the problem was sample size. It is very hard to make a conclusive statement of which one is the better player within a reasonable doubt. The ELO/mhsmith is good enough to give you a feel of where are you in the crowd from dabest spectrum into daworst spectrum (given both system pretty much agree on a lot of things), but head to head analysis is not trivial.

It was easy to make ELO system or Town of Salem in epic mafia because people are playing mafia like 30 minutes or 5 minutes a game.

Here it's like 2 months a game. Curse of small data.

However, I am in favor of a mafia tournament. I have seen BoardGameGeek make tournament with Resistance. They used "two loss and you are out"

But it was slow. Really slow.

[ironstove]

Well, assuming we keep all things the same and have these stats/rankings running in the background, I don't think it should change anything about the site. It's just a fun stat for people to look at.

[Thestatusquo]

Team game ELO where the team changes is essentially meaningless.

It becomes basically impossible to sort out the effect of team play over the sample size most players play. You could work out some sort of way to correct for team skill by taking into account elo of players on team vs players on other team, but that is way more complicated than you are suggesting you want to do, and its probably not worthwhile on the whole.

[Creature]

The problem is when players take these numbers serious. It's gonna be basically "this is unfairly unbalanced" "my team sucked" and stuff like that.

[Thestatusquo]

^^^^ that too.

[Realeo]

Well, assuming we keep all things the same and have these stats/rankings running in the background, I don't think it should change anything about the site. It's just a fun stat for people to look at.

Two responses, some people took it seriously. I

personally

received some PM about being asked to be the arbiter which one is the better.

Second, there is no point of having a ranking system if it doesn't have predictive value. We may as well roll dice to find the best player

Team game ELO where the team changes is essentially meaningless.

It becomes basically impossible to sort out the effect of team play over the sample size most players play.

You could work out some sort of way to correct for team skill by taking into account elo of players on team vs players on other team, but that is way more complicated than you are suggesting you want to do, and its probably not worthwhile on the whole.

So this is absolutely right. Low sample is a curse. You should check the thread (Project ELO is the title. Too lazy to link it.) and spectate some of the hurdle on maintaining the system. The ELO system that I maintained predicts the winner around 54% of the time (I forget the exact number) Obviously, it can be helped by using other techniques,

I personally just found out some idea

but the problem of ELO system is that it only works if the people trust the method. If the method gets too complex, people would like "Why would I trust it if I can't understand it?"

[Creature]

I could tell the story about a diferente forums that decided to add something called "Reputation" where you could either add or remove someone's reputation. It surely didn't end well.

[Psyche]

I think maybe an elo system would encourage people to play better, potentially more thoughtfully, and in that way make games more enjoyable. But it's impossible on a site like ours.

[skirt skirt]

yea, elo is kinda pointless for a sample size this small considering the luck and outside elements in mafia. you could probably rank people as mafia more accurately due to the team size, but even then theres complications due to shit towns vs good ones (rare nowadays but eh). I also don't think just eloing mafia as opposed to town is particularly important, especially because if you play 3-4 games a year (reasonable on this site if you do one at a time which many do) you'll roll mafia a whopping once, on average of course, and one game as a statistic especially when you have so many outside factors (town power, partner competence) is pretty close to completely useless.

I would love an elo system as a very competitive person, but I don't think it works for ms. feel free to try! I disagree with reaper that people need to understand it, I'm sure plenty of people don't understand their ranking in plenty of other systems thst use elo (scrabble, pokemon showdown, etc) other than generally "winning = going up, winning against better players = going up more, losing against worse players = going down more", you can have some complicated ass equation as long as the general rules of elo still apply. feel free to try and figure out a way where you can account for individual skill over team skill, but honestly I doubt it's possible with any reasonable amount of effort and cost vs effectiveness makes it likely that we will never have a good elo system here.

[Realeo]

If people would like to resume the ELO system or create a new system then by all means, go for it. We have compiled the cleaned data stored in the github somewhere in the Project ELO thread so you don't have to start from scratch. Just a reminder that it's complex and a lot of thing going behind the scene.

So the majority of the discussion in this thread is on ELO, but I would like to shift the discussion to the competition idea because it doesn't requires math. What do you think of the competition idea? I personally think it's a passable idea and deserves some air time.

---

Here is the Project ELO thread

[skirt skirt]

i think a tournament is a good idea... if you can get a 6 month min commitment from enough players. if we do a tournament id probably propose letting either a panel or players vote on the best player from the losing team to move on though, cause I'm sure lots of us have played amazing town games and still lost simply because that happens.

[Thestatusquo]

All that needs to happen is TEAM MAFIA THE REBOOT.

[davesaz]

I don't think the win/loss thing is useful.
If there were a way to -1 for spam in a game I'd go for that. I'd like to see that spam count as a public badge of shame.

Tracking replace out and completion rates would be a very good idea. I take a lot of pride in completing every game I start or replace into. If I have a RL emergency that can't be handled via V/LA then I'll consider replacing out, but that's the only reason.

[Realeo]

I'm pretty sure zoraster at one point announced that they considered giving replacement hero badge.

[ironstove]

Yea, I feel like replacing out is a mechanic that doesn't get discussed much in this thread and I wanted to see if we could create a system that incentivises good behavior/actions and discourages bad ones without any sort of meaningful impact besides a superficial one i.e. player score or mafia ranking.

I think a tournament would be another good solution which players need to be 'voted in' to join, it could possibly be something players want to work towards, but I feel like it's more selective so from the way I view it, good players will act better/ or continue to act good, whereas bad players who know they'll never be voted in will continue in their bad behavior with no real metric to see whether or not they're improving.

I'm not suggesting any sort of 'matchmaking' system to match players by their ratings. The ratings are by no means a display of skill or competence, you could say my suggestion for ELO might be misunderstood. I should not have used the word ELO. I guess 'behavior score' would be more appropriate.

One could argue that win/loss should not impact behavior score, and I can agree with that so I'd be fine with removing it alltogether and maybe just using the rating as a tool to track a MS player's consistency.

[Psyche]

you could just let people rate each other’s play after games

[Realeo]

I personally against behavior point if these behavior point are rewarded by other player. Players are not reliable scorer.

The status quo of moderator assigning punishment for excessive replacement is better.

[davesaz]

I've only seen a couple people restricted for excessive replacements.
Replacement should be for RL situations only, instead of the current I'm bored and want to join a different game standard.

[Psyche]

you’d rather bored people keep wasting space in their slots?