[Meta] Newbie Experience (Formerly IC replace rate thread)
- RadiantCowbells
-
RadiantCowbells He/himSmooth Criminal
- RadiantCowbells
He/him- Smooth Criminal
- Smooth Criminal
- Posts: 70855
- Joined: February 24, 2013
- Pronoun: He/him
- Contact:
- PenguinPower
-
PenguinPower He/Him.peng
- PenguinPower
He/Him- .peng
- .peng
- Posts: 24625
- Joined: June 15, 2016
- Pronoun: He/Him
- PenguinPower
-
PenguinPower He/Him.peng
- PenguinPower
- Toomai
-
Toomai Goon
- Toomai
- Goon
- Goon
- Posts: 734
- Joined: January 12, 2013
- Location: Ontario
- Contact:
I'll update it today then. And the linked spreadhseet is always up-to-date within a day or two.This should be required reading for...everyone for anything, really.- Zachrulez
-
Zachrulez Jack of All Trades
- Zachrulez
- Jack of All Trades
- Jack of All Trades
- Posts: 8550
- Joined: December 5, 2008
- Location: Minnesota
If you get to a point where newbies replace out like that the game has essentally served its purpose in filtering those players out. Keeping some kind of newbie integrity is silly. When I need to replace a newbie slot I don't care who takes it. Only se and ic slots need be restrictive with replacements.In post 66, Dunnstral wrote:
Disagree. Nothing turns me away from newbie games harder than having the majority of the game not be newbies at allIn post 17, RadiantCowbells wrote:I do my best to fill vacancies. For the record I think the 24 hour rule on newbie slots rule is counterproductive since it's mostly just causing newbie games to be short a slot for 24 hours more, and that's something that should change.
If I wanted to play in a game with a bunch of normal users, I'd join a micro. Which is what I do.
There should beharsherlimitations on replacements into newbies. At no point should se/ic players outnumber actual newbies- Gamma Emerald
-
Gamma Emerald AnySurvivor
- Gamma Emerald
Any- Survivor
- Survivor
- Posts: 69101
- Joined: August 9, 2016
- Pronoun: Any
- Location: Hell on Earth (aka Texas)
Do you even modIn post 104, Zachrulez wrote:
If you get to a point where newbies replace out like that the game has essentally served its purpose in filtering those players out. Keeping some kind of newbie integrity is silly. When I need to replace a newbie slot I don't care who takes it. Only se and ic slots need be restrictive with replacements.In post 66, Dunnstral wrote:
Disagree. Nothing turns me away from newbie games harder than having the majority of the game not be newbies at allIn post 17, RadiantCowbells wrote:I do my best to fill vacancies. For the record I think the 24 hour rule on newbie slots rule is counterproductive since it's mostly just causing newbie games to be short a slot for 24 hours more, and that's something that should change.
If I wanted to play in a game with a bunch of normal users, I'd join a micro. Which is what I do.
There should beharsherlimitations on replacements into newbies. At no point should se/ic players outnumber actual newbies<Embrace The Void>
“A flipped coin doesn't always land heads or tails. Sometimes it may never land at all...”- xyzzy
-
xyzzy they/themcomical third option
- xyzzy
they/them- comical third option
- comical third option
- Posts: 4970
- Joined: April 19, 2007
- Pronoun: they/them
- Location: northern VA
I think w/r/t any kind of incentive for being an IC, a good metric for deciding who gets that incentive should be the frequency of newbies staying on the site beyond their first game; if someone has a bad experience in their very first game due to a bad IC, they're unlikely to stick around, but if someone has an especially good experience in their first game due to a good IC, they're much more likely to join more games. this isn't a perfect metric, because some people will just leave the site after one game regardless of how the first one went, and some people will stick around even if they had a bad experience the first time, but I suspect that there's a correlation between games with especially good, effective ICs and games that lead to people continuing to use the site beyond their first game.- Psyche
-
Psyche he/theySurvivor
- Psyche
he/they- Survivor
- Survivor
- Posts: 10716
- Joined: April 28, 2011
- Pronoun: he/they
- chamber
-
chamber Cases are scummy
- chamber
- Cases are scummy
- Cases are scummy
- Posts: 10703
- Joined: November 20, 2005
- Zachrulez
-
Zachrulez Jack of All Trades
- Zachrulez
- Jack of All Trades
- Jack of All Trades
- Posts: 8550
- Joined: December 5, 2008
- Location: Minnesota
Whether or not I mod doesn't change the fact that it's probably even less realistic now to expect newbie slots to be replaced only by newbies than it was when I actually was modding. (There was a much more sizable pool of newbies to draw from, and still limited interest in responding to replacement ads.)In post 105, Gamma Emerald wrote:
Do you even modIn post 104, Zachrulez wrote:
If you get to a point where newbies replace out like that the game has essentally served its purpose in filtering those players out. Keeping some kind of newbie integrity is silly. When I need to replace a newbie slot I don't care who takes it. Only se and ic slots need be restrictive with replacements.In post 66, Dunnstral wrote:
Disagree. Nothing turns me away from newbie games harder than having the majority of the game not be newbies at allIn post 17, RadiantCowbells wrote:I do my best to fill vacancies. For the record I think the 24 hour rule on newbie slots rule is counterproductive since it's mostly just causing newbie games to be short a slot for 24 hours more, and that's something that should change.
If I wanted to play in a game with a bunch of normal users, I'd join a micro. Which is what I do.
There should beharsherlimitations on replacements into newbies. At no point should se/ic players outnumber actual newbies- Ircher
-
Ircher He / Him / HisWhat A Grand Idea
- Ircher
He / Him / His- What A Grand Idea
- What A Grand Idea
- Posts: 15190
- Joined: November 9, 2015
- Pronoun: He / Him / His
- Location: CST/CDT
- fferyllt
-
fferyllt Titan of Trajectory
- fferyllt
- Titan of Trajectory
- Titan of Trajectory
- Posts: 20499
- Joined: December 28, 2012
- Location: Left Coast
the sign up threads are still out there. it's easy enough to sample how many new players /inned, how many games launched per month. Might be interesting to pick 2 or three months and compare them across 3 or 4 years.- Sunlit Diamond
-
Sunlit Diamond Mafia Scum
- Sunlit Diamond
- Mafia Scum
- Mafia Scum
- Posts: 1247
- Joined: June 20, 2017
This is a thing I would enjoy doing. *Contemplates*In post 111, fferyllt wrote:the sign up threads are still out there. it's easy enough to sample how many new players /inned, how many games launched per month. Might be interesting to pick 2 or three months and compare them across 3 or 4 years.- fferyllt
-
fferyllt Titan of Trajectory
- fferyllt
- Titan of Trajectory
- Titan of Trajectory
- Posts: 20499
- Joined: December 28, 2012
- Location: Left Coast
Yeah, I'm thinking about doing it, too. I'd like some input about what would make a decent sample size. And also if there's an easier and more automated way to go about it.
I'd also be interested in doing something to track how many people keep playing after their first game, and what percentage are still playing a year later.
And do that over the same time windows maybe.
It would be a lot of manual labor.
Maybe psyche could make a bot!- Gamma Emerald
-
Gamma Emerald AnySurvivor
- Gamma Emerald
Any- Survivor
- Survivor
- Posts: 69101
- Joined: August 9, 2016
- Pronoun: Any
- Location: Hell on Earth (aka Texas)
- PenguinPower
-
PenguinPower He/Him.peng
- PenguinPower
He/Him- .peng
- .peng
- Posts: 24625
- Joined: June 15, 2016
- Pronoun: He/Him
This is something I think would be beneficial both from a stats pov, and an engagement pov. Meaning, I don't think that this info should just be pulled from historical data, but ongoing and through direct consultation with the players.In post 113, fferyllt wrote:I'd also be interested in doing something to track how many people keep playing after their first game, and what percentage are still playing a year later.<(") | (")>- fferyllt
-
fferyllt Titan of Trajectory
- fferyllt
- Titan of Trajectory
- Titan of Trajectory
- Posts: 20499
- Joined: December 28, 2012
- Location: Left Coast
Yeah, for sure. I'm thinking a baseline trend from a few years past would help put what's happening now into perspective.- Sunlit Diamond
-
Sunlit Diamond Mafia Scum
- Sunlit Diamond
- PenguinPower
-
PenguinPower He/Him.peng
- PenguinPower
He/Him- .peng
- .peng
- Posts: 24625
- Joined: June 15, 2016
- Pronoun: He/Him
- Sunlit Diamond
-
Sunlit Diamond Mafia Scum
- Sunlit Diamond
- Mafia Scum
- Mafia Scum
- Posts: 1247
- Joined: June 20, 2017
*salutes* I suspect I'll be here a lot. I've been digesting the Mastin Academy thread for the last week or so.
Let me see how automated I can get this, and get a spreadsheet going.
Edit:
My perfectionist mind wants to build a stats sheet for all of the last three years. My realistic mind just reminded me that I have a full time job, a three year old, I'm going to school, and I'm playing mafia. While I could do it, it would take foreeeever. What I can do is dump what I gathered tonight into a Google Sheet so interested parties can collaborate, if interested.
Two random facts I gleaned:
1. Number of Newbie games per month steadily climbed from 2013 to 2015; after that, they began to decline. The difference between 2016 and 2017 from January to June is 20%.
2. We have exactly one player remaining from the first Newbie game of 2013.
And now I toddle off to bed.- fferyllt
-
fferyllt Titan of Trajectory
- fferyllt
- Titan of Trajectory
- Titan of Trajectory
- Posts: 20499
- Joined: December 28, 2012
- Location: Left Coast
This is awesome info!
I'm interested in seeing what you've gathered so far, and I am planning to work on this a bit, too. Since you were able to nail the trend down objectively, I'll start on a detailed look at 2017 games.- fferyllt
-
fferyllt Titan of Trajectory
- fferyllt
- Titan of Trajectory
- Titan of Trajectory
- Posts: 20499
- Joined: December 28, 2012
- Location: Left Coast
This is a high level summary of the data I've captured about finished 2017 Newbie games.
These games were all announced by the Listmods in the Newbie Queue in those months. There was one game that started in early January, but because it was announced in December, I arbitrarily chose not to put it in the stats. It would be easy enough to add it, if tracking the games by when they actually started is a better approach.
I decided to track replacement data not only because a lot of first-time newbies start their first games as replacements, but because I think unusually large numbers of replacements probably indicate there was something anomalous about the game that may have turned off players from finishing the game and potentially playing additional games here.
As ongoing games finish, I'll update the spreadsheet. I don't plan to make the detailed spreadsheet generally available since I have to capture activity data on individual players in order to calculate aggregates. I feel like there's probably a window after which a first-time newbie who played their first game should be considered an established member and no longer tracked in terms of whether they're still visiting the site, playing games, or whatever.
Thoughts about additional information:
Instead of tracking whether the player is still active "today", track if they're active 1 month, 2 months, 6 months after their first game finishes.
Track how many games they play in the following 1 month, 2 months, 6 months after their first game finishes
There was no way to capture most of this information without doing a bunch of brute-force effort looking at newbie listmod posts, reading games, etc.
At the level of granularity I've captured the info, it would eventually be possible to see if particular mods, ICs, SEs are involved in games with high conversions of first time newbies to established members. It would also be possible to see if certain game "cohorts" go on to play more games together and stick around longer. And I hope it will be possible to tease out some best practices: identify established mods and players whose games have an unusually high conversion rate, and then look at what makes those games different and presumably better for newbies. It may also be possible to see what kinds of second, third, fourth game experiences (which queues, which types of games, etc) correlate with continuing to play on-site months later.
If you have thoughts about summary data that would be useful to track, let me know.
I haven't decided whether to track prior years in this detail. It's a lot of work digging through the data manually.
One correlation of note is that May, the month with the most Newbie slot replacements so far this year (though not by much) was also notable in that 4 players who were in one or more newbie games at the time were sitebanned.
The May numbers were inflated by one game that had 13 replacements, most of which were newbie slots.- Sunlit Diamond
-
Sunlit Diamond Mafia Scum
- Sunlit Diamond
- Mafia Scum
- Mafia Scum
- Posts: 1247
- Joined: June 20, 2017
One thing I have noticed is that newbie games appear to require a lot more brain power than other games. There's definitely a culture shock shifting from the road to other games and that might contribute to retention rate or lack thereof (for both newbies and ICs, really). I dunno if that's helpful, but ... you never know?- fferyllt
-
fferyllt Titan of Trajectory
- fferyllt
- Titan of Trajectory
- Titan of Trajectory
- Posts: 20499
- Joined: December 28, 2012
- Location: Left Coast
There may be something to that, though I think that the need for brainpower varies a lot from game to game even in the same queues. The game design and the player base in each game has an impact.
Some of the most challenging games I've played at MS happened in the Newbie queue. And some of the easiest, quickest wins happened there, too.- mastina
-
mastina She/HerFalse Prophet
- mastina
She/Her- False Prophet
- False Prophet
- Posts: 16670
- Joined: October 7, 2016
- Pronoun: She/Her
- Location: Between Snohomish and Monroe, WA
- Contact:
I still maintain my theory that running the same setup--even a semiopen one--with no alternatives may contribute to this because literally every single queue outside the newbie queue is going to have different game setups. Even the queues "closest" to newbies. (Opens for setup, Micros for size, Normals for type.)In post 122, Sunlit Diamond wrote:One thing I have noticed is that newbie games appear to require a lot more brain power than other games. There's definitely a culture shock shifting from the road to other games and that might contribute to retention rate or lack thereof (for both newbies and ICs, really). I dunno if that's helpful, but ... you never know?
They have variety of type and content within which newbies don't get exposure to. - mastina
Copyright © MafiaScum. All rights reserved.
- fferyllt
- Sunlit Diamond
- fferyllt
- fferyllt
- Sunlit Diamond
- PenguinPower
- fferyllt
- PenguinPower
- Gamma Emerald
- fferyllt
- Sunlit Diamond
- fferyllt
- Ircher
- Zachrulez
- chamber
- Psyche
- xyzzy
- Gamma Emerald
- Zachrulez
- Toomai
- PenguinPower
- RadiantCowbells