Vote Count Analysis: How Do You Do it?

[joqiza]

It is worth noting that NAI is itself kind of an interesting result and you probably want a methodology that can distinguish between things that are NAI and things for which the data is insufficient to categorize.

So it occurs to me that the way to test this kind of NAI result under your shuffling approach might be to, rather than assigning the alignments completely at random, instead assign the alignments

as if it were

towny or scummy, and then rerun the analysis, and then do that 5000 times or whatever and then see how likely it is to observe the NAI result.

I should say that I'm not sure that the shuffling is actually different from a typical approach with a beta distribution, though. It sort of seems like the same idea to me.

[joqiza]

- Correspondingly, it's probably better to frame analyses as examining how players navigate "forks in the road" throughout a Day instead of the Day's final state. We can measure players' propensities to join, leave, or stay on a wagon as a function of its size (or other wagons' size) instead of just looking at their position in a final snapshot. In this way, analyses use more data and provide more direct insight into players' decision-making and Days' overall trajectories.

Thinking bout this... might be worth extracting a "Time Before Phase Deadline" variable if possible.

[Psyche]

It is worth noting that NAI is itself kind of an interesting result and you probably want a methodology that can distinguish between things that are NAI and things for which the data is insufficient to categorize.

So it occurs to me that the way to test this kind of NAI result under your shuffling approach might be to, rather than assigning the alignments completely at random, instead assign the alignments

as if it were

towny or scummy, and then rerun the analysis, and then do that 5000 times or whatever and then see how likely it is to observe the NAI result.

cool idea!
it looks like the term for this is equivalence testing
maybe there's also a source directly describing how to do this in a nonparametric way like i think im doing to try to detect differences. i should probably already be tracking this stuff somewhere
but my guess is that it'll say something like what you described? -- that i can use the CI from an analysis applied this "equivalence" distribution and interpret complete overlap with the CI from applying the analysis to my actual data as unambiguous evidence that a behavior is NAI

[Psyche]

but what's the overarching idea or ideas behind all these patterns? that they're attention-avoiding? lack strong opinions?

I'd probably throw on a couple of other things too

1) When wagons are all on villagers, wolves are usually (though not always) less likely to be prominently driving any single wagon in particular (aggressive / powerwolfing wolves exist though)

2) When there's like two clear wagons and they're both town, and you have some number of players off of both wagons, they usually should get a lot of scrutiny - maybe it's something NAI (like a quickhammer or someone who 0 posted the day phase), maybe they're just really obvious towns in that spot (openly anti both wagons in a very prominent way), but otherwise that kind of positioning tends to be > rand wolf

3) Signs of bussing tend to be harder to pick up but are worth considered where they can show up as.

1) If the thread environment is such that wolves don't have to do anything to move the game forward in a favorable direction, then typically they find a way not to do anything - why take heat for an incorrect wagon when villagers will do it for you? (this then comes out later in game once you have more data to base things on, though sometimes role claims will give you info earlier)

2) similar idea - though here a lot of it is "ok everyone else wanted one of these folks dead, why didn't you, and why should we believe you're not full of it" - keep in mind that when a wolf is under pressure this is much less reliable (for instance, if the third wagon was a wolf but it faded away to make the top two wagons town, then wolves are likelier to have been materially involved)
also, frankly, in a spot where someone is off wagon in a v/v environment it can be REALLY easy to spot obvtowns in that spot, so "you were off wagon and you were NOT obvtown" is the more composite thing to be looking at

3) bussing signs are a different issue entirely, though i do think it can be instructive to look at overall game flow, who gained credibility from that wagon, and then how they acted subsequently - who's trusting who, who's engaging with who, is that what is "supposed" to happen if they were town, etc - this is more cross-referencing possible busses with behaviors that jar with how you'd expect a townie to act given that info, which is DIFFICULT for the most part, but from time to time you can get situations where it's relatively obvious

1) Cool. "When wagons are all on villagers" is concrete enough. But I wonder how we make concrete "[not] prominently driving any single wagon in particular " or "[finding] a way not to do anything"? In posts I guess this would look like weak (?) or infrequent advocacy for the wagon. But what about in voting patterns? Maybe more shifting? Less?

2) your description seems to cast it as a special case of 1 but w/ some useful caveats.

3) this resonates but still leaves it challenging to detect bussing yeah. there's a level of WIFOM to it that makes the whole deal challenging -- the more scum go out of their way to lim other scum, the townier they might look, making it hard to predict or read them based on an incentive analysis alone.

[joqiza]

It might be true that scum are less likely to bus players they perceive as valuable/ endgame pieces. We might be able to assign a "value" to scum players based on info like: how many votes they received throughout the game, how close they've been to elimination, for how long, etc. Then test whether scum players are less likely to vote in a way that would endanger other high value scum players. Not saying it's necessarily true but it might be testable.

[joqiza]

cool idea!
it looks like the term for this is equivalence testing
maybe there's also a source directly describing how to do this in a nonparametric way like i think im doing to try to detect differences. i should probably already be tracking this stuff somewhere
but my guess is that it'll say something like what you described? -- that i can use the CI from an analysis applied this "equivalence" distribution and interpret complete overlap with the CI from applying the analysis to my actual data as unambiguous evidence that a behavior is NAI

Yeah you should be able to calculate a CI by iterating over the different towny/scummy ratios essentially.

I guess the thing is, when you're shuffling alignments you're still basically doing a binomial experiment. Cuz it's the same rate each time. In the original example you had where you're assigning at random, it's just a 50/50 rate basically.

So I don't really see why the end result would be different from using a beta-binomial to model the uncertainty about the ratios. Except your approach is monte carlo so is going to be super computation heavy. I could totally be thinking about this wrong so definitely still worth doing as verification at the least, but I feel like it's the same approach at the end of the day.

Tbh I'm not sure exactly what nonparametric means in this context. At my old job the term mainly referred to the use of a historical distribution of an asset's log-returns versus a normal distribution in things such as VaR calculations. I'm trying to figure out what the parallel here is but I'm not quite sure.

[Psyche]

yeah probably more precise to say that my approach is "monte carlo" or "bootstrapping" than just to say it's nonparametric
i took this computation heavy approach because i figured it was a decent way to avoid thinking about the right way to frame analyses that, say, care about the identities of more than one slot at a time
i'd definitely be happy to take a less heavy approach if i could feel more confident going about it

[mhsmith0]

I think a broad point to keep in mind is that wagonomics analysis is (usually) a starting point not an ending point. Trying to brute force VCA tendencies might get you something but it's still going to be inherently noisy and isn't a substitute for doing actual analysis of what is happening inside the game.

[mhsmith0]

but what's the overarching idea or ideas behind all these patterns? that they're attention-avoiding? lack strong opinions?

I'd probably throw on a couple of other things too

1) When wagons are all on villagers, wolves are usually (though not always) less likely to be prominently driving any single wagon in particular (aggressive / powerwolfing wolves exist though)

2) When there's like two clear wagons and they're both town, and you have some number of players off of both wagons, they usually should get a lot of scrutiny - maybe it's something NAI (like a quickhammer or someone who 0 posted the day phase), maybe they're just really obvious towns in that spot (openly anti both wagons in a very prominent way), but otherwise that kind of positioning tends to be > rand wolf

3) Signs of bussing tend to be harder to pick up but are worth considered where they can show up as.

1) If the thread environment is such that wolves don't have to do anything to move the game forward in a favorable direction, then typically they find a way not to do anything - why take heat for an incorrect wagon when villagers will do it for you? (this then comes out later in game once you have more data to base things on, though sometimes role claims will give you info earlier)

2) similar idea - though here a lot of it is "ok everyone else wanted one of these folks dead, why didn't you, and why should we believe you're not full of it" - keep in mind that when a wolf is under pressure this is much less reliable (for instance, if the third wagon was a wolf but it faded away to make the top two wagons town, then wolves are likelier to have been materially involved)
also, frankly, in a spot where someone is off wagon in a v/v environment it can be REALLY easy to spot obvtowns in that spot, so "you were off wagon and you were NOT obvtown" is the more composite thing to be looking at

3) bussing signs are a different issue entirely, though i do think it can be instructive to look at overall game flow, who gained credibility from that wagon, and then how they acted subsequently - who's trusting who, who's engaging with who, is that what is "supposed" to happen if they were town, etc - this is more cross-referencing possible busses with behaviors that jar with how you'd expect a townie to act given that info, which is DIFFICULT for the most part, but from time to time you can get situations where it's relatively obvious

1) Cool. "When wagons are all on villagers" is concrete enough. But I wonder how we make concrete "[not] prominently driving any single wagon in particular " or "[finding] a way not to do anything"? In posts I guess this would look like weak (?) or infrequent advocacy for the wagon. But what about in voting patterns? Maybe more shifting? Less?

2) your description seems to cast it as a special case of 1 but w/ some useful caveats.

3) this resonates but still leaves it challenging to detect bussing yeah. there's a level of WIFOM to it that makes the whole deal challenging -- the more scum go out of their way to lim other scum, the townier they might look, making it hard to predict or read them based on an incentive analysis alone.

"early on the wagon" isn't a perfect proxy for "prominently driving the wagon" but if you're doing a pure stats type look it's probalby the closest thing you'll get.

[Psyche]

all this is probably enough detail for me. thanks very much

[gob]

VCA is rather simple

There are four-five outcomes of every wagon in standard setups

1. town mislimmed with no wolves on wagon
2. town mislimmed with wolves on wagon
3. mafia limmed with wolves on wagon
4. mafia limmed with no wolves on wagon
5. no votes (sometimes)

The goal of VCA is to try and figure out what the mafia did and why.

For instance, if a townie is mislimmed on a wagon day 1, with like 70% of the players on it, then the chances of a mafia being on the wagon is high.
FURTHERMORE (and this is the most important part), if the mafia ISNT on the wagon day 1, then the lims day 2 (along with NKA) then the off-wagon PoE is made small.

So basically, day 1 the lim happens. You make a read on which of the 4 outcomes you believe happened.
You basically have to believe and follow in that outcome as if its fact, as you can readjust later with better info. (if you dont die)

So with VCA you’re standing on one assumption looking at these four outcomes. You can stand on one assumption for ONE ingame DAY, any longer and the game can get away from you.


VCA’s biggest weakness is mafia teams who bus early. The mafia has to convert their vote-control with an extra member into town-control with credit for their towny plays.

-

This is the basic guide to VCA. These are the most likely outcomes, but only like 60% of the time. You cant assume things will go this swingly

Day 1 wagon townie mislim = at least one mafia on the wagon. The wagon that died for the mislim is more often than not, a wolf. Especially if the night kill is on the Day 1 mislim wagon.

So day 1 mislim -> day 2 lim the counter wagon that stinks the most -> wolf lim -> use outted wolf to find others.

If the day 1 is a correct lim on a wolf, in a game with 2 wolves or less, the remaining wolf is almost always either off-wagon or one of the last 3 people to hop on the scum-wagon.
In a game of 3 wolves or more, you need to prepare for bussing more. I cant really speak on this particular game scenario a lot cause I don’t understand it as well.

I think in big wolf games with early wolf-wagons, unless there was some role-action schenanigans, its almost certainly a bus. In 20p+ games, my playstyle is completely ineffective. So i am open for help here.

[Psyche]

thanks!

a couple questions

First, can you elaborate some on how you came to learn these practices for performing and reasoning about VCA?

Also,
> For instance, if a townie is mislimmed on a wagon day 1, with like 70% of the players on it, then the chances of a mafia being on the wagon is high.

Wouldn't it always be 50% + 1? Or are you counting everyone who appears on the wagon at all, even if they leave?

Overall, I'm looking for more context on how you infer whether mafia were on a town mislim or not.

[gob]

thanks!

a couple questions

First, can you elaborate some on how you came to learn these practices for performing and reasoning about VCA?

My friend explained it to me, and i put it into practice myself.

Also,
> For instance, if a townie is mislimmed on a wagon day 1, with like 70% of the players on it, then the chances of a mafia being on the wagon is high.

Wouldn't it always be 50% + 1? Or are you counting everyone who appears on the wagon at all, even if they leave?

Overall, I'm looking for more context on how you infer whether mafia were on a town mislim or not.

I'm not exactly sure what you mean by 50% + 1, but yes it does just indicate in that direction.

I'm only counting the people on the wagon when it hammers for this instance.

To determine whether or not mafia was on the wagon mislim or not. In almost all instances they are, its just how games play out on average. There are many ways to reach the conclusion (i usually just draw on my experience and what it 'feels' like. If i were to explain it and get other people on my wagon, then I would probably talk about the person's votes and how they dont come from a towny place).

If you use meta, you can also super-trap people. For example, certain players will commonly be on the end of a townie mislim wagon when they roll scum. Others will jump on a townie early and case them, and end up being the first or second on that wagon when it hammers.

VCA essentially cuts the game into pieces. After the day 1 lim, you have five outcomes. You can pretty easily infer (and if you're good and experienced, straight up know) which outcomes are impossible or not.
For example, if you have a strong feeling based on how the thread moved that there is multiple mafia off the mafia off the main wagon, you can ignore the outcomes that contradict that. So from the 5 outcomes, if you only deem 3 viable, then you solve each of those 3 "worlds" simultaneously. Or you can just solve the 'world' you find most likely (which is what I usually do cus im a lazy player).

But you want to keep these 'worlds' separate. Reads from one should not intersect with the other (until the day resolves at least), or else you'll confuse yourself since you're trying to solve the mafia team from different angles.

So VCA points you in the direction to look. Then you have to make actual reads on the players indicated by VCA.

Now there are instances, especially in smaller 9p games, where VCA does completely indicate one person as mafia. When a mafia is limmed day 1, its very common for the remaining mafia to have never voted their partner. Even doubly so if the person was scumreading the limmed-mafia without placing a vote. When the numbers get small, vca can only indicate one player. Usually the sole player off-wagon.

[JacksonVirgo]

Easy. VCA is making up shit that vaguely looks beneficial for what you want and push it with all the confidence in the world.

[KayJayQueue]

Easy. VCA is making up shit that vaguely looks beneficial for what you want and push it with all the confidence in the world.

See that’s what I thought…

[gob]

Easy. VCA is making up shit that vaguely looks beneficial for what you want and push it with all the confidence in the world.

See that’s what I thought…

It’s definitely not that.

[JacksonVirgo]

Easy. VCA is making up shit that vaguely looks beneficial for what you want and push it with all the confidence in the world.

See that’s what I thought…

It’s definitely not that.

It’s about right for what most people do, same with meta