Vote Count Analysis: How Do You Do it?

[Psyche]

Goal of this thread is to establish a concrete grasp of what VCA is on mafiascum.net.

I wonder if people who find this thread could, as precisely as they can while acknowledging the ambiguity of it all, walk me through the nuts and bolts of:

1) How they go through the process of performing vote count analysis
2) What kinds of assumptions and information they use to ground this analysis
3) What they think can and cannot be learned from VCA

Also:

People seem to talk about VCA as if it's some standardized process that can solve games under a lot of conditions even if people's reads are otherwise null or wrong.

How true is this?
How consistent is the process behind VCA from person to person?
And to what extent is it distinct from and can supplant the regular approach of just reading players' posts and looking for behavior that seems disingenuous?

[Klick]

Love this thread
I have nothing to contribute because I think most VCA removed from its context is super sketch
But I'll drop an ego out of interest

[Ythan]

[DragonEater70]

So 2 things:
1. I personally have no clue how to do VCA but at least I'm honest about it. When I do "VCA" I am basically just trying out a fun idea and definitely have no idea what the actual methodology behind it is. Which is why my "VCA" fails. And also why this thread could actually be useful.
2. I do think that when done properly, VCA can strongly support arguments for why someone is town and someone else is scum. For example in Mini Normal 2322, Flavor Leaf (who's known as being pretty good with VCA I think) was able to identify that Pavowski couldn't be scum and Hu Tao had to be scum based on the fact that the wagons were tied, there were 3 confirmed town (out of the required 5 votes for elimination) on Hu Tao, and she still wasn't eliminated. So while I am not an expert on VCA I can tell you it's pretty useful.

As a sidenote:
While looking at VCs and who was town and on which wagon is not something I usually manage to get alignment indicative information out of reliably, I want to address your last question of "to what extent is it distinct from and can supplant the regular approach of just reading players' posts and looking for behavior that seems disingenuous?", and say that while I don't think VCA (which I define as looking at the actual VCs in relation to the knowledge of alignments of flipped players) can supplant the regular approach, I DO think that looking at a person's voting pattern as well as the pattern of votes in relation to them (who votes them, who unvotes them, etc.), CAN supplant the regular approach of looking for disingenuous posts and often works BETTER than that approach. Like often you can catch scum just by noting the timing of someone's vote, or the lack of a vote where a vote is to be expected, where focusing on the actual content of the post wouldn't allow you to do that. To bring up again the example of Mini Normal 2322, in that game I noted that Hu Tao unvoted HPE with a bad reason, and with HPE having flipped red that indicated Hu Tao could be scum. Had I listened only to this and didn't also consider the tone, progression, etc., of Hu Tao's posts, I would've probably been able to catch her (which I ended up not doing because I was very pocketed).

[joqiza]

Perhaps not quite VCA, but about a year ago I ran an analysis on a particular voting pattern (unvoting without revoting). I didn't end up drawing any definitive conclusions, but I published the dataset and a discussion around it can be found here: https://www.mafiauniverse.com/forums/th ... tistics%29

[Psyche]

i have a lot of voting data scraped from this site that i've had on my todo list to analyze for a while if you ever want to push further

[joqiza]

Sure, I'd be happy to help advise if you're looking to run any kind of analysis. Honestly, I think the data collection and sanitization is the tougher part. For the analysis I linked, I picked only mountainous games to keep it simple. Once you start introducing different types of mechanical information, I think a lot more thought than I was willing to do needs to go into how you are going to standardize data across different types of games.

My general approach in the analysis I ran was pretty straightforward. For the "sonny tell" (unvoting without revoting >1 hr) I defined the experiment as "exhibits the tell at least once on a given day." So for example in Game 1 on Day 1, 3 town players exhibited the tell of 12 overall, so I record this as 3 town successes and 9 town failures. For the wolves on Game 1 Day 1, 1 wolf player exhibited the tell of 3 overall, so I recorded 1 wolf success and 2 wolf failures. Etc. for all game days for all games in the dataset.

I found that the simple tell without any criteria was not obviously indicative in either way. A player named soah proposed a list of criteria which, when applied, led to greater separation between the estimated wolf and town parameters. At that point we were potentially data snooping, though.

It was an interesting exercise and I'd be happy to talk to anyone more about it if they have data they want to test. That said, I suspect this is all probably more of interest to people who are into math/stats than really useful to anyone seeking some kind of competitive advantage in games.

[Psyche]

Yeahh I've looked at some things in a similar way. I used to be very attached to the program of just testing people's different ideas about what scumtells are but there are a few issues with this.

The main issue is that most scumtells are infinitely moving goal-posts, always more context-sensitive than how people initially specify them. When you dig into them, they turn out to only *really* be scumtells when players do the described behaviors "in a scummy way", which pushes the question of what makes a behavior scummy one step further away. This makes "testing" a scumread with data really challenging. And people trying to apply such tells in an actual game either still wrestle with the basic challenge of sorting whether what they're seeing is *really* a scumtell, or stop thinking beyond pattern detection. Not super useful either way. This feature of scumtells means they're primarily useful as a way of focusing attention -- either when trying to read someone or when trying to push a wagon. But they don't necessarily provide much sorting information beyond that.

Have come to think that we ultimately want to discover shared principles across (real) tells that explain

why

they are tells and can be flexibly applied across scenarios to sort players and justify reads. These principles could conceivably even imply more specific scumtells that have thus far been overlooked all their own. They might even tell us something about how people do social deduction and deception in general. This focus might at least help decide which among the massive pile of supposed scumtells to spend time testing...

[Psyche]

VCA seems like a good place to try finding some principles for two reasons.
First, votes are kind of concrete and easier to extract as a researcher than the stuff people say in their posts.
Second, the baseline theory of how having the scum faction's knowledge and win condition might shape how they vote is really clear, at least on some level:

Scum's most straightforward incentive during gameplay is to see that town are eliminated instead of their teammates. Even one teammate's death substantially reduces their chances of winning, all else equal. If scum could vote anonymously, they would always group all their votes on whichever town slot is closest to being eliminated. But their votes aren't anonymous. So they are also strongly incentived to vote in a way that is town-like. These incentives have to be balanced throughout the game, and run into the greatest conflict when the town-like thing to do is to vote out their teammates.

If right, this suggests that scumtells can be roughly grouped into three classes:
- A flawed idea of town-like behavior. They follow their notion of what town-like play looks like, but this notion is wrong enough that their play isn't town-like.
- Poor execution of town-like behavior. They have a great idea of what town-like play looks like, but they struggle to actually implement it for some reason (e.g., skill, difficulty, resource constraints).
- Scum motivated behavior. They have a great idea of what town-like play looks like and are perfectly capable of mimicking it, but diverge because they've judged it makes them more likely to win.

The latter seems easier to study at first because it doesn't necessarily require a lot of psychologizing to get started. Instead, we just have to examine incentives. For example: Even though scum are incentized to mimic town, the motivation to see town eliminated and their teammates survive is also strong. In aggregate, a decent baseline prediction from all this might that scum-on-scum votes are relatively rare, at least at certain junctures in the game where the votes would actually be impactful.

In the above example, the overarching principle is that scum will break 'character' when it suits their win conditions, and the more specific tell to look out for is hesitance to vote their buddies. This might show up in VCAs as absent, late, and/or shaky commitment to wagons on slots that are only now known by town to be scum. Of course town can also be hesitant to vote a player for any number of reasons, and scum can bus to avoid suspicion from or even to be positively townread by people sensitive to these dynamics. But maybe the incentive built into their win conditions to keep teammates alive overcomes these forces enough that there's at least one consistent pattern in how scum vote.

Finding these in voting data would improve our confidence in this overarching approach to finding scum, while gaps in evidence for these patterns could motivate us to add caveats to the approach or seek out alternatives. Having these higher principles makes it easier to reason about what the downstream tells would like like in odd scenarios, or when they wouldn't apply, or when they would be especially telling. And the reliability of these patterns can help weight how seriously we apply these principles in our reads. All this specifies some kind of research program for building an evidence-based approach to scumhunting and an account of how people play social deduction games in general, even if the initial predictions about what we'll find are wrong.

[Psyche]

This post has a lot of concrete ideas about vca that i overlooked but look pretty standard. will try to extract relevant stuff eventually (prob using chatgpt to some extent)

IC POST #2

**note: the following post is PRE WRITTEN, BEFORE I GOT MY ROLE PM. It has ZERO bearing on my alignment**

Since this is a newbie game, I figured I'd share some resources that I myself have found VERY useful (shamelessly copied/paraphrased from other sources, with a few of my own thoughts and examples mixed in as well). I'll be hiding them under spoiler tags, so feel free to ignore if you aren't really interested. This is 100% OPTIONAL, but I found it extraordinarily useful in improving my game. You may or may not.

Spoiler: Voting Analysis

Vote analysis is…vital
Nothing written in thread is as crucial as the names put in red and the reasons offered to justify those lynch votes. They are the best guide to player motive and thus to player alignment. And so it is the core of your scumhunting.

Vote analysis is huge in scope...
...because it encompasses everything from the micro-level of analysing individual votes to the macro-level of analysing voting patterns within and across multiple game days.

Spoiler: Micro vote analysis

Understanding how a scum’s motives and a townie’s motives differentiate their respective voting behaviours

So scum vote for townies right? Well, yeah…often enough, but that doesn’t get you very far. Firstly scum vote for other scum far more often than might be initially assumed, but your biggest problem is that townies vote for other townies…a lot!
So how does a townie tell the difference?
Well if each townie is bringing their best game then they should be intellectually invested in their lynch vote. They want that player killed. Consequently, they want to influence and persuade their fellow townies and drive their preferred lynch. In short, it matters to a townie who specifically is lynched that game day.
A townie might initially throw a vote or two around to see what happens, but essentially a townie lynch vote ought to be righteous

in the moment of being cast

.
Simultaneously a townie will be continuously reviewing their reads and triaging their suspects (everyone else in the game). They are genuinely searching for the best vote. That means if they change their vote…that process is righteous too.
…
Well preferably they [scum] want a townie lynched, but more than that

they want to blend in

. Ideally they also want someone else set up to take most of the heat for mislynches.
…
So…scum voting is therefore a part of their elongated active lurk – look busy, but don’t help. They want to surf along on the edge of your consciousness and low in your triage of suspects. Meanwhile townies throw suspicion at each other over misconstrued nothings.
…
So whether on or off the current leading mislynch, a scum in this mode is relatively passive and their lynch vote rationales underwhelming.
Also fake scumhunting is hard! It’s difficult to both contrive some reason to vote someone and then plausibly ‘sell' that contrivance. It is much easier to window-dress plagiarised arguments or just try and slip a token 'scumhunt' into thread without fanfare.
…
If a scum does try some fake scumhunting it will lack righteousness. It will smell feeble. You will struggle to see how or why this vapour is vote-worthy relative to alternative issues present in the thread. Knowing this some scum players might try and fake passion for their vote to disguise the deficit in intellectual integrity underpinning it. Then you are looking for a disproportionate level of zeal and/or confidence. A scum in this mode is not necessarily harder to spot, but can be harder to lynch since they can impress weaker-minded townies. They can shamelessly segue from one terrible vote underpinned by garbage to another and not miss a beat. Scum also tend to gravitate to this mode the closer they are to the win.

In summary

: If all is well for them, scum are just trying to find a quiet spot for their vote where they hope to avoid any scrutiny for their choice and its rationale and/or where they retain the maximum level of freedom to redeploy it if that suits their aims.
Either way,

it is a truism that a scum-advantageous town environment is one where it is an accepted norm for most lynch-votes to be cast for reasons other than the quality and history of the lynch target’s own voting choices

.
For example, scum might vote on people for alleged anti-town behaviours that usually amount to their (townie) lynch target saying/doing something daft about roles or mechanics. Or it might be some alleged incongruity woven from the target tripping over themselves in their mis-communication of something or other. Maybe even some misspeak that gets twisted into an alleged scumslip. All of this is normally absolute garbage! The reality is that most clumsy in-thread behaviour and expression labelled as anti-town is (regrettably) town-indicative. Typically, scum are more calculating and careful in what they say and how they say it.
…
The point is you want to parse out the difference between the player who really believes this guff is vote-worthy and the player who can hardly believe his luck that townies seem to believe this guff is vote-worthy.

Spoiler: Macro vote analysis

By definition, this is analysis in hindsight. It is looking into the past with fresh eyes based on the new information from death alignment reveal. Also by definition the further into the past you look the better the information you have because you have more death reveals on players active that game day.
This is why d1 in particular is gold-dust to any town. If you cannot spot all the pro-scum vote choice shenanigans I have listed above while d1 is happening you can

go back and re-read it

on subsequent days and spot those shenanigans in retrospect!
This approach is useful at any time, but really comes into its own on d3 and beyond.
Incidentally this is why townies should work really hard to get the maximum amount of data from the maximum number of players into thread on d1. As they play d1, they should be thinking about d3 when they and their team-mates will re-read d1 (and d2) both chronologically and after filtering by author to get an intensive re-read on their top suspects’ vote choices and behaviour on those days.
Only the very best mafia think of d1 in the same way whilst they play d1. More likely they have sort of drifted through d1 taking the high-odds easy mislynch with a minimum of effort. They often get away with that laziness during d1, but they can be exposed for it on d3 by a vigilant town!

So what types of Vote Analysis are there?

There are two broad types:
(i) Analysis within game days; and
(ii) Analysis between game days

(i) Chronological vote analysis within game days

What I will term type (i) vote analysis looks into the chronology of how votes came down and the fluctuations of different lynch trains rising and falling within a given game day. … Then colour it (red & blue) with known death reveals (and yourself in blue too is my habit
[

SMITH NOTE: I recommend that you do NOT follow TWC’s habit here; avoid coloring yourself in unless you’ve actually been confirmed as town mechanically, since this will ensure that the work you do is useful for other townies, as if they have to bother uncoloring you for anything they do with your analysis it’s a needless hassle

]
). Next sub-divide it into key phases. Finally try and spot ‘events’ or transitions of potential significance.

Catching scum with type (i) analysis

You are looking for any signs of scum tactics – for example, trying to save a scumbuddy under a lynch threat. This is rarer than you might imagine even amongst relatively unsophisticated 3-man scum teams, but more likely when a team is down to two.

Spotting fellow townies with type (i) analysis

However type (i) vote analysis can also help you pick up on town-indicative conduct. I will give an example hypothetically at first. Let’s say the clear mid-point d1 lynch vote leader, Player Z, is now known (through subsequent death reveal) to be a townie. Later in d1, Player X drives hard an alternative lynch on Player Y who is lynched that day and flips town.

Odds are that Player X is likely town too. Why?

Because what scum motive would Player X have to so prominently drive a mislynch on townie Y and get his hands dirty when sitting tight and doing nothing would have provided him with a townie death anyway?
…

Be careful - narratives like this are bewitching. Nothing about Player X is definitive based on this alone. You must re-read at the micro level too and cross-verify for righteousness.

(ii) Vote analysis between game days

What I will term type (ii) vote analysis looks at lynch counts at ‘snap-shot’ moments in time and works on presumptions about how scum will want to place their votes in relation to one another. Overwhelmingly, these snap-shots are the final lynch counts of each game day. After all, it is the votes at game day end which really count. It can be useful to take other snap shots at game day mid-points, but for now I will just focus on end of game day.
Place these end of game day snapshots side-by-side, with known alignments in red and blue and look for signs of scum ‘parking’ their votes tactically. Places that – at the time - seemed nice quiet spots for scum to hide out but which in retrospect stick out when you layer in alignments from death reveal.
Health Warning!
Note my use of the word presumptions. Not absolutes. Be careful - narratives woven from these presumptions can be bewitching. You always also have to synthesise your macro analysis with micro analysis of your suspect’s vote choices and reasoning.

Spoiler: D1 scumhunting

...Seriously, I want you all to write down your personal list of behaviours in-thread that on d1 you consider scum-indicative and which town-indicative
...
Once you have your lists, now I want you to try a new approach to the game. You are going to catch a mafia d1! Instead of believing this myth that d1 is always a crap-shoot and mafia will likely get a townie lynched that day believe something akin to the opposite.

I believe that d1 is the mafia's hardest day. They will leave a legacy of clues on this day specifically that will help you throughout the game.

If things are going badly for a town as they move into d3, d4, d5 there is more and more 'sludge' in thread - townies posting garbage, getting into fights, confirmation bias written large etc etc. The mafia have all sorts of camoflage to mask their insincerity. So you go back to d1 and you read it again through the lens of all that has happened since...

Oh and that's the clue - you are looking for

insincerity

most especially in why they vote for who they vote on. (They may be insincerely voting on a buddy) How do you spot insincerity - their reasons make no sense or their move from one person to a new target 'jars' you cannot see the progression or evolution in their thinking that is characteristic of a clueless townie or their vote is lazy in its level of scrutiny when you know this person as town is diligent and even sometimes insightful.

So mid-d1 I want you to sort by author and re-read each person's opus. Pick your top 2. Vote for one of them and drive that lynch. See what flows from that. Something might happen that will change your mind, but mostly you will stick with it.

Then every day subsequently most especially when you feel lost and frustrated that you haven't a f*cking clue who the mafia are they are so well hidden. Do the same again and re-find the scum. Things that would not have led you to lynch someone on d1 will now stand out as a clue based on what has happened d2 or d3 etc. Always do vote pattern analysis every day
…
Either way, d1 is 'pure'. Insincerity has a sharp relief. Mafia don't have as much townie generated 'sludge' to hide behind. Fake scumhunting is hard. bk as scum team lead was doing a good job, but even then ultimately his solution to getting through d1 was to vote on a buddy. I don't blame him. It's one of my favourites too. It has so much long-term value and a good mafia knows that d1 is tough because he is thinking deeply about what it is to be a mafia.
…
So catching mafia based on d1 is easy? Well of course not. You have a problem. largely useless townies who don't know how to scumhunt drifting into inactivity or just jumping on a bandwagon since they are so unconfident in their own ability to scumhunt or relying on some lame linear reason to vote for someone or its opposite - something woven out of too much complexity as if mafia were masterminds

Spoiler: Theories about how scum place their votes

The following scum-tendencies are decently reliable since they are based on experiential data and because they are a natural consequence of the micro-level behaviours articulated in part I – that scum are placing their votes tactically and trying as much as possible to fly under the radar.

They provide a way of framing your micro-level vote analysis, but remain only narrative without the it.

So…

a) On the whole, scum teams like to spread out their votes early on in the game because they are trying not to associate themselves with one another.

b) It is very rare for a scum team to have all their final votes on the same player early in the game. This is especially true on d1 and pretty reliable on d2 as well. An exception would be if the whole town (or close to it) is voting on the lynchee.

c) This latter point hints at a variable – how many votes did the lynch ‘winner’ have? The more votes, the more likely it will be that the scum will be willing to accept some bunching. For example 2 out of 6 votes is ok, but not 2 out of 4 votes. That kind of thinking.

d) Less reliably, this potentially implies something about the order in which those votes go on. Votes #1 and #2 are less likely to both be mafia since they cannot know whether there will even be a vote #3. Whereas one amongst votes 1-3 and another at 5 or 6 looks more likely.

e) Another variable that impacts here though is the size of the scum team itself. Point b) is very good guidance against a standard 3-man team. Against a 2-man team it is harder to say since we are only talking 2-0 or 1-1 as options for deployment rather than options of 3-0, 2-1 and 1-1-1. In the latter case 3-0 is very unlikely, but in the former 2-0 is far more plausible.

f) As a related point, you can say that if a scum is lynched relatively early in the game then it is pretty likely that at least one of his team mates was on that lynch for the credibility boost and for the tactical option (not taken) of jumping off if to help save his buddy if that play seemed viable. Again this varies depending upon how many votes were needed to lynch the scum. If it was only 4 votes to 3 then may be all 4 voters are town, but even then I’d say better than 50% odds that one colleague was bussing. Then again this is also better guidance against a standard 3-man scum team. A 2-man team has far more to lose from bussing but also more credibility to gain. Ergo the deciding factor is what kind of scum player are we dealing with here…

g) If a scum is being lynched in a near landslide then his buddy or buddies are very likely to be on the lynch and perhaps prominently so…

h) Throw-away votes that had only an indirect impact on the final lynch outcomes are more likely to be scum votes. In final votes counts on d1 and d2 of 6-3-1 or 5-3-1-1 or similar …I am interested in looking for scum amongst those 1s. Those solo votes were effectively voting on the lynch leader but possibly trying to keep out of the way while it happened. If a townie cannot get their first-choice lynch but have a chance at their second or even third-choice over someone they don’t want lynched at all then they need to act - not sit on the sidelines. This scum tendency can be another by-product of point a) – the desire to avoid bunching…Ergo by the same principle only one of those throw-away solo votes on a given game day is likely to be scum. It is a lot less likely that both the 1s in a 5-3-1-1 are scum.

i) Related to point h)¬ ‘parked’ votes that were tucked away in a reasonably sized pack of votes at one point but have then been left high-and-dry when the tide on that candidate went out…are more likely to be a scum left clinging to a stale vote backed by stale reasons…

[joqiza]

My own thoughts have gone down similar avenues in the past and I wish you luck. I'll check out your scraper when I get home. I'd built something to grab MU vote histories in the past but that's more structured I think. An MS dataset would be really interesting.

A word of caution--there's a rule on this site prohibiting players from creating machine learning networks to classify posts. I don't think that's what you're proposing, but the mention of ChatGPT may make some folks a bit nervous.

[joqiza]

Oh, this scraper is pretty cool. Nice work. Do you already have a script which constructs a votecount dataset from the json? I could give a shot at that if you'd like.

I imagine one approach might be to create a dataset of votes with variables for the following:
- Day Phase
- Postnum
- Time
- Voter
- Voter Alignment
- Voted
- Voted Alignment
- Vote Type (vote or unvote)
- Vote Count
- Votes Needed for Elimination
- # Town Alive in Day Phase
- # Wolves Alive in Day Phase
- # Town on Wagon
- # Wolves on Wagon
- Alignment of Prior Vote
- Count of Prior Vote
- Alignment of 2nd Highest Wagon
- Vote Count of 2nd Highest Wagon
- # Town on 2nd Highest Wagon
- # Wolves on 2nd Highest Wagon
... (etc for 3rd, 4th ...)
- # People Not Voting

[joqiza]

If you really want to develop a theory, you'll probably need to find some way to capture things like "oh, this guy just got red-checked so everyone moved on to him." Not sure how to do that in a programmatic way. I just stuck with mountainous games in the past which made things easy but limited my data.

[Psyche]

Spoiler: fear im moving a little offtopic so i'm spoilering

afaik the idea is that it's cheating to use such models in a game, and i'd never. am more psych guy than ml guy anyway -- goal is human understanding, not machine prediction

my stuff pulling out vote histories is kind of out of date at the moment and is also pretty messily written up
is on my todo list to rewrite more professionally (including tests) and also provide a utility for general use
once i do that i'll probably rethink my analysis plan. my previous approach was pretty messy and i think also underpowered

somewhere before or after that i'll revisit data collection; there are many more games i could include than before
and it seems very likely that an (open source) llm could make it much easier to automate some data cleaning tasks that were previously done manually
have always wanted to include games from other sites in the dataset so that site meta can be considered as a variable, but that is maybe a little downstream for now

have some longer term plans beyond all of this, but first it will all also require an EV simulator advanced enough to simulate a slot's knowledge and gameplay policies
for general use this would be a nice tool for assessing setup balance
and in a research context it would make it easier to shift into model-based research where instead of testing predictions about behavior in specific scenarios, we're measuring how well different formally specified theories of gameplay can fit to and account for voting data and beyond

but this would also indeed require more detailed data than LLM-free methods could ever track
my track record says i'll never get this far anyway -- at least in any timely sense. so i'm trying to make sure every increment has its own value
so right now...vca

EDIT: have a separate repo that pulls votes out across games, yeah. but i find the whole thing embarassing atm

[Psyche]

Someeeee thinking around analysis plan...

My analyses before looked at final votecounts, and measured the rate at which scum landed at each position on the hammered wagon as a function of factors like the hammered slot's faction, the Day, and so on.
For comparison, I generated a null hypothesis dataset where scum/town labels were repeatedly randomly shuffled per game, and applied the same analysis.
The idea was to set up a context for non-parametric statistical testing. But honestly I'm not sure I was even doing it properly, or if the tests were actually telling me much.
Effect sizes seemed small, and I'm not confident in anything I found, let alone in their usability.

Some reflections on that:

- The null hypothesis distribution thing is maybe still a good idea if implemented right, idk. But I'll probably try harder to verify that I understand the approach and its trade-offs and that I've implemented it correctly.

- While I always expected to go further, just looking at final wagons is wasteful (ignores most votes), and also seems sort of perspective-warping in a way harmful for the analysis and our ability to interpret outcomes. End-of-day analyses are still useful, but probably not best to treat as this kind of project's focal analyses.

- Like, there are likely many systematic differences between successful wagons and unsuccessful wagons. Without a full idea of what those are, it's not clear what we can learn about voting behavior in general from just the final votecounts of every game. And final votecounts are sort of a bit removed from the actual thing we're interested in: player decision-making. Players usually don't know when they first place their vote whether their wagon will fail or not, so conditioning analyses on success/failure obscures the context behind their decisions. A result like "days rarely end with scum as the first vote on their teammates' wagons" gives us some insight into their decision-making and might evince that they hesitate to stay on wagons that aren't advantageous to their win condition but this outcome is a few steps removed from the sequences of decisions made to actually lead to this result -- scum watching the wagon they're on grow and reason after each vote whether to move wagons or double down.

- Correspondingly, it's probably better to frame analyses as examining how players navigate "forks in the road" throughout a Day instead of the Day's final state. We can measure players' propensities to join, leave, or stay on a wagon as a function of its size (or other wagons' size) instead of just looking at their position in a final snapshot. In this way, analyses use more data and provide more direct insight into players' decision-making and Days' overall trajectories.

- I also think I'd more aggressively bin together similar outcomes than I did before. First and last on a wagon are super salient, sure, but I'm doubtful that we need to measure separate entry rates to the third and the fourth position on a wagon in a Day it takes 7 votes to achieve an elimination, for instance. Grouping positions means we'll have more data per bin and have an easier time detecting meaningful differences. Maybe four bins in the wagon-positional analyses, tops?

- I still have trouble conceptualizing how to pull it all together into a presentation that's easy to use and understand given the vast range of analyses that could be done on this data, and ways those analyses could be implemented. Focusing on a specific research question helps with the first half of this, but not the second. But I think this will get clearer as I get a better idea of what VCA already is like and how a tool based on this research could most directly support that.

[joqiza]

Don't worry about messy code lol, I'm sure anything I've written is worse. I'm an econ guy not CS, all my programs are an abomination unto god. Anyway I might give a shot tonight or tomorrow at writing something, if I don't get it done by then I might not get it done for awhile tbh but I'll let you know. FWIW I am not aware of any publically available large dataset of votes in mafia games, and I think just creating something like that would be a huge accomplishment.

[mhsmith0]

All yall might enjoy this thread on the topic
viewtopic.php?t=73863

[joqiza]

Someeeee thinking around analysis plan...

Reading through your analysis plan, I'm not sure I quite understand your approach, but I do like the idea of shuffling alignments around and re-running as a test. Yeah I am not immediately sure what that is telling you though. I would need to think about it.

I suppose if you had found that say, scum were much more likely to land on, idk, position #3 or something, and then you find that it is very unlikely to observe such a result when alignments are assigned at random, maybe that suggests it is a significant result? Hmm. Yeah got to think about it.

My guess is you found everything with scum position on EoD votecounts closer to NAI though. Yeah if that's the case when you start shuffling you'll probably find that it's pretty easy to generate something similarly NAI.

It is worth noting that NAI is itself kind of an interesting result and you probably want a methodology that can distinguish between things that are NAI and things for which the data is insufficient to categorize.

[Psyche]

are you stats savvy? this is maybe the trickiest part of all of this for me. the stats courses ive taken have almost been theoretical to a fault (and also i didnt do well)

[mhsmith0]

I'd probably throw on a couple of other things too

1) When wagons are all on villagers, wolves are usually (though not always) less likely to be prominently driving any single wagon in particular (aggressive / powerwolfing wolves exist though)

2) When there's like two clear wagons and they're both town, and you have some number of players off of both wagons, they usually should get a lot of scrutiny - maybe it's something NAI (like a quickhammer or someone who 0 posted the day phase), maybe they're just really obvious towns in that spot (openly anti both wagons in a very prominent way), but otherwise that kind of positioning tends to be > rand wolf

3) Signs of bussing tend to be harder to pick up but are worth considered where they can show up as.

[mhsmith0]

I think there are some sites / metas where wolves are meaningfully > rand to split votes, but there are plenty where that's basically just NAI entirely.

[Psyche]

but what's the overarching idea or ideas behind all these patterns? that they're attention-avoiding? lack strong opinions?

[joqiza]

are you stats savvy? this is maybe the trickiest part of all of this for me. the stats courses ive taken have almost been theoretical to a fault (and also i didnt do well)

relatively ig. not a phd or anything but i studied math/econ in undergrad and used to work in market risk. I can give my thoughts but don't take me as an authority

[joqiza]

It is worth noting that NAI is itself kind of an interesting result and you probably want a methodology that can distinguish between things that are NAI and things for which the data is insufficient to categorize.

So it occurs to me that the way to test this kind of NAI result under your shuffling approach might be to, rather than assigning the alignments completely at random, instead assign the alignments

as if it were

towny or scummy, and then rerun the analysis, and then do that 5000 times or whatever and then see how likely it is to observe the NAI result.

[mhsmith0]

but what's the overarching idea or ideas behind all these patterns? that they're attention-avoiding? lack strong opinions?

I'd probably throw on a couple of other things too

1) When wagons are all on villagers, wolves are usually (though not always) less likely to be prominently driving any single wagon in particular (aggressive / powerwolfing wolves exist though)

2) When there's like two clear wagons and they're both town, and you have some number of players off of both wagons, they usually should get a lot of scrutiny - maybe it's something NAI (like a quickhammer or someone who 0 posted the day phase), maybe they're just really obvious towns in that spot (openly anti both wagons in a very prominent way), but otherwise that kind of positioning tends to be > rand wolf

3) Signs of bussing tend to be harder to pick up but are worth considered where they can show up as.

1) If the thread environment is such that wolves don't have to do anything to move the game forward in a favorable direction, then typically they find a way not to do anything - why take heat for an incorrect wagon when villagers will do it for you? (this then comes out later in game once you have more data to base things on, though sometimes role claims will give you info earlier)

2) similar idea - though here a lot of it is "ok everyone else wanted one of these folks dead, why didn't you, and why should we believe you're not full of it" - keep in mind that when a wolf is under pressure this is much less reliable (for instance, if the third wagon was a wolf but it faded away to make the top two wagons town, then wolves are likelier to have been materially involved)
also, frankly, in a spot where someone is off wagon in a v/v environment it can be REALLY easy to spot obvtowns in that spot, so "you were off wagon and you were NOT obvtown" is the more composite thing to be looking at

3) bussing signs are a different issue entirely, though i do think it can be instructive to look at overall game flow, who gained credibility from that wagon, and then how they acted subsequently - who's trusting who, who's engaging with who, is that what is "supposed" to happen if they were town, etc - this is more cross-referencing possible busses with behaviors that jar with how you'd expect a townie to act given that info, which is DIFFICULT for the most part, but from time to time you can get situations where it's relatively obvious