Literature Dive: A Cognitive Approach to Lie Detection

This forum is for discussion related to the game.
Psyche
Jack of All Trades
 
User avatar
Joined: April 28, 2011

Post Post #0  (ISO)  » Mon Jul 12, 2021 6:08 am

There's a big scientific literature surrounding interrogation and lie detection that I think could be worth integrating into our approach to the game of Mafia. But I've never really made a full attempt to dive into it. I'm going to try to do that here, incrementally, by following up on my posts in this thread about a specific scientific work:

Vrij, A., Fisher, R. P., & Blank, H. (2017). A cognitive approach to lie detection: A meta‚Äźanalysis. Legal and Criminological Psychology, 22(1), 1-21.

It's a pretty influential paper and has already received lots of citations, so I think that with a deep dive into its content and all the works it's linked to, we can start to grasp the breadth of research that scientists have done so far around the topic. Once sufficiently characterized, we can start exploring the relevance of this literature to the game we all play - and maybe learn something.

Here's a cool tweet thread about the paper:

Spoiler:


Though these results have received (imo pretty reasonable) criticism:

Spoiler:


I'll add stuff to this thread as I get time to do it.
the sands under your feet were the stars in my heaven

Psyche
Jack of All Trades
 
User avatar
Joined: April 28, 2011

Post Post #1  (ISO)  » Sat Jul 17, 2021 9:40 am

Let me try to review my notes from the past thread so I can move past them.

Overview
The cognitive lie detection approach outlined in the paper consists of three techniques:

  • imposing cognitive load,
  • encouraging interviewees to say more, and
  • asking unexpected questions.

The initial paper and its 2017 critique find that this approach works around 60% of the time when humans are behind final decision-making about honesty. It's much higher when a formal process (i.e. an algorithm) is applied, success rates are a lot better. In a long text-based mafia game I think we might have the time and energy to achieve something closer to the ceiling set by formal approaches. If you do choose that framing (which I will going forward), then meta-analysis across studies finds a success rate approaching 80%.

Relevance for Mafia As We Play It
But then again, there's no way to know if it works as well in text-based mafia, where people get to prepare their posts, don't give off non-verbal cues, and aren't under the same sort of pressure as a lot of research conditions surrounding this technique foster.

And on the other hand, the method is pretty obvious from our perspective. Overall site meta already encourages people to use votes and questions to "pressure" people into producing content and giving off tells, etc. At the same time, I think a lot of players take a more passive approach to mafia than this research suggests is optimal. They read threads, produces some reads, maybe place down votes and then sort of check-out or press their case.

A better approach to sorting players along with generally putting pressure on them is probably to think up (ideally odd/unanticipated) questions that force them to elaborate further on stuff they've said/done in the thread. So like, rather than just making an accusation and watching someone squirm about it, actually interrogate them (along with making them squirm, yeah). But again, that's probably quite obvious to anyone who's played enough of this game.

Nonetheless, the medium by which we play this game makes this approach less likely to succeed though. In particular, people have room to workshop their answers, making cognitive load pretty hard to impose unless players are forced to respond immediately somehow. Alternatively, maybe time taken to produce a response becomes something worth tracking here. However, it's a very noisy variable and other research finds that hesitation isn't a good marker of lying anyway. Perhaps a combination of reaction time tracking and pressuring interviewees to respond promptly could maximize the approach's success here. Tough concept to test though!

But What Comes Next After Pressure?
More broadly, something missing from the "cognitive approach to lie detection" meta-analysis referenced above though is a direct account of what people should be looking for when they try the technique? What is it that people do when you apply this technique that differentiates truth-tellers and liars? There are some explanations of some of this in the cited paper, but they're pretty terse. Even the table in the spoilered tweet leaves mysterious how any classification scheme could have 60-80% accuracy in this context.

Wht I do see seems to depend on noticing that someone is struggling - i.e. to promptly generate accurate details about an event they should have familiarity with if it actually happened. I can see how that's a useful cue in a face-to-face interview where someone's gotta answer your questions immediately and you have the authority to impose cognitive load in odd ways like having them tell a story backwards, but tons of factors make this super hard in text-based games.

So atm I feel like the main themes of this paper can tell us about one piece of the equation (how to treat suspects so they give off tells) but not the other parts: what tells actually look like, and how to generalize this stuff to a meaningfully different format.

The studies within the meta-analysis applying a formal/algorithmic approach for identifying liars probably deserve the most attention going forward because they'll identify concrete features in participants' responses that contribute to these high success rates - i.e., tells. They'll also offer insight into how to properly aggregate these features to form a decision, an issue that's received less attention in this community, but is potentially just as fraught.

Reviewing those might provide actionable advice to people about how to sort players in their games. And they might give us a firmer basis for statistical projects people in this community might be interested in trying. So I'll look into those next.
the sands under your feet were the stars in my heaven

Bell
Sharp Note
 
User avatar
Joined: July 14, 2020
Location: Bad Player Jail
Pronoun: He

Post Post #2  (ISO)  » Fri Jul 23, 2021 5:16 am

I think the current site meta uses this frame work. At least for the games I've played. Site meta punishes inactivity. Players can be (but not always, and this includes me) apathetic to players that seem town but have low activity and don't intervene too strongly. This play-culture or norm setting provides the pressure required to impose cognitive load. Unwnd made a similar topic that approached this topic somewhat differently.

There's always the danger of reductionist thinking to explain behavior though. I caught myself thinking about whether site-norms could make towns better by just leaning on the variables that are difficult for scum to meta themselves out of.
Like plato's republic but for mafia. But that's just a distraction it's late.

Psyche
Jack of All Trades
 
User avatar
Joined: April 28, 2011

Post Post #3  (ISO)  » Fri Jul 23, 2021 6:49 am

yeah i mostly agree

i think the site has organically or not already arrived at the gist of what researchers might recommend

individual players might not implement the strategy in full or all that well and there could be some ways to improve the meta's execution in general but i think what's generally recognized as good play by the forum isn't all that different from what's generally recognized as effective by experts
the sands under your feet were the stars in my heaven

T-Bone
A Cut Above
 
User avatar
Joined: February 18, 2011
Location: Shrug City
Pronoun: He

Post Post #4  (ISO)  » Mon Jul 26, 2021 9:53 am

ego
I have a Twitch!
I have a Youtube!

"Playing in a Newbie game doesn't count" ~ PenguinPower, Feb 2019

Ythan
Welcome to the Haystack
 
User avatar
Joined: August 12, 2009
Pronoun: He

Post Post #5  (ISO)  » Mon Jul 26, 2021 1:04 pm

In post 4, T-Bone wrote:ego

T3
Jack of All Trades
 
User avatar
Joined: February 19, 2021
Pronoun: He

Post Post #6  (ISO)  » Mon Jul 26, 2021 2:31 pm

In post 5, Ythan wrote:
In post 4, T-Bone wrote:ego

Psyche
Jack of All Trades
 
User avatar
Joined: April 28, 2011

Post Post #7  (ISO)  » Mon Jul 26, 2021 3:03 pm

this sure is a lot of pressure
the sands under your feet were the stars in my heaven

Ythan
Welcome to the Haystack
 
User avatar
Joined: August 12, 2009
Pronoun: He

Post Post #8  (ISO)  » Mon Jul 26, 2021 3:28 pm

I'm just egoing to remind myself to process what you've already posted later no pressure. Very interesting thread already.

T-Bone
A Cut Above
 
User avatar
Joined: February 18, 2011
Location: Shrug City
Pronoun: He

Post Post #9  (ISO)  » Mon Jul 26, 2021 3:41 pm

I'm egoing to remind myself to post in this thread with my thoughts so TONS OF PRESSURE
I have a Twitch!
I have a Youtube!

"Playing in a Newbie game doesn't count" ~ PenguinPower, Feb 2019

Dannflor
Survivor
 
User avatar
Joined: May 25, 2012
Pronoun: He

Post Post #10  (ISO)  » Mon Jul 26, 2021 7:20 pm

ego no pressure :]


[ + ]

Return to Mafia Discussion