Using PredictionBook to improve read calibration

[Umlaut]

I haven't seen this site mentioned here before, and I think it might be useful for players who are trying to be more honest with themselves about the accuracy of their reads.

PredictionBook is a site where you can make arbitrary predictions, or look at predictions others have made, and put probabilities on them. As you do this, it will keep score and give reviews of your accuracy. Critically, you score better not just by being right more often, but by being right the correct proportion of the time: e.g., if all of your "80% confident" predictions come true, this is worse than if 4/5 of them had come true, because it means you were underconfident.

It's possible to make predictions privately, so I'm trying to get in the habit of recording my reads during a game and how confident I am in them. The idea is that I can adjust my confidence levels to better match my accuracy, by learning that when I feel

really sure

I'm actually only right, say, 50% of the time, or that when I feel a gut twinge that seems like almost nothing it actually picks out scum way more often than random, or that something I thought was a scumtell is empirically a towntell if anything.

You can also comment on predictions, and I think I should use this to list the reasons I have for and against scumreading players, so that later on I can review and figure out what's evidence and what's just garbage.

[Lycanfire]

The problem with improving your reads is that it takes effort.

Unless you're bad to begin with, learning a few tells won't improve you quickly. You need to do some kind of analysis.


In the 3 games I've spreadsheeted, I've called the scumteam on my first day in two of them. In the third, I called the right players, but not as a team. The only failing in my method was that secondscum lurked and was replaced 2x and firstscum hard bussed it. What's this miracle analysis I pulled out? I formed opinions of every single game post that could have an opinion formed on it, and counted interactions between every single player. I was in plainest terms... Exhaustive.

This thread kind of touches back to that one on gutreads. If the goal is to increase your ability to read correctly, you won't be predicting town or scum. You'll be finding town or scum. I was on an alt for one of these games and said (paraphrased) "I think games are solvable and I'm here to try to solve this game." and well, I did. In my second or third post.

[Umlaut]

I feel like improving your reads themselves is a bit orthogonal to what I'm talking about, which is improving your confidence estimates in your reads.

It may be that you're not very good at scumreading and so, even on a good day, your top scumread of day 1 will only be scum 30% of the time. PredictionBook isn't so much about improving that 30%, but about recognizing that it

is

30% so that you don't go around saying "I'm 90% sure this guy is scum" and being wrong 7 times out of 10.

[Umlaut]

(For that matter, recognizing that you're only 30% at best may lead you to realize you need to improve, but that's just a side effect)

[Infinity 324]

Thanks a lot for this, I'm going to try it

[Nachomamma8]

This is a really sweet resource, thanks for sharing.