Scummies Ideas, Suggestions and Comments Thread

[mastina]

Even after the event is over?

The judges are made public.
The nominees are also public.
And the ceremony for the current year is already over.

I don't see the problem?

[zoraster]

If you know it's being made public, you have to adjust what you say and do to avoid potentially negative consequences.

If you don't and then it is, that's a pretty crappy crappy surprise.

[zoraster]

For real, think about why YOU want to know. You were nominated. You didn't get it. That hurts. You want to know why and who voted against you. Someone who's a friend knows you're going to see they voted against you, knows that'll hurt your feelings, etc. and don't you think that can't help but influence the way they act?

And even if they're WRONG, the perception of risk is there and the perception is what causes the problem.

[mastina]

If you know it's being made public, you have to adjust what you say and do to avoid potentially negative consequences.

Anything said in private you fear being released is...something which shouldn't be said at all? I mean, that seems kind-of self-explanatory.

If it is something that you don't want them to see...then why are you saying it in the first place?

A judge can judge harshly, be blunt as they want, and still judge. If the user takes offense to the judge having judged them, then...that's not the fault of the judge? Unless what the judge said crossed a line, in which case...again, why the fuck would it have been acceptable to say in the first place?

Basically, if the judging process involves harsh words, there's a binary:
Those harsh words cross a line, the judge acted inappropriately regardless of public or private; those words should not have been said at all.
Those harsh words did not cross a line, the judge was acting within their job, and if the user takes issue with it, then that is the fault of the user, not the judge.
If the user feels like there is an issue, they can raise that issue to a higher power. If a user harasses a judge who was harsh to them, the judge reports the user to said higher power.

Nine times out of ten, either you'll find the user had a point but isn't entirely right, so their criticism can be construed as...constructive feedback, AKA, a critique, AKA, meant so that the judges can improve. (Or the user is entirely irrational and unreasonable and ends up throwing a tantrum, but any user likely to do so is...likely to do that anyways even without this, so I still don't see the harm?)

[zoraster]

feel free to disagree with the judges choices. That's half the fun of awards shows. But I am not subjecting the judges to public scrutiny for their individual choices. It's just not going to happen. No one is saying anything "inappropriate." This isn't the same as the release the PT discussion. It's a matter of how judge opinions affect their interpersonal relationships and the risk that has of coloring votes and discussions. Judging isn't a social game.

[mastina]

For real, think about why YOU want to know. You were nominated. You didn't get it. That hurts. You want to know why and who voted against you.

One, my suggestion wasn't meant for the current year--it's meant for future years.

Two, you're being a bit presumptuous here. I couldn't care less about the award, nor do I care about the names who denied it to me. What I DO care about is the reason why I didn't get it--that's important to me, because very critically, that lets me improve. That lets me see: "Oh. I did those things wrong. The judges saw that I did those things wrong, and those are things I can now work to improve, now that the judges have highlighted that these were an issue."

Furthermore, it lets me know why the winners were winners: it gives a metric for what we consider high-quality. This NOT ONLY raises the standard of the site on every level related to the scummies (which is...basically everything?), BUT ALSO allows me as a nominator to better describe why I think a player is worthy of a Scummy. You'll note I tend to make a lot of Scummies nominations. If I knew what warranted the award better on PREVIOUS years, that would allow me to better deem as a nominator who I think is worthy of an award and how to write a nomination for said person.

Right now, literally the only metrics available are: the description of the award itself, prior winners of the award, and the descriptions of the nominations of those players (especially those actually used in the ceremony). Seeing what the judges are actually judging would allow for a much better idea of what is considered a good nomination for each and every Scummy.

This, aside from the benefit of having a genuine interest in helping judges improve. I can't do that with no insight into how the judges judged, now, can I? Reviewers for games tend to open themselves up to criticism. Moderators for games tend to open themselves up to criticism. They willingly put their necks out on the line, asking for feedback so that they can improve.

Why would judging be any different?

And furthermore: even if a full release isn't practical...why not a partial release?
For instance, with names redacted, reasons behind the judging process being released? (Not a final vote. Just reasons.) That's just one example.
(Basically, instead of, say, "

Drixx:

Game X was good, but I found this fault in it", instead saying "Game X was good, but I found this fault in it", omitting the name of the judge from the process.)

[zoraster]

Scummies aren't intended as a self-improvement metric.

[mastina]

Basically,
-Releasing the judge process would allow the players who were judged to understand why they didn't win: this allows them to refine their craft, so that they can become even better than they were in the prior year. They can fix mistakes they were making. Not releasing means they have no idea and they could end up making the same mistake they made before because nobody told them it was a mistake.

-Releasing the judge process would allow the whole site to see what the judges consider worthy of the award: this allows people to have a better understanding of what each award actually

means

, leading to:

1. Better nominations
   
2. Better-described nominations
   
3. Higher quality of play for people striving to be worthy of nominated
   
4. A clear illustration making a statement

...As just a short list of benefits. By not releasing, nobody has any idea except the judges themselves what each award is really about.

-Releasing the judge process would allow for users to give constructive criticism to the judges, allowing the judges to improve their process for future years. They could voice disagreements in how the judges operated, stating they don't think that metric was good, or saying they think this metric is good but maybe this other metric would be better. They could voice these opinions in here, and have them be taken in neatly and nicely by the judges, to be considered.

ESPECIALLY if the release is not a full-release, ESPECIALLY if the release is only a partial release, I don't see how that could be a problem, but I see all of these benefits to the release.

[mastina]

Scummies aren't intended as a self-improvement metric.

And I firmly disagree with that.

Scummies serve as something to strive for.
Scummies serve as a badge of honor.
Scummies serve as a beacon of talent and skill.
Scummies are a reward for doing something.
Scummies are an achievement.
Scummies are a statement: "This user did something that was notable". "This user did something extraordinary".
This user did something which put them above all other users in the given year for the area their scummy is for.

How the fuck is that NOT an improvement metric?
Have you seen how players with scummies tend to have player perception of them be altered?
Mods with scummy achievements of modding nature have players DEMAND higher quality of their moderated games.
Players with Don Corelone will be plagued by players doubting them because they're paranoid of being fooled by the current reigning champion of scumplay.
Players with Paragon are plagued by Burden of Proficiency arguments because people expect them to live up to the scummy every single game.

You get the idea. Scummies hold a deep meaning as a statement of: "This is the best, and you should strive to be this way".

I know dozens, even hundreds, of scummers who view them that way--I don't think I am alone in my belief of what they stand for. They are a statement of our elite, of our best. By inherent nature of that, by inherently BEING our best, they are beacons of what to strive for, and as a result...they are a self-improvement metric.

[zoraster]

Scummy nominees don't not win because of mistakes or whatever. We are talking about pretty subjective stuff and nominees by their nature — and finalists in particular — represent top tier stuff. If you view Scummies as a reason to rack yourself over the coals you are completely missing the point.

The answer is no, and it isn't changing. Feel free to post more about it, but I've explained the reasons for not doing so, and it isn't going to change, whether you see the disadvantages or not.

[Psyche]

aren't the judging criteria already public?

[mastina]

Scummy nominees don't not win because of mistakes or whatever. We are talking about pretty subjective stuff and nominees by their nature — and finalists in particular — represent top tier stuff.

The use of the word mistake was used for lack of a better word--what I'm more going for is, releasing some of the judging process would allow for, more or less, an idea of:
"This award, to us, means this. And Game X, we feel, best demonstrates this, because of these reasons."

Yes, the process is subjective. Yes, the nominees in particular the finalists are top-tier stuff: but

what

made the finalists be finalists? And

what

made the winner be the winner? What was the magical process which caused the judges to feel, overall, this one nominated player/moderator/game stood out above all others in the year for the quality the award is for?

What made the townplay be the best townplay?
What made the scumplay be the best scumplay?

Having a better idea of that would allow:
-Players/moderators a better idea of what to strive for
-Players/moderators a potential way of self-improvement, eliminating weaknesses and capitalizing on strengths
-Players/moderators having stronger inspiration
-Players/moderators having a better idea of the qualities we on mafiascum wish to see more of in games, and also maybe even the qualities we on mafiascum wish to see less of in games
-Players/moderators worthy of nomination being easier to find because the criteria are better-known
-Players/moderators nominating moderators/players being able to better present their case for the above of why their candidate is worthy of the award
-Users to voice their opinions on how the judges can improve and get even better at their job in future years (especially when paired with the above point).

Basically, by having an increased understanding, all aspects of the site, from playing to moderation to nomination to judging, would be increased because everyone with an increased understanding now has that increased knowledge.

[mastina]

aren't the judging criteria already public?

Not really.

The judging criteria were posted by xRECKONERx...once. In this thread.
The criteria are in that one spot publicly only.

And even there. Reck kept it pretty generalized. He didn't go into many specifics. He was as broad as possible while having it still be giving criteria. He listed what the criteria are--not what the criteria mean to the judges.
Basically, posting the literal law without the accompanying interpretation of it.

[mastina]

Correction: Reck published the judging criteria for ONE award, Don Corelone, but had this to say about publishing the judging criteria in general:

We're not going to publicize the judging criteria unless it's a uniform system. Each judge basically has their own right now, and that subjectivity is like 90% of the problem. We're going to try to unify it or provide something more concrete so people know exactly what goes into an award.

[Firebringer]

Scummies aren't intended as a self-improvement metric.

Probably best quote for this thread.

[Firebringer]

I also don't think it's that horrible idea for more transparency, but i see how that's going to cause drama.
Not like the current system prevents drama, just more speculative drama.

[Kublai Khan]

Wow, mastina is so far down the path of obliviousness.

Has she ever volunteered to actually judge scummies?

[mastina]

Has she ever volunteered to actually judge scummies?

I made the decision years ago that I wouldn't be qualified for the job. Every year, I check on my skills and abilities, revisiting that stance, to ask if the status quo has changed, if I have improved in the areas I see as necessary for judging enough to be competent at the job.

Thusfar, I have yet to deem myself worthy. Common pitfalls?
What categories can I actually judge? (In a given year, I tend to be nominated for at least one or two scummies if not more--I obviously can't judge a category I have a personal bias in.)
What categories am I

qualified

to judge? (I cannot judge Mish-Mash games because I am not involved enough to have a sense there. That's an obvious one, but you get the idea.)
When you narrow the list down to those that I can judge which I am qualified to judge...
Do I have the ability to read the needed materials and treat them with enough respect where I can give the necessary opinion?

For instance, body of work awards require me to essentially use "third person meta" on a player, which I am notoriously bad at; single-game awards require me to read literally every aspect and every nuance of every game nominated and to give a hierarchy to them, which I am also notoriously bad at. Practice for this is reading games I am not actually a player in...but often, I find myself failing to meet my self-imposed minimums in that regard. (Namely, not actually reading said game I'm not in.)

I'm sure that, eventually, a year will come where I feel like I have the necessary skills--at that time, should I be allowed to, I would volunteer for the job. But I only volunteer for jobs on this site I feel I (1) am qualified for, that (2) I can actually handle. And my call is, being a judge is currently not within those boundaries for me.

So yes.

I am oblivious to the process of a judge.
So is 99.99% of the site.
Again, the only people who seem to know are people who are involved or have been involved in the process--yet your response here is generally the type of response ALL of them give. "You have no idea what it's like." That's the whole fucking point, though; we don't, and we WANT to know.

[Frozen Angel]

you ask to join

you will join

they will give you all the nominations and links and descriptions

you will start reading them all

[optional]you will post a summary of what you found

[optional]you will participate in a discussion about it

you will vote anonymously for what you think is right

I don't know why your game didn't won paperback. I might or might not have voted for it to win or to not win but regardless only zor knows the actual votes and thats for the best.

anymore questions?

[Andrius]

The people who have served as Scummies Judges and the people who are complaining about how the system works seem to occupy two separate groups.

Scummies Judging generally looks like this:

STAGE ONE: Organizing how you're going to tackle it and what awards to do first. This is when you're excited and want to do this. Some people post initial thoughts on games they already have formed opinions about or are familiar with. Judges also recuse themselves if compromised/etc from certain categories.

Weeks pass.

STAGE TWO: Putting off the reading. You're less excited, and you now have to actually READ every game. You might even start to dislike the people constantly nominating games because there's little vetting the games from anyone, and the judges are the few people who actually have to read through EVERY nomination. For categories like KODAK MOMENT, it is simple. You read a couple pages. For the big ones (Paragon/GOTY/Best [Team/Town/Godfather/Setup/Flavor] you have to read the entire game to understand what happened.

Time passes.

STAGE THREE: This is when the Scummies are actually here and the Writers are beginning their work. This is when you're supposed to be done, but you're not because the categories you saved for last are the ones that require the most reading: Paragon and the BEST categories. Someone from the Steering Committee generally says HEY GET A MOVE ON regularly. And, of course, the last-minute nominations can also throw wrenches into the system if you've already formed opinions.


I considered doing it this year, but the time investment is significant, especially when nominations are rolling in constantly and you know how long this takes from past tenure.

[Frozen Angel]

The fact you were nominated is a win

the competition was really close at the end and people voted for what they thought was right.

That's the current system.

as I said the only change I think might be productive is to vote for judges before judging starts. Making anything else public will ruin the whole system.

[zoraster]

To be fair, Andries, I think we greatly streamlined things this year.

[Andrius]

To be fair, Andries, I think we greatly streamlined things this year.

I'm sure you have! I'd expect no less of you.

But it still requires tons of reading. You can't just throw out noms without consideration etc.
My words may no longer be relevant because of time but the biggest thing I wished to convey was the sheer TIME required in doing this.

[xRECKONERx]

aren't the judging criteria already public?

Not really.

The judging criteria were posted by xRECKONERx...once. In this thread.
The criteria are in that one spot publicly only.

And even there. Reck kept it pretty generalized. He didn't go into many specifics. He was as broad as possible while having it still be giving criteria. He listed what the criteria are--not what the criteria mean to the judges.
Basically, posting the literal law without the accompanying interpretation of it.

No, the criteria is basically what it says in the description for each award post.

[xRECKONERx]

For what it's worth, the reason we keep things private is to prevent anyone from feeling like they can't speak their mind.

Many moons ago, my first year judging, I told one of my best friends on the site who was nominated for an award that they weren't going to win it before results were announced. I didn't really list reasons or anything, but it caused a huge backlash, and it even made me want to reconsider my vote.

Simply put, judges have the job of critiquing people's play. And if we want them to be objective, we need security that nobody will get hurt over what is said. Trust me: it's harder than it looks to maintain that objectivity. Even being backstage on SSC, there were a few times I saw myself being talked about for certain categories and it took everything within me not to jump and defend myself.

If we made the judging process public, here's what would happen:
- Who won the award wouldn't change
- Instead, the people who lost would find things to nitpick with the judges who didn't vote them

It's a bad fucking idea.