Let's talk Subjective challenges.

[kdowns]

I wanted to open up a little conversation on Subjective challenges. If you don't know what a subjective challenge is think of the creative challenges were you submit a picture/art/ etc... and a group of judges will rank your submission with the other submissions in the game with the best overall placing submission getting immunity or reward.

I personally do not like subjective challenges because it doesn't really feel like are in control and you're at the whims of the judges who are critiquing you. So you're not necessarily the sole reason on whether or not you win a challenge. I feel like when it comes to LSG/ORGs we should be using challenges that directly challenge the players skillset. Which while creative challenges do test the creativeness but they are left to whether or not they meet what the judge's like or dislike.

[Haschel Cedricson]

I'm usually not a fan; the team that loses will be upset that they lost, and the team that won will be upset that their victory wasn't unanimous.

My general rule is that if you are going to have them, only have one in a single game. Have the judges be publicly known in advance, and give the judges some sort of objective rubric.

[Ariel]

I enjoy subjective challenges being infrequently included as possible challenges in ORGs. Maybe because I view ORGs as by design usually involving other people's whims being a factor in one's elimination, it doesn't offend me for that to be true in challenges? Subjective challenges might be more appropriate in certain types of ORGs or more complex Survivor games than vanilla Survivor though. I think it's optimal when the challenge is designed so that players aren't spending a disproportionate amount of time on it. It's also best to provide players and judges as much information upfront as possible about the judging criteria. I personally see it being more of an issue that certain other skills are disproportionately tested in challenges. Even on the TV show, I'm bored from too many premerge challenges coming down to a puzzle.

[VashtaNeurotic]

I (obviously) agree with Ariel on this. I LOVE creative challenges. I think they allow people to make beautiful things, whether they be funny, pretty or intriguing. I also feel like things like creativity, persuasiveness, artistry and sometimes just being funny are in fact skills in themselves and while they are difficult to measure and will always have some variance, they're worth measuring just as much as other easier skills to measure are. I also agree that the keystone part of Survivor, Tribal Council is literally based on convincing people to keep you around/vote for you to win, so it's not like Survivor is a game that is built on complete objectivity (in fact you can easily argue the exact opposite). I'd also disagree on the claim that for other challenges you're usually the sole reason you lost, there's tons of reasons outside people's control they have the skills/resources you do and sometimes just having to do a challenge on an "off day" caused by something someone else did can alter your performance, it's just here the aspects outside of your control are a bit more obvious.

I assume Haschel is speaking from experience on the reaction from players (though honestly I'd love to see data on the reactions of people to different challenges, since if EVERYONE hates a challenge, then yeah, don't run it regardless of the principle) but in case it's more theoretical I will say it's not really the case for me? Like when I win....I'm usually more focused on the fact that I won than the idea that I won because 2 judges voted for me over another. And if I lose, I don't feel it's any different than losing another challenge (and hey I can even just go "nah, it's the judges who are wrong" as some small consolation).

I will say that a creative challenge done wrong can feel arbitrary, and if something feels completely arbitrary then it won't be fun. Which is why it is best to give some clear criteria and judging profiles for a challenge so players can play to a judge's interests and to make sure there is feedback given to the players so their efforts aren't just wasted with little chance to improve. After seeing how the challenges went this last game, I also realize it is better to make them more collaborative in nature, since then people can bond over the story/other thing they've made and it's a lot less likely there's identifying info in at least one submission that makes you go "maybe we can't just share these all publicly" since one of the huge benefits of these challenges is getting to see what everyone made and getting to go "oh, that's cool". When they're an individual challenge, I also do like putting them in places where multiple people winning is a fine outcome , since there's a decent chance that if you asked more people, 2nd place might swap with 1st.

[ORAM]

I would agree with Ariel and Vash, and add that the creative challenges I've been a part of have been some of the most memorable and fun ones. I've really enjoyed doing them even though I don't think I've ever placed above the median value; the actual process is much more interesting to me personally than solving a maze, crunching numbers, coordinating posts, etc. I do admit it doesn't feel good to get scores you feel are unexpected or unfair, so having a specific metric is better. Not so much "best drawing" or "funniest drawing" but something like "most outrageous drawing."

[kdowns]

I think I guess for me, if you're going to run a game that is based off a show that has no subjective challenges then you probably shouldn't add subjective challenges in a game that's trying to replicate the T.V show. For Survivor/Big Brother it's almost completely objective in all the challenges. Also there is super feels bad, I've never historically done great at subjective challenges and that may skew on how I feel. Though at this point you're introducing outside sources outside of the game that can indirectly have a huge impact on a players game. So it introduces elements that players can't control themselves. So it becomes a feel bad for a player that gets eliminated the round a subjective challenge happens.

I think the only time I've liked a subjective challenge was a challenge that was done in Hybrid which was a pair in which one person described the picture and the other player was trying to replicate them. While in LCA the Pitch the Idea challenge wasn't necessarily a good challenge because it was way too broad in what players could have done. So you have a situation where a video is getting compared to a script or power point presentation.

[Dannflor]

I mean, personally, I've historically done very well on subjective challenges to the point where in certain games people have painted me as a challenge beast for being good at... drawing a portrait. They are a different skill set but I think the subjective term sort of ignores the fact that you can see certain players do better at subjective challenges consistently, which must mean they aren't just like arbitrary, right? Like, humor, art, and writing are all skills that people can clearly be better at than others. I don't see why these skills can't be tested in challenges in the same way puzzle solving ability or flash game ability can be tested. Obviously, there aren't as many hard metrics you can use and you often need judges, but it's not arbitrary.

I *do* think it's probably wise to narrow the number of possibilities. I think creative challenges work best when the expected product is pretty narrow. Being able to use any medium you want breeds creativity sure, but so do restrictions. And I think the more restrictions you have, the more "fair" a creative challenge feels due to everyone being dealt a similar hand.

I liked Emoji Stories from Risk vs. Reward as a creative challenge a lot because it was pretty restrictive, and the objective was simple: write something the other players in the game will like. Everyone has to work with the same emojis and has the same restrictions. It's very easy to say "oh these stories were just better" and have that feeling fair and it doesn't feel like incomparable things were judged unfairly.

So, yeah, I think subjective challenges should exist but I also think the amount of freedom should be limited. This helps the challenges feel more fair and less like a crapshoot.

[VashtaNeurotic]

While I think "Sell Me This Pen"/Pitch The Idea ended up going fine (and produced some amazing pitches), I won't disagree it could have been improved in a number of ways. While the broadness of the mediums did results in some very good ads, it was also clear from Alberto/bippy's submissions (at the least) that some incredible pitches could be instructed with just text and images, and as such it probably would have been better to just restrict it to one medium (probably not videos since not every has easy access to video editing software and learning in a 36 hour window would be very stressful). Dannflor is also right that restrictions themselves can breed creativity.

I do disagree on the thought that we're JUST trying to replicate the T.V Show, we're adapting it to another medium and take inspiration from the show. After all we do several twists/challenges that predate their use on the show by a while (like the "un-merge" or even Haschel's FTC format) or just would never be on the show due to logistics or other reasons (like they're never gonna end up playing a rhythm game, geoguesser or Hanabi on the show). I personally think that creative challenges improve on the show format because they challenge skills that are harder to test, and you get some pretty things out of it. I also personally like getting to create things, even though I've mostly placed middle of the pack in the creative challenges I've been in. I do feel judging profiles solve most of the "out of my control" aspects of the challenge, and they have been shown to influence what people submit (unless people were already gonna write about dinosaurs a lot in a science fiction writing contest).

[Xalxe]

Survivor also has done creative challenges in the (very long ago) past.

[Haschel Cedricson]

Survivor also has done creative challenges in the (very long ago) past.

Rupert's Murder Bunker was amazing and I for one am shocked it did not win.

[Save The Dragons]

i think it is a crime i threw wine on my face for a creative challenge and didn't do well

[Cephrir]

I like them. I find that at least a solid percentage of the cast typically has fun with it, if they're tribal it provides a good bonding activity, and it tests a different skill which is something I pretty much always want. As a designer I feel that fun is an important component, and while some players don't have fun with them, it's a really memorable highlight for those who do. While it's true that there's an arbitrary aspect, it's also in a way arbitrary what skills each player comes into the game with, and I want to give people who aren't good at traditional challenges, or thinking quickly in general (which applies to nearly every challenge), a chance. It would be nice I guess if there were a way to objectively evaluate who the best writer is, but I don't want to give them up just because there isn't.

[Thestatusquo]

I like them.

I think a pretty important part of game design is challenges that favor a wide spectrum of different skillsets and creative is one of those. I think an issue with many ORG designs is that the challenges tend to fall down very predictable and samey lines in terms of which skills they're testing and this is problematic because it systemically advantages certain players in our games. In many ways the term "challenge beast" exists in part because we test the same things in challenges over and over again.

I think you can help with the objectivity in lots of different ways. One way I tried in RvR is we gave the judges a rubric with which to assign points so at the very least players knew the aim and what they were shooting foc.