Setup Review Guide for Reviewers and Hosts

Osie’s Setup Review Guide for Reviewers and Hosts

knBiCpq

This post is made to serve as a general guide on how to review mafia setups. Some of this has been taken from other people over the years, some of this is my own. Reviewers play an important role in mafia communities by ensuring that the games around which the community is centered are enjoyable for the players. I’ve included some notes throughout in italics which include additional thoughts on some of these concepts.

While this guide is primarily directed at reviewers, hosts should feel free to also use this as a tool for self-review to determine if they feel that their setup is ready to run.

Questions you should ask when reviewing a setup:

  1. Is the setup open or closed?
    A. Open - The setup is known to the players. These setups are potentially more breakable (by claims or otherwise) and puzzle-based, but particularly when very simple or very complicated, that can be okay. Open setups can be more enjoyable for players since they can judge for themselves ahead of time if they will enjoy it.
    B. Semi-Open - The setup is partially known to the players but not entirely. Having a pool of roles that is greater than the number of roles actually present in the game can be a way to reduce setup-gaming and still get some of the benefits of open setups.
    C. Closed - The setup is unknown to the players. Many people find this to be the most enjoyable out of these three, myself included. There’s something to be said for the finesse of a well-made closed setup that an open setup just doesn’t achieve in the same manner. That said, a closed setup can run into some issues. In particular, a closed setup is more likely to have balance concerns, even if it will be less likely to be outright broken.

  2. Is the setup Standard or Bastard?
    A. Standard - The host does not lie, but false information may occur in a reasonably expected manner, and only in regards to action results. The setup contains no alignment changes (e.g. cult), unusual wincons (e.g. jester), or traditionally unfun roles (e.g. silent, voteless, etc). I also consider it good form for a host to let the players have at least some idea of what false results the setup may contain.
    B. Bastard-Lite - The host will let players know that the setup has unusual mechanics but that the host still will not lie to the players outside of when the game presents false results. Role PMs are guaranteed to be accurate outside of potentially hidden abilities (at reviewer discretion). Role Reveals (“Flips”) are guaranteed to be accurate unless the host explicitly states that they are not. Alignment changes, unusual wincons, and traditionally unfun roles may be present. (e.g. Actress, Jester, Cult, Silent, Voteless are all okay.) Jester is not bastard if people expect it. Same with Cult, same with anything else. The question is whether or not the host directly or indirectly lies to the players.
    C. Bastard - The host will likely lie to the players. The players’ Role PMs may lie to the players. Roles can be anything. While running a bastard game can be fine, bastard games should still be fun. Review is less about balance at that point. Consider when someone has labeled a setup as bastard if it’s to say “I don’t care about the player experience”, if it’s to say “This game is still made to be fun but may have unusual mechanics”. All too often, hosts try to be clever by halves and just label the setup bastard so they don’t have to think too hard about balance or enjoyment of players. I recommend allowing bastard setups, but I recommend denying setups that appear to be labeled as bastard because the host is just being lazy. It can be a pleasant surprise for a new host who thought that they had to label their setup as bastard when you say “All you have to do is tell people that this role might exist.”

vote

  1. What form of voting does the setup use?
    A. “Majority” - During the Day, if a player receives votes from >50% of living players, that player is eliminated. This will skew the setup towards high activity.
    B. “Plurality” - At the end of the Day, if a player has the most votes, that player is eliminated. This will skew the setup towards lurking.
    C. “Majority+Plurality” - Both Majority and Plurality apply. This often still skews towards lurking as it can amount to just plurality, but the presence of majority ensures that votes mean something.
    D. “Hard Majority+Soft Plurality” - During the Day, if a player receives votes from >X% of living players, that player is eliminated. At the end of the Day, if a player has the most votes and has votes from ≥Y% of living players, that player is eliminated. This is a reasonably balanced system, but slightly favours higher activity.
    E. “Maj-at-EoD” - At the end of the Day, if a player has >50% of the votes, that player is eliminated. Even in relatively active groups, I would suggest pairing this with regular majority to ensure pressure from votes and reduce lurking.
    F. “Simple” - If at least 50% of living players are voting, at the end of the Day, if a player has the most votes, that player is eliminated. I haven’t really seen this in action and had only heard of it working in a relatively casual, 24/24 group. But I could imagine it working similarly to Soft Plurality.
    G. “Plurality-Midday” - During the Day, if at least X% of living players (typically 50%) are voting, the player with the most votes is eliminated. I think this takes agency away from players in a way that isn’t very fun, but it could be a functional system. This is mostly just a theoretical system for my own amusement.
    H. “Ranked” - Throughout the Day, each player makes a list of the other living players. The player with the highest spot across lists is eliminated at the end of the Day. This sounds like a pain in the ass, to be honest.
    I. “Kingmaker” - Players use one of the above methods of voting to pick someone who then chooses the elimination. While this can be cool, it is fundamentally a gimmick. I would only use this in settings where players know that they might expect it.
    J. Other. No need to get too clever here…

  2. What are the win conditions for each faction? Are the factions known?
    A. Town win condition - Is this public information? Maybe a sample role PM? My standard town win condition is “Ensure that no threats to town remain, as long as at least one member of town is alive.”
    B. Mafia win condition - Is this public information? My standard mafia win condition is “Ensure that no threats to mafia remain or nothing can prevent this from happening.”
    C. Serial Killer win condition - Is this public information? My standard SK win condition is “Ensure that no other players remain, regardless of whether or not you are alive.”
    D. Survivor win condition - Is this public information? My standard Survivor win condition is “Ensure that you are alive at the end of the game, regardless of who else wins or loses.”
    E. Creepy Doll win condition - Is this public information? My standard Creepy Doll win condition is “Ensure that another player dies holding your doll before you die. If you do, you achieve a win and leave the game.” Similar wording applies for any other neutral mission role that does not have to make it to endgame. Game-ending neutrals are usually bastard roles (e.g. Jesters).
    F. Other Neutral win conditions - Is this public information? Is this clear to the reader?

  3. How is the host handling Perpetual MyLO?
    A. No Elim - Is voting for No Elimination allowed? There are times in which it may be favourable for town to not eliminate someone. Most of the time, though, it’s not strategically favourable. That said, that’s generally better for the players to decide, at least in my personal experience.
    B. Optional Kill - Are anti-Town factional kill(s) optional or mandatory? There are times in which it may be favourable for scum to not kill someone. Most of the time, though, it’s not stategically favourable. That said, that’s generally better for the players to decide, at least in my personal experience.
    C. Perpetual MyLO - If the game goes into Perpetual MyLO, how does the host intend to handle it? I greatly favour making No Elim, No Kill, No Elim cause a random Town elimination. After spending a lot of time considering the three alternatives, none of them felt nearly as fair to both teams. Overall, XyLO situations are scum-favourable and if scum is forcing no elimination, then, well, they are controlling the elimination and should win the game.

  1. Other than the basic “People talk in the thread during the Day and in chats during the Night”, what forms of chatting is allowed?
    A. Daytalk - Is there daytalk in factional chats? In other chats? Daytalk all around is usually preferable.
    B. Nighttalk - Is there Strategic Public Nighttalk? Is there Social Public Nighttalk? In the latter case, can people other than living players join in? Zero Public Nighttalk is usually preferable.
    C. OOC - Is there Out-of-Game Communication? Is there a dedicated location for it? Zero Out-of-Game Communication is usually preferable. If Out-of-Game Communication exists, it should ALWAYS be in a space which the host can access.

  2. How is the host handling claims?
    A. Anti-Claim - Are there Hard Anti-Claim mechanics, ranging from a limited claim vigilante to the host modkilling anyone who claims? This will usually act in favour of minority factions.
    B. Copy-Pastas - What pieces of information are players allowed to copy-paste from roles? In most games, flavour, ability, and role names are allowed to be copied into the thread, but nothing else.

lastwill

  1. Are there any mechanical additions?
    A. ITAs - Are there ITAs? Ehh… Unless it’s a large game, I’m not a big fan of them. Make sure you consider the effect on the game, in particular the math of who has access to them.
    B. Last Wills - Are there Last Wills? “Bah” posts? Last Wills usually act in favour of majority factions. “Bah” posts are just fun for all with very little strategic impact unless they get abused.
    C. Hydras - Are Hydras allowed? This will usually act in favour of minority factions. Make sure the players know that hydras can exist, or the setup is firmly in bastard territory by most metrics.
    D. Vote Manip - Is Vote Manipulation disabled in XyLO? Usually, the answer should be yes. It doesn’t have to be, but the host should come to an active decision about it and the reviewer should ask the host about it if the host has not explicitly stated that it will be.
    E. Vote Lock - Are votes locked at any time? Some hosts and players prefer for votes to be locked in XyLO. I’m personally still not convinced of there being significant benefit to this, but oh well.
    F. Disabled Majority - Are parts of the vote system enabled/disabled at any time? Some hosts and players prefer for majority to be disabled Day 1. I personally hate having a Day phase in which players cannot be pressured by votes, so I would recommend against this, but I get why people do it.
    G. Self Votes - May players self-vote? I highly encourage banning self-votes. The vast majority of self-votes are placed in conjunction with OGI-level AtE.
    H. Oddities - Does the game include aliases (ala EiMM) or other significant game-changing core mechanical additions such as alternate factional actions, phases, etc? Most of these mechanics will act in favour of minority factions unless they affect kills, in which case they can significantly act in favour of majority factions. One way or another, they should be used cautiously as they can be incredibly unfun.

  2. How much may players utilize forum software?
    A. Editing - May players edit posts? May players use edits of posts during the game in forming reads? Minor edits are one thing, but major edits usually should not be allowed.
    B. Reactions - May players react to posts? May players use reactions during the game in forming reads? Reactions are just not that big a deal. Removing reactions can be used for cryptography, so that should typically not be allowed. If you liked a post, you liked the post.
    C. OGI - May players use activity elsewhere during the game in forming reads? This just shouldn’t be allowed. It gets gross and breaks the game experience really fast.
    D. Alts - Are alts allowed in the game? May players “alt-hunt”? Is the game Anonymous? Alts should ALWAYS be known to either the host and/or one or more of the site moderators, beyond than the player on the alt. Players should not be allowed to “alt-hunt”, as this veers quickly into game-breaking experiences. Anonymous games can be cool, but are definitely a different experience and require additional attention from the host(s) of the game.

NAR Contradiction-Handling Example.
  1. What system of action resolution is in place?
    A. “Natural Action Resolution” - The first unmodified action is applied before any modified actions. If a paradox occurs, follow a preset list. Personally, I take this a step further. I usually use the earliest layer in which an action will fit on the list, then a Jailkeeper goes after a Roleblocker and before a combined Roleblocker+Watcher, etc. I also put commuters and hiders on the same tier.
    B. “Linear Action Resolution” (e.g. SNARF) - The host has an organized list of possible actions and goes down the list in order to resolve actions. If an action has already resolved, it cannot be affected by another action. These systems, while the simplest, can be hard to remember, and cause frustration when there are awkward situations that come up when, for example, a roleblocker and a redirector target each other.
    C. “Reasonable Action Resolution” - The host asks whether an effect happen or not, based on the actions that might cause or interfere with said effect. Personally, while I understand this system, I find it counter-intuitive to explain to people, particularly when a roleblocker can affect what happens during a Night in a myriad of different ways depending on the other roles.
    D. Other - Most other systems fit under Linear Action Resolution.

  2. How is the host communicating common elements of action resolution?
    A. Roleblocks - If a player is roleblocked, what feedback do they receive if any? Is it distinct from if their action otherwise failed? I like saying “Your action failed” in any relevant context.
    B. Redirection - If a player is redirected, what feedback do they receive if any? I like only communicating that a player was redirected if that would receive information about their target.
    C. False Results - Do investigative roles know that mechanics which cause false results might exist? (Millers, Godfathers, Framers, Tailors, Lawyers, Traitors, Red Herrings, etc.) Red Herrings are an awesome way to handle false info: You tell the player with the investigative role that some number of players will return false results. You can even tell them how many players. Then it’s linked to their role and those players don’t know anything about it. Therefore there’s no issues with “If you have a miller you should claim in your first post” or “Godfathers aren’t fun”.

200

  1. What are the basic game details? (At least A through D should be in every OP for games which aren’t extremely bastard.)
    A. Players - How many players?
    B. Phases - What are phase deadlines?
    C. Prods - What are prod deadlines and expected posts? Personally, I go for 5 posts per 24 hours and an additional 5 overall in games which are shorter than 96 hours. In games which are at least 72 hours, I check every 48 hours until the last 24 hours, which I check separately. In games which are 96 hours, I go for 8 posts per 48 hours and 24 posts per 96 hours. In games which are 120+ hours, I go for 5 posts per 48 hours, and 2 in the last 24, with stricter added rules only if needed.
    D. Actions - What are action submission deadlines? Does the host give themself enough time to resolve actions? Do they give players enough time to submit actions? Players should always have at least 12 hours to submit actions. The host should usually have at least an hour to resolve actions. Ideally none of these alter the time at which the phases change.
    E. Factions - What factions are present in the game? Do the players know about the presence of any extra factions? Multiball games are a whole different animal than non-Multiball games. Players should usually know at least that a game might have neutral roles.
    F. Vanilla Percentage - What percentage of players are vanilla? Should usually be outside 0-20%.
    G. Scum Percentage - What percentage of players are anti-town? Should usually be inside 20-33%.
    H. Multiball Percentages - If there are more than two sizable factions in the game, what percentage of the game does each faction take up? It’s hard to put an exact number on this, but outside of single-player factions, there’s usually some knobs which can be twisted for balance in terms of number of players in each faction.

Thinking Cap Time

  1. Questions to ask yourself while reading a setup:
    A. Role Types - What percentage of the roles are informative, killing, manipulative, protective, communicative, voting, or other? Is there a lot of one type? Is there none or very little of one type? This may not be a problem, but it’s at least something to note. Informative, killing, and voting roles tend to be particularly swingy. Be careful about using a lot of them.
    B. Role Balance - Do the roles have similar power level? Having a 12-ability super-JoAT next to a plain Voteless Tree Stump can be okay, but there should be a method to the madness.
    C. Confirmability - How many roles are confirmable? A high number of confirmable roles will greatly favour majority factions.
    D. BCS/WCS in Detail - What’s the best case scenario and/or worst case scenario for each alignment and/or role? Minority factions can (and often should) take slightly longer for their best case scenario. Keep roles in mind as well. Can a role create an awful situation? Can it be overwhelming?
    E. Individual Roles - On a micro level, how well are individual roles balanced? If you look at a role and think “This is terrifying” or “This is awful”, then it often could use a tweak.
    F. Extremes - Is any role particularly swingy or bastardy (results in host lies which are unreasonable to anticipate)? An unrestricted Alignment Cop, an Actress, or a Psychotrooper, just to name a few examples, are roles which much of the time won’t be enjoyable for the players. Maybe for ONE player, maybe even for the players who happen to be in that particular game. But usually, for most players, not so much.
    G. Interactions - Are there any interactions between roles which might cause a problem? Sometimes you might want to have four eliminations in one Day phase. Sometimes you might not. Make sure that you think about how the roles interact and if they do anything that seems ridiculous.
    H. Comprehension - Is any role difficult to understand? Sometimes “Keep It Simple, Silly” can just be the best rule. Innovative roles are cool, but if a single ability takes up half of the screen in 10 pt font, it can probably be reworded and/or simplified. You don’t want players to TL;DR a role or be too confused to use it effectively.
    I. That just sucks - Is any role actively unpleasant to play? This can be startlingly subtle too, due to stuff that the host hints at but doesn’t intend to explicitly say. Try to empathize with a player receiving this role and thinking about it a little OR thinking about it a lot. A game mechanic of any sort that gives a player the option to do something that might seem good but is actually always terrible is a badly-designed game mechanic.

  1. Is the host organized?
    A. OP - Is the starting post written and satisfactory? The starting post is one of the primary sources for game information. It should be clear and complete.
    B. Role PMs - Are role PMs written and satisfactory? If the role PMs are hard to read for any reason, that needs to be solved before the game can pass. The reviewer is not there solely to analyze the balance of the setup.
    C. Role Reveals - Are role reveals (“flips”) and other flavour written? This is optional, but pre-written flavour can go a long way. It’s often one of the last things that hosts think to do, certainly something that I forget, but if the host remembers to write it, they’re decently likely to remember a lot of other smaller things.
    D. Action Tracking - How is the host intending to keep track of actions, prods, and votes? A spreadsheet? A list in Discord? Usually if there are 10 or more players or a high enough complexity, there should probably be a spreadsheet for action tracking.
    E. Role Confirmation - Does the host require each player to confirm their role and understanding of it in a reasonable manner? I usually include at the end of each of my role PMs “Confirm reading and understanding this message by explaining this role here in this PM.” and tie inclusion in scumchat/access to the factional kill to that as well. This ensures that players read their roles and ask me, intentionally or not, any questions they might have about their roles.

  2. Overall, I usually end up following the following pattern in some roughly similar order, sometimes switching and repeating steps:
    A. First Glance Analysis - “Does this setup immediately make me want to hit my head against a wall?”
    B. Role-by-Role Analysis - “Does any role in this setup make me want to hit my head against a wall?”
    C. Structural Analysis - “Does the composition of roles in this setup make me want to hit my head against a wall?”
    D. Best/Worst Case Scenario Analysis - “Does the setup make one of its alignments want to hit their heads against a wall?”
    E. Last glance analysis - “Is there anything I might have missed that makes me want to hit my head against a wall?”
    F. Repeat - Rinse and repeat any number of A-E any number of times.
    G. Conclude - If the speed at which bruises are forming on my head has decreased to low enough that I no longer want to scream at someone, the setup is likely ready to go.

%%%%%%%%%%%

On Review Dynamic and Denials:

The dynamic during the review is also important for the reviewer to gauge. As a general rule, the reviewer is there to make suggestions. If there is a problem, there are usually multiple ways to solve it and still make an enjoyable and balanced setup. I try to encourage hosts to come up with their own solutions to issues that I bring up, or I try to make simple suggestions that fit in with what I believe the host intends as the core concept behind the game.

Throughout this process, the back and forth between the reviewer and host should be genial and professional. I’ve personally designed/hosted/reviewed a lot of games (and other projects), period, not just mafia. Every design can feel like it’s your precious baby and you don’t want it changed in the slightest. Expectations or communication from a reviewer or a community can feel stifling, aggressive, or otherwise unpleasant. Hosts and reviewers should both be keeping in mind that the reviewer is there to help the host to make their game more appealing to the players.

Maybe 75% of hosts of setups I’ve reviewed/played/spectated/cohosted are not people who I would intrinsically call lazy. Design and hosting aren’t easy, even if they come easier to some of us than others. But some hosts act in lazy manners. If someone does not seem to have the necessary drive – they don’t seem to be putting in the effort to ensure that their game will be a success – that is reasonable grounds for denial of the game.

If the host is combative towards the reviewer, there’s no reason to believe they will not be combative towards the players. Some people do not have the personality to host games well, though often those people might be unrepentantly obnoxious in other contexts as well. At the same time, some of the most personally unpleasant people can design and host setups quite well. If the reviewer finds that the host is unpleasant to interact with, regardless of that host’s experience, that is reasonable grounds for denial of the game.

Just because a game is denied, does not mean that is the end of things. A host can always try again. But even if another reviewer passes a setup that the first one did not, whether or not any changes are made, the problems that the first one saw may come up, if not in that game, then in other games run by the host.

Not every game I’ve run across the various sites on which I’ve hosted has been reviewed. Some of them were approved by one person but altered or denied by another. (Usually just altered.) Hosts and reviewers should recognize that design and review are thorough processes, and not get impatient. When you rush, you are much more likely to make mistakes.

%%%%%%%%%%%

On Cohosts:

Even the nicest and most hard-working hosts are human and have their own lives. As a general rule, games with roughly more than 10 players are likely to benefit from a cohost, at the very least just to handle a few votecounts or be a backup when the host is busy IRL. Experienced hosts sometimes have this sense of bravado: “I’ve hosted a 30p role madness mash without a cohost before.”

It’s not an embarrassment to have a co-host. It’s not a failure. A host might truly not need one. But there’s rarely harm in having one. And the amount of benefit they can provide can be surprising. At the bare minimum, just knowing that there is a backup host ready to slide in and handle the thread for a little while if needed can reduce stress on the main host if something unexpected occurs in the real world.

I always would recommend a cohost to inexperienced hosts or for any game with more than 10 players. If you’re not sure, it doesn’t hurt to recommend one! Smaller games CAN still use a cohost, but it is usually less critical since it’s usually easier to keep track of everything. If a game has more than 15 players, I’d highly recommend or even require a cohost depending on the host. At more than 20 players, if I do not know that the host is extremely experienced, I might refuse to pass a game without a cohost.

%%%%%%%%%%%

On Co-Reviewers:

If a game has roughly 15 or more players or has a high enough complexity, it may make sense to have a second reviewer look at it. Sometimes this second review should be rigorous. This is relatively up to interpretation, but for perspective, a 20+ player role madness game in which everyone has multiple abilities, no matter how bog-standard those abilities are, should usually have at least two people rigorously reviewing it beyond the host.

%%%%%%%%%%%

In Conclusion:

Reviewing a mafia setup is an artful exercise in finding the knobs in a setup that can be turned without breaking what the setup is intended to be at its core. This can range from a single simple change to a series of significant changes all over the setup. The process discussion between the host and the reviewer is a chance for the reviewer to figure out what the host intends with the setup. No two reviews are exactly the same, even if similarities may abound. Being good at reviewing usually makes a person better at designing and hosting, which in turn usually makes them better at both reviewing again and at playing the game.

Some of reviewing comes through experience with setups, but a lot of it is via intuition and instinct. If something feels wrong, even if you can’t tell exactly what is wrong or how to fix it, there’s probably something wrong. Mafia as a game is so qualitative rather than quantitative and communities are so different that there isn’t any one perfect formula for a review.

I hope this guide proves helpful. Good luck with designing and reviewing!

Osie out.

10 Likes

I don’t see how this applies since players who would usually be low activity wouldn’t care about whether its majority or plurality.

I don’t think in terms of encouraging interesting play there’s any benefit, you could probably argue it hinders it.

I’m pretty sure its so that it doesnt turn someone voting in lylo into a button clicking competition between scum and the town who voted

If they aren’t active
Well
Oh no my execution

More just “better if you have higher activity” than “causes higher activity” from my understanding of Osie’s POV

2 Likes

Plurality-only encourages lurking in the early and middle parts of the day because votes aren’t worth nearly as much in mid to early day.

youre using your brain

one must remember the sage advice that slankers gonn slank

2 Likes