Method

Case
Playtesting

Simulate the room before you walk into it. Case playtesting is a method for pressure-testing decision materials - a teaching case, a proposal, a strategy memo - against a simulated panel of AI personas before those materials ever meet a real audience.

17simulated seats
3Claude model tiers
2voting rounds
1question: does the room move?
01 - The Idea

A room full of people is a topology. Topologies can be simulated.

A topology is the shape of a system of agents: which agent gets what context, which model runs in which seat, and how the seats coordinate. Get the shape right and a handful of model calls starts behaving like an organization.

The insight behind case playtesting is that this works in reverse, too. Any room where people read something and then decide - a classroom, a credit committee, an exec review - is already a topology: different backgrounds, different information, one coordination protocol. Model the seats, wire the protocol, and you can run the room before it exists.

fig. 1 - panel topology parallel / blind / structured output
DRAFT vN ORCHESTRATOR one workflow TIER · FAST ×8 TIER · MID ×5 TIER · DEEP ×4 VOTE TALLY 17 persona agents · one seat each · no seat sees another seat's answer
Parallel topology, deliberately. Every seat reads the same draft in isolation and returns a structured vote - no cross-talk, no anchoring on the loudest agent. Coordination happens only at the tally, exactly like a blind ballot in a real room.
/01

Context is a design decision

Each persona agent gets a professional background, a temperament, and the draft - nothing else. What a seat knows is chosen, not accumulated. That is what makes the simulation legible.

/02

Model choice is a casting decision

The seat's model tier sets how it reasons: fast tiers read like a skimming executive, deep tiers read like the one person who checks the exhibits twice. Casting the tiers is casting the room.

/03

Protocol is the room

Blind vote, then reveal, then re-vote. The wiring between the seats - who speaks when, who hears what - carries as much signal as the seats themselves.

02 - The Method

Five steps, run in a loop until the room moves the way it should.

The unit of work is a full playtest: one draft, one simulated panel, one complete run of the real room's timeline. Then you read the transcript, revise the draft, and run it again.

01

Cast a panel of distinct professionals

Write 17 personas, each with a different professional background - a risk officer, a CFO, a platform engineer, a policy staffer, a management consultant. Each persona is one agent with one seat. The spread of backgrounds is the point: a draft that only convinces one kind of reader hasn't been tested.

one agent = one seat = one worldview
02

Spread the seats across model tiers

Assign the personas across three Claude model tiers - fast, mid, and deep - so the panel doesn't reason alike. Seventeen copies of the same model produce seventeen flavors of the same read. Mixed tiers produce genuinely different failure modes: the skimmer who misses the buried detail, and the careful reader who catches it.

same persona + different tier = different reader
03

Run the real timeline

The panel runs the exact sequence the real room will: every seat reads the material cold and casts a blind vote with written reasoning; then the reveal - the information the room will only get live - is delivered; then every seat re-votes. One workflow, one agent per seat, structured output throughout so the tally is machine-readable.

R1·A Read cold Each seat gets the draft and nothing else.
R1·B Blind vote Position + reasoning, no cross-talk.
R2·A Reveal The withheld information is delivered.
R2·B Re-vote Same seats, updated positions, flip measured.
04

Iterate until the outcome flips the way it should

Decision materials usually have an intended dynamic: the blind vote should lean one way, and the reveal should move the room the other way. If the panel flips before the reveal, the draft is leaking. If it doesn't flip after, the reveal isn't landing. Revise the draft and re-run the full playtest - every version faces the same panel, the same protocol, the same tally.

revise → re-run → compare tallies → repeat
05

De-machine the prose

A final pass on the draft itself: strip the tells of generated text - the symmetrical paragraphs, the hedged summaries, the lists that want to be sentences - until the material reads as written by a person, because the last thing a simulated room can vouch for is how it feels to a real one.

the panel tests the argument; a human ear tests the voice
03 - Why It Works

Three properties do the work.

Diversity of simulated reasoners

Distinct personas on distinct model tiers disagree for different reasons - which is what a real room does. The disagreements are the diagnostic: where the panel splits tells you which readers your material loses, and why.

Blind / reveal isolates the signal

Voting before and after the reveal turns the playtest into a controlled comparison. Whatever moves between the two tallies is attributable to exactly one thing: the information you delivered in between. You learn what actually moves the room, not what you hoped would.

Iteration converges on materials that survive contact

Each run is cheap, repeatable, and scored the same way, so drafts improve against a fixed standard instead of a shifting gut feeling. By the time the material meets a real audience, it has already failed in front of seventeen simulated ones - and been fixed.

04 - Beyond the Classroom

Anywhere people read a document and then decide, the room can be rehearsed.

Panel evals for product decisions

Run a spec or launch memo past simulated customers, skeptics, and operators before it reaches the real ones.

Org simulation

Model a decision-making body - committee, board, review chain - as a topology and test how a proposal travels through it.

Pre-mortems

Cast a panel that wants your plan to fail and let the blind round surface the objections you'd otherwise hear too late.

Hiring-panel calibration

Playtest interview cases and debrief materials so the panel argues about the candidate, not about the exercise.