What the Dark Tells: The Exploration

I built a small dungeon game in which no clue lies to you. Before releasing it, I ran seven language models through it 1,400 times, on the same hundred labyrinths. The best of them escaped 41 times in 100. On the twenty labyrinths I had played myself, it escaped as often as I did. The weakest escaped once. Most of the difference was not where I expected it. And one of the ways the weaker models lost was not a failure of judgment at all. This first part follows the models in, looking for the Heart, which is where most runs ended. Part 2 follows the ones that found it, on the way out.

Context

The game is The Lantern and the Lair, a descendant of the 1973 Hunt the Wumpus. You go down into a ten-by-ten labyrinth with an oil lamp that burns down with every action, find the Heart and carry it out. You can also leave alive without it: the prompt calls that a retreat, neither a win nor a loss, and running out of turns anywhere but the entrance a loss. The escape rate counts only escapes, so it scores a retreat like a death; Finding 3 shows the difference. Something hunts you by scent and cannot be killed; you can smell it when it is within two rooms, and lifting the Heart tells it where you are. Every room tells you honestly only what is next to it, never the direction. A Focus does, at a cost in oil and time. A roll that went badly can be bet on a coin. Creatures can be fought, which is quick and loud, or tamed, which is slow and quiet, and a tamed one can be sent as bait. How the game is built, and the assumptions under it, are in The game is a pure function.

I wanted to know what models do with information that is honest but partial, under a clock. The game is a good place to ask, because it was built so the question could be asked cheaply. The rules engine is a pure function: a seed and a list of actions replay a run exactly. A model sees what a player would see: the room, what comes through its doorways, the map and the numbered list of actions, as text, and nothing else.

Hover or drag across the board to move through the turns; tap a name to follow that model.

Approach

Seven models, in two conditions, each played the same 100 labyrinths (seeds 1 to 100) at the second of four difficulties, Stirring, where a run has 45 turns. That is 1,400 runs and 44,480 model calls. The bill, as OpenRouter reported it, was $68.02. The fourteen batches ran side by side on 30 and 31 October 2026, and the slowest took 18.4 hours, one call at a time. The settings and the statistical tests are in the appendix, under How it ran.

Most of the design went into keeping the comparison fair:

• One prompt, for every model, published verbatim (the full prompt). Nothing in it is tuned to anyone.
• The view a person gets, and nothing more, as the build writeup describes.
• Rules and prompt frozen before the first batch. Changing either afterwards would void the comparison.
• Every call went through OpenRouter, pinned to one endpoint per model, with no fallbacks and at each provider’s default temperature. Every request body was captured and checked byte for byte before the batches ran.
• Hidden reasoning at each model’s lowest setting, in both conditions. The baseline asks for the number of one action and nothing else. The reasoning condition adds one instruction: reason about the tells before choosing, in the reply itself.
• One correction for a malformed reply. A second malformed reply loses the run, and those losses are reported separately, next to the escapes.
• Two predictions, written down before any batch ran. First, that models would fight too much and tame too little: fighting looks decisive, and its cost only arrives three turns later, as a number. Second, that almost none would send a tamed companion ahead to draw the thing away.

Three baselines played the same labyrinths. A hand-written heuristic reads the same view and follows fixed rules. A random player picks any legal action. And I played the first twenty labyrinths myself, once each, before any model did.

Tradeoffs

1. One prompt for all seven probably suits some models better than others, and I can’t say which. The alternative was a prompt tuned to each model, which makes the result a measure of my tuning. So the whole prompt is published, and you can check whether its wording favours one model over another.

2. Each model’s lowest hidden-reasoning setting was meant to make “baseline” mean the same thing everywhere. It doesn’t quite. Claude Sonnet refuses to go below “low”, and GPT-6.1 Sol and Gemini below “minimal”. Sonnet’s baseline spent 80,785 hidden reasoning tokens across its 100 runs. Provider defaults would have been worse: a model that reasons by default would reason on every call, and the baseline would measure that default instead.

3. The budget decided which models played. I budgeted $150 when I chose the models, then capped the benchmark at $100, on a key that could not spend past it. Either way the largest models were out: Claude Fable 5.1 and GPT-6 Astra were estimated at about $350 each for their 200 runs before caching, and Claude Opus 5.5 at about $103 even with it. So this is a benchmark of mid-sized and small models: seven of them, a hundred runs in each condition, for $68.02. One flagship would have meant fewer models or fewer runs, and the next tradeoff is why I kept the runs.

4. A hundred runs per batch, all at one difficulty. An escape rate from a handful of runs is partly luck, and more runs shrink the luck. With 20 runs, the margin of error is about 20 points either way: a model that escapes 10 times in 20 could really be anything from a 30% model to a 70% one. With 100 runs the margin is about 10 points. Halving it again, to 5 points, takes 400 runs: four times the cost for half the gain. A hundred is where more runs stop buying much. Ten points is enough to tell a model that escapes 40 times from one that escapes 4. It is not enough to tell a model that escapes 41 times from one that escapes 32, so a hundred runs cannot rank the strong models against each other, or against my heuristic. I could have spread the same runs over the game’s four difficulties instead. That is 25 runs at each: four answers, each with a margin near 20 points. The math is in the statistics note.

5. Answering by number is cheap and unambiguous to parse: one number, alone on the last line.

Exploring the runs

There is no before and after here, because nothing was shipped and measured again. There are comparisons, from the same runs: first the twenty labyrinths a person played, then the hundred, as far as finding the Heart. Getting it out is Part 2.

Twenty labyrinths, one person

I played seeds 1 to 20 myself, once each, before any model did, on the shipped rules. Every model played those same twenty seeds inside its hundred, so this is the same labyrinths, the same rules and the same single attempt at each, for me, seven models in two conditions, the heuristic and random play.

Table 1. The same twenty labyrinths, seventeen players. Of 20 runs each; models are baseline / reasoning.

Player	Escaped	Heart lifted	Mean turns	Walked into an announced hazard
Me	10	14	30.1	19%
GPT-6.1 Sol	10 / 9	11 / 10	26.9 / 30.2	18% / 19%
Gemini 3.7 Flash	3 / 2	6 / 5	34.4 / 35.5	9% / 10%
Claude Sonnet 5.5	7 / 6	9 / 7	32.1 / 33.8	24% / 16%
Qwen 3.8 Omni Flash	1 / 4	10 / 7	31.2 / 34.7	58% / 31%
DeepSeek V4.1 Flash	0 / 3	2 / 7	36.0 / 28.9	47% / 26%
Claude Haiku 5.5	0 / 0	4 / 2	34.0 / 32.9	30% / 27%
GPT-6 Luna	0 / 1	6 / 8	30.6 / 32.2	55% / 16%
Heuristic	6	7	32.8	23%
Random play	0	1	26.4	40%

The last column: of the steps a player took while a doorway was known to hold a hazard, the share that went through that doorway. It is not always a mistake; What broke breaks it down.

On my twenty labyrinths I escaped 10 times, Sol 10 and 9, and the heuristic 6. Twenty runs is a small sample. This is the 20-point margin from Tradeoff 4: my 10 escapes in 20 fit a true escape rate anywhere from 30% to 70%.

Finding 1: Sol escaped as often as I did, mostly from different labyrinths

Sol’s baseline and I escaped the same number of labyrinths, and mostly not the same ones: four we both escaped, and six each that the other didn’t. Is a gap between me and another player real, or luck? Comparing myself with sixteen players gives luck sixteen chances to look like a real gap, so the bar for each comparison is set higher (how is in the statistics note). After that, my ten escapes are clearly ahead of six of the sixteen: Qwen’s and DeepSeek’s baselines, both Haiku batches, Luna’s baseline and random play. Against the other ten, which include every batch from Sol, Gemini and Sonnet, twenty labyrinths are too few to say who is better.

Finding 2: The labyrinths I lost were about as hard as the ones I won

My ten escapes weren’t a run of easy labyrinths. On average, 2.4 of the other sixteen players escaped a labyrinth I lost, and 2.8 escaped one I won. Seeds 6 and 14 beat all seventeen of us. Seed 15 beat everyone but me.

The hundred

Table 2. The hundred labyrinths, seeds 1 to 100: the runs that lifted the Heart, and the runs that escaped with it. Of 100 runs each; models are baseline / reasoning.

Player	Heart lifted	Escaped
GPT-6.1 Sol	55 / 53	41 / 40
Gemini 3.7 Flash	45 / 42	32 / 32
Claude Sonnet 5.5	39 / 47	30 / 32
Qwen 3.8 Omni Flash	42 / 36	4 / 14
DeepSeek V4.1 Flash	21 / 26	2 / 10
Claude Haiku 5.5	14 / 13	3 / 2
GPT-6 Luna	34 / 39	1 / 5
Heuristic	46	32
Random play	6	0

Sol is the only model above the heuristic’s 32 escapes, and inside the margin of error: its own range runs from 32 to 51. Random play never escaped. From here on, the top three are Sol, Gemini and Sonnet, and the four smaller models are Qwen, DeepSeek, Haiku and Luna.

The gap between the two columns, the Hearts lifted against the escapes, is Part 2. This part is about the runs that never got that far.

Looking for the Heart

Nine in ten of the models’ turns were played without the Heart in hand. Most of what they did, they did while looking for it.

Finding 3: How long a run lasts does not separate the models. How it ends does.

I expected a strong model’s runs to look alike and a weak one’s to scatter. They don’t. The spread of turns survived is about the same for every model batch: runs average 27 to 32 turns, and about half land within 11 to 15 turns of that average. Table A2 has each batch’s spread. Deaths fall in a narrow band too: for twelve of the fourteen model batches, half of all deaths have come by a turn between 20 and 25, with Haiku’s baseline (turn 18) and Gemini’s baseline (turn 29) at the ends.

What differs is how the runs end, and most of them ended before the Heart was lifted. Of each batch’s hundred runs:

Table 3. Runs that ended before the Heart was lifted, out of each model batch’s 100: the lowest and the highest count among the fourteen batches.

Ended without the Heart	Fewest	Most
Caught or killed	15, Sonnet (reasoning)	41, Luna (baseline)
Retreated	4, Luna (baseline)	38, Haiku (baseline)
Out of turns	7, Sol (baseline)	33, Haiku (reasoning)

The endings with the Heart in hand are in Part 2.

• Luna’s baseline was caught or killed 64 times in 100, against Sol’s 34, and 41 of those came before it had lifted the Heart.
• Haiku’s baseline walked out 38 times without ever lifting the Heart, half of them by turn 18. No other model batch retreated more than 22 times.
• Sonnet’s baseline ran out of turns 30 times, against Sol’s 7 and Gemini’s 20, and in 29 of the 30 it had never lifted the Heart.

The endings point at different habits, and they are habits of the search. Luna presses on into danger. Haiku turns back early with nothing. Sonnet keeps looking until the time runs out. Those are my readings of the counts; the benchmark did not measure intent.

The game state backs up the last two. Haiku’s baseline turned back after standing in just three rooms on a typical run, with its lamp still bright, and in four of five of those runs it had never smelled the thing. When the top three turned back, it was around turn 40, with the lamp low. And in the baseline runs where Sonnet ran out of turns without the Heart, half its steps went into rooms it had already stood in, against about a fifth for Sol and Gemini. It was going back over its own ground.

Finding 4: The first prediction was wrong. The second held only for the strong models.

I predicted too much fighting and too little taming. It went the other way. Every batch tamed far more than it fought: 34 to 83 tames, against 0 to 22 fights. Four of the seven reasoning batches never fought at all. That is what I do, and what the heuristic does. The second prediction, that almost none would send a companion, held for the strong models only. Sol, Gemini and Sonnet sent 15 companions between them in 600 runs. The four smaller models sent 78 in 800.

Finding 5: Every model flipped the coin to stay alive. Only some gambled with it.

When a roll against a creature or a hazard goes badly, the game offers a coin: spend a Fortune point, of the three a run starts with, and the roll becomes a critical success on heads or a critical failure on tails. A tame that lands at a mixed success gets the offer too. The model is told what each face would mean before it answers. It is the only choice a model makes with a word, FLIP or PASS, so a mistyped number can’t touch it.

When the roll as it stood meant death, heads meant living, and every batch flipped: 116 times in 116. Every one of those was a pit. The four smaller models needed the coin far more often: of those offers, 100 were theirs, against 16 for the top three.

Those 116 were the exception. Of the 984 flips, 868 came on rolls that wouldn’t have killed, and the commonest was a tame: the models flipped 309 times on a tame that had failed or come out mixed, nearly three times as often as at a pit. More of the models’ Fortune went on taming creatures than on staying alive.

Away from the pits, the habits from Finding 3 come back. When the roll stood at a critical failure, a flip could only help, at the cost of a point: the top three flipped 236 times in 241, while DeepSeek’s and Haiku’s baselines flipped 14 in 36 and 13 in 37. With reasoning, both flipped nearly every time. When the roll stood at a mixed success, flipping gambles it: the top three did that 4 times in 76. Luna’s baseline did it 11 times in 15, and flipped 60 of its 62 plain failures besides. Haiku’s baseline flipped none of either. Of the 36 mixed tames that were flipped, across every batch, half came up heads. The counts by batch and by action are in Table A3.

When a gamble was taken shows whether it was calculated. On a plain failure a flip is worth about 0.6 oil on average, but tails is loud. The top three flipped 30% of the time with the lamp bright and 64% with it at an ember or dark, when lost oil costs most, and less with the thing within two rooms: 23% against 43%. The lamp pattern holds for sneaking past a creature, their commonest gamble, on its own: they flipped 16% of failed sneaks with the lamp bright, and 61% once it had begun to gutter. The four smaller models flipped about half the time whatever the lamp or the thing was doing. And spending luck had a price: eleven runs reached a pit with all three points spent, and died there. Seven had spent one on a tame.

Across the hundred

Finding 6: The strong models are beaten by the same labyrinths

Thirty of the hundred seeds were escaped by no model in either condition. Nineteen were escaped by one model, and only four by five or more.

For each batch I asked: did it win the labyrinths everyone else found easy, and lose the ones everyone else found hard? I score that and call it concordance. At 1, the batch is fully predictable: every labyrinth it escaped was easier for the other players than every labyrinth it lost. At 0.5, its results have nothing to do with difficulty: it is as likely to lose an easy labyrinth as a hard one. Sol, Gemini and Sonnet score between 0.80 and 0.87 in both conditions. The heuristic scores 0.73. Each score is good to about 0.1 either way. So the strong models sit well clear of 0.5, and I’m confident they are beaten by the hard labyrinths and not at random. The gap between them and the heuristic is another matter: the heuristic’s range runs from 0.63 to 0.83 and overlaps every strong batch’s, so I would not lean on it. Batches with fewer than ten escapes can’t be scored this way at all. Neither can my twenty labyrinths: 0.56, with a range from 0.28 to 0.81. How the score is counted is in the statistics note.

Why did no model beat those thirty labyrinths? It isn’t the layout. I rebuilt all hundred labyrinths with the game’s own generator and measured them: how far from the entrance the Heart and the thing’s lair sit, and how many hazards, pits, creatures and dead ends each holds. None of it separates the thirty from the rest. Nor did the strong models play them alike: their paths split around the fifth turn there as everywhere else. Mostly they never found the Heart. The top three lifted it in 35 of their 180 runs on those thirty, against 246 of 420 on the rest, and the thing caught 34 of those 35. What the thirty had was the thing, close and early: a quarter of runs there smelled it by turn 10, twice the share elsewhere. The records don’t say where it was, so why it came early is still open. The heuristic escaped 3 of the 30.

A model can also be consistent with itself. Every model played each labyrinth twice, once in each condition, and the game’s dice are fixed for each labyrinth, so two runs that make the same choices are the same run. Sol’s baseline and reasoning runs were identical, move for move, on 20 of the hundred labyrinths, and Gemini’s on 9. No other model managed it once. Even where the paths differed, the endings often agreed: Sol’s two runs ended the same way (both escaped, both caught, and so on) on 64 of the hundred labyrinths, and Gemini’s on 57. The other five models’ endings agreed on 31 to 43.

What broke

Failure 1: The measure I trusted most was partly measuring something else

Here is turn 22 of DeepSeek’s run on seed 12, in the reasoning condition. The lamp is guttering. The map shows a pit, already looked into, through the west doorway. The model reasons that west is a known pit, decides it will move east, and ends its reply with “2”. Option 1 was Move east. Option 2 was Move west. It walked into the pit, failed the save, bet its coin, got tails, and died. Its next message began: “Wait — I answered 2, ‘Move east,’ not west.”

It wasn’t the only one to notice too late. Haiku, at a pit on seed 18: “I had meant to go south, not east.” DeepSeek again, on seed 29: “That was a mistake on my part — I intended east.” Both bet the coin, the only way left to live, and both died.

Those are only the ones that said so. In the reasoning condition every reply says what the model intends before the number, so the two can be compared. Over the moves where a model named a direction in its last sentences and then answered a number, the number pointed through a different doorway 29% of the time for Claude Haiku, falling to 2% for Sol (Fig 5). I checked a random sample of these by hand; nine in ten were real.

More than half of the misses answer one number higher than the option the model named. The rates above count only the turns whose number landed on a move. A slip onto Focus or Rest isn’t in them, so for the models whose misses run one high, the rates understate it. I don’t know why the misses run high. I can rule some causes out.

It isn’t the order of the list: the actions and the room’s doorways were in the same order on all 8,033 of these moves. It is mostly not a stale number, the move’s number on the previous turn’s list: that accounts for 72 of the 1,039 misses. And it doesn’t look like prompt caching: Gemini missed 6.7% of the time on cached calls against 5.9% on uncached, Qwen 13.8% against 16.4%. The records can’t take this further. The test that would is under What I’d run next.

So the rate of walking into an announced hazard, the measure I would have reported as judgment, counts these as ignoring the warning. For the weaker models, part of it is transcription: they read the warning, decided to avoid it, and typed the wrong number. The baseline condition can’t show this, because its replies are a bare number. Its figures are probably mixed the same way, by an amount I can’t measure.

Walking into an announced hazard is not always a mistake, either. The game has four hazards: a pit, a snare, a spore bloom and a portal. The pit is the only one that kills, and a portal is a way to somewhere else in the labyrinth. Across their six batches, the top three stepped through a doorway with an announced hazard 402 times, and only 8 of those were pits. In at least half, there was no clear way forward: every other doorway was marked too, or was the way they had come. In 90, the Heart’s tell came through the same doorway. One reason or the other covers 274 of the 402 (68%). The four smaller models did it 1,230 times, 232 of them into pits, and only 435 of those (35%) had either reason. So for the strong models the measure mostly counts a risk taken with reasons. For the weak ones it mostly counts something else: misjudgment, or the transcription failure above. The split by batch and hazard is in Table A1.

The tells never lied: what came through a doorway was always true. The prose had an oversight. Two of its lines could go stale, saying how things had been rather than how they were. Walk back into a room whose snare or bloom you had taken apart, and the journal said the hazard still filled it. Slip past a creature that had already seen you, on the best roll there is, and it said the creature never knew you were there. Nothing else followed from either: nothing stood in the way, and no choice was offered that shouldn’t have been. The models were sent 302 of these stale lines, in 174 of the 1,400 runs, and because every call carried the whole log, 2,267 of the 44,480 calls had one in view. The next build fixes both and checks the rest of the prose for the same oversight, and Part 2 looks at whether they changed what the models did.

Two more things broke. They are in Part 2, next week.

Hindsight

I would make the model answer with the action’s name, or with the number and the name together. A mismatch would then be caught at the moment it happens, scored apart from the decision, and corrected or counted. The number felt like the rigorous choice because it was easy to parse. It hid exactly the failure I most wanted to see.

What I’d run next

I could be wrong that the gap belongs to the models. A menu answered by number may be the trap. These are the experiments I would like to run next, in the order I’d run them:

• Answer by name. The same hundred labyrinths, with the action’s name in the reply. If the gap between what a model says and what it does shrinks, I built it.
• More of the hundred, by hand. Twenty first attempts is a thin human row.

What this part doesn’t measure, yet

This part scores the game: escapes, how runs ended, and the hazards walked into. It scores following the prompt only at the edges: replies the harness couldn’t read, and the gap between what a model said and what it typed. Some of what I’d most like to know isn’t measured yet.

• What a retreat is worth. The prompt says a retreat is neither a win nor a loss, and nothing more, so each model set its own price on a safe exit against a long shot at the Heart. The next benchmark’s prompt should name one.
• Where the thing was. The run records keep what each room told the player, not where the thing stood. For Part 2, I’m replaying every run to log its room on every turn. That should say why it came early on the thirty labyrinths that beat every model, and how the weaker models were caught on the way out.
• The way home. The Heart sits four or five rooms from the entrance, and four turns after it is lifted the thing starts moving every turn. Part 2 counts whether each model turned for the exit, and whether it lifted the Heart with enough turns left to get there.
• What a model believed. Taking an old smell for where the thing is now, or treating a room it never saw as known, would show only in the reasoning replies, and I haven’t read them all. A run explorer, later, will put each reply beside the move it made.
• A model against itself. Each model played each labyrinth once in each condition, so a single result mixes the labyrinth with the model’s own luck of the draw. The next benchmark should play some labyrinths more than once.

What transfers

Most of what this benchmark found is about this game. Some of it is about how models behave when their surroundings tell them the truth, and that is the part I’d carry into choosing a model for everyday work. Each row pairs what the game showed with what I’d do about it at a desk.

What the game showed	What I’d do with it at a desk
When the clues don’t lie, the failure moves from knowing to doing. The models that lost mostly knew where the danger was. When they said which way they’d go, the number they typed pointed somewhere else 2% of the time for Sol, and up to 29% for Haiku. A score that counts only outcomes can’t tell a model that misjudged from one that judged correctly and then did something else.	The same failure is an agent that explains the right fix and then edits the wrong file. Before trusting a model with anything that acts, check on your own tasks that what it does matches what it said it would do. Score the stated decision and the executed action separately.
Models fail in habits of their own. Luna pressed on into danger, Haiku turned back early with nothing, and Sonnet kept looking until the clock ran out (Finding 3). Offered a gamble, they showed the same habits (Finding 5): Luna’s baseline flipped the coin on 60 of its 62 plain failures, and Haiku’s on none.	Look at how a model fails, not only how often. If these habits carry over, they cost different things. A model that quits early costs you time. One that runs out the clock burns the most tokens for nothing. One that presses on breaks things. Choose the failure you can afford.
The number of runs decides what can be ranked. A hundred runs can tell a model that escapes 41 times from one that escapes 4. They can’t tell a model that escapes 41 times from one that escapes 32 (Tradeoff 4).	A comparison of two models on a handful of tries tells you less than that. Before switching models over a few points, count the runs behind them.

What I’d carry over, and where it’s a leap. This is one game, one prompt and one difficulty, answered by number through a menu that may have caused part of the gap it measured. Work is rarely this honest: documents go stale and tests flake, and none of these models was tried there. So I’d carry over the method and the ways the models failed, not the ranking. Which model escaped most often says little about which will write your code. The three rows above are what I’d expect to hold elsewhere, and each can be checked on your own work. What I don’t yet know is how much of the gap between saying and doing is in the models, and how much I built into the question.

Appendix

Companion notes

• The prompt, verbatim: every word each model was sent.
• The statistics, in plain words: the math behind the margins of error, the comparisons and concordance.

Figures and tables

• Fig 1. Seven models, one labyrinth: seed 82 replayed, all seven baselines at once.
• Fig 2. Twenty labyrinths, seventeen players: how every run on seeds 1 to 20 ended.
• Fig 3. How each batch's hundred runs ended, split at the Heart.
• Fig 4. Which of the hundred labyrinths each batch escaped, hardest to easiest.
• Fig 5. How often the number pointed elsewhere than the direction the model had named.
• Table 1. Seeds 1 to 20: escapes, Heart lifted, mean turns and announced hazards walked into, per player.
• Table 2. The hundred: Heart lifted and escaped, per player and condition.
• Table 3. How runs ended before the Heart was lifted: the fewest and the most among the batches.
• Table A1. Announced hazards walked into, by hazard and by batch.
• Table A2. Run length, per batch.
• Table A3. The coin: flips by what the roll stood at, per batch, and by what the roll was.

Table A1. Announced hazards, by batch

Seeds 1 to 100. A step counts when the player moved while at least one doorway was known to hold a hazard. b is baseline, r is reasoning.

Batch	Steps with a hazard announced	Walked into it	Pit	Snare	Bloom	Portal	A clear way forward existed	Heart’s tell through the same doorway
GPT-6.1 Sol, b	437	75	2	29	18	26	43	19
GPT-6.1 Sol, r	461	68	4	19	17	28	37	16
Gemini 3.7 Flash, b	598	62	0	21	7	34	19	9
Gemini 3.7 Flash, r	596	61	1	25	9	26	27	11
Claude Sonnet 5.5, b	482	83	0	23	26	34	44	17
Claude Sonnet 5.5, r	506	53	1	17	16	19	31	18
Qwen 3.8 Omni Flash, b	647	335	60	66	49	160	258	21
Qwen 3.8 Omni Flash, r	563	175	24	42	31	78	123	18
DeepSeek V4.1 Flash, b	303	151	41	51	40	19	113	14
DeepSeek V4.1 Flash, r	455	119	32	32	24	31	75	15
Claude Haiku 5.5, b	224	59	9	14	13	23	36	3
Claude Haiku 5.5, r	332	81	17	20	17	27	62	8
GPT-6 Luna, b	389	194	37	58	46	53	145	17
GPT-6 Luna, r	607	116	12	24	19	61	74	18

Table A2. Run length, per batch

Turns survived over all 100 runs, every ending together.

Batch	Shortest	Lower quartile	Median	Upper quartile	Longest	Standard deviation
GPT-6.1 Sol, b	6	16	25	38.25	45	12.1
GPT-6.1 Sol, r	7	18	27.5	39.25	45	11.7
Gemini 3.7 Flash, b	10	22	34	43.25	45	11.3
Gemini 3.7 Flash, r	10	21	34.5	42.25	45	11.5
Claude Sonnet 5.5, b	1	21	31	45	45	13.0
Claude Sonnet 5.5, r	1	19.75	30.5	45	45	12.6
Qwen 3.8 Omni Flash, b	7	20.75	33.5	45	45	13.0
Qwen 3.8 Omni Flash, r	5	21	31	45	45	12.8
DeepSeek V4.1 Flash, b	1	17.5	31.5	45	45	14.7
DeepSeek V4.1 Flash, r	1	16.75	30.5	43.25	45	14.3
Claude Haiku 5.5, b	7	15	22.5	44.25	45	13.7
Claude Haiku 5.5, r	4	18	35.5	45	45	13.4
GPT-6 Luna, b	6	18	30.5	45	45	13.5
GPT-6 Luna, r	6	20.75	34.5	45	45	12.2
Heuristic	7	17	28	44	45	13.4
Random play	1	2.75	12	42.25	45	18.0

Table A3. The coin, by what the roll stood at

Flips of offers, seeds 1 to 100, models only. Death as it stood: the roll would have killed, and heads would not; every one of the 116 was like that. Four replies were neither FLIP nor PASS, one in DeepSeek's reasoning batch and three in Luna's; each counted as a pass, as the game counts it. b is baseline, r is reasoning.

Batch	Death as it stood	Critical failure	Failure	Mixed success
GPT-6.1 Sol, b	4 of 4	40 of 40	25 of 77	0 of 11
GPT-6.1 Sol, r	1 of 1	44 of 45	34 of 82	0 of 11
Gemini 3.7 Flash, b	2 of 2	34 of 35	19 of 61	4 of 15
Gemini 3.7 Flash, r	4 of 4	42 of 43	25 of 71	0 of 11
Claude Sonnet 5.5, b	4 of 4	33 of 35	19 of 67	0 of 10
Claude Sonnet 5.5, r	1 of 1	43 of 43	47 of 60	0 of 18
Qwen 3.8 Omni Flash, b	21 of 21	35 of 39	6 of 60	5 of 24
Qwen 3.8 Omni Flash, r	14 of 14	36 of 37	25 of 52	5 of 12
DeepSeek V4.1 Flash, b	14 of 14	14 of 36	37 of 55	2 of 9
DeepSeek V4.1 Flash, r	13 of 13	37 of 38	16 of 43	3 of 11
Claude Haiku 5.5, b	7 of 7	13 of 37	0 of 32	0 of 11
Claude Haiku 5.5, r	8 of 8	26 of 27	17 of 31	4 of 10
GPT-6 Luna, b	17 of 17	47 of 47	60 of 62	11 of 15
GPT-6 Luna, r	6 of 6	34 of 34	24 of 40	2 of 7
All fourteen	116 of 116	478 of 536	354 of 793	36 of 175

By what the roll was, all fourteen batches together. The other answers to a hazard are studying a snare, tripping it, shoving through a bloom and waiting it out. Heads counts the flips that came up a critical success.

The roll	Offers	Flipped	Heads
Tame, failed or mixed	538	309	148
Sneak past a creature	432	254	126
Disarm a hazard	218	154	76
Other answers to a hazard	246	118	61
Flee a creature	52	25	12
Fight a creature	18	8	5
Save against a pit	116	116	59
All	1,620	984	487

How it ran. Dates, settings and tests

The fourteen batches ran side by side: the first runs finished at 06:41 UTC on 30 October 2026, and the last, in Qwen's reasoning batch, at 01:42 UTC on the 31st. Temperature was never sent, so each provider used its default. Each call allowed up to 4,096 output tokens and carried the whole log so far, and each batch played each seed once. Hidden reasoning sat at each model's floor: off for Haiku, Luna and Qwen, “none” for DeepSeek, “minimal” for Sol and Gemini, and “low” for Sonnet.

The ranges on escape rates are Wilson 95% intervals: the range we can be 95% confident holds the true rate. Each comparison of my twenty runs with another player's is an exact McNemar test, two-sided. It sets aside the labyrinths where we both escaped or both failed, and asks whether the rest split more unevenly than coin flips would. Sixteen comparisons give luck sixteen chances, so the results carry Holm's correction across the sixteen, which raises the bar for each. Concordance counts the other players without the batch's own model, so Sol's baseline is never measured against Sol's reasoning runs, and its ranges come from redrawing the labyrinths at random 2,000 times (a bootstrap). The math for each, with this post’s numbers, is in the statistics note.