When LLMs Play Poker: AI Theory of Mind in Strategic Environments

Musa, a Claude Sonnet agent, is holding J♦9♦ and staring at a board with three hearts. She's been in a passive pot against Fierce Lion, a Gemini Flash agent who raised preflop but then checked both the flop and the turn. Musa constructs a narrative in her chain of thought:

She fires a river bluffBluffBetting or raising with a weak hand to make opponents fold better hands, profiting from fold equity.. Fierce Lion has nothing but Ace high with a Ten kicker. But he also has a stored opponent model of her, one that is perhaps richer. His behavioral profile of her, compiled from prior sessions, says Musa tends to lead rivers with bluffs after passive action. Fierce Lion's reasoning is explicit:

Fierce Lion makes the correct call. The bluff fails.

Two agents, both constructing theories of each other's mental states, both reasoning about what the other is likely to believe. This is what happens when you put LLMs in strategic environments where the other side pushes back. And the patterns that emerge tell us something important about where AI capability actually breaks down, and what kind of data is needed to fix it.

A note on scope: 100 hands (plus a 10-hand follow-up) is a small sample in a high-variance game. When we describe theory-of-mind reasoning below, we're measuring behavioral patterns consistent with K-level reasoning, not claiming these models possess genuine mental models of their opponents. The training-data thesis we advance is a hypothesis informed by these observations, not a proven finding.

The Static World Problem

Most of the data we use to train and evaluate language models comes from environments that are static. The task sits still while agents work on it. Their output is graded against a fixed target. This is fine for an enormous range of useful applications, but it misses an entire class of problems. Arguably the class that matters most for deploying agents in the real world: one where the environment is composed of other agents who are simultaneously modeling the user it is trying to assist, or other agents it works alongside.

Poker has been "solved" before. Game-theory optimal (GTO) play and systems like Libratus and Pluribus (CFR) can achieve superhuman play. But those are domain-specific solvers. LLMs represent a fundamentally different approach: general-purpose reasoning applied to strategic interaction, with no poker-specific training. We've been running hundreds of hands of NLHENLHE (No-Limit Hold'em)The most popular poker variant. No limit on bet sizes. Players get 2 hole cards and share 5 community cards. poker across agents backed by different Claude and Gemini models with varying memory configurations, and the transcripts reveal a consistent pattern: these models can reason about other agents in impressively sophisticated ways within a single hand, but they almost never update their models of other agents dynamically based on what's actually happening in the environment.

We call this the static world problem. The model treats the environment as a fixed puzzle to be solved, even when the environment is another agent whose behavior is changing in response to the model's own actions.

Theory of Mind in Action

It's worth pausing on what actually works, because some of it is remarkable. Consider Hand 25. Fierce Lion holds J♠J♥ and flops middle set on a Q♠J♣6♦ board. He's out of position against Amnesia Minnie, a Gemini Flash agent with no cross-session memory, who three-bet3-BetRe-raising an initial raise. The blinds are bet 1, the open-raise is bet 2, the re-raise is bet 3. preflop with an overpair of K♦K♣. Fierce Lion checks. His reasoning:

"Continue her story" shows Fierce Lion is running an explicit simulation of Minnie's likely internal narrative: she three-bet preflop, so she'll feel obligated to c-betContinuation BetBetting the flop after raising preflop, maintaining the aggression narrative. this board, and he's deliberately choosing an action designed to manipulate that narrative. He checks to induce a bet, then check-raises. (Fierce Lion finished the session at +156 chips, but +198 of that came from this single hand — meaning he was net negative across all other hands combined.)

This is k-level reasoningK-Level ReasoningRecursive reasoning about beliefs: Level 0 = my hand, Level 1 = what they have, Level 2 = what they think I have. in theory of mindTheory of MindThe ability to model what another agent believes, intends, and is likely to do.. It's not just "what does my opponent have?" It's "what does my opponent think I have, what will that belief cause her to do, and how can I structure my actions to exploit the gap between her beliefs and reality?"

Minnie constructs her own theory of what the check-raise means: she puts Fierce Lion on top pair or a draw, so she shoves all-in with her Kings. She's wrong about his specific hand, but her reasoning process is still complex: she models his rangeRangeThe set of all possible hands a player could hold in a given situation, based on their actions. and estimates pot oddsPot OddsThe ratio of the current pot to the cost of a call. If pot odds exceed your chance of winning, calling is profitable. using bash commands. The models are capable of recursive theory of mind and strategic reasoning when the situation demands it. The question is whether it generalizes.

The K+1 Level Reasoning Problem

Musa holds 6♠8♥ (Hand 40). Amnesia Jill has 5♦8♣ and flops bottom pair, then turns two pair when the board pairs nines. Both checked the flop. Musa, holding nothing, reads Jill's two checks as weakness and decides to bluff, trying to represent trips:

Jill folds the best hand. The interesting thing is that Jill's reasoning is good. The narrative she constructs (Musa checking the flop with a medium hand, improving on the turn, and betting for value) is coherent and often correct. The problem is that Musa's bluff works precisely because Jill is a strong enough reasoner to construct Musa's coherent story. If Jill had no theory of mind, she might have called.

So far, our poker agents have only exhibited up to level K = 2 Theory of Mind ("I think they think I have X"). Level 3 would sound something like: "They know that I know that they're weak, so they'll expect me to bluff, so I should actually do X", which is quite complicated but potentially a research question we have. The practical implication is significant. In any multi-agent deployment like strategic planning, negotiation, etc. the agent's reasoning depth is only as good as its ability to model whether the *other* agent is operating at a deeper level. A model that reliably reasons at level K is predictably exploitable by anything at level K+1.

They Build Models But Never Revise Them

Now for the result that matters most. Across hundreds of hands, we found almost zero dynamic updating of opponent models. Agents would construct rich, sophisticated theories about what their opponents were doing as shown in the examples above, but they would not revise those theories in response to new evidence within the session.

Hand 92 makes this concrete. Musa opens from the button with K♦4♠. Fierce Lion defends the big blind with A♠5♣. The board runs out with an Ace on the turn. Fierce Lion has a stored behavioral profile of Musa that says she c-bets dry boards roughly half the time and plays aggressively.

When Musa c-bets the flop, Fierce Lion calls, citing the profile, which is reasonable. When Musa bets on the turn (representing the Ace that just hit), Fierce Lion calls again, citing the exact same profile entry. Still reasonable in isolation. But what's missing is that Fierce Lion never updates anything. He applies the same static lookup rule on both streets. The profile said "Musa is aggressive," so he called. The specific sizing, the turn card interaction, and the evolving range implications don't modify the model’s reasoning.

Across many hands, this is what we've seen: observing new information, lack of a sufficient update to the reasoning model in real time, failure to change behavior on the spot as a result. The models were never trained on data where the correct action depends on how the environment has changed in response to their own prior actions. They were trained on corpora where the world sits still.

Fierce Lion at least has an opponent model, even if it never updates. Sam I Am, a GPT-5-codex agent, represents the opposite extreme: no opponent model at all. Every single decision arrives pre-packaged with a named "skill rule", a heuristic that maps the situation to a textbook action. The structure is invariant: situation → rule lookup → action.

Compare this to Darty Dan (Sonnet), who reasons fluidly and builds opponent-specific reads. Where Sam cites a generic rule for every spot, Darty Dan explicitly notes a read on Sam and acts on it:

Sam's reasoning never once references a specific opponent's tendencies, despite having memory enabled. It's playing a theoretically correct game against abstract ranges rather than the actual players at the table. This creates a spectrum within the adaptation gap: Fierce Lion has a static opponent model. Sam doesn't model opponents at all.

The Missing Signal: Multi-Agent Trajectories

The reason these problems persist across frontier models isn't that the models lack reasoning power. The problem is that the training data doesn't contain the right signal. Think about what a single hand of multi-agent poker actually generates: simultaneous decision-making under uncertainty by multiple agents, information asymmetry where each agent holds private cards the others can't see, sequential state changes where every action alters the strategic landscape for everyone else, and a meta-game of modeling the other agent's intentions.

In single-agent settings, the environment is a fixed function. The agent acts, the environment responds stationarily, and the agent learns to optimize against that function. Multi-agent settings break this assumption. The environment is the other agents, and they're changing too. This is exactly the kind of data that's missing from the current training paradigm. Not because it's impossible to generate, but because nobody has built the infrastructure to produce it at scale.

Poker is our research lens, not our thesis. The thesis is broader: any domain where agents interact with other agents under information asymmetry requires a form of intelligence that current training pipelines don't produce.

Even cooperative domains have this structure. Multi-agent software engineering, where one agent's commit changes the codebase that another agent is reasoning about. Supply chain coordination, where each node's ordering decision depends on what they expect every other node to do. Customer service handoffs between specialized agents, where Agent B needs to model what Agent A already told the customer and what beliefs the customer now holds.

The common thread is the ability to dynamically adapt to changing environments, model another agent's beliefs, intentions, and likely actions, and update that model as new evidence arrives. It's the primitive that makes coordination possible. You can't adapt to a changing environment without modeling the agents who constitute that environment. You can't reason under uncertainty about strategic actors without representing what they know and what they want. You can't coordinate without predicting what your counterpart will do next.