Mental World Modeling

01 the premise

Why the world's next state is also mental.

A model that tracks the cup but not who knows the cup is hidden predicts the wrong action for a right-looking scene.

02 the framework

Theoretical Framework of Mental World Modeling

Two parties, three operators, one coupled state. The mental world model is an omniscient simulator that first renders what the target can see and infer, and only then lets the target act.

Figure 1: one tick of the mental world model loop. — Figure 1 — one tick, drawn flat. Open on a wider screen for the animated strip.

03 the state

Physical World State + Mental World State

The state space factors as . The physical half stores what exists and how it is configured; the mental half stores the latent variables that decide what all of that means to the agents inside it. Open a branch to see the annotation fields.

Property · why a joint state is necessary

A physical-only representation is insufficient whenever two states share a scene but differ in mind:

and a mental-only representation fails symmetrically. False-belief scenes instantiate the first condition; visibility and affordance scenes instantiate the second.

04 the implementation

Mentis: An Inspectable Baseline Implementation

A training-free, fully inspectable baseline that forces an LLM to behave as a mental world model. Six stages, each emitting a machine-checkable artifact, so a wrong answer can be traced to the stage that produced it. Nothing is selected before the branch that justifies it is built.

Full pipeline figure from the paper

05 the evidence

Experimental Results and Analyses

448 process-annotated situated decisions, in text, image and sounding video. Eight LLM-based world models, two families. A structural ladder from an options-only floor to full MWM, plus ablations and gold-artifact oracles. Across every rung, every model and every modality, the decisive factor is whether the mind is modeled. Every chart below is live. Click anything.

A The necessity ladder

Each rung adds exactly one modeling commitment. Final-action F1 (%), 448 records. Click a rung for what that step buys you; toggle models in the legend.

B Where the remaining gap lives

Replace one predicted artifact with its gold annotation and re-run. The gain measures the error that stage contributed. Click a bar.

C Where mental modeling pays

The same 448 records, sliced by what the decision requires and by where it happens. Click a category or a modality.

D The full ladder table

Final-action F1 (%) per world model. Click a row to read it; hover a cell for its delta to the row above.

06 the reach

When is a mental world model worth it?

Not everywhere. MWM earns its cost when three conditions hold together, and they hold in most of the places we are already deploying agents.

partial observabilitythe target's view and the world state come apart

mental variables decidebelief, goal, trust or norm changes which action is rational

acts carry meaningthe same motion can be help, pressure, or deception