DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories
DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories
Authors: Neemesh Yadav, Palakorn Achananuparp, Jing Jiang, EeāPeng Lim Date: 2026-04-22 Paper ID: openalex:2604.20443
Summary
DialToM is a new human-verified benchmark designed to distinguish between superficial mental state recognition and robust functional reasoning in LLMs. The authors introduce Prospective Diagnostic Forecasting to evaluate whether models can leverage inferred mental states to accurately predict subsequent social dialogue trajectories. Experimental results demonstrate a clear reasoning asymmetry, where most state-of-the-art LLMs successfully identify mental states but fail to utilize them for accurate future trajectory prediction.
Key Contributions
- Introduces DialToM, a human-verified benchmark for assessing both Literal and Functional Theory of Mind in LLMs.
- Proposes Prospective Diagnostic Forecasting to measure whether models can use mental states to predict dialogue trajectories.
- Demonstrates a reasoning asymmetry in current LLMs, where high performance on state identification does not correlate with the ability to perform functional trajectory forecasting.
Open Questions & Future Work
Key Concepts
- prospective-diagnostic-forecasting: A technique for evaluating Theory of Mind by requiring models to predict future dialogue trajectories conditioned on inferred mental-state profiles.
Archivist Review
I approved the prospective diagnostic forecasting technique as a reusable evaluation methodology for latent state-driven prediction, and the DialToM dataset as a core resource for this specialized benchmark. The open question was refined to highlight the bottleneck between surface-level pattern matching and true functional reasoning in social intelligence models. The review adhered to the scarcity constraints while ensuring the methodological contributions were preserved.
Approved Concepts
- Prospective Diagnostic Forecasting: It provides a rigorous way to test if a model can use inferred latent states (like ToM) for downstream trajectory prediction, rather than just performing pattern recognition.
Approved Open Questions
- Isolating Functional ToM Reasoning: This distinguishes between predictive linguistic pattern matching and actual agentic reasoning, a requirement for reliable social intelligence in future models.
Rejected Candidates
- [open_question] Isolating Functional Theory of Mind (
distinguishing-functional-tom-from-pattern-matching) - duplicate_existing: Renamed for brevity and to align with existing vault naming conventions.
Datasets
Links
Metadata & Links
- url
- https://arxiv.org/abs/2604.20443
- paper_id
- 2604.20443
- paper_source
- openalex
- domain
- nlp
- tags
-
- architectures
-
- datasets
- dialtom
- concept_slugs
- prospective-diagnostic-forecasting
- dataset_slugs
- dialtom
- skill
- TimeSeriesSkill
- created_at
- 2026-04-25T05:37:42Z