Can Transformers predict system collapse in dynamical systems?
Can Transformers predict system collapse in dynamical systems?
Authors: Zheng-Meng Zhai, Celso Grebogi, Ying-Cheng Lai Date: 2026-05-05 Paper ID: openalex:2605.04024
Summary
This paper investigates whether Transformer architectures can serve as reliable digital twins for nonlinear dynamical systems, specifically focusing on their ability to extrapolate to unseen parameter regimes. By using the prediction of catastrophic collapse in dynamical systems as a benchmark, the authors compare Transformer performance against reservoir computing. The findings show that while reservoir computing succeeds in capturing the underlying physics of parameter-dependent transitions, Transformers consistently fail, suggesting limitations in their ability to model complex temporal structures across varying parameter regimes.
Key Contributions
- Evaluates the extrapolation capability of Transformer architectures for nonlinear dynamical systems through the benchmark task of predicting catastrophic collapse.
- Demonstrates that Transformers, despite their success in dynamics reconstruction, consistently fail to predict system collapse when tested on unseen parameter regimes.
- Identifies a performance gap between Transformer-based models and reservoir computing in capturing parameter-dependent temporal dynamics.
Links
Metadata & Links
- url
- https://arxiv.org/abs/2605.04024
- paper_id
- 2605.04024
- paper_source
- openalex
- domain
- time-series
- tags
- time-series-forecastingdynamical-systemsfoundation-models
- architectures
-
- datasets
-
- concept_slugs
-
- dataset_slugs
-
- skill
- TimeSeriesSkill
- created_at
- 2026-05-08T05:44:54Z