Skip to content

Can Transformers predict system collapse in dynamical systems?

Home / Papers / Can Transformers predict system collapse in dynamical systems?

Can Transformers predict system collapse in dynamical systems?

Authors: Zheng-Meng Zhai, Celso Grebogi, Ying-Cheng Lai Date: 2026-05-05 Paper ID: openalex:2605.04024

Summary

This paper investigates whether Transformer architectures can serve as reliable digital twins for nonlinear dynamical systems, specifically focusing on their ability to extrapolate to unseen parameter regimes. By using the prediction of catastrophic collapse in dynamical systems as a benchmark, the authors compare Transformer performance against reservoir computing. The findings show that while reservoir computing succeeds in capturing the underlying physics of parameter-dependent transitions, Transformers consistently fail, suggesting limitations in their ability to model complex temporal structures across varying parameter regimes.

Key Contributions

  • Evaluates the extrapolation capability of Transformer architectures for nonlinear dynamical systems through the benchmark task of predicting catastrophic collapse.
  • Demonstrates that Transformers, despite their success in dynamics reconstruction, consistently fail to predict system collapse when tested on unseen parameter regimes.
  • Identifies a performance gap between Transformer-based models and reservoir computing in capturing parameter-dependent temporal dynamics.

Metadata & Links

url
https://arxiv.org/abs/2605.04024
paper_id
2605.04024
paper_source
openalex
domain
time-series
tags
time-series-forecastingdynamical-systemsfoundation-models
architectures
datasets
concept_slugs
dataset_slugs
skill
TimeSeriesSkill
created_at
2026-05-08T05:44:54Z