S2TX: Cross-Attention Multi-Scale State-Space Transformer for Time Series Forecasting
S2TX: Cross-Attention Multi-Scale State-Space Transformer for Time Series Forecasting
Authors: Zihao Wu, Juncheng Dong, Haoming Yang, Vahid Tarokh Date: 2026-04-21 Paper ID: openalex:2502.11340
Summary
S2TX is a novel time series forecasting architecture that addresses the isolation of variate processing and multiscale representation learning. By combining the long-range modeling capabilities of Mamba with local window-based Transformer attention, the framework captures comprehensive temporal dependencies. An integrated cross-attention mechanism facilitates synergistic communication between global and local features, resulting in improved accuracy and efficient memory usage across diverse benchmarks.
Key Contributions
- Proposes S2TX, a hybrid architecture integrating Mamba for long-range cross-variate context and Transformers for local window attention.
- Employs cross-attention mechanisms to enable variate-level interactions and bridge global-local feature representations.
- Achieves state-of-the-art forecasting performance across six standard benchmarks while maintaining low computational memory overhead.
Open Questions & Future Work
Archivist Review
I approved two open questions that address significant architectural bottlenecks in multivariate time series forecasting, specifically regarding local-variate correlation and multi-scale temporal modeling. I rejected the S2TX architecture itself as it is a specific hybrid of existing state-space models and Transformers, rather than a sufficiently novel, reusable concept for the vault.
Approved Open Questions
- Modeling Local Cross-Variate Correlations: This represents a significant architectural bottleneck in multivariate time series forecasting that prevents models from capturing the complex, localized interaction patterns between different variables.
- Learning Intermediate Time Scales: Capturing intermediate scales is essential for extremely long sequences where significant temporal variations occur between the extremes of global and local contexts.
Rejected Candidates
- [concept] S2TX Architecture (
s2tx-architecture) - not_novel: The architecture is a specific implementation of existing building blocks (Mamba/Transformer) rather than a novel, generally reusable primitive.
Links
Metadata & Links
- url
- https://arxiv.org/abs/2502.11340
- paper_id
- 2502.11340
- paper_source
- openalex
- domain
- time-series
- tags
- forecastingattentionmulti-scale
- architectures
- TransformerMamba
- datasets
-
- concept_slugs
-
- dataset_slugs
-
- skill
- TimeSeriesSkill
- created_at
- 2026-04-24T05:49:49Z