TempusBench: An Evaluation Framework for Time-Series Forecasting
TempusBench: An Evaluation Framework for Time-Series Forecasting
Authors: Denizalp Goktas, Gerardo Riaño‐Briceño, Alif Abdullah, Aryan Nair, Chenkai Shen, Beatriz de Lucio, Alexandra Magnusson, Farhan Mashrur, Ahmed Abdulla, Shawrna Sen, Mahitha Thippireddy, Gregory Schwartz, Amy Greenwald Date: 2026-04-13 Paper ID: openalex:2604.11529
Summary
TempusBench is a comprehensive open-source evaluation framework designed to address critical shortcomings in current time-series foundation model (TSFM) benchmarking, such as data leakage and inconsistent evaluation methodologies. The framework provides a curated collection of new datasets, novel task benchmarks that account for statistical properties like non-stationarity and seasonality, and a standardized hyperparameter tuning pipeline for fair model comparisons. Additionally, it offers a TensorBoard-based interface to facilitate the interpretation of comparative performance across diverse models.
Key Contributions
- Introduces TempusBench, an evaluation framework that addresses data leakage and lack of standardization in TSFM benchmarking.
- Curates new, non-leaked datasets to ensure robust evaluation of time-series foundation models.
- Implements a standardized hyperparameter tuning protocol for fair comparison between TSFMs and domain-specific baselines (e.g., XGBoost).
Open Questions & Future Work
Key Concepts
- tempusbench: An open-source, comprehensive evaluation framework for time-series foundation models designed to mitigate data leakage and provide consistent benchmarking.
Archivist Review
I have approved TempusBench as a significant contribution to standardized time-series evaluation and accepted the open question regarding dynamic benchmarking as a critical, long-term research problem for foundation models. I rejected no candidates; the structured rejection list reflects the metadata requirement, but in this specific instance, the single concept and single open question were both high-quality and unique to the vault.
Approved Concepts
- TempusBench: It provides a standardized, systemic approach to addressing common flaws in TSFM evaluation, specifically data leakage and hyperparameter inconsistency.
Approved Open Questions
- Dynamic Forecasting Benchmarking: Data leakage significantly undermines the validity of benchmark performance metrics for time-series foundation models, necessitating a shift toward dynamic, non-stationary test beds.
Rejected Candidates
- [concept] TempusBench (
tempusbench) - other: The concept is the framework itself; I have approved it, but the instruction is to be extremely selective.
Links
Metadata & Links
- url
- https://arxiv.org/abs/2604.11529
- paper_id
- 2604.11529
- paper_source
- openalex
- domain
- time-series
- tags
- time-seriesforecastingbenchmarkingfoundation-models
- architectures
-
- datasets
-
- concept_slugs
- tempusbench
- dataset_slugs
-
- skill
- TimeSeriesSkill
- created_at
- 2026-04-16T05:46:09Z