Evaluating Counterfactual Forecasting without Ground Truth
Background: Time series forecasting models often struggle to evaluate predictive quality in counterfactual settings, where true future values remain inherently unobserved.
Question / Future Work: Future work is required to develop robust evaluation metrics for forecasting models when the ground truth for future trajectories is unavailable or non-existent, particularly for varying lengths of forecast sequences. The current model-based evaluation approach relies on semantic consistency metrics, but there is a recognized need to improve the reliability and breadth of assessment for these counterfactual, ground-truth-free environments.
Why It Matters: Lack of ground truth is a fundamental bottleneck in counterfactual time series forecasting. Improving evaluation methodologies is essential for establishing reliable benchmarking and model validation in scenarios where true outcomes cannot be realized.
Metadata & Links
- created_at
- 2026-05-17T06:07:04Z