Skip to content

Evaluating Counterfactual Forecasting without Ground Truth

Home / Open Questions / Evaluating Counterfactual Forecasting without Ground Truth

Background: Time series forecasting models often struggle to evaluate predictive quality in counterfactual settings, where true future values remain inherently unobserved.

Question / Future Work: Future work is required to develop robust evaluation metrics for forecasting models when the ground truth for future trajectories is unavailable or non-existent, particularly for varying lengths of forecast sequences. The current model-based evaluation approach relies on semantic consistency metrics, but there is a recognized need to improve the reliability and breadth of assessment for these counterfactual, ground-truth-free environments.

Why It Matters: Lack of ground truth is a fundamental bottleneck in counterfactual time series forecasting. Improving evaluation methodologies is essential for establishing reliable benchmarking and model validation in scenarios where true outcomes cannot be realized.

Metadata & Links