Skip to content

TSFM Normalization Architectural Sensitivity

Home / Open Questions / TSFM Normalization Architectural Sensitivity

Background: Time series foundation models often struggle with domain-specific scale variations and non-stationary data, which can significantly hinder zero-shot transfer capabilities. While normalization techniques like RevIN have shown promise, the optimal integration of these methods across different architectural paradigms (e.g., LLM-based vs. distribution-based) remains an active area of investigation.

Question / Future Work: Determine how the interaction between specific normalization layers (like RevIN) and model architectural choices (point-forecast, distribution, or LLM-based) affects the loss function sensitivity and zero-shot performance in foundation models.

Why It Matters: Clarifying this architectural sensitivity is critical for designing future time-series foundation models that robustly handle distribution shifts.

Evidence: Yet its effective utilization depends on architectural design choices and optimization objective, particularly with respect to training loss scale sensitivity and model type (point-forecast-, distribution- or LLM-based models).

Metadata & Links