Dataset-Driven Channel Masks in Transformers for Multivariate Time Series
Dataset-Driven Channel Masks in Transformers for Multivariate Time Series
Authors: Seunghan Lee, Taeyoung Park, Kibok Lee Date: 2026-04-21 Paper ID: openalex:2410.23222
Summary
This paper addresses the limitation of generic channel dependency modeling in Transformer-based time series forecasting by introducing Partial Channel Dependence (PCD). The authors propose integrating learnable channel masks (CMs) into attention mechanisms, which combine global similarity matrices with dataset-specific domain parameters. This approach allows Transformers to adaptively refine their channel-wise attention based on inherent dataset characteristics, improving performance across multiple backbones and multivariate time series tasks.
Key Contributions
- Introduces Partial Channel Dependence (PCD) to incorporate dataset-specific priors into Transformer-based channel dependency modeling.
- Proposes learnable Channel Masks (CMs) that integrate similarity matrices and domain parameters via element-wise multiplication into attention matrices.
- Demonstrates consistent performance improvements across various time-series backbones and benchmark datasets.
Key Concepts
- partial-channel-dependence: A method to refine channel dependency modeling in Transformers by integrating dataset-specific constraints into attention matrices via channel masks.
Archivist Review
I have approved ‘Partial Channel Dependence’ as a novel contribution to time-series Transformer architecture, as it shifts the paradigm from purely architecture-driven channel attention to incorporating learned, dataset-specific inductive biases. Other candidates, such as the specific ‘Channel Masks’ module, were rejected as they are implementation subcomponents of this broader concept.
Approved Concepts
- Partial Channel Dependence: It provides a novel framework for incorporating dataset-specific constraints into standard channel attention mechanisms, moving beyond pure architectural modification.
Rejected Candidates
- [concept] Channel Masks (
channel-masks) - subcomponent_of_broader_mechanism: The concept of channel masks is a specific implementation detail; Partial Channel Dependence covers the methodological innovation.
Links
Metadata & Links
- url
- https://arxiv.org/abs/2410.23222
- paper_id
- 2410.23222
- paper_source
- openalex
- domain
- time-series
- tags
- time-seriesforecastingtransformer
- architectures
-
- datasets
-
- concept_slugs
- partial-channel-dependence
- dataset_slugs
-
- skill
- TimeSeriesSkill
- created_at
- 2026-04-24T05:50:39Z