Skip to content

Dataset-Driven Channel Masks in Transformers for Multivariate Time Series

Home / Papers / Dataset-Driven Channel Masks in Transformers for Multivariate Time Series

Dataset-Driven Channel Masks in Transformers for Multivariate Time Series

Authors: Seunghan Lee, Taeyoung Park, Kibok Lee Date: 2026-04-21 Paper ID: openalex:2410.23222

Summary

This paper addresses the limitation of generic channel dependency modeling in Transformer-based time series forecasting by introducing Partial Channel Dependence (PCD). The authors propose integrating learnable channel masks (CMs) into attention mechanisms, which combine global similarity matrices with dataset-specific domain parameters. This approach allows Transformers to adaptively refine their channel-wise attention based on inherent dataset characteristics, improving performance across multiple backbones and multivariate time series tasks.

Key Contributions

  • Introduces Partial Channel Dependence (PCD) to incorporate dataset-specific priors into Transformer-based channel dependency modeling.
  • Proposes learnable Channel Masks (CMs) that integrate similarity matrices and domain parameters via element-wise multiplication into attention matrices.
  • Demonstrates consistent performance improvements across various time-series backbones and benchmark datasets.

Key Concepts

  • partial-channel-dependence: A method to refine channel dependency modeling in Transformers by integrating dataset-specific constraints into attention matrices via channel masks.

Archivist Review

I have approved ‘Partial Channel Dependence’ as a novel contribution to time-series Transformer architecture, as it shifts the paradigm from purely architecture-driven channel attention to incorporating learned, dataset-specific inductive biases. Other candidates, such as the specific ‘Channel Masks’ module, were rejected as they are implementation subcomponents of this broader concept.

Approved Concepts

  • Partial Channel Dependence: It provides a novel framework for incorporating dataset-specific constraints into standard channel attention mechanisms, moving beyond pure architectural modification.

Rejected Candidates

  • [concept] Channel Masks (channel-masks) - subcomponent_of_broader_mechanism: The concept of channel masks is a specific implementation detail; Partial Channel Dependence covers the methodological innovation.

Metadata & Links

url
https://arxiv.org/abs/2410.23222
paper_id
2410.23222
paper_source
openalex
domain
time-series
tags
time-seriesforecastingtransformer
architectures
datasets
concept_slugs
partial-channel-dependence
dataset_slugs
skill
TimeSeriesSkill
created_at
2026-04-24T05:50:39Z