Skip to content

Cognitive load in multimodal annotation

Home / Open Questions / Cognitive load in multimodal annotation

Background: Annotation of multi-modal robotic data, particularly the inclusion of time-series signals alongside video, has been shown to increase the cognitive load for human annotators compared to vision-only approaches.

Question / Future Work: It remains an open question whether the observed increase in annotation time when incorporating time-series data is an inherent limitation of human cognitive processing of multi-modal information or if it can be mitigated through interface design improvements. Systematic investigation is required to determine how to present proprioceptive signals, force-torque measurements, and visual data simultaneously without increasing the cognitive burden on the annotator.

Why It Matters: Understanding the trade-off between annotation cognitive load and temporal precision is critical for the development of scalable, human-in-the-loop data labeling systems for complex robotic tasks.

Metadata & Links