Skip to content

Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment

Home / Papers / Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment

Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment

Authors: Sayantan Kumar, Shahriar Noroozizadeh, Juyong Kim, Jeremy C. Weiss Date: 2026-05-14 Paper ID: openalex:2605.15168

Summary

This paper presents a retrieval-augmented multimodal alignment framework designed to reconstruct precise clinical timelines by combining unstructured narratives with structured EHR data. The approach utilizes a graph-based, multi-step process that builds a temporal scaffold from narrative anchor events and calibrates them using retrieved structured tabular records as temporal evidence. Experiments on the i2m4 benchmark demonstrate that integrating these modalities significantly enhances absolute timestamp accuracy and temporal concordance over text-only baselines.

Key Contributions

  • Introduces a graph-based, retrieval-augmented multimodal alignment framework for precise clinical timeline reconstruction.
  • Achieves superior absolute timestamp accuracy (AULTC) and improved temporal concordance compared to unimodal text-only approaches on the i2m4 benchmark.
  • Quantifies the clinical data gap, revealing that 34.8% of text-derived events are missing from structured tabular records.

Open Questions & Future Work

Key Concepts

Archivist Review

I approved the core retrieval-augmented multimodal alignment concept as a novel mechanism for longitudinal clinical data integration. The open question regarding its generalization was approved because the current reliance on sepsis-specific, limited-scale benchmarks is a known obstacle in the field. Other items were rejected for being domain-specific benchmarks or overly generic future work.

Approved Concepts

  • Retrieval-augmented multimodal alignment: It serves as a novel paradigm for synchronizing heterogeneous data sources (narratives and structured EHRs) to enhance temporal fidelity.

Approved Open Questions

  • Generalization of Clinical Timeline Reconstruction: Scaling beyond constrained, sepsis-specific benchmarks is essential to validate whether the proposed multimodal scaffold is a robust solution for heterogeneous patient trajectories.

Rejected Candidates

  • [dataset] i2m4 benchmark (i2m4-benchmark) - not_reusable: Routine benchmark specific to the paper that does not justify a standalone vault entry.

Metadata & Links

url
https://arxiv.org/abs/2605.15168
paper_id
2605.15168
paper_source
openalex
domain
nlp
tags
clinical-predictionmultimodal-learningretrieval-augmented-generation
architectures
datasets
concept_slugs
retrieval-augmented-multimodal-alignment
dataset_slugs
skill
TimeSeriesSkill
created_at
2026-05-17T06:09:24Z