Skip to content

Anticipating Innovation Using Large Language Models

Home / Papers / Anticipating Innovation Using Large Language Models

Anticipating Innovation Using Large Language Models

Authors: Enrico Maria Fenoaltea, Filippo Santoro, Giordano De Marzo, Segun Taofeek Aroyehun, Andrea Tacchella Date: 2026-05-06 Paper ID: openalex:2605.04875

Summary

This paper introduces TechToken, a transformer-based model designed to forecast innovation by modeling the emergence of new technological combinations. By treating International Patent Classification (IPC) codes as words in a vocabulary, the model learns the semantic structure of technological development. The authors demonstrate that linguistic shifts in patent descriptions provide long-range predictive signals for technological convergence, effectively anticipating combinations decades before they occur. TechToken achieves superior performance in patent-related representation tasks compared to existing state-of-the-art approaches.

Key Contributions

  • Introduces TechToken, a transformer-based model that treats IPC codes as language tokens to capture technological semantic evolution.
  • Demonstrates that forthcoming technological combinations leave detectable linguistic signals in patent corpora decades in advance.
  • Defines context similarity of technology embeddings as a metric for predicting first-time technological pairings.

Open Questions & Future Work

Key Concepts

  • techtoken: A transformer-based model that embeds classification codes as semantic tokens to forecast technological innovation and convergence.

Archivist Review

The paper introduces an innovative way to treat hierarchical classification systems (IPC) as semantic tokens for predictive modeling. I have approved ‘TechToken’ as a core concept because it generalizes well to other classification-heavy domains, and ‘Predicting Higher-Order Technological Combinations’ as a critical open question because it addresses a fundamental structural limitation in how innovation is currently forecasted.

Approved Concepts

  • TechToken: It represents a novel approach to treating categorical classification systems as semantic tokens for time-series forecasting of innovation.

Approved Open Questions

  • Predicting Higher-Order Technological Combinations: This is a fundamental limitation in current technology forecasting; most real-world innovation is non-reducible to binary interactions. Advancing this is critical for modeling complex emergent systems.

Metadata & Links

url
https://arxiv.org/abs/2605.04875
paper_id
2605.04875
paper_source
openalex
domain
nlp
tags
architectures
datasets
concept_slugs
techtoken
dataset_slugs
skill
TimeSeriesSkill
created_at
2026-05-09T05:58:02Z