Suggested Citation:
Zhao, Jingyuan, Fulin Chu, Lili Xie, Yunhong Che, Yuyan Wu, Andrew Burke (2026)

A Survey of Transformer Networks for Time Series Forecasting

Time-series data support critical functions in forecasting, anomaly detection, resource optimization, and real-time decision making across finance, healthcare, energy systems, and networked computing. Classical statistical approaches and early deep-learning architectures (RNNs, LSTMs, CNNs) have achieved notable progress, yet they exhibit structural limitations: recurrent models struggle with long-range dependencies, convolutional models require deep stacking to enlarge receptive fields, and both scale suboptimally to high-dimensional and high-volume data. Transformer architectures—characterized by global attention mechanisms, flexible temporal receptive fields, and growing adoption within foundation-model paradigms—have consequently gained increasing prominence for modern time-series analysis. Drawing on a systematic review of IEEE Xplore, ACM Digital Library, and Scopus (2020–2025), this survey offers a unified, theoretically grounded synthesis of Transformer-based methods for time-series learning. The survey summarizes defining characteristics of time-series data and analyzes core architectural elements, including attention formulations, encoder–decoder structures, hyperparameter design, and domain-specific adaptations. An architecture-centered and task-aware taxonomy is presented to organize recent advances across forecasting, representation learning, anomaly detection, and multimodal fusion. Persistent challenges—spanning computational scalability, data-efficiency constraints, distributional heterogeneity, overfitting risks, hyperparameter instability, interpretability, and reproducibility—are examined in depth. A forward-looking research agenda is outlined, highlighting opportunities in physics-informed architectural design, hybrid neural–mechanistic modeling, resource-efficient real-time inference, multi-resolution spatiotemporal learning, and emerging human–AI collaborative paradigms. By consolidating these methodological developments, this survey aims to provide a structured reference point for ongoing research on Transformer models for time-series machine learning.