Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

by   Haixu Wu, et al.

Extending the forecasting time is a critical demand for real applications, such as extreme weather early warning and long-term energy consumption planning. This paper studies the long-term forecasting problem of time series. Prior Transformer-based models adopt various self-attention mechanisms to discover the long-range dependencies. However, intricate temporal patterns of the long-term future prohibit the model from finding reliable dependencies. Also, Transformers have to adopt the sparse versions of point-wise self-attentions for long series efficiency, resulting in the information utilization bottleneck. Towards these challenges, we propose Autoformer as a novel decomposition architecture with an Auto-Correlation mechanism. We go beyond the pre-processing convention of series decomposition and renovate it as a basic inner block of deep models. This design empowers Autoformer with progressive decomposition capacities for complex time series. Further, inspired by the stochastic process theory, we design the Auto-Correlation mechanism based on the series periodicity, which conducts the dependencies discovery and representation aggregation at the sub-series level. Auto-Correlation outperforms self-attention in both efficiency and accuracy. In long-term forecasting, Autoformer yields state-of-the-art accuracy, with a 38 improvement on six benchmarks, covering five practical applications: energy, traffic, economics, weather and disease.


page 1

page 2

page 3

page 4


Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution

Long-term time-series forecasting (LTTF) has become a pressing demand in...

ETSformer: Exponential Smoothing Transformers for Time-series Forecasting

Transformers have been actively studied for time-series forecasting in r...

Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecasting

Transformer-based methods have shown great potential in long-term time s...

Are Transformers Effective for Time Series Forecasting?

Recently, there has been a surge of Transformer-based solutions for the ...

Split Time Series into Patches: Rethinking Long-term Series Forecasting with Dateformer

Time is one of the most significant characteristics of time-series, yet ...

Deep Switching Auto-Regressive Factorization:Application to Time Series Forecasting

We introduce deep switching auto-regressive factorization (DSARF), a dee...

Time Series Analysis of Big Data for Electricity Price and Demand to Find Cyber-Attacks part 2: Decomposition Analysis

In this paper, in following of the first part (which ADF tests using ACI...

Please sign up or login with your details

Forgot password? Click here to reset