Adaptive Cyclic Learning Rate for Long-Term Time-Series Forecasting on Encoder-Decoder and Encoder-Only Deep Learning Model Architectures

Long-term time-series forecasting remains a critical challenge, with deep learning models often facing persistent training instability. This study reveals a critical insight: the effectiveness of a dynamic optimizer is fundamentally dependent on the underlying model architecture. We find that the autoregressive decoding process in encoder-decoder models clashes with cyclical learning rate adjustments, while the direct, nonautoregressive structure of encoder-only models proves highly compatible. To demonstrate this, we investigate the Adaptive Cyclic Learning Rate (ACLR) across five representative models from both encoder-decoder (Transformer, Informer) and encoder-only (iTransformer, FRNet, PatchTST) families on three benchmark datasets. The results are statistically significant, showing that ACLR consistently enhances stability and prediction accuracy for encoder-only models while inducing instability and performance degradation in their encoder-decoder counterparts. These findings provide a new guiding principle for practitioners, emphasizing that architectural compatibility should be a primary consideration when selecting an optimization strategy for long-term time series forecasting.