The article presents a novel method for time series forecasting, the Sparse Vector Quantized FFN-Free Transformer (Sparse-VQ). This approach leverages sparse vector quantization and Reverse Instance Normalization to reduce noise and capture sufficient data for forecasting. It offers an alternative to the Feed-Forward layer in transformer architectures, reducing the parameter count, enhancing computational efficiency, and preventing overfitting. The Sparse-VQ Transformer outperforms leading models in tests on ten benchmark datasets. The method can also be integrated with existing transformer-based models to improve their performance.
Publication date: 9 Feb 2024
Project Page: https://anonymous.4open.science/r/Sparse-VQ-DC28
Paper: https://arxiv.org/pdf/2402.05830
