SparseTSF

Welcome to the official repository of the SparseTSF paper: "SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters"

Updates

🚩 News (2024.06) SparseTSF paper has been selected for an Oral presentation at ICML 2024.

🚩 News (2024.05) SparseTSF has been accepted as a paper at ICML 2024, receiving an average rating of 7 with confidence of 4.5.

Introduction

SparseTSF is a novel, extremely lightweight model for Long-term Time Series Forecasting (LTSF). At the heart of SparseTSF lies the Cross-Period Sparse Forecasting technique, which simplifies the forecasting task by decoupling the periodicity and trend in time series data.

Technically, it first downsamples the original sequences with constant periodicity into subsequences, then performs predictions on each downsampled subsequence, simplifying the original time series forecasting task into a cross-period trend prediction task.

Intuitively, SparseTSF can be perceived as a sparsely connected linear layer performing sliding prediction across periods

This approach yields two benefits: (i) effective decoupling of data periodicity and trend, enabling the model to stably identify and extract periodic features while focusing on predicting trend changes, and (ii) extreme compression of the model's parameter size, significantly reducing the demand for computational resources.

SparseTSF achieves near state-of-the-art prediction performance with less than 1k trainable parameters, which makes it 1 ~ 4 orders of magnitude smaller than its counterparts.

Additionally, SparseTSF showcases remarkable generalization capabilities (cross-domain), making it well-suited for scenarios with limited computational resources, small samples, or low-quality data.

Getting Started

Environment Requirements

To get started, ensure you have Conda installed on your system and follow these steps to set up the environment:

conda create -n SparseTSF python=3.8
conda activate SparseTSF
pip install -r requirements.txt

Data Preparation

All the datasets needed for SparseTSF can be obtained from the Google Drive provided in Autoformer. Create a separate folder named ./dataset and place all the CSV files in this directory. Note: Place the CSV files directly into this directory, such as "./dataset/ETTh1.csv"

Training Example

You can easily reproduce the results from the paper by running the provided script command. For instance, to reproduce the main results, execute the following command:

sh run_all.sh

Similarly, you can specify separate scripts to run independent tasks, such as obtaining results on etth1:

sh scripts/SparseTSF/etth1.sh

Usage on Your Data

SparseTSF relies on the inherent periodicity in the data. If you intend to use SparseTSF on your data, please first ascertain whether your data exhibits periodicity, which can be determined through ACF analysis.

We provide an example in the ACF_ETTh1.ipynb notebook to determine the primary period of the ETTh1 dataset. You can utilize it to ascertain the periodicity of your dataset and set the period_len parameter accordingly.

It is important to note a special case where the dataset's period is excessively large. For instance, in ETTm1, due to dense sampling, its period is 144. Resampling with too large a period results in very short subsequences with sparse connections, leading to underutilization of information. In such cases, setting period_len to [2-6], i.e., adopting a denser sparse strategy, can be beneficial. For more details, refer to the discussion in Appendix C.2.

Citation

If you find this repo useful, please cite our paper.

@article{lin2024sparsetsf,
  title={SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters},
  author={Lin, Shengsheng and Lin, Weiwei and Wu, Wentai and Chen, Haojun and Yang, Junjie},
  journal={arXiv preprint arXiv:2405.00946},
  year={2024}
}