Is Less More? Do Deep Learning Forecasting Models Need Feature Reduction?

Time series forecasting is a powerful tool in data science that offers insights into future trends based on historical patterns. However, handling the multitude of features in time series data can be a challenge. Feature reduction is a crucial step in refining forecasting models, as it simplifies the feature set and retains predictive power. This process is like cleaning up a workspace to make it easier to find what you need. Feature reduction can reduce complexity, improve generalization, make the model easier to interpret, and increase computational efficiency. Most time series packages in Python for forecasting do not perform feature reduction automatically, so it is a step that typically needs to be handled on your own before using these packages. A practical example using real-world daily data from the Federal Reserve Economic Data (FRED) database demonstrates the importance of feature reduction. After making the data stationary, the number of variables that have at least a 95% correlation coefficient with another variable is counted. In this case, 260 out of 438 variables have a correlation of 95% or more with at least another variable, indicating significant multicollinearity in the dataset. To address this issue, feature evaluation and selection techniques can be used. Principal Component Analysis (PCA) is a common and effective dimensionality reduction technique that identifies linear relationships between features and retains the principal components that explain a predetermined percentage of the variance in the original dataset. In this example, PCA reduces the number of features from 438 to 76 while keeping 90% of the variance explained. Other techniques, such as the Temporal Fusion Transformers (TFT) model, can also be used for feature reduction. The TFT model includes the Variable Selection Network (VSN), which is specifically designed to automatically identify and focus on the most relevant features. By using these techniques, feature reduction can be an effective way to improve the performance of time series forecasting models.