Subscribe

Issue No.03 - March (2011 vol.23)

pp: 350-359

Dymitr Ruta , British Telecom Group CTO, Ipswich

Bogdan Gabrys , Bournemouth University, Poole

Christiane Lemke , Bournemouth University, Poole

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2010.137

ABSTRACT

Rapidly evolving businesses generate massive amounts of time-stamped data sequences and cause a demand for both univariate and multivariate time series forecasting. For such data, traditional predictive models based on autoregression are often not sufficient to capture complex nonlinear relationships between multidimensional features and the time series outputs. In order to exploit these relationships for improved time series forecasting while also better dealing with a wider variety of prediction scenarios, a forecasting system requires a flexible and generic architecture to accommodate and tune various individual predictors as well as combination methods. In reply to this challenge, an architecture for combined, multilevel time series prediction is proposed, which is suitable for many different universal regressors and combination methods. The key strength of this architecture is its ability to build a diversified ensemble of individual predictors that form an input to a multilevel selection and fusion process before the final optimized output is obtained. Excellent generalization ability is achieved due to the highly boosted complementarity of individual models further enforced through cross-validation-linked training on exclusive data subsets and ensemble output postprocessing. In a sample configuration with basic neural network predictors and a mean combiner, the proposed system has been evaluated in different scenarios and showed a clear prediction performance gain.

INDEX TERMS

Time series forecasting, combining predictors, regression, ensembles, neural networks, diversity.

CITATION

Dymitr Ruta, Bogdan Gabrys, Christiane Lemke, "A Generic Multilevel Architecture for Time Series Prediction",

*IEEE Transactions on Knowledge & Data Engineering*, vol.23, no. 3, pp. 350-359, March 2011, doi:10.1109/TKDE.2010.137REFERENCES

- [1] M. Aiolfi and A. Timmermann, "Persistence in Forecasting Performance and Conditional Combination Strategies,"
J. Econometrics, vol. 127, nos. 1/2, pp. 31-53, 2006.- [2] V. Assimakopoulos and K. Nikolopoulos, "The Theta Model: A Decomposition Approach to Forecasting,"
Int'l J. Forecasting, vol. 16, no. 4, pp. 521-530, 2000.- [3] D.W. Bunn, "A Bayesian Approach to the Linear Combination of Forecasts (1975),"
Operational Research Quarterly, vol. 26, no. 2, pp. 325-329, 1975.- [4] M. Casdagli, "Nonlinear Prediction of Chaotic Time Series,"
Physica, vol. 35, pp. 335-356, 1989.- [5] E.S. Gardner, "Exponential Smoothing: The State of the Art—Part II,"
Int'l J. Forecasting, vol. 22, no. 4, pp. 637-666, 2006.- [6] C.L. Giles, S. Lawrence, and A.C. Tsoi, "Noisy Time Series Prediction Using Recurrent Neural Networks and Grammatical Inference,"
Machine Learning, vol. 44, no. 1, pp. 335-356, 2001.- [7] T. Dietterich and R. Michalski, "Learning to Predict Sequences,"
Machine Learning: An Artificial Intelligence Approach, R. Michalski, J. Carbonell, and T. Mitchell, eds., vol. 2, pp. 63-106, Morgan Kaufmann, 1986.- [8] S. Hansen and P. Salamon, "Neural Network Ensembles,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 12, pp. 993-1001, Oct. 1990.- [9] R.J. Hyndman and B. Billah, "Unmasking the Theta Method,"
Int'l J. Forecasting, vol. 19, no. 2, pp. 287-290, 2003.- [10] C. Igel and M. Husken, "Improving the RPROP Learning Algorithm,"
Proc. Second Int'l Computer Science Conventions (ICSC) Symp. Neural Computation, pp. 115-121, 2000.- [11] S. Makridakis and M. Hibon, "The M3-Competition: Results, Conclusions and Implications,"
Int'l J. Forecasting, vol. 16, no. 4, pp. 451-476, 2000.- [12] "Delft Center for Systems and Control—Software," http://www. dcsc.tudelft.nl/researchsoftware , 2007.
- [13] A. Harvey, "Forecasting with Unobserved Components Time Series Models,"
Handbook of Economic Forecasting, G. Elliott, C. Granger, and A. Timmermann, eds., pp. 327-408, Elsevier, 2006.- [14] C. Lemke and B. Gabrys, "Do We Need Experts for Time Series Forecasting?"
Proc 16th European Symp. Artificial Neural Networks (ESANN '08), 2008.- [15] J.G.D. Gooijer and R.J. Hyndman, "25 Years of Time Series Forecasting,"
Int'l J. Forecasting, vol. 22, no. 3, pp. 443-473, 2006.- [16] C. Granger and R. Ramanathan, "Improved Methods of Combining Forecasts,"
J. Forecasting, vol. 3, no. 2, pp. 197-204, 1984.- [17] S.G. Makridakis, S.C. Wheelwright, and R.J. Hyndman,
Forecasting: Methods and Applications. Wiley, 1998.- [18] S. Mukherjee, E. Osuna, and F. Girosi, "Nonlinear Prediction of Chaotic Time Series Using Support Vector Machines,"
Proc. IEEE Workshop Neural Networks for Signal Processing, pp. 511-520, 1997.- [19] P. Newbold and C. Granger, "Experience with Forecasting Univariate Time Series and the Combination of Forecasts,"
J. Royal Statistical Soc., Series A (General), vol. 137, no. 2, pp. 131-165, 1974.- [20] NISIS Competition, http://www.nisis.risk-technologies.com/tte tte5.aspx?tb=t4, 2006.
- [21] NN3 Forecasting Competition, http:/www.neural-forecasting-competition.com , 2007.
- [22] J. Peng and J.A.D. Aston, "The SSM Toolbox for MATLAB,," Inst. of Statistical Science, Academia Sinica, 2007.
- [23] A.J.C. Sharkey,
Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems. Springer-Verlag, 2001.- [24] J.H. Stock and M.W. Watson, "A Comparison of Linear and Nonlinear Univariate Models for Forecasting Macroeconomic Time Series. NBER Working Papers 6607," Nat'l Bureau of Economic Research, http://ideas.repec.org/p/nbr/nberwo6607.html , 1998.
- [25] J.H. Stock and M.W. Watson, "Combination Forecasts of Output Growth in a Seven-Country Data Set,"
J. Forecasting, vol. 23, no. 6, pp. 405-430, 2004.- [26] F.E.H. Tay and L.J. Cao, "Modified Support Vector Machines in Financial Time Series Forecasting," vol. 48, no. 1, pp. 847-861, 2002.
- [27] J.W. Taylor, "Exponential Smoothing with a Damped Multiplicative Trend,"
Int'l J. Forecasting, vol. 19, no. 4, pp. 715-725, 2003.- [28] T. Teräsvirta, D. van Dijk, and M.C. Medeiros, "Linear Models, Smooth Transition Autoregressions, and Neural Networks for Forecasting Macroeconomic Time Series: A Reexamination," Pontifícia Univ. Católica de Rio de Janeiro, Dept. de Economía, 2004.
- [29] A. Timmermann, "Forecast Combinations,"
Handbook of Economic Forecasting. G. Elliott, C. Granger,, and A. Timmermann, eds., pp. 135-196, Elsevier, 2006.- [30] R.S. Tsai,
Analysis of Financial Time Series. John Wiley & Sons, 2002.- [31] Z. Vojinovic, V. Kecman, and R. Seidel, "A Data Mining Approach to Financial Time Series Modelling and Forecasting,"
Int'l J. Intelligent Systems in Accounting, Finance, and Management, vol. 10, no. 4, pp. 225-239, 2001.- [32] G. Zhang, B. Patuwo, and M. Hu, "Forecasting with Artificial Neural Networks: The State of the Art,"
Int'l J. Forecasting, vol. 14, no. 1, pp. 35-62, 1998. |