To reliably predict customer demand, companies predominantly rely on statistical forecasting by selecting the most suitable model(s) for their products. However, the thought of forecast model selection is outdated. Combining the power of different models is the proven way forward to take forecast accuracy to new levels.

The Struggle of Forecast Model Selection

For demand planners, the essential objective is to increase forecast accuracy. To reliably predict demand, most companies use statistical models, e.g., exponential smoothing methods or the moving average. Although these models have been around since the 1950s, they are still widely adopted in companies as they are known for their robust forecast performance and ease of understanding. On top of these still popular models, a growing number of models has been developed over the past 70 years to achieve even better forecast accuracy, and new models are still being developed. These are often significantly more complex and increasingly based on machine learning (ML).

As a result, demand planners are now faced with a multitude of available models. It is highly relevant for them to know which of these alternatives they should use for forecasting the demand of their products. To address this issue, model selection rules have been suggested.

The most common selection approach is the so-called “best-fit” method, which is supported by most advanced planning systems. Here, several competing forecast models are simulated on the historical sales figures for each product. The model that has the lowest forecast error for past sales is selected for the respective product to create the future forecast.

Selection approaches like the best-fit exist in abundance. What they have in common is that they are looking for that one model which is the most suitable. However, the thought of forecast model selection is outdated, as we will show in the following section.

One Model is Not Enough

The selection of the most suitable model results in an all-or-nothing decision, which has two major drawbacks:

1.     Loss in information:

Typically, statistical forecast models have been developed to capture specific behaviors and patterns in demand, such as trends and seasonality. Selecting the best-fitting model out of the many options implies automatically to disregard any patterns that the selected model does not capture. Therefore, selecting only one model results in a loss of information about the true demand pattern of a product.

2.     Nervous forecasting behavior:

In cases where, for example, two models perform similarly well on historical demand, a model selection approach such as the best-fit usually results in regular switching between models at each planning cycle. Worst case, the two models even capture very different patterns. Thus, when jumping back and forth between models, forecast values change dramatically, especially regarding long-term forecasts. This nervous forecasting behavior is a real problem for all subsequent planning steps (production planning, inventory planning, etc.) and commonly leads to organizational distrust towards the forecast. Figure 1 visualizes such a case.

Constant and seasonal model perform similarly well on historic sales figuresFigure 1: Constant and seasonal model perform similarly well on historic sales figures

The Power of Combination

The fact that a combination of forecast models can lead to more precise forecasts was already noticed in the early course of the M-Competitions, the world’s biggest forecasting competitions. With the M4-Competition, the dominance became finally indisputable, leading to one of the key take-aways of the competition that is summed up as “the improved numerical accuracy of combining” (M4-Competition, 2020). For more information on the competition itself, see our last article “What We Can Learn from the World’s Biggest Forecasting Conference”.

Approaches for forecast model combinations are diverse. They can be distinguished by means of different “maturity levels”, which were identified in the course of the M-Competitions:

1.     Simple combination of statistical models

The “Comb S-H-D” method was one of 24 (mainly statistical) methods studied in the M3-Competition (2000). It involves the simple average of three commonly used exponential smoothing models, namely Single Exponential Smoothing, Holt’s Linear Exponential Smoothing, and Dampen Trend Exponential Smoothing. This simple averaging showed a superior performance to the individual models in most cases and performed well in general when compared to the other methods within the competition. These results could be observed for different forecast horizons (up to 18 months), error measures and data types (micro, industry, macro, etc.), aiming for generalizable results.

2.     ML combination of statistical models

The dominance of forecast model combinations already observed in the M3-Competition was again and very clearly confirmed by the M4-Competition, conducted in 2018: among the 17 most accurate methods in the competition, 12 were combination approaches – predominantly from statistical models.

One of these is the so-called “feature-based forecast model averaging” (FFORMA) framework (Montero-Manso et al., 2019), which came in second in the M4-Competition. Unlike a simple average, the idea uses an ML model to develop a dynamic weighting scheme for combining forecast models. For this, the ML model is trained to minimize the forecasting error by evaluating the forecast accuracy of each model in respect to a set of features evident in the sales history (spikiness, stability, strength of trend and seasonality, etc.). Hence, given any new product (and its set of calculated feature values), individual model weights are determined.

3.     Combination of ML models

In addition to the trend of forecast model combination, ML models receive increasing recognition in the forecasting community. Therefore, it is not surprising that the winning model of the M5-Competition is indeed a combination of several ML models. More precisely, the framework considers the simple mean of multiple advanced models from the area of gradient boosting.

Bottom line: regardless of how sophisticated the forecasting models or combination algorithms (statistical or ML-based) are, the results of the M-Competitions clearly demonstrate that combining multiple models outperform individual ones. Figure 2 illustrates exemplarily the effect of forecast model combination in comparison to the use of individual models.

Forecast model averaging enables the happy medium between the extremesFigure 2: Forecast model averaging enables the happy medium between the extremes

Boosting Forecast Accuracy Has Never Been Easier

Transferring findings from research into practice often fails. But the undeniable evidence recurringly coming from the M-Competitions should no longer be overlooked by demand planning departments.

Independent of the maturity of your forecasting approach, whether using simple statistical models or advanced ML models, it is definitely worth a shot to evaluate the forecasting performance of their simple average. This does not require any additional model but can easily be accomplished by making a few adjustments in the forecasting profile of your existing planning system. Achieving a higher forecast accuracy has never been easier!

Nevertheless, for your long-term roadmap, you should consider an intelligent model combination approach. This requires more complex algorithms from external systems. The technical details of their integration will be the subject of our next blog article. Stay tuned!

Race for Value: Towards the Digital Supply Chain

To leverage the full potential of digital supply chains, there are still challenges to overcome. This thought paper provides insights on how to drive the digital transformation of the supply chain.

Download our Thoughtpaper on Digital Supply Chains

Recommended articles

Data & Analytics

ILM Capabilities – Outlook on IT

Risk assessments and retention requirements drive companies to reengineer their data strategy. In order to meet the challenging requisites of regulations …

read more
Data & Analytics

How Your Organization Benefits from Lean Data Management

This article aims to provide guidance and should enable organizations to optimize data and processes by following lean data management principles.

read more
Innovation

Blockchain enabled Accounting Automation Platform

Sharing revenue and according reconciliation processes connected to digital services are a time-consuming matter. Deutsche Telekom, SAP and Camelot ITLab teamed …

read more

Reimagine your Value Chain with us

Contact us