Forecasts are like school children — some are well behaved and do what is expected, but some need supervision. When implementing automated demand planning, it is important to know which SKUs you expect to receive accurate forecasts and which ones will require special treatment.

Making the effort to keep this assessment fresh will streamline your process and improve your decision-making all the way through to who is accountable for the item’s performance and whether to continue offering it. Poor forecasts can also be valuable because they can inform you that demand patterns are changing.

Forecast Accuracy & Variation

If you use demand planning software, you probably have access to a few simple metrics you can use at a glance. Let’s start with forecast accuracy and error. Although software has minimized forecasting error as much as it could through automatically selecting the best statistical method, sometimes the error will still be significant.

If you rank your items by MAPE (mean absolute percent error), you should see which items have the worst forecasts. Even better, if you have a “dollars of error” measure, this ranking will make the biggest impact of these errors obvious. A real pro will want to see these metrics over time to know whether the forecastability of an item is getting better or worse.

Of these less reliable forecasts, we need to take a closer look at how much of demand the forecast is explaining. This can be done with looking at MAPV (mean absolute percent variation), which is an expression of a product’s variations in demand (average absolute deviation from mean, divided by the mean and given as a percentage). Comparing MAPV with MAPE, you can get an idea of how much of demand is being accounted for by the forecast.

For example, if MAPV is 20% and MAPE is 10%, the forecast is explaining half of the variations. But if the MAPE is higher than the MAPV (not likely), the forecast is making things worse. This kind of analysis can also be useful in comparing the statistical forecast with manual forecasts, to see who or what is improving the forecasting process.

If you can look at this over time, you really start to get a sense of what’s going on. PVE (percent variation explained) is 100 x (1 – MAPE/MAPV) and shows what portion of the variations in demand are being anticipated by the forecast. If this suddenly changes, it means something fundamental in buying patterns for this SKU is changing. PVE is a respectable measure of forecast accuracy because it takes into account these changes. If you see MAPE increase alone, you might assume a decrease in accuracy. But if MAPV has increased more, the forecast accuracy has actually improved.

Forecastability Profiles

With this knowledge in hand, you can create a forecastability profile for important items or product lines and analyze what factors are influencing the variations in demand and what changes to forecast and process are prudent going forward. For example, looking at CV (coefficient of variation), you can get an idea of whether demand is more “steady” or erratic. If erratic, and seasonality doesn’t explain it, this might be an item that is mostly sales and marketing driven and one should look for evidence of this and make this part of the product line’s forecastability profile.

Knowing which products are well behaved and which are more erratic due to other factors is key to reducing risk and establishing credibility with your team. Now your analysis will lift the uncertainty fog around these products and give everyone a common starting point for making improvements.