Probabilistic Demand Forecasting

Most large organisations use machine learning demand forecasting solutions to predict future demand. For probably all of those organisations, they would agree that their demand forecasts are almost always wrong. Yet many see the answer to this status-quo being a more sophisticated demand model, leveraging everything from weather to events to macro-economic indicators. But to what end? No matter how sophisticated your demand model, there will always be an underlying level or uncertainty, or variability, that you cannot eliminate. It is very challenging (near impossible) to get forecast accuracy close to that underling uncertainty, as you cannot capture all the inputs required to eliminate other causes of demand variation.

The demand forecast is the bedrock of any supply chain. It is used for everything from raw material procurement to production and supply planning. Forecast errors propagate along the supply chain and result in lost opportunities (stock-outs) or excess costs (excess inventory, expired or obsolete stock). Supply Chains are all about trade-offs. Zero stock-outs imply excessive inventory, minimal safety stock will result in poor customer experience. Finding the sweet spot with a traditional demand forecast is near impossible, hence the increasing interest in probabilistic forecasting.

The traditional demand forecast is a deterministic forecast – it gives a single demand prediction for each point (each month per product per store for example). To create and improve this forecast, you probably clean your dataset to remove all historical outliers to avoid over-fitting your model to the extremes. In doing this, you lose hugely valuable information about what your future demand could be. 

Instead of taking the most likely demand (deterministic forecast), you can get much richer insights by looking at the possible range of outcomes we should expect based on all of the historical demand scenarios. With this we don’t remove outliers from historical data, we use that valuable information to understand the likelihood of future ‘outliers’ or demand extremes. This enables us to understand the likelihood of different future events and factor these into supply chain planning. This gives us the risk of stock outs, waste and inventory levels based on different supply or production plans, which is transformational in optimising supply chain decision making. Below are some specific use cases where probabilistic forecasts should be used.

  When should I order more inventory? How much inventory do I need to order? What should my safety stock level be? While all of these questions can be answered with a deterministic forecast, you are probably getting a sub-optimal answer. This is because the costs of over or under predicting demand are not equal, and a deterministic prediction means you have no idea how likely small or large over or under prediction errors are.

A low cost, high margin SKU will have a high cost of stock-out and a low cost of excess inventory and waste. A high cost, low margin SKU will have a low cost of stock-out (it doesn’t make much margin anyway) and higher costs of excess inventory and waste due to the higher COGS. It is also important to consider the costs of poor customer service, and to set minimum service levels, which we discuss in our Inventory Optimisation article. 

A probabilistic forecast shows you the probability and associated costs of different scenarios, covering direct supply chain costs (inventory, logistics), opportunity costs (lost sales) and waste costs. You can see that increasing the delivery size initially decreases the expected stockout costs more than the supply chain and waste costs increase by, so you should do it. Then there comes a point where the risk of stockout is minimal, and ordering more increases the overall cost, so you should not continue to increase the order size. With a deterministic forecast, all you would have seen was that you had enough inventory to cover the predicted demand, and that ordering more would increase inventory costs.

  Similar principles can be applied to calculate optimum safety stock targets, optimum production quantities and optimum raw material sourcing.

When deciding how much volume to contract (either raw materials for a manufacturer or finished products for a wholesaler or retailer), you are following a similar process to looking at how much to deliver to a store or distribution centre, but there is much more uncertainty given the longer time horizons. The process usually involves the planner taking the current inventory and the forecast demand over the contract and maybe adding a safety margin to avoid stocking out. When purchasing fresh materials or products over a period of months, the impact of forecast errors is large. The ability to trade-off the cost of excess inventory going to waste against the opportunity cost of stockouts enables a far superior optimisation and in most scenarios far less excess inventory going to waste.

There are a huge range of opportunities to leverage the improved insights from probabilistic forecasts across an organisation, not just for supply chain planning. But not all use cases want a probabilistic forecast. A key concept is that probabilistic forecasts maintain all of the information used to create your regular deterministic forecast. A probabilistic forecast can easily be used to calculate your deterministic forecast (most likely scenario), but once you have converted a probabilistic forecast to a deterministic one, you can’t get that extra level of information back. The fact that you can get back to a deterministic forecast is great for implementation. You can change your forecast as a first phase, and then incrementally cascade the benefits across different use cases at your own pace, keeping existing processes running on the deterministic conversion in the meantime. 

Another challenge is that users don’t know how to make decisions based on a 10% probability of X demand and a 20% probability of Y demand. When we created a probabilistic forecast for a leading food retailer, and used it to optimise inventory and order plans, users based decisions on expected costs and benefits scenarios (like the chart shown earlier). The solution presented users with an optimised inventory replenishment plan, which they could drill into and see the impact of changing when or how much was ordered using the scenario charts to make truly data-driven decisions. The result was >10% reduce in both inventory and stock-out costs.

Purpose AI is an AI and Analytics Consultancy delivering data-driven solutions for complex Supply Chain and Commercial decision making. We work across the Consumer Products and Retail Industries and have worked with many Top 20 CPGs and Retailers in Europe and North America. Our USP is that we have developed industrialised solutions to common industry problems, reducing the time and costs to deliver value. But unlike SaaS providers, there is no licensing costs, we customise our solutions to your requirements and embed a strong business process to ensure they deliver value.