All forecasts are wrong: tips for improving accuracy

All forecasts are wrong. This infamous statement may be among the first things professors preach in many university-level business courses on forecasting. Most of us are familiar with the frustration felt when we are unable to close a project within the budgeted scope, year-end business volumes are nowhere near the January forecasts, summer vacations are ruined by rain appearing out of nowhere and savings yield little profit in the stock market.

With forecasting so prone to error, why then do we continue to rely so much on them for critical decision-making? A reasonable hypothesis is that to be able to start planning the future, we feel that our planning efforts need to have a basis. It would be fair to say that most decision-makers are aware that predicting the future with 100-percent accuracy is impossible. Widely accepted is the fact that forecasting contains some margin of error and that forecasts with a lower margin of error are the ones that make a difference.

A healthier approach would therefore be to stop questioning why forecasts are always off and instead, try to minimize the margin of error. After all, in a world where chimpanzees beat Wall Street professionals in competitive salaries, room for improvement must be possible.

How then to maximize the accuracy of your forecasting? Here are some useful tips:

Know and embrace your errors
No one is Nostradamus. Errors are an inherent part of your forecasting. So, minimizing the margin of error while staying loyal to your overall data quality – making assumptions, rounding up or down statistics, etc. – will determine the success of your error handling. You may recall high-school science experiments when using one margin of error for one variable that was in direct correlation to another, it was useful to stick to the same margin of error with inter-dependent variables.

Another useful and often overlooked rule of thumb is to remember that the further we try to predict in the future, the greater normally the margin of error. Think weather forecasts: tomorrow’s forecast is often more accurate than that of next week’s.

Improve the quality of your data
Gathering, refining and analyzing data for forecasting is an art in itself. In this day and age with the use of big data skyrocketing, more and more data can be tracked, stored and crunched in a multitude of ways. Compared to forecasting in the 70s and 80s, finding the most relevant data – not just any data – has never been more important and is a challenge that forecasting professionals face today.

A good way to increase your relevant data sampling is through crowdsourcing. Sports Interactive (SI), the makers of the digital game, Football Manager, recently released a B2B version to professional football clubs, which successfully used and popularized the crowdsourcing model. Contrary to the traditional scouting model of sending individual scouts to different national leagues, SI’s crowdsourcing network of over 1,300 scouts (who constantly get feedback from fans and Football Manager players) – give much more updated and reliable data on individual players and eliminates the subjective views of individual scouts.

If you’re unable to outsource your data gathering in any way, a similar strategy could be to outsource your data analysis: i.e. crowdsource the forecasting process to different individuals in an organization rather than just delegating it to a few people. I’ll discuss this crowdsourcing aspect at length in a later blog.

Forecast more often
“If you have to forecast, forecast often.” This quote should be a no-brainer. The more frequently you forecast, the more you narrow down your forecasting horizon. This, in turn, allows you to do a reality check and reduce your margin of error.

Train and/or recruit “super forecasters”
“The herd instinct among forecasters makes sheep look like independent thinkers” (Edgar R. Fiedler in The Three Rs of Economic Forecasting-Irrational, Irrelevant and Irreverent, June 1977)

Super forecasters are not aliens or uniquely smart people. The talent to forecast can be taught, nurtured or, if training existing personnel is unaffordable, recruited.

In a recent forecasting blockbuster book, Superforecasting: The Art and Science of Prediction, author Philip Tetlock states that most super forecasters have above-average IQs (not necessarily geniuses), strong statistical mindsets and the courage to go against pre-determined decisions or prejudices, that is, the power to say no and present their findings despite top management expecting them to forecast a particular scenario that supports their own arguments.

Gamification – i.e. arranging forecasting tournaments and competitions – is key to crowdsourcing the forecasting effort, increasing its accuracy as well as motivating and developing super forecasters.

The right software
You’ve improved the quality of your data, become more aware of the margin of error, decided on the optimal forecasting frequency and trained/recruited the right personnel. What’s the missing link in the chain? It’s investing in the right software, which, like the Big Lebowski’s rug, is the element that ties the room together. Briefly, the four main characteristics of good forecasting software are as follows:

• It uses the right language and algorithm (contingent on the scientific expertise of the software provider)
• It is stable for continuous use
• It provides a good user interface – critical for training personnel
• It offers good reporting tools (enabling visualizing findings for top management)

Inherent randomness
You may have done everything right but you must still embrace the inherent randomness of every forecast. Sure, the level of randomness differs from forecast to forecast but accept the fact that luck – i.e. force majeure events – plays an important role in every forecast. It is therefore essential to prepare for the worst.