Building Better Product Plans with Forecasting

---

Building a product plan is a challenging and often costly endeavor. The value of a good plan is immeasurable. Plans help you facilitate communication, hold teams accountable, and confidently make commitments to partners.

How can we be confident that the plan we create is realistic? That is, how do we know we can realistically execute according to the plan? Normally, engineers and product managers use a combination of estimates for both the effort required to complete an item in the plan and the capacity of the team doing the work to come up with reasonable completion dates. The estimates, however, are usually based on heuristics and the gut feelings of experienced engineers, the quality of which can vary drastically from person to person and project to project. Low-quality estimates produce inaccurate plans, which in turn add risk to the business. Given that there is tremendous business value relying on the timely execution of our plans, is there anything we can do to tighten up these estimates?

On the financial side of the business, analysts face a similar problem of predicting the future during financial planning. Financial analysts use a statistical technique called forecasting to make predictions about things such as sales volumes or cash flow. In particular, forecasting typically combines historical financial performance data with educated guesses about future performance or trends to estimate performance at a future date. This all sounds like a familiar problem, doesn’t it?

In this article, we will look at how forecasting techniques can be used to build better product plans by taking inspiration from our friends in the finance world. We will present some easy ways for you to integrate these techniques into your planning process.

Why Use Forecasting?

To start, you might be wondering, “Why would I want to do forecasting?” The answer is quite simple. As product leaders, we build various different kinds of plans for different applications, such as communication and making commitments. One thing remains constant between these applications: we use our plans to make decisions. As a result, the quality of our decisions is directly influenced by the quality of our plans. Thus, it behooves us to do our best to build accurate plans.

Forecasting is a general technique in statistics where historical data is extrapolated to make predictions about future events. Finance teams at established companies have historical performance data at their fingertips, so leveraging forecasting enables better predictions about future financial performance.

Engineering teams, like finance teams, have a wealth of historical execution data; the difference is that this data goes largely unused. Adopting forecasting into the planning process allows us to leverage this data to build better plans.

In this discussion, we will focus on two simple techniques that will improve your plans: comparables and forecasting.

Comparables: Fine-tuning estimates

Talk to any software engineer, and they will tell you that estimating effort is a really tough thing to do. There are a host of unforeseeable problems that always come up, and generally, the human mind is not well equipped to consider these complications in an ad hoc way. In order to make better estimates, we need to lean on statistics and data-driven techniques to take our bias out of the picture.

A simple technique for doing this is using comparables. Although no two engineering projects are exactly alike, they tend to rhyme, both in terms of code changes required and the general structure of the work. Fortunately, we can use this to our advantage when estimating effort. For a given project, we can look back at previously completed projects for ones that are similar in nature, e.g., code changes or structure. We call this list of similar projects comparables. Comparables are useful as they allow us to anchor the estimation process in something concrete, as we already know how long those projects took to complete. From this data, we can derive a more accurate estimate of our current project using one of a number of strategies, e.g.,

• The maximum of all comparables.
• The average of all comparables.
• A weighted average where projects are weighted relative to their relevance.

The strategy that you pick is up to you and your team based on your needs. The strategy that you pick can be tuned in future iterations so that it produces better results.

If you are having trouble finding comparables for a new project, if the scope of the project is large, it can help to subdivide the problem into components, look for comparables for those components, and sum the estimates together.

At PlanEngine, we automate the process of identifying and preparing a list of comparables for a proposed project, as well as using those comparables to provide an estimate for the work, saving your team a huge amount of time digging through your historical data. Sign up for early access to PlanEngine to unlock efficiencies in your product planning process.

Retros: Comparing Forecasts to Actuals

In the finance world, actuals is the term used to refer to the actual financial performance for a given time interval, usually a quarter or a half.

Similarly, product teams should adopt a process of comparing their forecasts to their actual execution performance. Though this change is more operational than mechanical, the process is immensely important for tuning future forecasts. For example, if all projects for a team are completed roughly according to plan, the forecasting model is likely correct for that time. In the more common scenario, there will be some drift on at least one project between the forecast and actuals, and this can tell you a lot about what happened. For example, some common scenarios are:

1. If there is uniformity in the deltas between forecast and actual for every project, this points to operational overhead that was not accounted for in the forecast. In future planning cycles, accounting for this overhead, computed using your forecasted and actual amounts, will produce a better plan.
2. If a few project forecasts differ greatly from the actuals, this is likely a result of unknowns in those projects that came up during execution and caused delays. Identifying these projects is immensely useful, as they make perfect candidates for comparison in the future when tuning estimates.
3. If every forecast was wrong by random amounts, it points to fundamental issues in the estimation process, and it is likely that the entire process would need to be reworked for future iterations to get more uniformity in the results.

As you can see from the examples, ensuring that your team has a robust retro process in place to analyze forecasts after the fact is crucial to ensuring better results in future iterations.

Conclusion

Adopting techniques of forecasting into your product planning will help you make data-driven planning decisions. In particular, we recommend augmenting your estimation process by looking for comparables to anchor your estimates, introducing a retro process to compare your planned execution to actual execution, and using that to tune your planning in future iterations. Thank you for reading, and I hope you learned something useful for your team.

---