When working with business KPI software, the mantra “fail fast or scale fast” often surfaces as a crucial decision-making guideline. This principle advocates for starting small with software pilots and, if these initiatives meet or exceed expectations, quickly scaling them across the organization to maximize benefits. Conversely, if a pilot does not deliver as hoped, the recommendation is to move swiftly to the next software solution. However, accurately gauging the success or failure of these pilots can be complex, with important factors frequently overlooked. This exploration aims to highlight key considerations necessary for understanding the full impact of business KPI software pilots and investments.
Go or No-Go?
When a pilot of the latest business KPI software in a department is showing promising results, with improvements that surpass initial expectations, it might seem like a cause for immediate celebration. However, confirming true success requires a comprehensive review of the business case, examining both quantitative (financial) results and qualitative impacts.
For an objective evaluation, the same business case that justified the investment in the software pilot should be used as a foundation for its assessment. This critical evaluation step determines whether to expand the software’s use or to explore alternative solutions. Essential considerations include:
- Maintaining original success criteria: After significant investment in selecting and piloting software, adjusting success benchmarks can compromise the integrity of the evaluation process.
- Balancing quantitative and qualitative analysis: While financial metrics may be straightforward, the broader impacts, such as organizational support needs, team acceptance, training requirements, and feedback from IT and maintenance departments, are equally critical.
- Prompt decision-making: Armed with comprehensive information, the next steps—whether to proceed with wider implementation, adjust the pilot, or explore other solutions—should be decided swiftly.
Key Reporting Steps and Ingredients
The challenge with effectively reporting on a pilot is putting together something that is to the point, yet comprehensive enough to accurately illustrate the costs and benefits of a solution. Of course, it’s important to use your company’s standard report format and style, which ideally should include the following:
- Executive Summary—a go or no-go recommendation with a bit of context
- Project/pilot/investment summary— what did you set out to do and what technology and vendor did you go with
- Outcomes:
- Financial summary
- Qualitative summary— how were people, process and technology impacted?
- Suggestions for improving the use of the technology
- Risks and mitigations
These tips will help you ensure that the content of your report best serves the needs of your company:
- Use the language of your finance/accounting department. Every company has a different process for evaluating capital investments. Your business case and subsequent evaluation should use the process of your current company, not a former employer or some industry-recognized process.
- Make sure you include a financial representative on the team. Senior management (or whoever has the power of “go” or “no go”) will likely turn to finance/accounting for guidance. That’s why it pays to involve a finance representative in the evaluation, especially during the development and evaluation of the business case.
- Capture the costs and benefits through multiple lenses. There are many popular ways to capture the costs and benefits of a technology investment. Most companies that I’ve worked with over the last 10 years like to use payback period because it’s a fairly straightforward measure of value. But it’s still worth exploring return on investment (ROI) and net present value (NPV) in most cases.
- Don’t sugarcoat things. Being straightforward and honest about anything that could be a future issue or headache is key to avoiding unnecessary pain and frustration down the road for you and potentially the entire company. For example, if a vendor seems great but is also small and could be acquired or go out of business, note that along with how you could mitigate the risks.
- Skip steps at your own risk. Missing or avoiding key steps in the measurement of a pilot can result in a project blowing up in all kinds of different ways. For example, a pilot project team without a finance person or other objective representatives could get overly enthusiastic and myopic about a technology. Maybe they get excited about moving a key measure from point ‘A’ to ‘B’ and decide to move forward without adequately considering the bigger picture. When the other teams attempt to repeat or scale the solution, they may discover downsides related to integration or support any number of things.
Ultimately, It’s All About Data
In some ways, there’s a chicken and egg component to measuring the value of a technology investment. After all, in complex operations, you need baseline data about operations to compare against the results of a pilot. And you need a way to capture data related to the pilot at the right intervals, be it every second, minute, day or week. A clipboard or spreadsheet won’t cut it. If you’re looking for a way to transform decision-making with advanced business analytics, Dploy Solutions can help. We provide the real-time performance insights and analytics capabilities you need to advance operational excellence no matter where you are in your digital journey.