Starting a BI Initiative? Choose Agile Over Waterfall
In our podcast episode, BI is Not the Same as an ERP Implementation, our VP of Delivery Operations Caleb Ochs and CEO Brick Thompson discuss the benefits of an agile strategy in BI implementation over the traditional waterfall approach and how to get quick wins while establishing a state-of-the-art BI system.
Having worked with hundreds of clients, we have noticed a consistent pattern: clients who utilize an agile approach in their dashboard implementations experience a higher rate of success than those who use the waterfall approach.
Research confirms this trend. Ambysoft’s Project Success Rates Survey found that the agile method has a 15% higher rate of success than waterfall across IT projects. That gap is particularly meaningful in BI, where the nature of the work strongly favors an iterative approach.
Drawbacks of Waterfall Methodology
If waterfall delivers a lower rate of success, why do companies continue to use it? The conventional process goes something like this. When setting out to convert your data into automated dashboards, scope creep is a constant temptation. You meet with a group of stakeholders to discuss creating dashboards to advance a goal or area of constraint in the business. The potential impact becomes evident, along with some seemingly reasonable impulses: we need to get this right the first time, we cannot forget to address work order efficiency, we might want to add data that spotlights labor utilization, and the finance team will want a version that fits their needs too. Consequently, your group drifts into a one-and-done mindset and rallies around a much larger project, energized by the potential to address a broader set of issues in one fell swoop. The budget and timeline swell, the project plan grows in complexity, and the stakeholder group expands. That is the waterfall approach.
Waterfall can be appropriate for complex organizational projects, but that tendency often drifts into BI projects where point-solutions can be implemented and form the building blocks of a data-driven organization. The outcomes of waterfall projects in terms of cost, timeline, and delayed benefits are commonly unfavorable. Most organizations cannot afford these drawbacks, particularly in a volatile economy. Quick wins are essential, and in BI, those wins inform each subsequent sprint while delivering value and reducing project risk.
Benefits of Agile Methodology
Agile methodology focuses on a short development cycle, delivering insights quickly, and iterating in response to customer feedback. Agile works especially well for projects that require learning and adjustments along the way, which makes it a natural partner for BI, where users typically need to get their hands on a working version before they can fully clarify their requirements. Investing in design before development is part of what keeps agile sprints focused and prevents the scope drift that causes waterfall projects to stall.
Consulting firm PwC shifted from a traditional waterfall approach to agile delivery in 2016 as part of its own digital transformation. Using sprint-based delivery with short cycles and continuous iteration, the firm found the agile model improved employee experience, accelerated value delivery, and produced better outcomes than the waterfall approach it had used previously.
At Blue Margin, we follow best practices at the forefront of data engineering and dashboard design, resulting in rapid delivery and high-impact projects. Our managed data service is built on this sprint-based model, giving clients working dashboards quickly rather than asking them to wait through a lengthy design and build phase before seeing any results. For teams that are new to BI and looking for a practical starting point, the quick path to winning with data and dashboards covers what a focused first initiative looks like in practice.
Practically, what does agile BI look like in action? Begin each initiative by clearly defining the business need and the measurable impact: you do not need to know everything, but you do need to know what your business wants out of the project. Intentionally choose who is and who is not a key player for decisions, so the process remains as nimble as possible. And apply a disciplined process for keeping the scope of the project focused and finite to stay on time and on budget while reducing project risk.
Learn more about getting the most value out of your BI projects from our podcast: The Dashboard Effect. If you would like to talk through what an agile BI initiative would look like for your organization, contact our team here.
