Overview
In this episode of The Dashboard Effect, Brick Thompson and Caleb Oaks take an honest look at where generative AI in analytics actually stands, measured against the expectations that were set when the technology arrived with considerable fanfare at Microsoft Build. The assessment is candid and grounded in real experience: the tools have improved, but the gap between what was promised and what reliably works in production remains significant.
For organizations that have been waiting for generative AI to transform their analytics workflows, this episode offers a grounded calibration of what is reasonable to expect right now and what the path forward might actually look like. See how Blue Margin’s Managed Data Platform builds the semantic layer, metadata structure, and data governance that determine whether your organization is positioned to take advantage of tools like Power BI Copilot when they mature enough to deliver reliably on their promise.
What This Episode Covers
Power BI Copilot and Natural Language Querying (0:19 – 0:47, 5:07 – 5:50)
Natural language querying is a genuinely compelling prospect, and the hosts reflect on Microsoft’s Power BI Copilot demo with a mixture of interest and realism. The current reality is that making these tools useful for business users requires significant backend configuration, including defining metadata, synonyms, and semantic structure, before the natural language interface can deliver reliable results. The demo experience and the production experience are meaningfully different, and that gap is worth understanding before committing to it as a user-facing capability.
The Gap Between Enthusiasts and Business Users (2:47 – 4:05, 5:42 – 6:10)
The hosts are direct about the distinction between how these tools perform for technically engaged users who understand their limitations and how they perform for the average business person who needs reliable, consistent outputs without managing the underlying complexity. Current implementations, including those using advanced models like GPT-4, often require substantial data engineering investment to reach a level of accuracy and reliability that business users can trust. The technology is not yet ready for prime time in a general business analytics context, and the hosts are clear about that assessment.
Machine Learning Automation (9:08 – 9:52)
One of the more promising near-term developments the hosts identify is the automation of machine learning model selection and application. AI-assisted predictive analytics, where the system handles the work of choosing and configuring models against a dataset, is an area the hosts view as accelerating and as more practically accessible than some of the natural language querying applications that have received more attention.
AI in Video and Voice (10:24 – 11:51)
The conversation broadens to cover AI developments outside of analytics, including voice cloning and video dubbing capabilities that the hosts describe as increasingly realistic and sophisticated. These applications are advancing faster than many business analytics use cases and offer a useful reference point for understanding just how quickly the broader AI landscape is moving even when specific enterprise applications lag behind.
AI in Biotechnology (12:37 – 13:20)
The hosts highlight AI’s role in medical research, particularly protein folding and drug discovery, as an area where the technology is showing immediate and tangible impact that exceeds what is currently possible in general business analytics. The comparison is useful: AI is not uniformly at the same stage of maturity across domains, and the applications where it is delivering the most dramatic results today are not always the ones that get the most attention in enterprise technology conversations.
Who It’s For
This episode is worth your time if you are a technology or data leader trying to set realistic internal expectations about what generative AI in analytics can deliver today versus what it will deliver as the technology matures, a BI or data engineering team evaluating whether to invest in natural language querying capabilities now or wait for the implementation overhead to come down, an executive who has seen compelling AI demos and wants a more honest assessment of what production deployment actually requires, or anyone who follows AI broadly and wants a practitioner’s perspective on how the analytics applications compare to the progress happening in other domains.
Why It’s Worth a Listen
The gap between AI demos and AI in production is one of the most consistent sources of misaligned expectations in enterprise technology, and this episode addresses it directly without dismissing the technology’s potential. The hosts are enthusiastic about AI and honest about its current limitations in business analytics, which is a more useful combination than either uncritical optimism or reflexive skepticism would produce.
The backend configuration point about Power BI Copilot is particularly valuable for organizations that are evaluating natural language querying as a near-term capability. Understanding that the demo experience depends on significant upfront semantic work that most organizations have not yet done changes the timeline and investment calculation considerably, and knowing that before starting is significantly better than discovering it partway through an implementation.
And the biotechnology comparison is worth sitting with as a reminder that AI’s most significant near-term impacts may not be in the business analytics domain at all. Maintaining perspective on where the technology is actually delivering versus where it is mostly being discussed helps organizations make better decisions about where to invest attention and resources as the landscape continues to evolve.
