For years, “self-service analytics” was the holy grail of the data organisation. The promise: give business users the tools to answer their own questions, without routing every request through an already-stretched data team. The reality: conflicting dashboards, inconsistent metric definitions, and a growing backlog of ad-hoc requests that no tool has quite solved. AI self-service analytics has changed that equation.
AI has changed the equation. Not because it replaced the hard work of building good data foundations, it hasn’t, but because it has finally provided a natural language layer that bridges the gap between a business question and a SQL query. When deployed thoughtfully, AI-powered self-service analytics is delivering something genuinely new.
“The bottleneck was never SQL. It was the gap between what a business leader wanted to know and what a data engineer knew how to query.”
Anthropic, the AI safety company behind Claude, recently published a detailed account of how they automated 95% of their internal business analytics queries using Claude, achieving around 95% accuracy in aggregate. Their experience mirrors what we see across our clients: the wins come from a structured approach to context, verification, and retrieval, not from simply pointing an AI at a database and hoping for the best.
The initial enthusiasm for “chat with your data” tools often gives way to frustration. Answers look plausible but contain subtle errors. Users can’t tell when to trust a result. Data teams spend more time validating AI output than they did answering questions manually.
The root cause is rarely the AI’s coding ability. Modern large language models can write sophisticated SQL. The problem is context: without deep, accurate, up-to-date knowledge of your specific data model, any AI will make wrong assumptions that produce wrong answers confidently.
The AI can't reliably map a plain-language question to the single correct field, table, or metric definition when multiple plausible candidates exist.
Business definitions and schemas change constantly. Without active maintenance, AI knowledge drifts out of sync, producing subtly wrong answers with no error signal.
The right information may exist in the model but be buried in thousands of fields. Without structured navigation, the AI simply doesn't find it.
Microsoft’s Copilot for business intelligence and Anthropic’s own internal deployments both point to the same conclusion: semantic layers and governed metadata are the most important enablers of accurate AI analytics. The AI is only as good as the context you give it.
Organisations achieving high accuracy with AI analytics share a common architecture, one built in layers, each addressing one of the three failure modes above.
Canonical, single-source-of-truth datasets with clear ownership. Fewer tables, better governed, aggressively deprecating near-duplicates. Metadata treated as a first-class product.
Semantic layer for governed metrics, lineage tracking for data provenance, business context (org charts, roadmaps, decision logs) to resolve ambiguous references.
Domain-specific knowledge files that give the AI procedural knowledge: which sources to consult, in what order, with what verification steps. The difference between 21% and 95%+ accuracy.
Offline evals, adversarial review sub-agents, provenance footers on every answer, and correction harvesting loops that turn user feedback into documentation improvements.
If there is one component that separates analytics AI deployments that work from those that don’t, it is the semantic layer. A semantic layer defines your business metrics in a governed, reusable way, what “revenue” means, what counts as an “active user,” and what the approved lookback window is for any given KPI.
Anthropic’s team found that when a question maps cleanly to a defined metric, accuracy approaches 100%. The AI calls a function, gets one number, the same number that every other surface in the company produces. The problem of ambiguity disappears before the AI even has to search.
Both Anthropic and Microsoft’s Power BI team make the same recommendation: generate documentation with AI, but have a human own the definitions. Auto-generated metric definitions encode the very ambiguities you’re trying to eliminate. The investment in human curation of a smaller, well-governed semantic layer consistently outperforms a larger, auto-generated one.
analytics queries automated at Anthropic
aggregate accuracy in production
accuracy improvement with structured skills
One of the most underestimated challenges in AI analytics is freshness. A system that achieves 95% accuracy at launch can drift to 65% within a month if documentation isn’t maintained as the data model evolves.
The teams that solve this don’t treat documentation maintenance as a separate workstream. They colocate skill documentation alongside transformation models in the same repository, and enforce that any change to a reporting model must update the relevant documentation in the same pull request. Roughly 90% of data model changes automatically include a documentation update.
Think about this: data pipelines form the operational foundation of modern organizations that compete on data. You build them correctly; with proper error handling, scalability, security, and governance; they transform from constant pain points into competitive advantages. Organizations that prioritize data pipeline architecture gain faster insights, better compliance, lower costs, and happier engineering teams. The question isn’t whether to invest in data pipelines; it’s whether your organization can afford the consequences of neglecting them.
✓ Skill documentation lives in the same repo as transformation models
✓ CI checks flag any reporting model change without a corresponding doc update
✓ Offline evals are anchored to snapshot dates so they don't drift with live data
✓ Eval results stored as telemetry (with model version, git SHA, pass rate), not just test logs
✓ User corrections in Slack/Teams automatically harvested as new eval candidates
✓ Weekly review of semantic layer coverage vs incoming questions
Enterprise AI analytics doesn’t require a multi-year transformation programme. The organisations achieving results fastest share a common starting point: a handful of canonical datasets, a few dozen offline evals, and a thin AI skill layer. That foundation captures most of the value. Everything else is optimisation.
The key strategic questions to answer before building:
How tolerant is your organisation of errors? If business leaders will use AI output in board presentations, you need provenance footers and a human sign-off on high-stakes numbers. If the audience is data-literate analysts who can spot wrong answers, you can move faster and iterate.
What’s the complexity of your data model? A company with a clean, well-governed warehouse of 20 core tables has a very different starting point from one with 2,000 tables built up over a decade of acquisitions. The messier the foundation, the more important canonicalisation becomes before layering AI on top.
What is your compute and latency budget? Adversarial review sub-agents, where a second AI model challenges the first’s assumptions before an answer is returned, significantly improve accuracy, but add latency and token cost. The accuracy gain is real; so is the price.
At Expert-Extend, we’ve built AI analytics capabilities across industries including manufacturing, professional services, financial services, and SaaS. Our approach combines deep data engineering with AI implementation, because the two are inseparable.
We don’t sell a product. We design and build the system: data foundations, semantic layer, AI skill architecture, validation pipelines, and the governance processes to keep it accurate. We also transfer capability, training your team to maintain and extend the system independently.
If you’re exploring AI-powered analytics for your organisation, the conversation starts with your data foundations, not with the AI. Get in touch to discuss what’s realistic for your context.
✓ 95% queries automated (Anthropic)
✓ 21% → 99% accuracy via skills
✓ 90% PRs include doc updates
✓ 6% accuracy gain via adversarial review
Navigation
