Challenge

Leadership still waited on weekly packs while analysts chased ad hoc SQL, duplicate extracts, and one-off slide decks. Self-serve BI covered common cuts but broke down on cross-functional questions that required joins across domains and careful definitions of revenue, margin, and pipeline.

Context

The organization had invested in a modern warehouse and a semantic layer, yet most stakeholders lacked the vocabulary to query safely. Prior attempts at generic chat tools produced confident wrong numbers because they lacked grounding in approved metrics.

Approach

• Co-designed a governed metrics catalog with owners, grain, and refresh SLAs for every KPI exposed to the copilot.
• Built retrieval over approved semantic views and pre-vetted SQL templates; blocked free-form database access from the model.
• Implemented human-in-the-loop review queues for novel question shapes before answers could be shown outside pilot groups.
• Added semantic caching with invalidation tied to pipeline completion events to keep hot questions fast without stale data.
• Shipped exportable answer cards with lineage links into the BI tool for auditors.

Technical foundation

The stack combined an LLM orchestration service, embedding indexes over documentation and metric definitions, and a policy service that evaluated group membership before any retrieval or SQL assembly step. Latency budgets drove aggressive summarization of large result sets and streaming partial responses where UX allowed.

Outcome

Median time-to-answer for executive questions fell from days to minutes; ad hoc analyst tickets dropped sharply during the pilot. Stakeholders retained control of definitions while non-technical users explored within policy boundaries—and every response remained attributable for audit.