AI agents can build a dashboard in minutes. Can you trust the number on it?

At Tecknoworks we built an agent harness that takes a business request all the way to a verified Power BI report on Microsoft Fabric, and refuses to show any number it can't prove. Speed required little engineering effort. The real work went into proving every number.

Ask a general-purpose AI assistant for "churn rate by region" and it'll happily write SQL and render a chart, picking a definition from whichever file is nearest. The chart will look right, but nobody can say which definition it used or whether the grain is correct, and the same question asked tomorrow may return a different answer. For any business that makes decisions on its numbers, what looks like a productivity gain is a liability.

How it works: one supervisor, three specialist agents

Everything starts with the supervisor. You give it a request in plain language, and it works backwards from the deliverable through the stack: report first, then the semantic model, then Gold, then Silver. At each layer it checks what's already there, running the cheapest checks first, and it only calls in a specialist agent when a layer is missing or invalid.

Each agent plays one role a data team already has: data engineer for ingestion, data engineer for the governed data product, analyst for the report. None of them can approve its own work, and each stops at its gate and waits for a person.

Five principles that make the output trustworthy

The harness is opinionated by design. Every rule below exists because we watched an agent, ora person, get it wrong without one.

1.    One source of truth for definitions. Metrics, parameters, grains and governance rules live in a versioned knowledge bundle(OKF). Gold SQL, the semantic model and report specs are generated from it and never feed back into it. Only a human edits the bundle.

2.    Ingestion runs on configuration. One standard pipeline loads every table from a validated config record. Adding a source table means reviewing a YAML file; nobody writes a new job. If the config can't express a table, the framework changes once, under code review.

3.    Agents abstain instead of guessing. No agreed data contract: ask. Ambiguous grain: ask. A metric the semantic layer doesn't govern: say "not governed" and stop. We treat abstention as a correct outcome.

4.    Every layer proves itself. Nothing gets trusted until it's checked against reference values: row parity from source to Silver, metric parity in Gold and in the semantic model, small-cell suppression, and every report visual re-queried with DAX and exported to an image that is inspected.

5.    Humans hold the gates. Load pattern, data contract, duplicate-key rules, every code diff and every report layout are approved by a named person, and the approval is written to a decision log.

The semantic layer and the ontology: where meaning lives

Most AI-for-data tools point an agent at tables and hope. Tables hold values. They carry no meaning on their own. A column called status or date says nothing about which business event it records, how it may be aggregated, or what it can be joined to. Our harness puts two layers of meaning between the agent and the data, and both are generated from the same governed definitions.

The semantic layer is the contract for every number. Each metric is defined once: its formula, its grain, its parameters, its filters and its privacy rules. Every report and every agent queries that one model, and so does anyone asking an ad hoc question, so the same question returns the same answer in every tool. The agent never writes a metric; it can only select one that exists. When a request asks for a number the semantic layer doesn't define, the answer is "not governed", with the owner named.

The ontology is the map of the business. It names the entities the business talks about, such as customers, products, orders or episodes of care, and the relationships between them. That map is what lets an agent turn a plain-language request into precise concepts, see that "by region" is a valid cut for one metric and an ambiguous one for another, and refuse a join that doesn't exist in the business, even when it exists in the database.

Together they move intelligence out of the prompt and into a governed asset. Swap the modelor the BI tool, even the agent itself, and the meaning stays. Change a definition and everything downstream picks it up from one reviewed place. It's this layer that turns AI from a fast guesser into a reliable colleague.

Two requests, two correct answers

We put the harness through two business requests on the same day. Both outcomes were correct, but only one of them built anything.

Request one: a set of core KPIs broken down by year, plus a median duration. The supervisor matched every number to a governed metric in the semantic layer and used the ontology to confirm that "by year" was a valid cut. It then checked each layer, cheapest check first. The data, the governed tables, the semantic model and an existing report were all valid, so it built nothing. One person approved the existing layout, and an agent re-verified it: every visual re-queried and every filter proven to work, with the rendered page inspected at the end. The answer arrived in minutes, each figure traced to its definition.

Request two: a cost KPI and a compliance KPI, broken down by a business segment. The supervisor stopped before any agent ran. Neither KPI was defined in the semantic layer, and the segment depended on reference data still waiting on a business decision. It said so and named who could unblock each item. The decision went into the log.

A typical assistant would have shipped a plausible page for request two. Ours shipped a precise list of what's missing and who owns it, which is the behaviour we want in front of a client.

What clients gain

Why this is the path forward

Agents will write more and more of the data stack. The organisations that benefit will be the ones whose agents stay inside a governed context. Ours generate from a single definition layer and verify every hop. Where judgement matters, they hand the decision to a person.

If your team is weighing AI agents for data engineering and needs numbers that stand up in the boardroom, we'd like to show you the harness running on your own questions. Get in touch with Tecknoworks.

‍