I spent twelve months building semantic models across a large-scale Microsoft Fabric project using an AI agent for measure generation, relationship design, and TE2 script creation. The agent ran on the same model the entire time. No upgrades. No fine-tuning. No provider switch.
The system’s output quality improved anyway. Measurably.
By report twenty-five, first-pass accuracy on measure definitions was noticeably higher than at report five. Not because the model got smarter. Because the harness file grew from forty lines to over two hundred, encoding naming conventions, DAX patterns, relationship constraints, and documentation requirements that earlier reports had surfaced the hard way. The memorybank directories accumulated twelve months of stakeholder corrections, domain gotchas, and implementation decisions. The model read richer context every session.
That experience confirmed something I keep seeing in the broader industry: agentic systems improve at three distinct layers, and the model is the slowest of them.
Three layers, three improvement speeds
Every agentic system shares the same basic architecture, whether it is a coding assistant, a data platform automation tool, or a customer-facing support agent:
The model. The language model doing the reasoning. Sonnet, GPT, Gemini, Llama, Qwen. Whatever sits at the centre.
The harness. The code surrounding the model that connects it to the environment. Tool definitions, workflow orchestration, routing logic, guardrails, execution constraints. The scaffolding that turns a language model into a working system.
The context. The instructions, memory, and accumulated knowledge fed to the harness for each task. Persistent instruction files, skills, domain-specific rules, evaluation history. What the system knows about your particular problem.
All three can be improved. Most teams improve only one: they wait for the next model release and ignore the two layers they control.
The model layer: real but not yours
Model improvements are real. The gap between GPT-3.5 and current frontier models is enormous. Each generation handles more nuanced reasoning, longer context, better tool use. But model improvements happen on the provider’s schedule, not yours.
Two trends are starting to shift that.
Open models are closing the gap. Base performance on open-weight models (Llama, Qwen, Mistral, DeepSeek) is approaching frontier levels for many practical tasks. Benchmarks still show gaps on the hardest reasoning problems, but for structured output, code generation, and domain-specific work, the difference is narrowing fast.
Cost pressure is making open models practical. Coding agents burn through tokens at a rate that makes frontier model pricing a material concern. An agent that generates and tests dozens of code iterations per task can consume hundreds of thousands of tokens in a single session. Open models on cheaper infrastructure offer a way out, particularly for high-volume, moderate-complexity work.
The third angle is domain-specific fine-tuning. Companies accumulating traces from agent runs (every interaction, every correction, every success) can fine-tune open models on that data. An expense-processing model trained on a company’s own transaction patterns will outperform a general-purpose model on that specific task, often at lower latency and a fraction of the inference cost.
This is genuine continual learning at the model layer. For data engineering teams building AI-ready platforms, it is the most appealing path because it feels like the kind of infrastructure work they already know. But it requires tooling (training pipelines, evaluation datasets, deployment automation) that most teams do not have yet. The feedback loop is long. The iteration cycle is measured in weeks or months. It is the slowest layer to improve, not because it is unimportant, but because the infrastructure bar is high.
The harness layer: where your leverage lives
The harness is the code around the model. It is what turns ‘ask an LLM a question’ into ‘run a reliable system.’
In my Fabric project, the harness was an agents.md file defining naming conventions, DAX style rules, relationship principles, and documentation requirements. The agent did not know these rules natively. The harness made them structural. A prompt says ‘please follow these conventions.’ A harness makes it difficult to violate them.
Harness improvements compound faster than model improvements because the feedback loop is tight. You run the agent, observe the output, identify where the harness failed to constrain or guide, edit the harness, and the next run is better. This cycle can happen multiple times per day. Model improvements happen on quarterly release cycles at best.
The industry data supports this. Published research on automated harness optimisation (including work from MIT and Stanford) has demonstrated that tuning the code around a coding agent, without changing the model, can match or exceed hand-tuned configurations on software engineering benchmarks. Same model. Different harness. Materially different results.
I have seen the same pattern at smaller scale across my own projects. Over twenty-five Fabric reports, harness improvements accounted for more output quality gain than any model upgrade would have delivered in the same period.
What harness improvement looks like in practice
Harness engineering is not prompt engineering. It is software engineering applied to the system around the model:
Tool definitions. Which tools the agent can access, what parameters they accept, what outputs they return. Tighter tool schemas produce more reliable tool calls. Both Anthropic and OpenAI publish detailed guidance on structuring tool definitions for predictable agent behaviour: precise parameter descriptions, explicit return types, constrained enums where the domain allows.
Model routing. Not every task needs a frontier model. A routing strategy that sends simple tasks (documentation, formatting, lookups) to lightweight models and reserves frontier models for complex reasoning reduces cost without reducing quality where it matters. The harness encodes this routing, not the model.
Guardrails. Explicit constraints on scope and behaviour. File boundaries, action approval gates, output format requirements. The difference between an agent that stays on task and one that rewrites your test framework when you asked it to fix a single failing test is a guardrail in the harness, not a better prompt.
Structured outputs. Forcing agents to respond in typed formats (JSON schemas, function call signatures) eliminates an entire class of parsing failures. Both major provider ecosystems support this natively. The harness enforces the schema. The model fills it.
Evaluation loops. Automated checks that run after agent actions, comparing output to expected behaviour. These are the forcing function. In classical machine learning, gradient descent updates model weights based on a loss signal. In harness engineering, evaluation results update the harness code. The evals provide the training signal. The harness is the thing that learns.
The context layer: accumulated knowledge
Context is what you feed the harness to make it specific to your problem. Persistent instruction files, skills, memory systems, per-task knowledge.
In my Fabric project, this was the memorybank: per-report directories storing implementation decisions (‘chose SUMX over CALCULATE because the grain requires row-level evaluation’), stakeholder feedback (‘finance confirmed internal recharges excluded from revenue totals’), known issues (‘Direct Lake falls back to DirectQuery on the rolling 12-month variance measure, accepted for now’), and cross-source relationship mappings.
The memorybank was not a chat log. It was curated context, maintained and updated after every review cycle. Raw conversation history is noisy. Curated context is signal.
This layer improves through a different mechanism than the harness. Harness improvements come from engineering: identifying structural failures and fixing them in code. Context improvements come from domain knowledge: learning what the finance team means by ‘revenue’, discovering that project codes map differently across source systems, noting that a specific DAX pattern triggers DirectQuery fallback in a way the documentation does not warn about.
This is where the argument for domain experts contributing to agent systems becomes practical. The people who know that ‘revenue’ excludes internal recharges are the finance team, not the engineers. The people who know that a specific equipment hierarchy follows a non-standard parent-child relationship are the maintenance planners. When those domain experts contribute context (through instruction files, skill definitions, or structured feedback), the system improves at a rate that no amount of model fine-tuning matches for that specific domain.
The pattern I see separating teams that get compounding returns from those that plateau: teams treating instruction files and context as living artefacts, updated after every significant failure, reviewed on a regular cadence, see steady improvement. Teams that write their instruction files once and never revisit get the same results six months later, regardless of model upgrades.
Evals are the training gradient
The analogy to classical machine learning is worth making explicit.
In supervised learning, you have training data, a model, and a loss function. The loss function measures the gap between predicted and actual output. Gradient descent uses that gap to update model weights. The model improves because the loss provides a directional signal about what to change.
In agentic systems, evaluations serve the same structural role, but they update the harness and context layers rather than model weights. You run the agent against a set of tasks. You measure accuracy, constraint adherence, latency, cost. The gap between expected and actual output tells you what to fix. You fix it in the harness or the context. You run again.
This is the same principle behind test harnesses for data pipelines: build the verification scaffolding first, then iterate against it. The evals surface regression. They also surface improvement opportunities. The system gets better because the feedback loop is fast and the signal is clean.
Both Anthropic and OpenAI publish evaluation frameworks and guidance for measuring agent performance. The specific tooling matters less than the discipline: write evals before you optimise, run them after every change, treat the results as the primary feedback signal. A team without evals is optimising on vibes. Vibes do not compound.
Agent identity is coming and most teams are not ready
As agents move from developer tools into business systems, a question emerges: when an agent acts, on whose behalf is it acting?
Two patterns are forming. In the first, agents use delegated user credentials. My agent accesses Slack with my permissions, sees what I see. A colleague’s agent sees different results. In the second, agents have fixed credentials (a service account), and everyone interacting with that agent gets the same view.
Neither is universally correct. Delegated credentials suit personal tools (coding assistants, research agents). Fixed credentials suit shared resources (team support bots, automated triage). Some SaaS providers are making it straightforward for agents to create their own accounts, which accelerates the second pattern.
The unsolved part: most tooling does not make this distinction explicit. Agents inherit credentials from whoever configured them, which is neither a deliberate delegation nor a deliberate fixed identity. As agents gain access to email, CRM, code repositories, and deployment pipelines, getting identity right becomes a governance requirement. Not optional. Not a nice-to-have.
What this means for how you build
Improving an agentic system is three activities, each with its own feedback loop, cadence, and responsible parties.
Model selection and routing is an engineering decision, revisited when new models release or cost constraints change. For most teams, this means picking the right model tier for each task class and tracking open-model benchmarks for domain-specific opportunities.
Harness engineering is software development. The harness belongs in version control, reviewed in pull requests, tested against evals, updated after every significant failure. It is code. Treat it like code. For data platform teams adopting agentic workflows, this means the same CI/CD discipline applied to pipeline code should apply to agent harnesses.
Context curation is collaborative. Engineers maintain the structure. Domain experts supply the knowledge. Instruction files, skills, and memory systems should be living documents, not write-once configuration. The domain expert who corrects an agent’s interpretation of ‘revenue’ is improving the system in a way that persists across every future session.
The teams making the fastest progress have separated these three activities, assigned clear ownership for each, and built feedback loops running at different speeds. Weekly for model routing reviews. Continuous for harness iteration. Ad hoc for context updates triggered by real failures. This is what technical leadership on agentic systems looks like in practice: not picking the best model, but designing the improvement system around it.
Trade-offs and honest limits
Harness engineering has diminishing returns. The first fifty lines of constraints deliver enormous improvement. The next two hundred deliver less. At some point, the harness becomes so prescriptive that it constrains useful agent behaviour alongside the harmful. Finding the right level of constraint is an ongoing judgement call, not a solved problem.
Context curation has a noise problem. A memorybank with three hundred entries is richer than one with thirty. But if half those entries are outdated, contradictory, or irrelevant, the model spends context window capacity processing noise instead of signal. Context curation requires periodic pruning, not only adding.
Evals are expensive to write well. Measuring what matters (not what is easy to measure) takes domain expertise and iteration. A poorly designed eval gives false confidence. I do not have a shortcut for this. Start with the three metrics that matter most. Expand when those stabilise.
Domain expert contribution has practical limits. Domain experts supply irreplaceable knowledge for the context layer. They do not typically design tool schemas, write evaluation pipelines, or debug routing logic. The division of labour matters: domain experts own context, engineers own the harness, providers own the model.
The compounding advantage
Model capability advances on the provider’s schedule. Harness quality advances on yours, every session. Context richness grows with every stakeholder correction, every edge case documented, every failure fed back into the system.
When I started that Fabric project, the agent produced reasonable but generic output. Twelve months later, the same model produced measures matching our naming conventions, respecting our relationship constraints, following our documentation patterns, and accounting for domain-specific edge cases it could not have known about without the accumulated context.
The model did not learn any of that. The system did. The harness and context layers improved on a cycle measured in hours, not quarters.
That compounding is the advantage. It belongs to whoever builds it.