There is a quiet danger in modern AI systems that has nothing to do with them breaking.
In conventional software systems, failure is relatively binary, immediately noticeable and staff receive waves of texts, emails and pings, with somebody on it within the hour. However, an AI system that has been handed a poor account of the business behaves nothing like that. It answers fluently, explains its reasoning, and is confidently incorrect in precisely the register a decision-maker is inclined to trust.
Failure announces itself. Wrongness does not. It surfaces only when somebody who knows the business looks closely, and often not even then.
The model is rarely the fix
When an LLM-based system starts producing poor outputs, our first instinct is generally to reach for a newer, more expensive, more capable model. It is the option every provider keeps pushing in front of us, and it is rarely the one with the most impact.
The better place to start is with the variables and inputs you control. There are many levers such as enriching the data available, polishing the context assembled from it, and evaluating the outputs rigorously. Evaluation tells you whether any of it is working, but it leans on the same foundation. A reliable eval set must be graded against a trustworthy reference, and that reference comes from the same estate.
Competitors can be calling the same model, with the same capabilities, in the same way. The one with a richer, polished and correct account of its business sitting behind it will get materially better results and materially better insight. Same model, same price, different ceiling.
Frontier models are incredible amplifiers. Feed one a clean, rich signal and it will resolve detail nobody knew was there. Feed it noise and it will give you louder, more confident noise. The gain is the same either way.
A better amplifier can move you closer to the ceiling of what your data can support. It cannot raise that ceiling.
How legacy estates end up this way
Nobody sets out to build a poor data estate. It accumulates, and for entirely rational reasons. A finance system is procured because it is the right tool for the job. A CRM is chosen because sales needed a CRM. Each decision is sound in isolation. What emerges over a decade is six systems with six definitions of a customer, several of which disagree about who is still active.
Layered on top is the debt that never reaches the top of the backlog - the manual reconciliations that happen every Friday, the niche quirks of a system that is not broken enough to justify fixing. All of it is additional noise.
A seasoned analyst navigates this without much difficulty, because they carry the missing context in their head. They know which of the six customer tables is the one to pull from. That knowledge is not written down, and that is the problem. It exists in a form no automated system can reach, supplied silently by someone who was never asked for it. That also means the estate only holds together while they are still there, unless someone captures what they know.
Where the medallion architecture earns its place
The medallion architecture is a way of structuring a data estate in three progressively refined layers, named bronze, silver and gold. It gives each of those problems a specific place to be solved, and it is not a new concept. Structured, modelled, deliberately shaped data has been the foundation of trustworthy reporting for decades.
What has changed is the consequence of not having it. A report built on shaky data tends to look obviously wrong to anyone who knows the business. A model handed the same data fills the gaps with whatever is most probable, and presents the result with exactly the same confidence it presents fact.
Bronze is the raw capture. Data lands exactly as the source produced it, unmodified, including the parts that are inconvenient. The value of this layer is not analytical, it is evidential. When a figure is challenged six months later, bronze is what lets you trace it back to what the source actually said rather than to what a transformation assumed it said. It also means that when your understanding of the data improves, and it will, you can reprocess history rather than have lost it.
As more decisions come to rest on systems built on LLMs, explainability stops being a courtesy and may well become a requirement. Being able to show what the source actually said, separately from what a transformation or a model made of it, is what makes an audit defensible at all.
Silver is where the noise is filtered. Schemas are enforced, types standardised, duplicates resolved, and entities reconciled so that a customer in the finance system and a customer in the CRM are recognisably the same customer. This is also where the seasoned analyst’s knowledge stops being a bottleneck. Every rule they applied silently becomes explicit pipeline logic - written down, tested, reviewed and applied identically to every record.
Silver is the layer LLMs should be reasoning over. It is clean but still at row level, so the grain is intact and a model can answer questions nobody thought to build a report for. That is a higher bar than most silver layers are held to. Column names must be intelligible without a data dictionary, relationships have to be explicit rather than implied, and definitions have to be documented, because there is no layer downstream to supply the meaning.
Gold is the business-facing layer - aggregated, shaped to answer known questions, with definitions agreed rather than assumed. This is the reporting and semantic layer, and serving reports is a different job from serving AI. The moment you aggregate, you have decided which questions can be asked. That is exactly what you want for query-intensive reporting graphs and tables, and exactly what you do not want for a system whose value lies in answering the question nobody anticipated.
Polishing is a capability, not a project
Data quality work is usually framed as a project with an end date, when in reality, sources drift, schemas change without warning, new systems arrive, and a one-off cleanse begins decaying the day it ships.
None of that diminishes the foundation, but it is worth being precise about what the foundation is. It is not a cleaned dataset, which is a snapshot and starts ageing the moment it is produced. It is the infrastructure and architecture that moves, conditions and prunes the data on the way through. That is the part that must be right, because once it exists, quality becomes a property of the system rather than a periodic effort.
The mistake is treating that build as a phase to be signed off before anything interesting can begin. There is no point at which the data is finished, so the foundation and the systems depending on it are better built in step, each exposing what the other actually needs. The fastest way to find out that a definition is wrong is to put it in front of something that relies on it.
The ceiling
The organisations that get real value from AI over the next few years will not be the ones with the best prompts, or the ones that moved first, or the ones that picked the right model. They will be the ones that did the diligent work of getting their data into a state where an automated system can be trusted to reason over it.
A better model can move you closer to the ceiling. The data underneath is the only thing that raises it, and a well-built medallion data solution is one of the most reliable ways to get it there. Only one of those is something you can buy, and it is not the one that will set you apart.