The pattern is always the same. A team spends six months building a pilot. The demo is impressive. Leadership is excited. Then someone asks: “How do we connect this to our actual systems?”
And the room goes quiet.
The pilot worked beautifully in isolation. Clean data. Controlled environment. No legacy systems to fight. No compliance requirements. No one asking who’s on call when it breaks at 2am.
This isn’t a few unlucky companies. It’s the norm. m. 80% to 95% of enterprise AI initiatives never reach production. IDC measured it. RAND confirmed it. Gartner keeps updating the numbers, and they keep getting worse.
I’ll be honest. For years, I thought the model was the hard part too. We’d celebrate accuracy numbers and algorithm breakthroughs. Then I watched a team spend eight months building a model that performed beautifully on test data and collapse within weeks of meeting real users, real integrations, and real data quality. Not because the model was wrong. Because no one had engineered the system around it. That was the moment it clicked: the model is 20% of the problem. The other 80% is production systems engineering.
Let’s be specific. It’s not bad algorithms. It’s not the wrong use cases. AI pilots die for three reasons.
Most AI initiatives start with the wrong question: “Can we build a pilot that proves this concept works?”
The right question is: “Can we engineer a system that survives contact with reality?”
These aren’t the same thing. Pilots are optimized for clean data, controlled environments, and impressive accuracy metrics. Real systems must survive dirty data. Legacy integrations built 15 years ago. Users who find every edge case. Compliance audits no one planned for.
When you optimize for one, you don’t get the other for free.
“The model works great. Now we just need to integrate it. ” That word —“just” — is where pilots go to die.
Integration isn’t a step at the end of the project. Integration IS the project. Your AI needs to pull data from systems built before anyone said “machine learning.” It needs to handle the 47 different date formats your legacy systems use. (I wish I were exaggerating.) It needs to write outputs to databases with schemas no one fully understands. And it needs to fail gracefully when any of this breaks.
We once spent the better part of two years helping a pharmaceutical cooperative turn data from over 600 member pharmacies into something usable. When we started, the analysis lived in spreadsheets. Prescriptions, insurance claims, inventory movements. All trapped in Excel files and disconnected systems. No warehouse. No pipeline. No way to see what the data was actually saying.
The first years were unglamorous. Data pipelines. Quality frameworks. Inconsistent formats from hundreds of sources. No AI. No demos. Just the foundation nobody wants to build and everybody needs.
Today that organization runs five AI use cases in production, and one of their data products generates seven-figure annual revenue. But none of it works without those unglamorous first years.
Most AI teams have never made software work inside a 20-year-old environment that processes a million transactions a day. That’s not AI work. That’s systems engineering.
The pilot worked. The integration is… in progress. Then compliance walks in.“Who approved this model?”
“How do we audit the decisions it makes?”
Silence.
AI pilots are built by people who are brilliant at building pilots. But they rarely build the operational scaffolding that the real world demands: monitoring, incident response, model drift detection, audit trails, versioning and rollback, performance SLAs.
These aren’t nice-to-haves. Once you deploy, they’re requirements. And building this infrastructure for AI is a different discipline entirely
So who’s supposed to do this work?
Look at the landscape honestly. AI consultancies are brilliant at building models. And most of them hand off before the model ever meets a legacy system. Software firms understand production engineering, but they’re bolting AI onto existing capabilities like an afterthought. Data science teams go deep on the math and the algorithms, but ask them to integrate with a system that hasn’t been documented since 2008 and you’ll watch the
confidence drain from the room
System integrators have the scale, but scale without production discipline just means you spend 18 months and $5 million arriving at the same pilot graveyard. Cloud platforms sell managed AI services, and they’re genuinely useful, but a platform doesn’t solve your integration, your governance, or the legacy systems you can’t rip out.
Each of these groups does real work. Good work, even. But none of them owns the full stack.
Getting AI to survive in the real world isn’t an AI problem. It’s a systems engineering problem that happens to involve AI. It requires people who understand how software behaves under load. How data platforms maintain quality at scale. How AI models degrade in real-world conditions. And how to build the infrastructure that keeps it all running when no one’s watching.
That combination doesn’t appear overnight.
It takes 25 years of building production software. 15 years of engineering data platforms. A decade of deploying AI that works in the field. Notice the order. AI comes last, not because it’s less important, but because it doesn’t survive without the foundation.
Notice the order. AI comes last, not because it’s less important, but because it doesn’t survive without the foundation.
Ten years ago, we started building a data platform for one of the world’s most prestigious consulting firms. It serves the mining industry and the investment banks that finance it, ingesting data from public sources, combining it with proprietary intelligence, and giving consultants the tools to build custom analysis that puts finished insights directly in front of their clients.
That platform has generated nine-figure revenue for our client. It’s been running for ten years. Not just running. Evolving. New data sources. New analytical capabilities. New user workflows. And right now, we’re integrating AI capabilities into it.
A medical device platform we helped build from a hackathon prototype now holds over 200,000 breathprints across 100+ research institutions worldwide (from Dutch clinics to research labs in Sydney). From a single 30-second breath sample, it predicts which lung cancer patients will respond to immunotherapy, sparing them from ineffective treatments before symptoms even appear.
These aren’t projects that ended. They’re capabilities that compound. And that distinction, between a project and a capability, is exactly what production systems engineering is about.
And now those capabilities face their biggest test yet.
We’ve entered the age of AI agents, systems that don’t just answer questions but act on their own. They approve invoices. They adjust treatment protocols. They draft contracts and route them for signature. They make decisions at speeds no human can review in real time.
Think about what that means for a moment. A predictive model gives you a recommendation. A human still decides. An agent takes the action. The feedback loop between “the AI suggested something” and “something happened in the real world” just collapsed from hours to milliseconds.
When a chatbot hallucinates, you get an embarrassing screenshot on social media. When an agent hallucinates with write access to your ERP, it processes refunds that shouldn’t exist. When it hallucinates with access to clinical systems, someone gets the wrong treatment protocol. When it hallucinates inside a legal workflow, a contract goes out with terms no one agreed to.
The difference between a well-engineered system and a pilot held together with hope isn’t just expensive anymore. It’s dangerous.
The integration has to be tighter. Agents don’t just read from your systems, they write to them. The governance has to be deeper. You need audit trails for decisions no human made. The monitoring has to be real-time. By the time you catch a bad decision in a weekly review, the agent has already made a thousand more.
And the people building this need to have done it before. Not necessarily with AI agents specifically, but with production systems where failure has consequences. Where uptime isn’t aspirational but contractual. Where “it worked in testing” is the beginning of the conversation, not the end.
The engineering discipline that was important for predictive models just became existential for agentic AI.
Each one answers a question your board will eventually ask. Each one breaks independently. And each one breaks differently at 2am than it does in a demo.
Most failed AI initiatives addressed two or three of these dimensions well. Reaching production requires all ten. Not perfectly, but deliberately, with clear ownership and operational plans for each.
These aren’t theoretical categories. They’re the things no one thought about during the pilot. The things that determine whether your AI system is still running a year from now, or joins the graveyard alongside every other pilot that looked great in the demo room.
I won’t pretend there’s a simple answer. But there is a clear discipline: start with production requirements, not model requirements. Engineer the system around the AI, not the other way around. Build governance before you need it, not after the audit. And treat deployment as the beginning, not the finish line.
80% to 95% of AI pilots never reach production. That’s not a statistic. It’s a system failure, and system failures have engineering solutions.