TL;DR: The AI strategy document lists three approved models. The production stack runs eleven or more. This happens because the standard evaluation framework (accuracy, cost, latency) actively drives sprawl. The fix is not stricter governance or model bans. The fix is a passive model registry that discovers reality first. Then add a living evaluation harness that scores continuously. Finally, add a single inference gateway that makes shadow AI unnecessary.

Key Takeaways: - The strategy/reality gap is universal. Google engineers publicly mock their own 75% AI-generated code strategy. They ship it at the same time. This proves it is a structural problem, not a maturity problem - Banning models backfires into underground adoption. This happens through personal API keys. It also happens via browser playgrounds. Freemium tiers route data outside corporate perimeters. As a result, the ban fails - The three-axis evaluation (accuracy, cost, latency) is the engine of sprawl, not the solution. Engineers add models because the framework rewards them for doing so

Google engineers are publicly mocking their own company's 75% AI-generated code strategy. They are shipping that very roadmap at the same time. The rest of the industry copies it anyway. The gap between sanctioned AI architecture and deployed reality has never been wider. Most CTOs do not see it until a model they have never heard of breaks production.

Strategy documents list three models. One GPT-class, one Claude-class, one open-source fallback. The slide looks clean.

Engineering repos tell a different story. They hold fine-tuned variants, embeddings models, rerankers, classifiers, and three different LLM APIs. Nobody on the architecture board approved these. Eleven models minimum. The pattern repeats across enterprise teams. Sometimes thirty.

This is not a governance failure first. It is an evaluation failure. Teams adopt new models because the current ones break in ways nobody tracked. The three-axis framework (accuracy, cost, latency) drives the sprawl it claims to prevent. Every AI committee writes this framework down.

Research on model sprawl describes this as "an inability to quickly and easily discover where models are deployed." It is the same inability to track them throughout the organization. That inability is the root cause of duplicate effort. It also causes missed security patches. It also creates orphaned production systems that no one owns anymore.

If the problem were simply rogue engineers, the fix would be easy. It is not. The next section shows why the obvious remedy makes things worse.

Shadow AI is not a policy violation. It is an availability reflex. When a sanctioned model returns a hallucination during a late-night incident, the on-call engineer reacts fast. They swap in a backup before the incident review starts. By morning, the swap is in production. By next quarter, it is load-bearing.

Banning model access without a working replacement creates underground adoption. Personal API keys appear in CI pipelines. Browser-based playgrounds get used for prototyping. Freemium tiers route customer data outside the corporate perimeter.

The org chart says AI governance is enforced. The network logs tell a different story. The gap between policy and packet capture is where risk compounds fastest.

The conversation inside engineering is rarely "which model did the architecture board approve?" It is "which model handles this edge case without breaking?" The two questions have different answers. The first optimizes for compliance. The second optimizes for uptime.

Engineers are not rebelling against governance. They are solving real failures that governance never acknowledged existed. Once you understand that, the next question becomes unavoidable. Which failures, exactly, and which evaluation framework missed them?

The standard three-axis evaluation covers accuracy, cost, and latency. It is what every AI governance committee writes down. It is also what drives sprawl.

Consider accuracy first. It is non-stationary. A prompt that works on Monday fails on Wednesday. This happens because the upstream provider shipped a silent update. Engineers respond by pinning versions. They also test alternatives. They keep the previous model warm as fallback. The "approved" model has quietly become two. Coverage in Why Your LLM Evals Approve Models That Fail treats this as a measurement problem, not a model problem.

Latency requirements are task-specific, not application-specific. Different task classes demand different latency budgets. Summarization tolerates relaxed thresholds. Agentic tool calls need tight ones. One approved model cannot serve both ends of the spectrum. So teams route different request classes to different providers. Two models, then a third when the second one's API rate limits kick in at peak hours.

Cost behaves the same way. A chatty agentic workflow at a few cents per call becomes uneconomic at scale. Teams reach for quantized variants, smaller models, and specialized endpoints. Each one is "cheaper" in isolation. Together, they are sprawl.

The flailing loop, as one engineering lead described it, is when "test cases that work today break next week." Meanwhile, broken cases start working. Engineers chase moving targets by adding models, not removing them.

If the evaluation framework is the engine of sprawl, then version management is where the damage compounds. Eleven models quickly become thirty checkpoints.

Each sanctioned model spawns variants across every axis of specialization. There are fine-tunes for domain language. There are LoRA adapters for specific tasks. There are quantized versions for cost control. There are prompt-template branches that behave like separate models. They tokenize, route, and respond differently. The "three models" in the strategy document is, in production, three families. Each family contains siblings that diverge from the base model.

No single registry tracks them. Related analysis in The Blind Spot in Your AI Stack That Audits Always Find First shows the same condition from a security angle. Here, each missing model is an untracked OAuth scope.

Version explosion makes the model count a lagging indicator. The real number includes checkpoints, embeddings indexes, and prompt templates. That number is always higher than what leadership was quoted. A board presentation that says "three models" describes only a fraction of what is serving traffic.

Systems still running in production long after deployment are not the cleanest. They are not the ones with the cleanest strategy documents. They are the ones where someone built a registry that survived the next reorg. Discovery is the first step toward control. However, building a registry from scratch is where most in-house teams stall for long periods. There is a faster path.

AI governance succeeds when it starts from reality, not policy. The pattern below ships across enterprise deployments. It pairs well with Your Model Registry Passes Tests. It Will Still Fail Audit.. That post describes what happens when the registry captures models but not accountability.

Step 1: Deploy a passive model registry. It scrapes inference logs, API gateways, and vector stores. It enumerates every model in production, including the ones no one will admit to using. Engineers do not self-report. The registry finds them. This is the part that takes long when done wrong. It is fast when done right.

Step 2: Tag each model with three required fields: business owner, data classification of inputs, and rollback path. Anything missing a tag is auto-quarantined after a grace period, not deleted. Engineers keep working while visibility improves. Quarantine is reversible. Erasure is not.

Step 3: Replace the three-axis evaluation with a living evaluation harness. It re-runs golden test sets on every model version and every upstream API change. Accuracy, cost, and latency are scored continuously, not quarterly. The eval harness is the registry's immune system. Your AI Ships Daily. Your Risk Reviews Don't. describes why quarterly cannot survive contact with weekly releases.

Step 4: Route all inference through a single gateway that enforces the registry at request time. New models can be registered quickly. The primary motivation for shadow AI disappears. That motivation was waiting through a slow procurement review. In practice, this is the pattern that ships, not the pattern that lives in slide decks.

When this lands, the number on the slide and the number in production converge.

Incident response collapses. Every model in the blast radius is known, owned, and versioned. The "which model is serving this request?" question gets answered instantly. It no longer triggers a long investigation. The on-call engineer stops guessing and starts fixing.

Model retirement becomes routine. A registry with business owners means sunsetting a deprecated embeddings model is a ticket. It is no longer a multi-month archaeology project. Technical debt stops accumulating because nothing is orphaned. The AI governance loop closes.

The strategy document starts to mean something. The board sees three models because there are three models. The engineering team trusts the roadmap because the roadmap reflects how they actually work. It does not reflect how a vendor pitch deck suggested they should. That trust shows up in production. It looks like systems still running long after deployment, with roadmaps that survived contact with reality.

The pattern works because it inverts the usual sequence. Most teams try to govern models they cannot see. This pattern makes models visible first. Then it governs what is actually there.

How many AI models is too many for an enterprise?

Counting models is less useful than measuring the ratio of sanctioned models to discovered models. When production has outpaced the strategy document, sprawl is active and risk is compounding. This happens regardless of the absolute number.

What is the difference between AI sprawl and shadow AI?

Shadow AI is one cause of sprawl. It means teams using unsanctioned tools. AI sprawl is the broader condition. It is the uncontrolled growth of sanctioned, semi-sanctioned, and unknown models across the stack. You can have sprawl without shadow AI. This happens if governance never caught up to legitimate adoption.

How do you discover AI models that are already in production?

The fastest method is passive enumeration. Tap API gateway logs, cloud billing exports, and inference endpoint telemetry. This surfaces models that no spreadsheet knows about. That is most of the true production footprint.

Does AI model sprawl increase security risk?

Yes. Each unsanctioned model is an unmanaged OAuth scope. It is also an unpatched dependency and an undefined data-handling boundary. Attack surface scales linearly with the number of models that lack owners.

How long does it take to build a model registry from scratch?

In-house teams stall for long periods before the registry is trusted by engineering. Specialized deployments complete the discovery and tagging layer quickly. They do this by starting from passive enumeration rather than waiting for engineers to self-report.