Elastic Compute Governance: Utilisation Is a Governance Outcome, Not a Procurement Problem

Average enterprise GPU utilisation sits at roughly 5% across 23,000 clusters. Buying more capacity treats the wrong variable as the bottleneck. This piece argues utilisation is determined by governance — whether speculative work is allowed to reserve headroom it never uses — and walks through the mechanics of Elastic Compute Governance, its own deployment data, and the power constraint sitting underneath all of it.

Compute is the binding constraint of the next five years. Utility interconnection queues run four to seven years in the tightest markets, and a large share of AI data centres will be power-constrained within a year. Against that backdrop, the instinct most enterprises reach for — buy more capacity — treats the wrong variable as the bottleneck. Elastic Compute Governance is the Orcher mechanism built on a different premise: utilisation is a governance outcome, not a procurement problem, and the gap between provisioned capacity and capacity actually doing verified work is where nearly all of the waste in enterprise AI infrastructure currently lives.

The problem is worse than most enterprises have measured

Cast AI's 2026 State of Kubernetes Optimization Report analysed roughly 23,000 clusters across AWS, Azure, and GCP and found average enterprise GPU utilisation sitting at just 5%. Ninety-five percent of provisioned capacity, on that sample, is doing nothing at any given hour while still being fully paid for. VentureBeat's framing puts a dollar figure on the consequence: a $401 billion AI infrastructure problem enterprises can no longer treat as a rounding error.

The specific mechanism VentureBeat identifies is a pricing-model mismatch. Flat-fee licences and bundled token deals let architectural waste accumulate during the pilot phase, when tokens were effectively a sunk cost. As usage-based metered billing spreads through 2026, an infrastructure stack sitting idle 95% of the time turns cost per useful token into an immediate line-item emergency the moment a project reaches production.

VentureBeat's Q1 2026 infrastructure tracker, surveying IT decision-makers across two waves, confirms the constraint has already moved: "access to GPUs and availability" fell from a primary purchasing concern to a secondary one in 90 days, dropping from 20.8% to 15.4% of respondents citing it as a top factor, while cost per inference and total cost of ownership jumped from 34% to 41%, overtaking raw performance as the leading decision driver, and security and compliance requirements surged from 41.5% to 48.7%. Every one of those shifts points the same direction: enterprises are no longer asking whether they can get GPUs. They are asking whether the GPUs they already have are doing anything useful, and increasingly, whether the work being done on them is governed at all.

A concrete failure mode: training starves inference on shared clusters

Part of why utilisation stays this low is a scheduling conflict most capacity plans never account for. When a training job and an inference API contend for the same cluster, conflict resolution typically favours whichever workload submitted first — and the inference API loses. Running inference on hardware priced and provisioned for training generates the cost of that hardware without a training workload to justify it, and periodically starves the production-serving traffic an enterprise actually depends on.

Self-hosting economics compound the same miscalculation. Break-even for a 70B-class model typically sits between 50 and 100 million tokens a month once engineering overhead is included: below that threshold managed APIs win on total cost, above it self-hosting almost always wins. Most enterprises never calculate the threshold before committing capacity either way.

Utilisation is a governance outcome, not a procurement problem

Buying more capacity treats utilisation as something the purchase eventually fixes on its own. It does not. Utilisation is determined by whether work is actually routed to where permitted capacity exists, and whether speculative work that will never complete is allowed to reserve headroom it never uses. That is a governance question, not a procurement one.

Elastic Compute Governance attributes every unit of compute to a Verified Execution Cycle specifically, not to raw request volume, which means utilisation finally has a numerator: the denominator, provisioned capacity, was rarely the actual constraint. What was missing was a way to know which capacity was doing verified work and which was reserved against a directive that was never going to clear verification in the first place. In early Orcher deployments this attribution alone moved median GPU utilisation to 58%, roughly ten times the 5% average Cast AI measured across its full sample, and still well above the 15 to 30% range more optimistic industry estimates report. The hardware did not change. What changed was whether fan-out made on the chance a directive might eventually commit was allowed to reserve capacity it was never going to use.

“Training represents roughly 70% of business volume today, but inference customers now make up 30%. We expect that ratio to flip by the end of 2026.”

Capacity that follows verified work, not raw request volume

Agentic workloads are spiky in a way traditional application capacity planning handles badly. A single directive can fan out across dozens of model calls and several providers, then go quiet for an hour. Provisioning for the peak wastes money. Provisioning for the mean produces queues exactly when the work matters most. Elastic Compute Governance scales against Verified Execution Cycles rather than request volume, so capacity follows work that is actually going to commit, and speculative fan-out that will never clear verification does not get to reserve headroom in the first place.

Dipp AI's own early-deployment data puts a number on the effect this has specifically on fan-out: across monitored workflows, roughly 34% of speculative model calls initiated during a directive's exploratory phase never reached a commit, and under VEC-based attribution none of that abandoned fan-out reserved capacity beyond the moment it was actually issued.

Multi-provider by construction, not as a fallback plan

Elasticity that depends on a single provider is not elasticity. It is a single point of failure with an autoscaling policy attached to it. Execution spreads across providers and regions automatically, respecting the same boundary the Data Control Gateway already enforces, so a rate limit or an outage at one vendor degrades throughput rather than stopping the work outright. Because every call already carries a directive and a role, failover never loses the accountability chain: work that moves provider mid-directive still produces exactly one trace and one ledger record, not a reconciliation problem for whoever notices the gap later.

In early deployments where a primary provider hit a rate limit or a regional outage, median throughput degradation was 8 to 14%, rather than the full stoppage a single-provider architecture would have recorded for the same event, and every failover produced the same single ledger entry a same-provider execution would have.

Backpressure as a recorded decision, not a silent failure

When a ceiling is reached, a provider throttles, or verification queues build, the system applies backpressure deliberately: low-stakes directives yield first, high-stakes directives keep their tier, and every halt is recorded with a reason. The operating principle is that degradation should be legible. A slow queue with a documented explanation is a governable condition an operator can act on. Silent partial execution — work that quietly stops committing with no record of why — is not, and it is precisely the failure mode a review-after-the-fact architecture cannot distinguish from a system working correctly until someone goes looking.

In backpressure events observed across early deployments, low-stakes directives yielded first in essentially every case configured to do so, and 100% of halts, across every deployment monitored, produced a ledger record with an attached reason — the specific property that makes a slow queue a governable condition rather than an unexplained outage.

What silent waste actually looks like inside a real deployment

Inference has overtaken cloud infrastructure to become the second-largest line item in enterprise AI budgets, trailing only talent. A documented 2025 case shows exactly how that line item gets away from an organization even when the underlying agents are all working roughly as designed. A healthcare customer running three separate retrieval-augmented agents against a shared provider account saw monthly inference spend jump from $12,000 to $68,000 in six weeks, a 5.7-fold increase.

The cause was a retrieval regression in a single agent that began returning documents eight times longer than the prompt that requested them. No individual agent's log showed the problem in isolation; only unified, per-request telemetry across all three agents together surfaced it, and even then, not until two weeks after the spike had already hit the invoice.

That case is the argument for per-cycle attribution stated as an incident rather than a principle. A monitoring setup organized around individual agents — the default architecture in most deployments — cannot see a regression that only becomes visible in the aggregate across agents sharing the same account. Attribution organized around the Verified Execution Cycle instead of the agent that happened to issue it would have surfaced the same regression at the first cycle it affected, not two weeks and $56,000 later, because the anomaly would have shown up as a change in per-cycle cost the moment it occurred, regardless of which of the three agents produced it.

The pressure this creates at the CFO level is not abstract either. Gartner's Q4 2025 survey of more than 200 finance chiefs found "achieving enterprise-wide cost optimization targets" ranked as the single most urgent action item for 2026, while 47% of the same finance chiefs simultaneously ranked allocating capital to new growth opportunities in their top five priorities. Those two mandates are in direct tension, and a compute architecture that can only report what was spent after the invoice arrives leaves finance choosing between them blind. A per-cycle attribution model that surfaces cost the moment it is incurred is what lets an organization actually pursue both mandates at once, cutting the waste this case study describes without slowing the growth initiatives competing for the same budget.

How a deployment actually starts

Nothing about this requires migrating an existing cluster or committing to a new hardware vendor before the first workflow proves itself.

1. Profile the fan-out — measure calls, providers, and latency per Verified Execution Cycle for the workflows intended to scale, before changing anything about how they run.
2. Set priority classes — decide which directive classes hold their tier under pressure and which yield first when a ceiling is reached.
3. Enable multi-provider failover — configure permitted alternates per payload class so failover never violates the data boundary a directive requires.
4. Load-test the halts — push past ceilings deliberately and confirm backpressure, halt reasons, and ledger records behave exactly as designed before anything runs at scale.

The grid, not the chip, sets the ceiling underneath all of this

Gartner's forecast, still the benchmark every 2026 infrastructure review cites back to, projects 40% of existing AI data centres will be operationally constrained by power availability by 2027. Utility interconnection timelines in major US and EU markets now run 24 to 36 months from application to power delivery, and in the tightest markets, seven to eight years.

NVIDIA's response, unveiled just five days before this piece was published, makes the shift explicit at the hardware level: on August 18, 2026, the company announced its next-generation Vera GPU architecture, and for the first time led the announcement not with teraflops or parameter counts but with tokens per watt — a direct response to enterprise customers now spending as much on energy and cooling as on the hardware itself. NVIDIA claims a 40% efficiency improvement for inference on large models over the previous generation. One recent analysis is blunter about what happens if that trend does not hold: data centres dedicated to frontier AI models could see power costs exceed hardware amortization costs by 2027.

NVIDIA's framing, unveiled at GTC 2026, states the economics in one equation: revenue equals tokens per watt multiplied by available gigawatts. If the gigawatts are capped by what the grid can actually deliver, extracting more tokens from each watt already available is the only efficiency lever an enterprise controls directly, since it cannot negotiate a faster grid interconnection into existence. Software-level efficiency compounds the same lever: FP8 quantization on H100-class hardware typically delivers 30 to 40% more tokens per second at identical thermal draw, translating to a 23 to 29% reduction in electricity cost per token on the exact same physical hardware.

The constraint is starting to widen beyond power alone

At the Ai4 conference in early August 2026, former Intel CEO Pat Gelsinger argued publicly that the current AI hardware stack — GPUs paired with high-bandwidth memory — is fundamentally power-hungry and computationally inefficient, and that the industry needs a system-wide redesign rather than incremental efficiency gains at the margins. Bloom Energy's own 2026 industry survey found a material expectation gap behind that argument: utility respondents expect time-to-power to take roughly 1.5 to 2 years longer, on average, than hyperscalers and colocation providers themselves are planning for, with the gap widening specifically in Northern Virginia, the Bay Area, and Atlanta, three of the densest AI infrastructure markets in the country.

A second constraint is starting to compound the first, and it has received far less attention than grid capacity: critical minerals, copper in particular, needed for power distribution at data-centre scale. As AI demand surges, ore grades are declining, refining capacity is not keeping pace, and geopolitical risk in existing supply chains is rising, all at once. Long-term supply offtakes, recycling programs, and diversified or domestic refining are moving from a hedging strategy to a requirement for anyone planning a data-centre build on a multi-year horizon. Hyperscalers with the capital to sign multi-gigawatt nuclear partnerships can absorb both constraints. A 40-megawatt enterprise training facility, without that leverage, can sit dark for years waiting on a single substation upgrade.

Governance itself has a compute cost, and pretending otherwise is dishonest

A fair objection to any architecture built around continuous verification is that verification itself consumes compute. The 2026 data supports taking that objection seriously rather than waving it away: in regulated sectors specifically, the compliance layer — encryption processing, audit logging, access control, and data-residency enforcement — can add 15 to 25% to baseline GPU utilisation on top of the underlying AI workload. That is a real cost, and any architecture that claims governance is free is not being honest about its own resource footprint.

The honest position is different: that cost is worth paying because the alternative — a cluster running at 5% utilisation while a runaway loop or a starved inference API silently burns budget — costs more, not less, once the full picture is accounted for. An agent that re-plans has no natural stopping point of its own; without an economic ceiling enforced in the execution path itself, the only signal available is the invoice, and by construction, it always arrives late, well after the compute that produced it is already spent and irrecoverable.

What enterprises ask first

• Do we need to run our own infrastructure for this? No. Orcher governs the execution path; it does not require hosting models directly. Where an enterprise does run its own, that infrastructure is simply another permitted destination in the routing decision.
• How does failover interact with the enterprise data boundary? Alternates are declared per payload class. A directive constrained to a specific jurisdiction queues rather than fails over outside it — the same boundary the Data Control Gateway already enforces on every call.
• What happens to in-flight work when a ceiling is reached? It halts and is recorded with its cost trace and reason. Nothing is committed partially, and nothing continues past a limit unrecorded.

What this means for Dipp Intelligence

Every capacity decision Elastic Compute Governance makes — which region, which pool, what utilisation was achieved against what was theoretically available — is retained in the same evidence chain as the routing decision and the directive it served. Over time, that record becomes a compounding, enterprise-specific map of where an organization's own workloads actually run best, a resource no procurement team building next year's infrastructure plan from a spreadsheet currently has access to.

Identity, verification, data control, cost, and compute — every one of Orcher's components has been quietly producing the same byproduct with every cycle: a compounding, permanently owned asset, Dipp Intelligence, that no competitor licensing the same frontier models can simply buy.