Sovereign AI: Why Governments and Banks Require On-Premise Deployment

Sovereign AI means AI systems that run entirely within one jurisdiction's legal and physical control: the model, the data it processes, the logs it produces and the infrastructure it runs on. For a bank or a government institution, it is not an architectural preference. It is what makes deploying AI legally possible at all when the data involved cannot leave the country or the perimeter.

The term has been diluted by marketing to the point of near-uselessness — vendors describe hosting a region-locked instance of a foreign cloud service as "sovereign." That is a meaningful improvement over nothing, and it is not sovereignty. This article sets out what the word has to mean technically for it to carry weight in a supervisory conversation.

The money says the distinction is now mainstream. The sovereign AI infrastructure market was valued at around $61.4 billion in 2025 and is projected at roughly $78.6 billion for 2026, with national governments accounting for about 32% of end-user demand and defence and security agencies a further 22%. NVIDIA's sovereign AI revenue passed $30 billion in its fiscal 2026, roughly 14% of total revenue. States are buying compute they control, and the reason is the same one your legal department will give you.

What sovereignty actually requires

Four properties. A deployment that satisfies three of them is a good deployment, not a sovereign one.

1. The data never leaves the perimeter. No prompt, no document, no embedding, no log line containing customer data crosses a border or reaches a third party's infrastructure. This includes the parts people forget: telemetry, error reporting and the vector store.

2. The model runs on infrastructure you control. In your data centre, or a facility subject to your jurisdiction, on hardware you can physically point at. Not an API endpoint, however region-locked.

3. The operator is subject to your jurisdiction. If the entity that can technically access the system answers to a foreign legal process, control is contingent on the goodwill of a legal system you do not participate in. This is the property most "sovereign cloud" offerings do not deliver, and it is the one that survives contact with a serious legal review.

4. It continues to work when the connection does not. A system that degrades to unavailable when international connectivity drops is not sovereign infrastructure; it is a dependency with a local cache.

Test any vendor claim against all four. The gap between "sovereign" and "hosted in your region" usually shows up at three and four.

Why regulated institutions cannot simply use an AI API

The reasoning is unglamorous and it holds.

Data localisation obligations. Azerbaijan's Law on Personal Data subjects information systems that process personal data to state registration with the responsible executive authority, and permits cross-border transfer only where the receiving state ensures adequate protection, the data subject has consented, or the transfer is necessary to perform a contract — with transfers prohibited outright where they would harm national security or public order. Sending a customer's file to a foreign inference endpoint is a transfer, whatever the retention policy at the other end says. The detail is in data residency and personal data law in Azerbaijan.

Supervisory expectations on information security. For banks, the Central Bank of Azerbaijan's Regulation on Information Security Management in Banks, in force since 1 April 2022 and modelled on the ISO/IEC 27000 series, sets minimum requirements that an uncontrolled external processing path is difficult to reconcile with. CBAR has since added a Cybersecurity Strategy for Financial Markets for 2023–2026 and requirements for information security across supervised entities.

Supervisory answerability. A regulator asking how an automated decision was reached expects an answer grounded in evidence. "The vendor's model produced it and we cannot inspect the weights or the version that was current on that date" is not an answer. Sovereign deployment gives you a pinned model version, your own logs and reproducibility.

Contractual reality. Enterprise API terms have improved considerably — most major providers now offer no-training commitments and zero-retention options. Those commitments are real and worth having. They are also contractual rather than physical, they can change at renewal, and they do not survive a foreign legal order compelling disclosure. A bank's legal counsel will make this distinction, correctly.

Concentration and continuity risk. Supervisors have grown pointed about dependence on a small number of foreign providers for critical functions. An AI system embedded in credit decisioning or fraud detection becomes exactly that kind of dependency.

Sanctions and geopolitical exposure. For institutions in this region, the possibility that access to a foreign service is interrupted for reasons unrelated to the institution's own conduct is a live operational risk, not a theoretical one.

The three deployment postures

In descending order of control and ascending order of convenience.

Air-gapped

No network path to the outside world. Models, weights, dependencies and updates arrive on physical media through a controlled process.

Appropriate for classified government systems, defence, and the most sensitive banking workloads. It is genuinely hard operationally: every dependency must be mirrored internally, every model update is a logistical procedure, and diagnosing an issue without being able to reach documentation or a vendor is a real constraint on the team.

Nothing else provides the same guarantee. If the requirement is absolute, this is the only posture that meets it, and the operational cost is the price.

On-premise, connected

Runs in your data centre. Outbound connectivity exists for updates and package retrieval, controlled by policy and egress filtering. Data does not leave; software comes in.

This is the practical sweet spot for most banks. It satisfies localisation, keeps inference inside the perimeter, and preserves a workable operational model. The critical control is egress: it must be enforced at the network layer rather than by application configuration, because an application-level setting is one deployment mistake away from silently reversing.

Sovereign hosted

Infrastructure in-country, operated by an entity under local jurisdiction, dedicated rather than multi-tenant. This is the posture behind the "national AI cloud" model, which now accounts for roughly 35% of sovereign AI deployments worldwide.

Reasonable for organisations without their own data centre capability. Whether it qualifies as sovereign depends entirely on property three above — who legally controls the operator. A local subsidiary of a foreign parent is a different answer from a domestic entity, and the distinction matters when it is tested.

What it costs, honestly

Sovereignty is not free, and the trade-offs are worth stating plainly rather than discovering after commitment.

Capital cost. GPU infrastructure has to be bought rather than rented. For a serious deployment this is a substantial capital outlay before any value is produced. The full sizing discussion, with current hardware and break-even figures, is in deploying LLMs on-premise.

Model capability gap. The largest frontier models are not available for self-hosting. Open-weight models have closed much of the distance and are entirely sufficient for the majority of enterprise workloads — retrieval-grounded question answering, classification, extraction, summarisation, routing. They are not equal to the best hosted models on the hardest reasoning tasks. Whether that gap matters depends on the workload, and honest scoping at the outset prevents disappointment later.

Operational burden. You own uptime, updates, scaling and incident response. This is a standing capability, not a project cost.

Speed. An API call is available today. A sovereign deployment is a procurement, an installation and a hardening exercise.

Against these: data that legally could not otherwise be processed becomes processable; cost per token at sustained volume falls dramatically below API pricing — current benchmarks put a self-hosted 70B-class model at roughly $0.18 per million output tokens at batch 8 on an H100, against dollars per million for frontier hosted models; latency is local; and there is no third party whose pricing, availability or terms can change underneath a system you have made critical.

The architecture

A sovereign AI deployment is not just a model on a server. The components that make it useful are the ones around it.

Inference layer. Open-weight models served through a production inference engine with batching, quantisation and multi-tenancy. This is the part everyone thinks about and the part that requires the least ongoing judgement.

Retrieval layer. Almost every enterprise use case is retrieval-grounded — the model answers from your documents rather than from its training. This means a vector store and an embedding model, both of which must also be local. An organisation that self-hosts the language model and calls a hosted embedding API has moved the leak rather than closing it, and this is one of the most common errors in first deployments.

Orchestration layer. Routing, tool access, state, permissions and audit. This is what turns a model into a system that can do work, and it is where the governance controls live. See what is AI orchestration.

Governance layer. Prompt and response logging, model version pinning, evaluation harnesses, human-in-the-loop gates for consequential actions, and an audit trail that reconstructs any decision after the fact. In a regulated institution this is not optional tooling; it is the thing that makes the system defensible, and it maps directly onto the controls an ISO/IEC 42001 audit will ask to see.

Identity and access. Integration with the existing directory, so an agent acts with the permissions of the person on whose behalf it acts — never with a shared service account that can see everything. This single design decision prevents the most serious category of failure in enterprise AI deployments.

The mistakes that recur

Self-hosting the model and nothing else. Local LLM, hosted embeddings, hosted vector database, hosted observability. Three leaks and a sovereignty claim that will not survive review. Audit the full data path, including telemetry.

Sizing hardware for peak concurrency from day one. Organisations buy for an imagined load and run at low utilisation for a year. Start with the workload you actually have, measure, then scale.

Skipping evaluation. Without an evaluation set built from your own data, you cannot tell whether a model change improved or degraded anything, and you will be changing models regularly. Build the eval harness before the first production use case, not after the first incident.

Treating open weights as a fixed choice. The open-weight landscape moves quickly. Architect so the model is a swappable component behind a stable interface. Organisations that hard-wired a specific model in early deployments paid for it within a year.

Giving agents shared credentials. An agent operating under a service account with broad access is a privilege-escalation path with a natural-language interface. Permissions must derive from the requesting user.

Where sovereign AI is going

Three developments worth planning around.

Open-weight models continue to close the gap. The distance between the best open and best closed models has narrowed consistently, and for retrieval-grounded enterprise tasks it is now small enough that most organisations will not notice it. This trend is the single largest factor making sovereign deployment practical.

Hardware efficiency is improving faster than model size is growing. Quantisation, distillation and better serving stacks mean the hardware needed for a given capability keeps falling — a 70B model that needs about 140 GB of memory at full precision fits in roughly 35 GB at 4-bit, and FP8 serving on current accelerators retains close to full quality at half the memory. A deployment that was uneconomic two years ago frequently is not now.

Regulation is tightening, not loosening. The EU AI Act, sector-specific supervisory guidance and national data protection regimes are all moving toward more documentation, more traceability and more accountability. The Digital Omnibus — Regulation (EU) 2026/1744 — deferred the AI Act's high-risk obligations to December 2027 and August 2028 while leaving transparency duties and GPAI enforcement live from 2 August 2026. Deadlines moved; direction did not.

Locally, the same direction is visible in policy: Azerbaijan's Artificial Intelligence Strategy for 2025–2028 names data governance, security and ethics as priorities, and the Action Plan for the Acceleration of Digital Development for 2026–2028, approved by presidential decree on 27 February 2026, carries 58 initiatives across digitalisation, AI and cybersecurity.

The combined direction is clear: the capability cost of sovereignty is falling, and the regulatory cost of its absence is rising.

Key points

• Sovereignty requires four properties: data stays inside, infrastructure is yours, the operator is under your jurisdiction, and it works without international connectivity. Three out of four is not sovereignty.
• The sovereign AI infrastructure market is around $78.6 billion in 2026, with governments and defence agencies together making up over half of end-user demand.
• Contractual no-training guarantees are real and are not the same as physical control. Legal counsel in regulated institutions will make this distinction.
• Air-gapped, on-premise-connected and sovereign-hosted are three genuinely different postures with different costs. Choose deliberately.
• Self-hosting the model while calling hosted embedding or vector services is the most common way a sovereignty claim fails review.
• Agents must act with the requesting user's permissions, never a shared service account.

HAVAA is built for this posture — AI orchestration that runs entirely inside your perimeter, including retrieval and embeddings. Yukon Labs deploys it on-premise and air-gapped for banks and government institutions in Azerbaijan.