Enterprise AI Security: Threat Model and Controls

Securing enterprise AI is a programme problem before it is an engineering one. It requires an inventory of every AI system in the organisation, a risk classification for each, seven control families applied proportionately, and an incident process that can handle a failure mode no existing runbook covers.

The engineering controls matter and are covered elsewhere. This article is about the layer above them: how a security function establishes what it is defending and demonstrates that it is.

Why does AI need its own security programme?

Not because the threats are unprecedented — most of them are recognisable variants of injection, privilege escalation, data disclosure and supply chain risk. It is because the assets are invisible.

An AI system is easy to create and difficult to see. A team enables an AI feature inside an existing SaaS tool, and the organisation now has a data flow to a foreign model provider that no architecture review approved. A developer builds an assistant over an internal document store, and there is now a retrieval path around the access controls on that store. Neither shows up in an asset register designed for applications and servers.

Three consequences follow, and they define the programme:

You cannot secure an inventory you do not have. Discovery comes first.

Risk is not uniform. A chatbot answering HR policy questions and an agent with write access to a payment system need very different controls, and applying uniform controls means either over-controlling the first or under-controlling the second.

The failure mode is quiet. AI systems degrade rather than break, and a compromised or manipulated system continues producing plausible output. Detection has to be designed, not assumed.

What belongs in the AI system inventory?

Seven attributes per system, and the inventory is the first deliverable of the programme.

What it does, in one sentence, and who owns it.

What data it can reach — the retrieval scope and the tools, with the classification of each source. This is the attribute that most often has no recorded answer, and establishing it is where the security programme meets the data governance programme.

Whose identity it acts under. A per-user identity, a service account, or a scheduled principal.

What actions it can take. Read-only, reversible writes, or irreversible or externally-visible actions.

Where inference runs and where logs go. Two separate questions, frequently with two different answers, and both matter for residency as set out in data residency and personal data law in Azerbaijan.

Which model and version, including whether it is hosted internally or externally.

What the audit trail contains and how long it is retained.

Discovery is harder than it sounds, because AI features arrive inside products the organisation already owns. Practical methods that work: network egress analysis for traffic to known model providers, procurement review of SaaS renewals for newly-enabled AI features, and simply asking each department what they have switched on. The last one finds more than the first two.

How should systems be risk-classified?

Two dimensions, which together produce the control level.

Data sensitivity — what the system can reach. Public information, internal information, personal data, or restricted or regulated data.

Action capability — what it can do. Read-only, reversible writes, irreversible or externally-visible actions.

A read-only system over public content needs light controls. A system with irreversible action capability over regulated data needs the full set, plus human approval on every consequential action. Most systems sit in between, and the classification determines which controls apply rather than leaving it to each project's judgement.

This classification is also the input to the risk assessment that management-system standards expect, and it maps cleanly onto the impact-assessment expectations in ISO/IEC 42001.

Which control families apply?

Seven, applied proportionately to the classification.

1. Identity and access. User identity propagates into every retrieval and every tool call, with downstream systems applying their own authorisation. Where the system holds a service account instead, any successful manipulation inherits it. This is the foundational control and the one most often missing in systems built as prototypes.

2. Input and content controls. Every input to a model is untrusted — from the user and from retrieved content alike. Prompt injection sits at the top of the OWASP Top 10 for LLM applications, and it cannot be eliminated by instruction. The control is constraining what a successful injection can do, as set out in building secure AI workflows.

3. Output controls. Model output is untrusted input to whatever consumes it. Validate types and ranges, never interpolate into queries or commands, scan for credential-shaped strings and sensitive data before release.

4. Action controls. Reversibility classification, approval gates on irreversible actions, policy evaluated at execution time on actual parameters, and fail-closed behaviour when the policy service is unavailable.

5. Isolation. Sandboxed execution for anything that runs code, no network egress from sandboxes, tenant isolation enforced at the data layer rather than in application code, and separation of high-privilege tools from contexts that ingest untrusted content.

6. Monitoring and detection. Anomalous tool-call sequences, unusual retrieval patterns, output referencing actions nobody requested, and retrieval quality drift. AI systems fail quietly, so detection is a design requirement rather than an operational nicety.

7. Supply chain. Model provenance and integrity verification, review of tool and connector packages, and assurance over any hosted service in the path. Model weights and container images arriving into an air-gapped environment need a controlled path with integrity checking agreed in advance.

How does third-party model risk work?

The question a security function is asked and most often cannot answer precisely.

Where does processing happen, and under what terms. A hosted model provider processes whatever is in the prompt. Whether it retains it, whether it trains on it, where its infrastructure sits and what its subprocessors do are contractual questions with concrete regulatory consequences. For personal data leaving Azerbaijan, the transfer rules apply regardless of the answers.

What happens on model deprecation. Hosted models are withdrawn and replaced on the provider's schedule. A system whose behaviour was validated against one version faces revalidation on the provider's timeline rather than its own — an availability and change-control risk, not only a quality one.

What the assurance position is. Whether the provider offers audit reports, and whether they cover what you need to evidence.

For supervised institutions here, these questions frequently resolve to running inference inside the perimeter — not as a preference but because the alternative is hard to defend under transfer rules and outsourcing scrutiny, as covered in CBAR IT and data requirements and sovereign AI. That decision moves the risk from third-party assurance to internal operations, which is a trade rather than an elimination.

What does AI incident response look like?

Existing incident processes assume a system either works or fails. AI systems produce wrong output while operating normally, which breaks that assumption.

Four incident classes worth pre-defining, each with a different response:

Manipulation. The system was induced to act outside its intended behaviour. Response: contain by disabling the affected tool or capability, review the audit trail for what was executed, assess the injection vector.

Disclosure. The system surfaced information to someone not entitled to it. Response: determine scope from retrieval logs, assess whether it is a reportable personal data breach, and fix the entitlement path rather than the prompt.

Systematic error. The system has been producing wrong output for a period — usually from stale content, a source change or retrieval degradation. Response: determine the period, identify decisions taken on that output, and correct downstream.

Degradation. Quality has declined without a specific failure. Response: retrieval evaluation against the fixed question set, corpus currency check, source change review.

Three requirements make any of this possible: an audit trail complete enough to reconstruct what happened, a kill switch per system that a duty officer can use without a deployment, and a defined owner reachable outside working hours. The audit trail requirement is the same one described in AI orchestration architecture, and it is why the trail is an architectural property rather than a logging feature.

Notification obligations also need pre-mapping. A disclosure incident may be a personal data breach with a notification deadline, and for a supervised institution it may carry sectoral reporting duties as well. Working that out during an incident is too late.

How does this map to existing frameworks?

Deliberately, and the mapping is what keeps the programme from duplicating work.

Most of the seven control families are existing ISO 27001 control families applied to a new asset class — access control, secure development, supplier relationships, logging and monitoring, incident management. An institution running a real ISMS already has the machinery.

What AI adds sits mainly in the management-system layer: an inventory of AI systems, impact assessments, model lifecycle and change control, and evidence about how systems behave. That is what ISO/IEC 42001 formalises, and its 38 Annex A controls under nine objectives overlap substantially with 27001 rather than replacing it.

For risk management practice specifically, the NIST AI Risk Management Framework provides the functional structure — govern, map, measure, manage — and the comparison between the two is in NIST AI RMF and ISO 42001 compared.

The practical instruction: extend the existing control set and map it, rather than building an AI security programme beside the security programme.

Key points

• The distinguishing problem is invisibility. AI systems are easy to create and hard to see, so discovery and inventory come first.
• Record seven attributes per system, including what data it can reach and whose identity it acts under.
• Classify by data sensitivity and action capability. The classification determines the control level rather than leaving it to each project.
• Seven control families: identity and access, input controls, output controls, action controls, isolation, monitoring, supply chain.
• Identity propagation is foundational. A service account with broad access turns any manipulation into privilege escalation.
• Third-party model risk turns on where processing happens, deprecation timelines and assurance coverage — and often resolves to in-perimeter inference.
• Pre-define four AI incident classes and ensure a complete audit trail, a per-system kill switch and a reachable owner.
• Extend ISO 27001 machinery rather than duplicating it; ISO/IEC 42001 adds the management-system layer, NIST AI RMF the risk structure.

Yukon Labs advises on AI policy and governance and deploys HAVAA on-premise with these controls configured as platform capabilities. A readiness assessment typically starts with the discovery step, because most organisations do not yet know how many AI systems they have.