Analysis Cybersecurity AI

AI incident response: what changes when AI is part of the attack

AI has not replaced the incidents your response plan already covers. It has added two things most plans do not account for: attacks that move faster than the response times your current controls were set up for, and a new category of asset, the AI agents, integrations and machine credentials your business has adopted, that can itself become the point of compromise. This guide covers what actually changes in detection, containment and investigation, and where to test it.

14 September 2026
9 min read
Updated 10 November 2026
Key takeaways
  • The NCSC assesses that AI is mainly enhancing tactics attackers already use rather than creating new ones, and that it will almost certainly shorten the time between vulnerability disclosure and exploitation, which is already down to days
  • Two distinct categories now show up in incidents: AI used by the attacker to move faster, and AI as part of the attack surface when your own tools or agents are the thing compromised
  • Machine identities, including AI agent credentials and API keys, outnumbered human identities by around 45:1 in the enterprise environments Rubrik Zero Labs studied in 2025 (ratios vary with architecture and with which identity types are counted), and most incident response plans have no playbook for revoking them
  • Shadow AI is now its own incident category: a data exposure that starts inside an approved workflow and ends in a personal AI account nobody in IT knew was in use
  • A tabletop exercise built only around scenarios from a few years ago will not test whether your team can make the right call when AI is the entry point

The two ways AI shows up in an incident

Most conversations about "AI and incident response" collapse two different problems into one. It helps to separate them before building or testing a plan.

The first is AI used by the attacker. This covers automated reconnaissance, phishing and business email compromise written well enough to pass as internal correspondence, deepfake voice or video used to authorise a payment or grant access, and AI-assisted vulnerability research and exploit development. The NCSC’s assessment of AI and the cyber threat to 2027 expects this last category, AI-assisted vulnerability research and exploit development, to be the most significant near-term AI-driven shift in the threat landscape. That is a forward-looking assessment, not a measurement of incidents so far. CrowdStrike’s 2026 Global Threat Report recorded an 89% year-on-year rise in operations by AI-enabled adversaries, who use AI for reconnaissance, credential theft and evasion. The same report puts the average eCrime breakout time, from initial access to lateral movement, at 29 minutes. That figure covers eCrime intrusions in general, not only those where the attacker used AI. None of this changes what the incident is. A phishing-led compromise is still a phishing-led compromise. It changes how quickly it moves and how convincing it looks along the way.

The second is AI as part of the attack surface. This is newer and less well understood inside most incident response plans. It covers prompt injection against a customer-facing AI assistant, data poisoning of a model or its training data, an AI agent with more system access than it should have, a compromised AI integration or API key, an insecure vector store or RAG pipeline, and Shadow AI, staff using AI tools your IT team has never assessed. Here the AI system itself is the asset that gets compromised or misused, in the same way a server or an account would be in a conventional incident.

The right response depends on which component was compromised. A leaked agent credential calls for revocation and a review of everything the credential was used for. A prompt injection calls for containing the input channel and checking which actions the model took on the attacker's instructions. Suspected poisoning of training data or of a RAG knowledge base calls for quarantining the affected data and rebuilding or re-indexing from a known-good source. Shadow AI calls for establishing what data left the organisation and through which account. NIST's taxonomy of adversarial machine learning attacks (NIST AI 100-2 E2025) is a useful reference for naming these attack types consistently in your playbooks.

Our guides to agentic AI security risks and deepfake identity fraud cover how these specific attack types work in depth. This guide focuses on what they mean for the people running the response: what changes in detection, containment, investigation, and what to rehearse.

29 min
average eCrime breakout time in 2025, from initial access to lateral movement; the fastest observed was 27 seconds (CrowdStrike 2026)
45:1
machine-to-human identity ratio in enterprise environments studied by Rubrik Zero Labs (2025)
39%
of breaches involve credential abuse at some stage (Verizon DBIR 2026)

How AI changes detection and triage

AI-written phishing makes language-based warning signs less reliable. The spelling mistakes, awkward phrasing and generic greetings that staff training has relied on for years are disappearing from attacker emails, and messages can be tailored to the person receiving them. Email security that already uses sender authentication and reputation still works, but the human and behavioural signal has to shift from "does this look badly written" toward behavioural indicators: unusual sending patterns, mismatched reply-to addresses, requests that bypass normal approval steps regardless of how well they read.

Triage gets new questions. When an alert comes in, start with what you can observe: which systems, accounts and data are affected, and how serious the impact is. Whether the attacker used AI is worth recording, but it rarely changes the first steps and is often impossible to confirm early on. The question that does change the playbook is whether one of your own AI tools or agents is involved. If it is, the team needs to assess what that agent can reach: its permissions, its integrations, the instructions it runs under and the actions it has already taken. An AI-accelerated phishing incident is still a phishing incident: contain the account, reset credentials, check for lateral movement. A compromised AI agent or a prompt injection incident needs a different first move, because the "account" in question might be a service identity with broad API access and no obvious way to lock it out of a session the way you would a human user.

There is also a volume problem. In organisations that use them, AI agents and automated workflows can generate a meaningful share of normal traffic: scheduled API calls, automated data pulls, agent-to-agent requests. A SOC calibrated on human traffic patterns will either miss malicious automation hiding in that noise or drown in false positives every time legitimate automation looks unusual. Detection tuning has to account for what your own AI tooling does on a normal day before it can reliably flag what an attacker's AI tooling does on a bad one.

Identity is now an incident-readiness problem

Machine identities, API keys, service accounts, OAuth tokens and now AI agent credentials, outnumbered human identities by around 45 to 1 in the enterprise environments Rubrik Zero Labs studied in 2025, and by as much as 100 to 1 in some. Ratios vary with architecture and with which identity types are counted. Most organisations have inventoried only a fraction of them. Our machine identity security guide covers how that sprawl builds up and how to bring it under control. From an incident response standpoint, the relevant fact is simpler: if your plan only has a fast path for resetting a human employee's password, you do not have a fast path for the credential type most likely to be involved in an AI-related incident.

Stolen credentials remain one of the main ways attackers get in. In the 2026 Verizon Data Breach Investigations Report, exploitation of vulnerabilities overtook credential abuse as the most common first step, but credential abuse still appears at some stage in 39% of breaches. Adversary-in-the-middle techniques that steal session tokens rather than passwords have made multi-factor authentication alone an incomplete defence. Our MFA bypass and credential compromise guide covers this in detail. AI changes the volume and credibility of the phishing and social engineering that leads to that initial credential theft. It does not change the fact that your playbook for revoking and rotating credentials, human and machine, is the thing that determines how fast you contain it.

Containment, investigation and rehearsal

Two companion articles cover the response itself.

Containing and investigating AI incidents covers why switching off a compromised AI agent is harder than isolating a laptop, what evidence to record and preserve during an AI agent incident, and how Shadow AI turns into an incident, including in the middle of an unrelated one.

Rehearsing AI incidents walks through a fictional, minute-by-minute incident involving an over-permissioned AI agent and sets out four tabletop scenarios worth adding to your next exercise.

Readiness checklist

Before your next review

Nine things worth checking: an inventory of AI agents and integrations with what each one can access; least-privilege permissions for each agent, with human approval required for high-impact actions; a documented, pre-agreed process for revoking AI agent and API credentials quickly, naming who is authorised to suspend each agent (not only who is technically able to), with alternates; a playbook step for finding and reversing what a compromised agent has already done, such as records created, messages sent, settings changed or new credentials issued; a tested path for restoring an agent, covering how it is validated and who authorises resumption without reintroducing the compromise; logging and retention requirements for AI tools that match what you already require of conventional systems; a stated instruction covering Shadow AI use during incident activation; a verification procedure for high-value requests that does not rely on recognising a voice or face; and at least one AI-specific scenario in your last twelve months of tabletop exercises.

None of this replaces the fundamentals covered in our incident response plan guide. It extends them to cover the asset types and attack paths that plan of even a year or two ago was not written to account for.


How Cyvra helps

Cyvra's Cyber Incident Readiness service builds and rehearses incident response capability that accounts for the full range of triggers a business faces today, cyber and AI-driven, not just the scenarios a plan was originally written around.

  • AI asset inventory: map the AI agents, integrations and machine credentials in your environment, what each can access, and who can revoke it
  • AI-specific incident playbooks: containment and investigation procedures for compromised AI agents, prompt injection and Shadow AI exposure events
  • Deepfake verification procedures: practical out-of-band verification steps for high-value approvals that do not rely on recognising a voice or face
  • Tabletop exercises with AI-specific scenarios: rehearsed alongside your existing ransomware and data breach playbooks, not as a separate exercise nobody attends
  • Credential and machine identity readiness: pre-agreed authority and process for revoking API keys, service accounts and AI agent credentials during an incident

Contact our Cyber Incident Readiness team or our AI practice to start with an Incident Readiness Assessment.

For the announcement of this service and the broader threat landscape it was built for, see our post on introducing Cyber Incident Readiness.

Frequently asked questions

Does AI actually change incident response, or is this mostly marketing?

Both are true to a degree. The fundamentals of incident response, detection, containment, investigation, recovery, notification, have not changed and are not going to. What has changed is the speed of some attacks and the arrival of a genuinely new asset type, AI agents and integrations, that most existing plans were never written to cover. NCSC's own assessment is that AI is mainly accelerating existing attacker tactics rather than inventing new ones, which is exactly why speed and asset coverage, not a wholesale rewrite, are the right places to focus.

What is the difference between AI used by an attacker and AI as an attack surface?

AI used by the attacker means AI tools make an existing attack type, phishing, reconnaissance, deepfake fraud, faster or more convincing. The incident itself is still a phishing or fraud incident. AI as an attack surface means one of your own AI systems, an agent, an integration, a customer-facing assistant, is the thing that gets compromised or manipulated, through prompt injection, a leaked credential, or an over-broad permission grant. That requires different containment and investigation steps because the "asset" in question behaves differently from a conventional account or server.

Ryland Deakin
About the author
Lead Consultant, Cyvra · CISM · CompTIA Security+ · MCP

Ryland has delivered cybersecurity, compliance, and IT management programmes for regulated organisations across the UK and the Netherlands for over 20 years. View full profile

Talk to Cyvra

Build a plan that accounts for AI

We help you inventory your AI agents and machine credentials, build containment playbooks that cover them, and rehearse it with a tabletop scenario built around how these incidents actually unfold.

Disclaimer: This article is for general informational purposes only and does not constitute legal, regulatory, or professional advice. Cyvra makes no warranty as to the accuracy or completeness of this content. Readers should seek independent advice appropriate to their specific circumstances. Cyvra accepts no liability for any loss arising from reliance on this content.