AI Incident Response

The AI Incident Response Plan establishes the organisational structure, procedures, and resources for detecting, responding to, and recovering from AI-related security incidents. It integrates with the enterprise Incident Response Plan and addresses AI-specific incident types.

Incident Response Phases

1. Preparation: Maintain incident response capabilities, training, tools, and documentation. Conduct regular tabletop exercises and red-team simulations.
2. Detection & Analysis: Monitor AI systems for anomalous behaviour, security alerts, and user reports. Validate and classify incidents by severity.
3. Containment: Implement short-term containment to stop immediate damage. Develop long-term containment strategy for sustained operations.
4. Eradication: Remove the root cause of the incident, including compromised models, poisoned data, or injected prompts.
5. Recovery: Restore affected systems to normal operations with enhanced monitoring. Validate integrity before full restoration.
6. Post-Incident: Conduct lessons-learned review, update controls, and communicate findings to stakeholders.

All AI incidents shall be classified by severity to ensure appropriate resource allocation, escalation, and communication.

Severity Levels

Prompt injection is a critical attack vector against large language models. This playbook provides step-by-step response procedures for confirmed or suspected prompt injection incidents.

Response Playbook

1. Detect: Monitor for anomalous outputs, user reports of unexpected behaviour, or automated alerts from input validation systems.
2. Confirm: Reproduce the suspicious input in a sandboxed environment. Document the exact prompt, model version, and system configuration.
3. Contain: Immediately disable the affected model endpoint or route traffic to a fallback model. Preserve logs and model state for investigation.
4. Assess Impact: Determine whether the injection resulted in data exfiltration, unauthorised actions, or harmful output generation.
5. Notify: If personal data or regulated information was accessed, initiate regulatory notification timelines (72 hours for GDPR, as applicable).
6. Remediate: Patch prompt templates, strengthen input validation, update system prompts, and retrain or fine-tune if model weights are compromised.
7. Validate: Test fixes in staging with red-team prompt injection techniques before restoring production traffic.
8. Document: Complete the Incident Report (Template 9.13) and update the Prompt Register with the attack vector and mitigations.

AI hallucination — the generation of plausible but factually incorrect or unsupported content — poses significant risks for organisations using AI in decision-support, customer-facing, or content-generation roles.

Hallucination Response Procedures

• Detection: Implement automated fact-checking, confidence scoring, and retrieval-augmented generation (RAG) grounding checks.
• User Reporting: Establish clear channels for users to report suspected hallucinations with structured feedback forms.
• Impact Assessment: Classify hallucinations by severity — factual error, harmful misinformation, or safety-critical incorrect advice.
• Correction: For high-impact hallucinations, issue corrections to affected users, update knowledge bases, and retrain or fine-tune models.
• Prevention: Enhance RAG retrieval quality, implement citation requirements, constrain output to verified sources, and use ensemble models for critical queries.

Data leakage from AI systems can occur through model memorisation, prompt echoing, insecure API responses, or insider actions. This section establishes detection and response procedures.

Data Leakage Response

1. Detect: Monitor API responses for unexpected inclusion of training data, PII, or confidential content. Use DLP tools on AI outputs.
2. Confirm: Reproduce the leakage in a controlled environment. Identify the source data, model version, and triggering input.
3. Contain: Restrict access to the affected model, disable the vulnerable endpoint, and block the triggering input patterns.
4. Assess Scope: Determine which data elements were exposed, to whom, and for how long. Review access logs comprehensively.
5. Notify: If personal data was leaked, notify the Privacy Officer immediately. Comply with mandatory breach notification timelines.
6. Remediate: Retrain or fine-tune the model with differential privacy, implement output filtering, and strengthen data sanitisation.
7. Review: Assess whether the leakage indicates a broader control failure requiring systemic improvement.

Timely, accurate, and coordinated communication is essential during AI incidents. This section establishes notification requirements, communication templates, and approval workflows.

Notification Matrix