AI Security Operations, Incident Response & Recovery, 25 min read

AI telemetry, logging and correlation

In this lesson you will:

  • Identify the AI telemetry needed for security monitoring
  • Correlate AI events across SIEM, SOAR and threat-intelligence pipelines
  • Protect log integrity and handle logs as evidence
  • Mask sensitive data in telemetry while preserving investigative value
  • Monitor AI behaviour, performance drift and agent actions in real time

You can't detect what you don't record. AI systems generate new kinds of telemetry — prompts, completions, retrievals, tool calls — and security teams need to collect it, correlate it with everything else, and protect it.

1. What to collect

Sources of AI telemetry flowing into a SIEM.
Figure 8.3 AI telemetry sources. The gateway and agent harness are the richest collection points.
Telemetry Key fields Why it matters
Prompts and inputs User, session, app, size, classifier scores, (masked) content Injection attempts, abuse, data leakage into AI
Completions / outputs Size, filter results, citations, (masked) content Data disclosure, harmful output, exfiltration attempts
Retrieval events Query, documents returned, their classifications and ACLs Over-broad retrieval, poisoned document access
Embeddings and index changes Who added/changed what, when Poisoning, tampering
Tool calls Agent, user (on-behalf-of), tool, parameters, result, approval status Excessive agency, misuse, lateral movement
Model API traffic Endpoint, identity, tokens in/out, latency, errors Abuse, extraction, denial of wallet, shadow AI
Guardrail events Rule triggered, action taken Attack trends, guardrail health
Platform logs Registry, pipeline, deployment, configuration changes Tampering, unauthorised change
Identity events Agent token issuance, scope changes, OAuth grants Credential misuse

The AI gateway (for model calls) and the agent harness (for tool calls) are the best places to capture consistent, structured events.

2. Correlation across SIEM, SOAR and threat intelligence

AI events are most useful when joined with other signals:

  • A prompt-injection detection followed by an unusual tool call from the same session.
  • A new OAuth grant to an AI app followed by bulk mailbox reads.
  • A data-scientist login from an unusual location followed by model downloads from the registry.
  • Token-usage spikes from an identity also seen in credential-stuffing alerts.

SOAR playbooks can automate first responses: throttle an identity, disable a tool, open a ticket for the system owner, snapshot relevant logs. Threat intelligence feeds (new jailbreak techniques, malicious MCP servers, compromised model artefacts) become detection rules and watch-lists.

3. Log integrity and evidence

  • Centralise logs quickly; protect them with append-only or write-once storage, hashing or signing, and time synchronisation.
  • Restrict who can delete or alter logs; log access to the logs.
  • Record model, prompt and guardrail versions in each event so behaviour can be reconstructed later.
  • Treat logs as potential evidence: preserve chain of custody when an incident might lead to legal or regulatory action.

4. Privacy-preserving telemetry masking

Telemetry can become the largest store of sensitive data in the AI system. Balance investigation needs with privacy:

Tiered telemetry: metadata for everyone, masked content for analysts, full content under break-glass.
Figure 8.4 Tiered access to AI telemetry. Most detection works on metadata and masked content; full content needs justification.
  • Tier 1 — metadata (identities, sizes, scores, tool names, timings): broad SOC access, longer retention.
  • Tier 2 — masked content (PII and secrets tokenised or redacted): analysts during investigations.
  • Tier 3 — full content: break-glass access with approval, logged, short retention.
  • Use consistent tokenisation so the same identifier can be correlated across events without revealing it.

5. Real-time monitoring

  • Behaviour — refusal rates, filter hits, topic distribution, output length, citation rates.
  • Performance drift — quality scores from sampled evaluations; input distribution shifts.
  • Agent actions — tool-call rates, new tool sequences, approval-queue volumes, spend per agent, failed authorisation attempts.

Dashboards help, but alerts should fire on deviations from baseline defined at release (Lesson 8.1).

Worked example — correlating a slow exfiltration

Time Event (source) Alone Correlated
09:02 Injection classifier score 0.62 on an uploaded PDF (gateway) Below block threshold —
09:03 Agent calls search_contacts 40 times (harness) Within quota Unusual for a summarisation task
09:05 Agent output contains 12 external image links (gateway output filter) Not blocked Links encode data in query strings
09:05 Egress proxy sees requests to a newly registered domain (proxy) Low severity Matches the image links

SIEM rule created: injection score > 0.5 and tool-call pattern deviation and external URLs in output within 5 minutes → high-severity alert; SOAR disables external rendering for the session and suspends the agent pending review.

🔐 Security lens

Log the versions with every event. When an incident happens three weeks later, "which prompt and guardrail config were live at 09:03?" must have an answer.

Key takeaways

  • Collect prompts, outputs, retrievals, index changes, tool calls, API traffic, guardrail events, platform and identity logs — gateways and harnesses are the best collection points.
  • Correlate AI events with identity, endpoint, network and threat intelligence; automate first responses with SOAR.
  • Protect log integrity (append-only, signed, time-synced) and record versions in every event.
  • Tier telemetry access: metadata broadly, masked content for investigations, full content under break-glass.
  • Monitor behaviour, drift and agent actions against release baselines.

Self-check

  1. Why are the AI gateway and agent harness good telemetry collection points?
  2. Give one example of correlating an AI event with a non-AI event.
  3. Why should full prompt content be in the most restricted telemetry tier?