AI telemetry, logging and correlation
In this lesson you will:
- Identify the AI telemetry needed for security monitoring
- Correlate AI events across SIEM, SOAR and threat-intelligence pipelines
- Protect log integrity and handle logs as evidence
- Mask sensitive data in telemetry while preserving investigative value
- Monitor AI behaviour, performance drift and agent actions in real time
You can't detect what you don't record. AI systems generate new kinds of telemetry — prompts, completions, retrievals, tool calls — and security teams need to collect it, correlate it with everything else, and protect it.
1. What to collect
| Telemetry | Key fields | Why it matters |
|---|---|---|
| Prompts and inputs | User, session, app, size, classifier scores, (masked) content | Injection attempts, abuse, data leakage into AI |
| Completions / outputs | Size, filter results, citations, (masked) content | Data disclosure, harmful output, exfiltration attempts |
| Retrieval events | Query, documents returned, their classifications and ACLs | Over-broad retrieval, poisoned document access |
| Embeddings and index changes | Who added/changed what, when | Poisoning, tampering |
| Tool calls | Agent, user (on-behalf-of), tool, parameters, result, approval status | Excessive agency, misuse, lateral movement |
| Model API traffic | Endpoint, identity, tokens in/out, latency, errors | Abuse, extraction, denial of wallet, shadow AI |
| Guardrail events | Rule triggered, action taken | Attack trends, guardrail health |
| Platform logs | Registry, pipeline, deployment, configuration changes | Tampering, unauthorised change |
| Identity events | Agent token issuance, scope changes, OAuth grants | Credential misuse |
The AI gateway (for model calls) and the agent harness (for tool calls) are the best places to capture consistent, structured events.
2. Correlation across SIEM, SOAR and threat intelligence
AI events are most useful when joined with other signals:
- A prompt-injection detection followed by an unusual tool call from the same session.
- A new OAuth grant to an AI app followed by bulk mailbox reads.
- A data-scientist login from an unusual location followed by model downloads from the registry.
- Token-usage spikes from an identity also seen in credential-stuffing alerts.
SOAR playbooks can automate first responses: throttle an identity, disable a tool, open a ticket for the system owner, snapshot relevant logs. Threat intelligence feeds (new jailbreak techniques, malicious MCP servers, compromised model artefacts) become detection rules and watch-lists.
3. Log integrity and evidence
- Centralise logs quickly; protect them with append-only or write-once storage, hashing or signing, and time synchronisation.
- Restrict who can delete or alter logs; log access to the logs.
- Record model, prompt and guardrail versions in each event so behaviour can be reconstructed later.
- Treat logs as potential evidence: preserve chain of custody when an incident might lead to legal or regulatory action.
4. Privacy-preserving telemetry masking
Telemetry can become the largest store of sensitive data in the AI system. Balance investigation needs with privacy:
- Tier 1 — metadata (identities, sizes, scores, tool names, timings): broad SOC access, longer retention.
- Tier 2 — masked content (PII and secrets tokenised or redacted): analysts during investigations.
- Tier 3 — full content: break-glass access with approval, logged, short retention.
- Use consistent tokenisation so the same identifier can be correlated across events without revealing it.
5. Real-time monitoring
- Behaviour — refusal rates, filter hits, topic distribution, output length, citation rates.
- Performance drift — quality scores from sampled evaluations; input distribution shifts.
- Agent actions — tool-call rates, new tool sequences, approval-queue volumes, spend per agent, failed authorisation attempts.
Dashboards help, but alerts should fire on deviations from baseline defined at release (Lesson 8.1).
Worked example — correlating a slow exfiltration
| Time | Event (source) | Alone | Correlated |
|---|---|---|---|
| 09:02 | Injection classifier score 0.62 on an uploaded PDF (gateway) | Below block threshold | — |
| 09:03 | Agent calls search_contacts 40 times (harness) |
Within quota | Unusual for a summarisation task |
| 09:05 | Agent output contains 12 external image links (gateway output filter) | Not blocked | Links encode data in query strings |
| 09:05 | Egress proxy sees requests to a newly registered domain (proxy) | Low severity | Matches the image links |
SIEM rule created: injection score > 0.5 and tool-call pattern deviation and external URLs in output within 5 minutes → high-severity alert; SOAR disables external rendering for the session and suspends the agent pending review.
🔐 Security lens
Log the versions with every event. When an incident happens three weeks later, "which prompt and guardrail config were live at 09:03?" must have an answer.
Key takeaways
- Collect prompts, outputs, retrievals, index changes, tool calls, API traffic, guardrail events, platform and identity logs — gateways and harnesses are the best collection points.
- Correlate AI events with identity, endpoint, network and threat intelligence; automate first responses with SOAR.
- Protect log integrity (append-only, signed, time-synced) and record versions in every event.
- Tier telemetry access: metadata broadly, masked content for investigations, full content under break-glass.
- Monitor behaviour, drift and agent actions against release baselines.
Self-check
- Why are the AI gateway and agent harness good telemetry collection points?
- Give one example of correlating an AI event with a non-AI event.
- Why should full prompt content be in the most restricted telemetry tier?