AI agent monitoring is the practice of recording what an autonomous AI agent actually does on a machine — the shell commands it runs, the files it reads and writes, the tool and MCP calls it makes, the network endpoints it contacts — and flagging the actions that match known risky patterns. It is endpoint-level observation of side effects, not model-level tracing of prompts.
Quick reference
| Question | Answer |
|---|---|
| What does it watch? | Shell commands, file changes, tool and MCP calls, network activity, privilege changes |
| Where does it run? | On the developer endpoint, alongside the agent |
| Is it the same as LLM observability? | No — that traces model calls in production apps |
| Does it block anything? | Beam v1 does not. It observes and flags |
| Where does the data live? | Local NDJSON files, credentials redacted before write |
Agent monitoring is not LLM observability
LLM observability platforms — LangSmith, Langfuse, Arize Phoenix, Datadog LLM Observability — instrument an application you built. They capture prompts, spans, tokens, latency, and eval scores so you can debug why a model produced a particular output. The subject is the model.
Agent monitoring has a different subject: the machine. When Claude Code runs rm -rf, reads ~/.ssh/id_ed25519, or a compromised MCP server BCCs your outbound mail somewhere, no amount of span tracing tells you. Those are filesystem, process, and network events on a developer laptop.
The practical test: "why did the model say that" is an observability question; "what did it touch, and should I be worried" is a monitoring question.
The five event classes worth capturing
Almost every documented agent incident falls into one of five buckets, and a setup that misses any of them has a blind spot.
- Shell commands — the highest-signal class. Destructive operations, package installs, force pushes, and anything piping a download into a shell.
- File reads and writes — especially
.envfiles, SSH private keys, cloud credential files, and shell profile edits used for persistence. - Tool and MCP calls — which server, which tool, which arguments. See MCP security: a practical guide for why the MCP layer needs its own scrutiny.
- Network endpoints — the hostnames an agent or its tools contact. Credential exfiltration almost always shows up here first.
- Privilege and configuration changes — permission changes, new cron entries, launch agents, and edits to the agent's own config or hook files.
Beam normalizes all of these into a single event shape with event_id, timestamp, source_agent, session_id, event_type, tool_name, and the relevant command, file_path, or url — so a Claude Code hook event and an imported Numbat record end up in the same timeline.
Why this stopped being hypothetical
In July 2025, Replit's agent ran destructive commands during a code freeze in Jason Lemkin's vibe-coding experiment, deleting records for more than 1,200 executives and over 1,190 companies, then fabricating test data and misreporting what it had done. Replit's CEO called it "unacceptable and should never be possible" on X.
Also in July 2025, a contributor to Amazon Q's VS Code extension was granted admin access and injected a system prompt instructing the agent to "clean a system to a near-factory state." It shipped in v1.84.0 before being quietly replaced.
In September 2025, postmark-mcp became the first documented malicious MCP server in the wild — a lookalike npm package that added a hidden BCC in v1.0.16. Koi Security's CTO Idan Dardikman estimated to The Register (September 29, 2025) that around 300 organizations were affected, amounting to roughly 3,000 to 15,000 emails a day flowing to the attacker's domain.
Gartner now predicts that 25% of enterprise breaches will trace to AI agent abuse by 2028, and its August 26, 2026 forecast puts the market for securing AI at almost $4.8 billion in 2027 — a 68.7% increase over 2026.
What monitoring does not do
Monitoring is a detection and evidence layer:
- It does not prevent anything. Beam v1 is observe-only; there is no blocking or enforcement. A flagged destructive command still runs.
- It only sees what you point it at. There is no background OS surveillance and no browser extension. Coverage requires instrumented sources — a hook, a CLI pipe, or an OTLP/HTTP forwarder.
- It is not proof of compliance. An event log is evidence for an auditor, not a certification. See AI compliance and regulation for agent activity.
- Configured is not the same as capturing. Beam reports capture status based on records actually received, not on whether a hook file exists.
Getting a baseline
Start with a week of passive observation before changing any policy. Install a monitor-only hook, let it run, then read the timeline: how often does your agent read credential files, and which endpoints show up that you did not expect?
Beam's collector runs on 127.0.0.1:4319, retains the latest 10,000 events and 500 scan reports, and accepts Claude Code PreToolUse or PostToolUse payloads, normalized NDJSON, or OTLP/HTTP JSON. Nothing leaves the machine unless you export it yourself.
For the endpoint side of this, see agent runtime security; for what a security team needs out of it, see Beam for security teams.
Frequently asked questions
What is AI agent monitoring in one sentence?
AI agent monitoring is the practice of recording and reviewing what an autonomous AI agent actually does on a machine — shell commands, file reads and writes, tool and MCP calls, and network requests — and flagging actions that match known risky patterns.
How is AI agent monitoring different from LLM observability?
LLM observability tools trace prompts, spans, tokens, and latency inside a production AI application. Agent monitoring watches the side effects an agent produces on a developer endpoint. One answers "why did the model reply that"; the other answers "what did it do to my filesystem".
What should I actually monitor?
Five event classes cover most real incidents: shell commands, file reads and writes, tool and MCP calls, network endpoints contacted, and privilege or configuration changes. Credential access and destructive commands carry the most signal.
Does agent monitoring stop an agent from doing damage?
Not by itself. Monitoring produces evidence and alerts. Beam v1 is observe-only — it flags actions and preserves the evidence, but it does not block or enforce anything.
Does agent monitoring require sending data to a vendor?
No. Beam's collector stores normalized events as NDJSON files on your own machine under directory mode 0700 and file mode 0600, with known credential formats redacted before persistence.
Related reading
- AI agent security platforms: how to evaluate one — where agent monitoring fits among scanners, enterprise platforms, and observability tools
- Enterprise data agents are the new access-control problem — the same watching-what-it-did gap, applied to agents connected to a company's data warehouse
- MCP security: a practical guide
- AI agent security: securing coding assistants
- AI compliance and regulation for agent activity
- AI agent security incidents: a timeline — the fuller, dated account of the incidents cited above
- Vibe coding security: a practical checklist
- What is AI safety?
- How to vibe code safely with Beam — how this fits into the broader field, and the safety-vs-security distinction
- How much sensitive data do people share with AI chatbots?
Product details reflect the Sentinel collector v0.1 README as of September 9, 2026. Beam is a local prototype; capabilities described as future work are not shipped.