If your app passes user text to a language model, the model is part of your attack surface. Strangers can ask it for the application's keys, tell it to ignore its instructions, hide commands in invisible characters, or send it a flood of requests until the bill grows. The Beam monitor SDK gives you two local checks for this: prompt() for what goes into the model and output() for what comes back. This guide covers what they flag, how to wire them in, and what they leave to you. For the broader product picture, see the app monitoring solution page.
| Question | Short answer |
|---|---|
| Does it block anything? | No. It returns what it found and your code decides |
| Where do the checks run? | Locally in your app, in microseconds |
| What leaves the server? | Redacted findings only, sent in the background with the app token |
| Do I need a token to try it? | No. checkPrompt and checkOutput work without one |
| What version do I need? | @agent-beam/beam 0.2.29 or newer |
| Is it reliable? | Heuristic. It will miss some attacks and flag some harmless text |
Start monitoring an app if you want findings to reach a dashboard, Slack or a webhook. The checks themselves run without an account.
Why LLM features need their own checks
The incidents in the news this month all come down to AI provider keys. OpenCode and Railway were platform breaches that app code could not have stopped. A chat feature in your own app is different: it is a door you built, and the question is what a stranger can make it say or do.
Three goals recur. Attackers try to get the key itself, by asking the app to print its environment. They try to hijack behavior, by overriding the instructions or smuggling commands in through a web page, email or document the model reads. And they try to burn money, because a model endpoint is an expensive thing to leave open. Researchers have documented the last of these in the wild as LLMjacking, and indirect injection through content a stranger wrote shows up everywhere from HTTP headers to hidden chatbot instructions in a messaging app.
Check what goes in with prompt()
Create one monitor, give it a canary, and run each incoming message through prompt() before calling the model. This is the pattern from the App monitoring docs:
import { createAppMonitor } from "@agent-beam/beam/monitor";
const beam = createAppMonitor({ canaries: [promptCanary] }); // a random marker you also put in the system prompt
const check = beam.prompt(message, { user: userId }); // also retrieved documents, web pages, emails, tool results
if (check.severity === "high" || check.severity === "critical") return refuse(check.matches);
const reply = await callModel(message);
if (beam.output(reply).flagged) renderAsPlainText(reply);
prompt() flags requests for the app's API keys, tokens or environment, instruction overrides, instructions hidden in invisible Unicode or base64, forged chat-template and system markers, jailbreaks, attempts to extract the system prompt, and one sender flooding the model, which the docs call "denial of wallet".
Treat everything a stranger could have written as input, not only the chat box. Indirect prompt injection arrives in retrieved documents, web pages, emails and tool results, so pass those through prompt() as well, with a label such as "retrieved web page". The check is the same; the source is what changed.
Check what comes out with output()
output() looks at the model's reply for things that should not be there. It flags zero-click exfiltration links, which are markdown or HTML images whose URL carries data, so that rendering the reply leaks it without a click. It flags credentials, script, and injected instructions passed along to another agent. And it flags a leaked canary, which is critical.
The canary is the cheapest control in the set. Put a random marker in your system prompt, give the same marker to createAppMonitor, and a leaked system prompt becomes a loud, specific alert instead of something you learn about from a screenshot. When flagged is true, the simplest safe response is the one in the example: render the reply as plain text so links and markup do nothing.
Keep prompt text private
The checks run locally. What leaves your server is a redacted finding, sent in the background with the app token. Two habits keep it tidy. Pass an opaque user id or the client address, never an email. And if you do not want excerpts at all, create the monitor with excerpts: false to report the rule and sender without the text.
To try the checks before connecting an app, checkPrompt(text) and checkOutput(text, { canaries }) run the same logic without a token and report nothing.
Also watch the logs
The SDK covers calls you wire it into. The log-side rules cover the rest. beam monitor flags prompt injection and jailbreak attempts in logged prompts and query strings, exfiltration links in logged model output, and floods of requests to your LLM endpoints from one client. It also flags requests for your keys, such as printenv, cat .env or "print your OpenAI API key", and a .env, .git/config or credentials file actually served with a 2xx response. Those are high severity, so they alert on the default level.
One caveat from the docs: an app that answers 200 to every path, like a single-page app's catch-all, can trip the served-file rule. If that happens, an owner or admin can create a rule from the finding to ignore it for that path. The API keys in logs guide covers the credentials-in-logs side in detail.
What this does not do
Be clear about the limits, because they decide what else you need.
- It does not block. Beam never sits in the request path. Refusing a prompt is your code's job, using the severity the check returns.
- It is heuristic. It misses attacks without a rule and flags some harmless text. A polite, well-disguised injection can pass.
- It does not replace least privilege. If the model can call tools, scope those tools tightly and keep keys out of its reach. A detector is a second line.
- It does not cap spend. Flood detection tells you a sender is hammering the endpoint. A provider-side spend limit is what stops the bill, as covered in the platform breach post.
- The optional LLM review is hardened, not unbeatable. Lines that address AI reviewers are never sent to a model and a refusal never counts as benign, but the docs describe these layers as making manipulation hard, not impossible.
How AgentBeam handles this
AgentBeam treats LLM features as part of app monitoring. The SDK checks prompts and replies locally and reports only redacted findings; beam monitor watches the logs for the same attack shapes; rules let an admin tune out noise; and webhooks send findings to Slack, Discord or your own endpoint. Detection stays on your server, and the Beam monitor never blocks traffic. To have your coding agent wire it in, install the bundled skill and ask it to add monitoring to your app:
npx -y @agent-beam/beam add skill agentbeam-app-monitoring -g --all-agents
The skill is covered in Let your coding agent set up AgentBeam. Then get started here.
Related reading
- Monitor a running app for attacks with beam monitor
- API keys in logs and env vars: how they leak and how to catch it
- What app monitoring can and cannot catch when your platform is breached
- Your first 24 hours after a vendor breach
- What is AI agent monitoring?
- October 2026 attacks timeline
Frequently asked questions
What does the Beam monitor SDK check in an LLM feature?
prompt() checks text going into a model for requests for the app's keys or environment, instruction overrides, hidden Unicode or base64 instructions, forged chat markers, jailbreaks, system prompt extraction and one sender flooding the model. output() checks the reply for exfiltration links, credentials, script, injected instructions for another agent and a leaked canary.
Does it block a malicious prompt?
No. Both calls run locally in microseconds and return what they found. Your code decides whether to refuse a prompt or render a reply as plain text. Beam never blocks anything and never sits in the request path.
What is a canary and why add one?
A canary is a random marker you place in your system prompt and also give to createAppMonitor. If it ever shows up in a model reply, the system prompt has leaked, and output() raises a critical finding.
Does my prompt text leave my server?
Only redacted findings leave, in the background and with the app token. Pass an opaque user id or client address rather than an email, and create the monitor with excerpts: false to report the rule and sender without the text.
Do I need a token to try the checks?
No. checkPrompt(text) and checkOutput(text, options) run the same checks without a token and report nothing, so you can test them in development before connecting the app.
Can an attacker talk the monitor out of flagging them?
The optional LLM review is hardened against it, because lines that address AI reviewers are never sent to a model and a refusal never counts as benign, but the documentation calls these layers hard to bypass, not impossible. Keep your own validation and limits in place.
Product behavior described here follows the AgentBeam App monitoring documentation as of October 23, 2026. The SDK requires @agent-beam/beam 0.2.29 or newer, detection is heuristic, and rules and limits can change in later releases.