To monitor an app hosted on Railway for attacks, run beam monitor as a separate service next to the app, so the app itself is never modified. The monitor reads the service's HTTP and application logs, flags lines that look like injection, traversal, reverse shells or leaked credentials, and sends only redacted findings to your AgentBeam dashboard and Slack. This guide walks through that setup, then covers the token, the monitoring level, beam monitor test and beam monitor status, tuning the first week, and what to do on the day your host announces a breach. The same approach applies to other platforms-as-a-service, with a different way of running the monitor.
| Question | Short answer |
|---|---|
| Do I change my app? | No. The Railway setup is a separate service |
| Does it block attacks? | No. It only watches and never sits in the request path |
| Can it see inside Railway? | No. It sees the logs of the service you point it at |
| Do raw logs leave? | No. Only redacted findings, with an excerpt of up to 600 characters |
| How accurate is it? | Heuristic. Expect some false positives and some misses |
| What does it need? | An AgentBeam workspace with App monitoring on, plus a per-app token |
| Is there a catch on Railway? | Yes. The Railway token it needs is broad. See the section below |
If you want to follow along, create or open your AgentBeam workspace first, since the monitor needs a per-app token from it.
Why monitor an app on a PaaS at all
A platform-as-a-service takes servers off your plate, and with them most of your view. You deploy, the platform runs it, and what you see is what the dashboard shows. That is a fine trade until you need to answer a question the dashboard does not answer: who has been probing this service, did anything reach a route it should not have, did our own code print a credential into a log line.
On any public service, probing starts within the first hour. Requests for .env, ../../etc/passwd and /wp-login.php are routine. The useful signal is the small share of lines that carry a real payload, or a request that was answered with a file it should never have served. Reading raw logs for that by hand does not scale, even for a small app.
There is a second reason, and it is the evergreen one. You depend on the host for storage, secrets and routing, and a host can have a bad day. On October 21, 2026, Railway disclosed that an unauthorized party may have accessed its systems that store customer environment variables, and obtained decrypted contents for some of them. We covered the details in the Railway breach post. It is one example, not the point. The point is that when something goes wrong on the platform, you want your own record of what your app was seeing, held somewhere the platform does not control.
What beam monitor does and does not do
beam monitor follows application logs and HTTP access logs and matches lines against attack patterns: SQL, command and template injection, JNDI lookups, path traversal, reverse shells, cloud-metadata access, probes for .env or .git files, credentials written to logs, and attacks on LLM features such as prompt injection. The full list is in the App monitoring docs, and the longer explainer is Monitor a running app for attacks with beam monitor.
It has real limits, and you should plan around them:
- Watch-only. It never blocks requests, changes the application or sits in the request path. Blocking stays with your firewall, WAF or code.
- Heuristic. A match means a line looks like an attack, not that one succeeded. It misses attacks it has no rule for, and it flags some legitimate traffic. An app that answers 200 to every path, like a single-page app's catch-all, can trip the served-file rule.
- Logs only. It cannot see anything that never reaches a log it reads.
- Blind to the platform. It cannot see inside Railway or any other host. A platform-side intrusion, such as someone reading stored variables, produces no line in your app's logs.
Treat it as a smoke alarm in your own hallway, not a lock on the host's door.
The approach: a separate monitor service
You have two ways to get logs into beam monitor. If you can change the app and it is Node.js or Next.js, the in-app SDK (@agent-beam/beam/monitor) needs only the app token. If you would rather not touch the app, or it is not Node.js, use the separate service.
The Beam monitor service lives in the beam-cli repository under deploy/railway-monitor. Its README describes it plainly: it streams the target service's HTTP request logs and app logs from Railway into beam monitor, detection runs in the monitor service, and only redacted findings go to your workspace. The monitored app's code, image, start command and deploys are untouched, and if the monitor stops, the app does not notice. It needs no public domain, volume or inbound port, and uses a few tens of MB of memory.
Step 1: add the app and get its token
An owner or admin opens App monitoring in the dashboard, turns it on, and adds the app by name (the docs use bgremover and explainx as examples). Each app gets its own token, starting with beam_app_, shown once. Copy it now. It is your BEAM_APP_TOKEN.
New to AgentBeam? The getting-started link works whether or not you already have an account and lands you on the App monitoring page.
Step 2: create a Railway project token
In the Railway project that runs your app, the README says to go to Settings, then Tokens, and create a token for the environment you want to watch.
Read this before you do. The README is direct about it: a Railway project token is not read-only. Besides logs, it can read every variable in its project and environment, which includes other services' credentials, and it can deploy. That is a lot of reach to hand to a small service. Seal the token, scope it to the one environment you need, and rotate it if anything looks off. If you can change a Node.js or Next.js app, the SDK path avoids this entirely because it only needs the app token. This tradeoff is yours to make, and on a day when a host has just disclosed a breach of its variable storage, it is worth thinking about twice.
Step 3: add the monitor service
Per the README, add a service to the same project, set its source to the beam-cli repository, and set the Dockerfile path variable:
RAILWAY_DOCKERFILE_PATH=deploy/railway-monitor/Dockerfile
Then set these variables, sealing the two tokens:
| Variable | Value |
|---|---|
BEAM_APP_TOKEN | The app token from step 1 (sealed) |
RAILWAY_TOKEN | The project token from step 2 (sealed) |
MONITOR_SERVICE | The service to watch, for example BGR_PROD |
MONITOR_ENVIRONMENT | Optional; defaults to production |
BEAM_LLM_BASE_URL, BEAM_LLM_MODEL, BEAM_LLM_API_KEY | Optional: review flagged lines with your own LLM |
The container runs as an unprivileged user and, per the README, never writes the tokens to disk.
Step 4: deploy, then prove it works
Deploy the monitor service. Then open a shell on it and run the test, as the README instructs:
node /app/dist/cli.js monitor test
A test finding should appear on the App monitoring page within seconds, and at the webhooks for that app. It travels the same route a real finding takes, so a wrong token or a blocked outbound connection shows up now, not during an attack. To check the app, whether monitoring is on, the level, the LLM in use, the number of webhooks and this month's usage, run beam monitor status wherever the CLI is installed.
Other hosts
For a server, a container platform or another PaaS that lets you run a process, install the CLI where the app runs, put the token in the platform's credential store as BEAM_APP_TOKEN, and point it at the logs the app writes:
npm install -g @agent-beam/beam
beam monitor --file /var/log/app/app.log
beam monitor test
The docs suggest running it under whatever already keeps your app alive: a systemd unit, a sidecar container that shares the log volume, or pm2. Prefer --file in production so the app does not depend on Beam's pipe; --stdin is available for piped output. The ready-made no-change service is specific to Railway, and I would not claim an equivalent for other hosts that the docs do not describe.
Choose a level
The level is set in the dashboard by an owner or admin, so you can change it while the monitor keeps running.
| Level | Alerts on | Repeats fold for | LLM review |
|---|---|---|---|
| Light | Critical | 60 minutes | Never |
| Standard | High and critical | 15 minutes | High-severity findings |
| Strict | Medium and above | 5 minutes | Every alert |
Start at Standard. Traffic from one client address is grouped, so a scan from one IP becomes one alert with a count, and above 200 new findings in 10 minutes the rest are summarized in a single finding.
Send findings to Slack
Owners and admins can add up to five webhooks in App monitoring settings. Paste a Slack incoming-webhook URL and you get Slack's native message format; Discord works the same way. Any other public HTTPS URL receives JSON with a findings array, signed with X-Beam-Signature and X-Beam-Timestamp so your receiver can verify it.
Each webhook has its own minimum severity (critical only, high and critical, or everything the level alerts on), its own signing secret, a test button and a rotate-secret button. Webhooks can also be limited to specific apps, so the team that owns one service does not get alerts about another. Use the test button once, then run beam monitor test again so you see the whole path end to end.
Tune the first week
A gradual path avoids alert fatigue.
- Day one: run
beam monitor test, keep Standard, and look at the dashboard with your own eyes before adding any webhook. - Days two and three: add one Slack webhook at a high minimum severity. For findings you decide are routine, such as a scanner you run yourself, use Create rule on the finding: Ignore it drops matching findings before they are stored or alerted on, and Keep it here, don't alert leaves them on the dashboard but out of Slack and webhooks.
- End of week: try Strict for a day to see what medium findings look like, and decide whether you want them. Optionally add an LLM second opinion for flagged lines. A "benign" verdict keeps a finding off your webhooks, a refusal never counts as benign, and a "benign" verdict never silences a critical finding.
Treat each finding as a lead. A flagged injection string in a request that returned 404 tells you who is probing. The same string followed by a 200 and an odd response size deserves a look at the code behind that route.
The day your host discloses a breach
This is where a separate record earns its keep. Take it in this order.
- Rotate first. Do not wait for certainty. In the Railway case, the company advised customers to roll all environment variables, API keys and private registry credentials. Our rotation plan puts AI provider keys first because stolen keys are monetized quickly. Rotate the app token and your Railway project token too, if you use the monitor service.
- Get the window. Ask the host for the exposure window so you can bound your review. In the Railway disclosure, how values were decrypted and the entry point were not disclosed, so assume nothing about the edges.
- Review findings and logs for that window. Filter the dashboard by app, look for credential-file requests answered with a 2xx, reverse-shell or metadata-access findings, repeated failed sign-ins, and anything from a client address you do not recognize. Cross-check against your provider-side usage and billing, and your database connection logs.
- Keep the trail. Export or note the findings and the matching log excerpts with timestamps, and record which keys you rotated and when. Rule creation, token rotation and app changes are audit-logged in AgentBeam.
Be honest with yourself about what that trail shows. Findings tell you what your app was served and logged. A clean week is reassuring, but it does not prove the host's variable store was untouched, because that activity does not pass through your logs. It also cannot prove an attacker did not use a stolen key from somewhere else. Use it to support your investigation, not to close it.
Paste this to your coding agent
AgentBeam ships a skill called agentbeam-app-monitoring for exactly this job. Install it, then hand your agent a prompt. The install command comes from the docs and from Let your coding agent set up AgentBeam:
npx -y @agent-beam/beam add skill agentbeam-app-monitoring -g --all-agents
Then paste something like this:
Add AgentBeam monitoring to this app using the agentbeam-app-monitoring skill.
It is hosted on Railway. I do not want to change the app, so use the separate
Beam monitor service from the beam-cli repo (deploy/railway-monitor).
Do not ask me to paste any token into this chat: tell me the variable names
to set (BEAM_APP_TOKEN and RAILWAY_TOKEN), keep both out of the repository,
and have them sealed in Railway. Finish by running beam monitor test and
tell me what you changed.
Review what the agent does before you accept it. Per the docs, the skill finds where the app runs, keeps the token out of the repository, and finishes with beam monitor test.
How AgentBeam handles this
AgentBeam's App monitoring does the log-watching job that a platform's own dashboard does not: it runs detection where your logs are, sends only redacted findings (a rule, a severity, a file, a count and an excerpt of up to 600 characters), and keeps raw lines on the server. Each app has its own least-privilege token that only sends that app's findings, shown once, stored hashed and revocable, and every change to apps, rules and tokens is audit-logged. Roles matter too: owners and admins configure it, while security managers and auditors can view findings. Detection is heuristic and watch-only, so it complements your firewall and patching rather than replacing them. See the app monitoring solution page for the short version, or start with an app token to try it on one service.
Related reading
- Monitor a running app for attacks with beam monitor, the full explainer.
- Railway breach: environment variables accessed, rotate every secret.
- Let your coding agent set up AgentBeam.
- Check dependencies against known CVEs in CI, which uses the same app token.
- October 2026 attacks timeline for the month's incidents.
- What is AI agent monitoring? for the agent-side counterpart.
- What app monitoring can and cannot catch when your platform is breached
- API keys in logs and env vars: how they leak and how to catch it
- Your first 24 hours after a vendor breach: a rotation and monitoring runbook
Frequently asked questions
How do I monitor a Railway app for attacks without changing the app?
Run the Beam monitor service from the beam-cli repository (deploy/railway-monitor) as a second service in the same Railway project. It streams the target service's HTTP and application logs into beam monitor, so the monitored app's code, image, start command and deploys are untouched. If the monitor stops, the app does not notice.
Does beam monitor protect my app or block attacks?
No. It only watches. It never blocks requests, changes your application or sits in the request path. Detection is heuristic, so it can flag harmless lines and miss attacks that never reach a log. Keep your firewall, input validation and patching.
Can beam monitor see what happens inside Railway's platform?
No. It sees the HTTP and application logs of the service you point it at. It cannot see a platform-side intrusion, such as someone reading your stored environment variables inside the host. Its value there is a separate record of what your app was served and logged.
What does the Railway token in the monitor service allow?
The README warns that a Railway project token is not read-only. Besides logs it can read every variable in its project and environment, including other services' credentials, and it can deploy. Keep it sealed, and prefer the in-app SDK when you can change a Node.js or Next.js app.
Does this work on hosts other than Railway?
The ready-made no-change service is for Railway. On other platforms you run beam monitor under whatever keeps your app running, such as a systemd unit, a sidecar that shares the log volume or pm2, or use the SDK inside a Node.js or Next.js app.
What should I do the day my host discloses a breach?
Rotate the secrets your host stored, starting with AI provider keys, then review your own logs and monitoring findings for the exposure window the host gives you. beam monitor findings and your logs form an evidence trail of what your app saw, though they do not prove what happened inside the host.
If you have a Railway service running today, the quickest next step is to add it as an app in AgentBeam and run beam monitor test. You will know within a few minutes whether the path works.
Details are accurate as of October 23, 2026. The Railway monitor service, App monitoring availability and allowances, and Railway's own token behavior can change, so check the linked README and docs before you deploy. Detection is heuristic.