Beam turns the activity it already records — commands, file edits, tool calls, flagged findings — into a per-session scorecard: did this agent run stay inside policy, and where did it not. Evaluation against your rules, not a benchmark of model quality.
This scores behavior against policy, not output quality — it won't tell you if a generated answer was correct, only whether the actions taken to get there were allowed.
Every check reads from the same activity record Beam already keeps — nothing is inferred about intent, only what the agent actually did.
Policy checks
Define what an agent run is allowed to touch — paths, commands, network destinations — and each session is checked against it.
Risk score
Sessions with credential access, destructive commands, or permission changes score higher risk, with the evidence behind the number one click away.
Trend
Compare scorecards across sessions to see whether a given agent, project, or team is drifting toward riskier behavior over time.
Coverage
A session with no recorded activity yet, or activity outside what Beam is watching, is marked unscored rather than silently passed.
01
No separate setup — evaluation reads the same activity record Beam captures for every monitored session.
02
Each rule you've defined — allowed paths, blocked commands, network destinations — is checked against what actually happened.
03
Credential access, destructive commands, and permission changes weigh into one score per session, evidence attached.
04
Review one session, or compare scorecards across an agent's history to catch drift before it becomes an incident.
No. Beam's scorecard checks behavior — what commands ran, what files changed, what was flagged — against policy. It does not judge whether a generated answer, PR, or piece of code was correct; that's a different problem from the one Beam is built to solve.
The same flagged findings Beam already surfaces for runtime monitoring — sensitive-data access, credential exposure, destructive commands, permission changes — rolled into one number per session, with the underlying evidence still one click away.
It's marked unscored, not passing. A scorecard is only as complete as the activity record behind it, and Beam never fills a coverage gap with an assumed result.
Not currently. Like the rest of Beam, evaluation observes and flags — it does not block. A scorecard is meant for review, the same way flagged runtime activity is today.
No separate integration — if Beam is watching an agent, that session can be scored against the policy you set.