AI Asset Inventory and Model Security: A Repository Review

An AI asset inventory should tell a reviewer which components exist, what evidence supports that conclusion, and what those components can touch. A list of model names leaves out the skill that edits files, the retrieval connector that supplies context, and the container that grants access to both.

This guide defines a repository-first review you can use with Beam's AI asset scanner skill. The inventory format and worked example below are Beam's practical review convention, not a claim of compliance with an external bill-of-materials standard.

What belongs in the inventory?

Start at the application entrypoints and follow references into instructions, tool configuration, and data handling. Include these categories even when the result for one is unknown:

Inspect hidden configuration inside the agreed root. Exclude caches and generated output unless they answer a specific review question. Record file-count, individual-size, and total-byte limits, skipped paths, and any truncation; never follow a symlink outside the scope.

Discovery does not authorize cloud queries, database reads, endpoint probes, or a search of the operator's home directory. Prefer credential names and secret-store references to values. Prompt templates and memory exports can also contain sensitive data, so keep evidence excerpts short and redacted.

How should evidence states work?

Use three explicit states. Referenced means an identifier appears in source, documentation, or a sample. Configured means an executable path or deployment configuration wires it into an application. Runtime-observed requires a dated supplied runtime artifact; record its environment and observation window.

A README mentioning a model is referenced evidence. A service configuration naming that model is configured evidence. Neither proves a request reached it. Keep tests, examples, and retired files labeled so that a sample configuration does not inflate the deployed inventory.

Assign a stable report ID and record source path plus line number or JSON pointer. Add publisher, declared version, resolved revision or digest when available, consumers, owner, data categories, and allowed operations evidenced by code or configuration. Preserve unknowns rather than guessing them from filenames.

What does a useful review find?

Consider a fictional repository for a support assistant. Its code includes a summarizer, an optional local embedding worker, and an MCP search tool. This compact inventory records configuration evidence only:

Trace the path from the support question through search results, prompt assembly, and the provider request. If the application passes complete search documents into the summarizer, record that data flow. Whether those documents contain customer secrets remains unknown without appropriate evidence; do not inspect a live dataset to fill the gap automatically.

The model worker deserves early review because the configuration permits supplier code to execute. A useful finding reads: high severity, high confidence in configuration evidence; workers/embed.py:17 enables custom code without an immutable revision; a changed supplier revision could execute different code in the worker; remove the requirement where possible, or review and pin the necessary code before use. Deployment and exploitation remain unverified.

The missing memory policy is a separate gap. Ask the owner to establish retention and deletion behavior; do not assign critical severity merely because documentation is absent. Unknown package versions similarly require resolution before any advisory can be claimed to apply.

How do you inspect models safely?

Pickle deserialization can execute arbitrary code. Discovery should never call a model loader or import downloaded Python just to identify an artifact. Inspect filenames, model cards, manifests, and loader source; record binary contents as unassessed when no trusted static analyzer is available. Hugging Face also cautions that its pickle import scan is not a complete safety assessment. Hugging Face pickle security guidance.

Transformers custom models can require trust_remote_code=True, which is a reason to inspect the accompanying Python and its dependencies. Treat that switch as an execution decision requiring review, not a convenience for inventory collection. Transformers custom model documentation.

Safetensors stores tensor data without pickle's arbitrary-object execution mechanism. That narrower format property does not certify the surrounding application code, model behavior, or data provenance. Keep those review questions separate. Safetensors format and rationale. See also CycloneDX ML-BOM for a standard way to exchange the result.

Supplier identity, artifact integrity, and dependency records also matter beyond the loader. OWASP's supply-chain guidance recommends component inventories and verification of model sources and integrity. Use those principles to request the missing provenance evidence; a hash calculated from an unknown file identifies those bytes but does not establish who supplied them. OWASP LLM supply-chain guidance.

Why pickle is the usual place to stop

The reason this guide keeps repeating "do not load it" is mechanical, and Hugging Face's documentation explains it well. A pickle file is a stack of opcodes that the unpickler executes in order. Two properties make it unsafe. Importing a module can run code, and the GLOBAL and STACK_GLOBAL opcodes tell the unpickler what to import. The REDUCE opcode then calls the imported function with supplied arguments. A pickle cannot embed new function definitions, but it can import a builtin such as exec or eval and call it on a string. Hugging Face demonstrates this with the fickling library, which inserts an exec call into a pickle that otherwise looks like ordinary data. Loading the file prints the attacker's message and then returns the expected data, so nothing looks wrong.

That is why torch.load on an unknown checkpoint is an execution decision, not a read. An inventory should record the file, its format and its origin, and it should not open it.

What a pickle scan can and cannot tell you

Reading a pickle without running it is possible. Hugging Face says its Hub scanner extracts the list of imports from pickled files using pickletools.genops, which reads opcodes without executing them, and highlights suspicious imports next to the file. Hugging Face also runs ClamAV scans, but its documentation notes that the antivirus does not cover pickle exploits.

The same page carries a plain disclaimer: the import scan is "not 100% foolproof", the safe and unsafe import lists are maintained on a best-effort basis, and it is the user's responsibility to check whether something is safe. So an inventory entry should say what was checked. "Imports listed by the Hub, none highlighted" is a different statement from "reviewed and safe". Record the first and leave the second as unknown unless someone with the skills has reviewed the file.

Hugging Face's suggested mitigations are worth copying into the review notes as questions:

• Does the file come from a user or organisation you trust, and is the commit signed? Hugging Face says a signed commit does not guarantee the file is safe, only its origin.
• Is there an alternative format? The documentation describes loading TensorFlow or Flax checkpoints through from_tf and from_flax, and lists safetensors as a simpler format that avoids the pickle problem.

The safetensors project describes itself as a way to store and distribute tensors safely, as opposed to pickle, while staying fast with zero-copy loading. That answers the loading question only. The surrounding application, the training data and the model's behaviour still need their own review.

Custom model code and mutable revisions

The A-03 finding in the example, a loader that requests custom code from a mutable revision, comes from a documented switch. The Transformers documentation says that because a custom model does not use the same modeling code as a Transformers model, you need trust_remote_code=True in from_pretrained() to load it. That flag means Python from the model repository will run in your process.

Two questions follow from the flag. Who controls the repository the code comes from? And does the loader name an immutable revision, or a branch that its owner can change? A branch name or the default revision can point to different code tomorrow than it does today. That is why the example finding reads "without an immutable revision" and why the recommended action is to review and pin the code you need. If the repository has no pinned commit in source or configuration, record that as an unresolved item with the file and line.

What supply-chain guidance adds

OWASP's LLM03:2025 entry on supply chain lists risks that map to inventory columns. It notes that models are "binary black boxes" for which static inspection offers little assurance, and that there are currently no strong provenance assurances in published models: model cards describe a model but guarantee nothing about its origin, and an attacker can compromise a supplier account or publish a lookalike. It also names vulnerable LoRA adapters, collaborative model-merge and conversion services, and on-device models as separate attack surfaces, along with unclear terms and privacy policies.

For mitigations it recommends vetting suppliers and their terms, scanning and patching components, red-teaming third-party models, keeping an up-to-date and signed SBOM, using file hashes and signing to compensate for weak provenance, and monitoring collaborative environments. It points to CycloneDX for AI and ML bills of materials as an emerging option.

CycloneDX's ML-BOM page says the format can represent datasets, models and configurations of AI and ML systems, including dataset provenance, so that organisations can assess risks related to bias, data integrity and model security. If your organisation already exchanges bills of materials in that format, export the inventory table into it. The table in this guide is a working review format, not a replacement for it.

How to prioritise what the inventory shows

An inventory with twenty rows is only useful if it produces an order of work. A simple approach based on the evidence states above:

1. Code that can execute from a supplier. Custom model code, adapter loaders and pickle-format weights come first, because the failure is arbitrary execution and not degraded quality.
2. Components that hold credentials or can write. An MCP server with a broad token, or a skill with shell access, can do more damage than a read-only retrieval connector.
3. Data flows that carry sensitive categories. Trace what reaches the provider request and where persistent memory sits.
4. Unknown owners. A row with no owner is a finding on its own. Nobody will resolve its open questions.
5. Everything else, sorted by whether the evidence is referenced, configured or runtime-observed.

Keep confidence separate from severity, as in the worked finding above. A high-severity item with only configuration evidence is a reason to ask for runtime evidence, not to declare an incident.

Keeping the inventory current

An inventory is a snapshot. Date it, record the repository revision it covers, and re-run the review when the dependencies, model identifiers or deployment files change. Record skipped paths and limits every time so that a reader can tell the difference between "nothing found" and "not looked at". When a later review resolves an unknown, change its state and keep the old entry in history rather than overwriting it.

For related incidents that show why unpinned and unreviewed components matter, see the npm supply-chain worms in AI tooling and AI agent application and supply-chain security.

Where do skills and monitoring fit?

For reachable skill instructions, use Beam's agent skills specification and security review. For server launch commands and tool permissions, use the MCP security guide. Review both as untrusted data before enabling their capabilities.

With an already trusted Beam CLI installation, these commands add offline heuristics for individual text artifacts:

beam scan /absolute/path/SKILL.md
beam scan /absolute/path/mcp.json --mcp

Inspect the returned findings and report rejected or skipped inputs. These commands do not enumerate repositories, inspect model weights, or discover running cloud services. Keep optional report persistence separate from discovery.

Use the inventory's unresolved actions in your coding-agent security review and rollout checklist. For why weights deserve provenance checks, see the backdoored open-weight Qwen model demo. When you need runtime evidence, start with what agent monitoring measures. Visit agentbeam.com for advanced monitoring and control options, and further self-hosting and ongoing monitoring guidance. That handoff does not upload the inventory or configure instrumentation automatically.

Reviewed September 14, 2026 and extended October 20, 2026 with additional primary sources. The example is illustrative; runtime coverage depends on supplied evidence and instrumentation.