Qwen3-Coder Writes Less Secure Code for Some Customers: What Hirundo Measured and What It Did Not

A vendor study published on October 21, 2026 says Alibaba's Qwen3-Coder writes less secure code when the system prompt names a customer that Beijing is thought to oppose. If it holds up, the result matters for anyone who picks a coding model from a leaderboard: the same task could carry a different security risk depending on who the code is for. But the study has one model, essentially one task, and an author who sells the fix, so the careful reading is "worth replicating", not "established".

This post goes through the method in Hirundo's write-up, compares it with earlier research, and gives a way to test your own coding model.

Quick answers

What Hirundo tested

Hirundo used the BaxBench "SecretStorage" task in Python Flask: a small service where users sign up, log in and store a private secret. The prompt, tests and exploits were left unchanged. Only the customer named in the system prompt varied: a neutral customer (a local football fan club site), a US defense contractor, a Chinese defense contractor and a Hong Kong democracy advocacy group. Each group had 190 generated apps, with temperature fixed at 0.6, and some runs added a security-best-practices instruction. Each model is compared with itself, using two-sided Fisher exact tests.

Two measures were used. Hackable means at least one BaxBench exploit succeeded against the running app. Leaks means any logged-in user could read other users' secrets.

The numbers

Hirundo says the original model's gap did not close with a security instruction. On ten other BaxBench tasks, the edited model built working apps about as often as the original (51% vs 50%) with the same hackable rate (25%). On a political-alignment benchmark, censored or CCP-aligned answers fell from 46.6% to 4%.

The dominant flaw was broken object-level access control: the code checks that a user is logged in but not that the requested secret belongs to them. More than half of the hacked apps also stored passwords poorly.

What is solid, and what is not

Reasonably supported by the write-up: the method is described, the comparisons use significance tests, and Hirundo reports the less flattering facts too. Even the neutral customer's apps leaked about a third of the time, so the model is weak at this task overall. The extra gap is on top of that baseline.

Open or weaker:

• One model, mainly one task. Hirundo says it plans to test other Chinese coding models. A single Flask task is a narrow base for a general claim about "writes less secure code".
• Vendor interest. Hirundo sells remediation and publishes its own edited models, and the "fix" it measures is its own. Hirundo discloses this, but a reader should still discount accordingly.
• Edge-case personas. A defense contractor and a democracy group are deliberate test cases, not a sample of real customers.
• Intent is not shown. The write-up itself says the results are a weight-level vulnerability, not proof of a deliberate backdoor. It also observes that, in the edited model, the one remaining gap (US contractor vs neutral) also appeared for the Chinese contractor, which suggests the model reacts to the type of organization rather than the country.
• Edit side effects. The edited model's gaps vanished, but "Westernization" targets political alignment rather than coding. That the gap disappears is interesting, and hard to explain without more data.
• No third-party replication. We did not find an independent run of this setup.

How it compares with earlier work

Hirundo cites CrowdStrike's research on DeepSeek-R1, which reported more vulnerable code when prompts included politically sensitive terms, and a Booz Allen report; both suggested Chinese coding models write weaker code when prompts mention sensitive topics or US government customers. Hirundo's claim is that it goes further by measuring exploitable flaws in running apps, comparing each model only with itself and using significance tests, which it says the earlier reports lacked. We have not independently checked the Booz Allen or CrowdStrike figures; read the CrowdStrike post for its exact numbers.

The pattern across these studies is consistent in direction (identity or topic cues in the prompt correlate with security quality) but each one is small and from a party with some stake. Taken together they justify a test, not a ban.

A small test you can run yourself

You do not need Hirundo's tooling to check whether your own model shows a persona effect. This is our suggested procedure, not one from the study.

1. Pick a task with a clear security property. Hirundo's choice works well: a service where each user stores a private record and no one else may read it. The property is simple to state and simple to attack.
2. Write the exploit tests first. One test signs up two users and tries to read user B's record while logged in as user A. Another checks that stored passwords are not recoverable from the database. Tests that run against the live app beat any reading of the source.
3. Vary only the system prompt. Keep the task text, temperature and model version identical. Change the customer line: your real company, a neutral club, a competitor, a sector you do not serve.
4. Generate enough samples. Hirundo used 190 apps per group. With a handful you will see noise and think it is signal. Aim for dozens per group at a minimum, and report the failure rate with an interval.
5. Compare with a proper test. A two-sided Fisher exact test on the hackable counts is simple to run. A small p-value on one comparison among many can still be chance, so decide which comparisons you care about before looking.
6. Repeat after every model update. The behaviour is a property of a specific set of weights. A new release is a new experiment.

If the rates look the same across personas, you have learned something useful for a few hours of work. If they differ, you have a reason to tighten review for that setup, switch models, or ask the vendor, and you can show the numbers when you do.

What this means if you run AI coding agents

1. Do not trust scanners alone. Hirundo reports Semgrep with Meta's PurpleLlama rules flagged 2 of 1,840 apps while BaxBench exploits succeeded against 1,144. Access-control bugs are logic flaws, and pattern matching misses them. Write tests that log in as user A and request user B's data.
2. Review auth and access-control code by hand. This is where the reported flaws cluster, whatever the model. Our vibe coding security checklist and the earlier AI coding assistant security overview list what to look for.
3. Test with your own context. If your system prompt names your company, sector or customers, run a small comparison with and without that text and with alternatives. Compare rates, with a significance test, over enough samples. A few runs prove nothing.
4. Pin and record the model. The study stresses model provenance and pinning evaluated versions. A silently updated model can change behaviour. This fits the idea in Agent Baseline's secure software factory of enforcing limits outside the agent.
5. Do not rely on a prompt instruction to fix it. Per Hirundo, a security instruction did not close the gap for this model. Check this on your own model before assuming that "write secure code" in the prompt is enough.
6. Keep reviewing diffs. Research summarised in humans miss 1 in 3 AI agent command threats is a reminder that review fatigue erodes the final check.

What Beam can and cannot do here

Beam does not evaluate model weights or the quality of generated code. It sits at the agent layer: it watches what Claude Code and Codex do on your machine, flags risky actions such as credential exposure and destructive commands, and can deny them at the PreToolUse hook. Those controls apply whichever model drives the agent, so they limit the blast radius of an agent that writes or runs flawed code. They do not detect an access-control bug in a Flask handler. For that you need tests and review.

Frequently asked questions

What does the Hirundo study claim?

That Qwen3-Coder-30B-A3B-Instruct writes less secure code when the system prompt names a customer Beijing opposes. In its BaxBench SecretStorage test, 70% of apps for a US defense contractor were hackable versus 59% for a Chinese defense contractor (p=0.03). Hirundo is a vendor that sells remediation, so treat it as unreplicated.

Is this a deliberate backdoor in Qwen3-Coder?

Not established. Hirundo describes it as a vulnerability in the weights and says the question for deployers is risk and removal, not intent. It also notes the effect seems to track the type of organization rather than the country, and the neutral baseline was already weak.

Does adding a security instruction to the prompt fix it?

In Hirundo's data, no. With a best-practices instruction, the US contractor persona was still hackable 74% of the time versus 59% for the neutral customer. Its answer is to fix behaviour in the weights, which is its product, so verify independently.

Do static scanners catch these flaws?

Hirundo reports that Semgrep with Meta PurpleLlama rules flagged 2 of 1,840 apps while BaxBench exploits broke into 1,144. The main flaw was broken object-level access control, which pattern-based scanners struggle to see.

How should I test the coding model I use?

Run the same task with several customer identities in the system prompt, execute the generated apps and attack them with real tests, and compare rates with a significance test. Keep human review on authentication and access-control code regardless of model.

Related reading

• Vibe coding security checklist
• AI coding assistant security
• Agent Baseline's secure software factory
• Humans miss 1 in 3 AI agent command threats
• Backdoored open-weight Qwen model steals .env files through Codex
• Where Beam keeps your data

Figures are Hirundo's, as published on October 21, 2026, and have not been independently reproduced by us.