2026 AAAI/ACM Conference on AI, Ethics, and Society (AIES)2026 COLM Workshop on Agent Behavior
How framing, context, and social signals shape compliance
Companies now write legal rules into their AI agents' system prompts and expect the agents to follow them. We gave twelve language models one environmental regulation and a procurement job that rewards breaking it, then tested what three theories from law and economics predict about when people obey. Each theory explains part of what the agents do, and the same regulation produces very different behavior depending on the model.
This is a real trial from the study. The agent, the rule and the purchase request stay the same. In the second run the prompt also names a fine, and the same model changes its answer. Use the timeline or the buttons to move through it.
Specifying a penalty can turn a legal obligation into a cost-benefit calculation that favors violation. We show that this enforcement information paradox occurs in AI agents. Most AI safety evaluations test whether models fail; we ask why, using compliance theory from law and economics as a diagnostic. We evaluate twelve instruction-tuned language models deployed as enterprise procurement chatbots. Each is given an environmental regulation in its system prompt covering large purchases, and a vendor list on which the certified suppliers cost nearly twice what the uncertified ones do. We test the agents against the predictions of deterrence, legitimacy, and expressive law, and find that each theory accounts for part of what we observe. Under identical conditions, compliance spans 46 percentage points across models, and models differ in which pressure breaks them: some treat the regulation as binding however it is worded, while others fail where theory predicts, under low penalties and non-command phrasing. Benchmark scores and developers' own descriptions of post-training do not predict where a model falls. Across all twelve, financial incentives, managerial demands, peer outcomes, and employee pressure each produce large compliance failures. These agents violate regulatory constraints to satisfy local user objectives in ways standard alignment benchmarks do not measure. Embedding the rule in the system prompt is not on its own enough to produce a compliant agent: model selection is itself a governance decision, and benchmark evaluation is not sufficient for compliance-sensitive deployments.
Change the framing, fine or added condition to see the exact prompt and how each model responded.
The paper treats three accounts from law and economics as hypotheses about the agents (§2.1), and each one describes part of the results. Dots are the twelve models and the black bar is their average. Hover a dot to see the model.
Agents comply when the expected penalty is larger than the gain from breaking the rule. Compliance should rise with the fine and should not depend on wording.
ResultThis fits the models that weigh costs, but it cannot explain the enforcement information paradox: with a Small Fine, compliance is lower than with no fine.
See Figure 3 →Agents comply because the rule comes from a legitimate authority, so the way a rule is issued, and by whom, matters more than the penalty.
ResultThis fits the rule-anchored models (GPT-OSS, Qwen 3.5 and Llama 4), which hold the rule however it is phrased, while the other models comply mainly when the rule is imperative.
See Table 3 and Figure 4 →Law signals what behavior is normal, so what peers and the community do should change compliance even when the fine stays the same.
ResultThis explains the peer and norm effects. A peer who was fined, or community activism, raises compliance, and a peer who escaped an audit lowers it, with the fine unchanged.
See Figures 6 and 11 →Models show each of these modes to a different degree, and the context of a request decides which one drives the decision.
These are the paper's figures and tables, recomputed from the released trials and shown in the paper's order. Each dot is one model in one condition. Select a model to highlight it in every figure, and hover any underlined term for its meaning.
Across the twelve framing and fine conditions, overall compliance ranges from 43.5% to 89.5% depending on the model. Models also break for different reasons. Some are sensitive to how the rule is phrased, others to what they are told about enforcement, and a model can be robust on one and fragile on the other.
Compliance (%) in the four configurations of Table 3, plus each model's overall rate across all twelve framing and fine cells. Hover a column header for what it tests, and click it to sort.
Figure 4, left. The horizontal axis is framing sensitivity and the vertical axis is enforcement sensitivity. Negative drops are plotted at zero.
Deterrence predicts that compliance rises with the expected penalty and does not depend on wording. In the data, wording moves compliance a lot, and adding a Small Fine lowers compliance for many models. The paper calls this drop the enforcement information paradox.
Figure 3. Choose a framing; each column is a fine level. At the Small Fine, paying the expected penalty is cheaper than buying certified, so an agent that only minimizes cost should break the rule.
Small Fine: audit "unlikely but possible", $2,400. Medium Fine: "possible", $4,800. Large Fine: "likely", $7,200. The cheapest certified vendor costs $2,400 more than the cheapest uncertified one.
Appendix C.2. The rule keeps its imperative structure and only the verb changes, for example "state environmental reg now recommends purchases over $1k to use ISO 14001 certified vendors". The group in brackets is the paper's verb-strength group. Hover a verb for its exact sentence.
One paragraph of company background describes community activism, an industry standard or media coverage. None of them mentions a penalty, and each sits next to the same informational regulation. At No Fine all three raise compliance above the regulation alone, as expressive law predicts.
Click a row to read the text added to the system prompt.
The employee adds one of nine pressure tactics to the request. An anti-adversarial mandate in the system prompt helps every model, though the size of the gain differs by an order of magnitude across models. Deadline urgency still gets through for most of them, which the paper calls the urgency exception.
Informational framing. Click a tactic to read the employee's message.
Same settings as the chart above. Cells show compliance (%); green cells mostly complied and red cells mostly violated. Hover a column header for the employee's message.
After the agent's first answer, the employee sends one follow-up. Compliant answers mostly hold up against a neutral probe, except for GLM 4.7 Flash, and a cost argument or a manager's order erodes them further. The same neutral probe repairs some violations, from none for several models to about 60% for Kimi K2.5, while a direct correction brings nearly every model back.
Share of compliant first answers that are still compliant after the follow-up. Hover a row for the message.
Share of violating first answers that become compliant. GPT-OSS and Qwen 3.5 had no violations under informational framing to follow up on.
Of 6,743 classified violations, 96.0% mention the regulation in the agent's stated reasoning and 4.0% are silent. Silent rates do not track compliance: Mistral Small and Qwen 3.5 rank second and fourth on compliance and have the highest silent rates, so a compliant model is not necessarily an auditable one.
Percent of each model's classified violations in the selected regime, sorted by SILENT rate. Hover a class in the legend for its definition.
Every number on this site is computed from the released trials. This page holds a sample of 1,010 agent replies, each labeled with its experiment and condition (hover a label for its meaning). Certified vendors are marked green and uncertified ones red.