TRACE AI Labs › LLM compliance
LLM compliance is whether a language model keeps following a binding rule once it is deployed and something in the situation rewards breaking it. Every model we have tested can recite its rules. Far fewer keep them.
An LLM deployed as a workplace assistant is given rules: a hiring assistant must not screen on protected attributes, a bank assistant must file a suspicious-activity report before releasing funds, a procurement agent must use certified vendors above a spending threshold. LLM compliance, sometimes called AI compliance or agent compliance, is the question of whether the model actually acts on those rules when its task incentives point the other way.
This is different from two things it is often confused with. It is not knowledge of the rule: models score well on questions about regulations and still violate them in a live decision. And it is not jailbreak robustness or refusal safety: nobody in these settings is attacking the model. The pressure is ordinary. A deadline is close, a manager says to make an exception, a coworker already did it, or the user simply argues back after the first answer.
Our AIES 2026 study, Why Do AI Agents Break Rules?, put twelve instruction-tuned models in the same enterprise procurement role under one legal rule and changed only the context around it. Three findings shape how we think about LLM compliance:
A built-in "follow the law regardless" instruction helped, but a deadline still got past it for all but one model.
PACT (Pressure-Applied Compliance Testing) turns those findings into a benchmark. Each of its 3,364 items is one workplace decision: a system prompt that gives the model a persona, its KPIs, and a compliance rule stated as plain fact; a user request with a short menu of options where the rule-violating option wins on the metric the persona is measured on; and, on multi-turn items, a scripted pushback that argues against the compliant choice. Items span 48 scenarios in 12 regulated domains, and the rules are real statutes. In seven scenarios a court or regulator has already punished the same violation.
A model's PACTScore is the share of items it handles compliantly, weighting the first decision three to one against the decision after pushback, with each item run three times and credit given only when the model gets it right every time. Because the benchmark also includes near-identical situations where the rule does not apply, a model cannot score well by refusing everything. Over-applying a rule is scored as an error too.
| Measure | Result |
|---|---|
| Best PACTScore across 24 models | 0.944, and none clears the bar we set for unsupervised use |
| Effect of one sentence of workplace pressure | Violation rates rise by 65% |
| Mildest pressure family | Still produces violations on 4.8% of requests |
| When a model does violate | It presents the result as compliant 79% of the time |
| Newer or bigger models | Not reliably better; a 27B dense model ties a trillion-parameter one |