Code Review Risk Labels
Every PR gets the five-lens review swarm automatically. On top of that, CI tags each PR with a risk:low / risk:moderate / risk:high label, so you know when a change is worth a deeper read of your own before you approve it. The label is a signal; the decision is yours.
The high-risk path list does double duty. Adding an entry to it both re-weights this label and wakes the swarm's security reviewer on every file beneath that path, at its own model and timeout — so editing the list to tune a label also changes reviewer selection and per-round review cost.
.github/code-review-heuristic.yml; the logic is in .github/scripts/classify-risk.ts. Numbers on this page are illustrative — when in doubt, read the YAML, not this doc. The rule
Two independent conditions, evaluated over the PR diff:
cond_path— a non-test file of any status, added files included, falls under a high-risk path (the RBAC engine, sign-in, the external user API, ramps, playbook draft service, data fields, the workflow/playbook UI, portal security surfaces, CI identity, agent hooks and settings, …), or the diff touches a migration (in either Alembic tree), a route table or route policy, an IAM change, a sweeper-style tool, or deletes 20 or more test files.cond_size— the number of modified source files exceeds the configured threshold.
The label is the XOR of the two:
cond_path | cond_size | Label |
|---|---|---|
| ✗ | ✗ | risk:low |
| ✓ | ✗ | risk:moderate |
| ✗ | ✓ | risk:moderate |
| ✓ | ✓ | risk:high |
Neither → low. Exactly one → moderate. Both → high.
Why the two conditions count differently
cond_size counts modified source files only — files whose GitHub status is anything other than added, minus renames of 4 or fewer changed lines. A new file has no blast radius on existing call sites by definition, so a feature that adds 80 brand-new files is safe in a way that a 20-file refactor of existing code is not. A pure move is not an edit either.
cond_path counts added files too. New code in an authorization directory, a new migration or a new route table has no existing caller to modify, yet it is where the risk lives.
Tests and non-code (docs, config, lockfiles) never count toward cond_size, and a test-only edit to a high-risk path does not trip cond_path. The path list names a few config files, such as app/config/environments.yaml and infra/app/envs.ts, that count whatever their extension; the heuristic file's header says which.
Two triggers read names or diff text rather than a fixed path. An IAM change is an infra/ .ts file named like an IAM or boundary file, or one whose diff adds or removes a line with aws.iam., a policy statement keyword (Effect:, Action:, Resource:, Principal:, Condition:), a quoted service:Action string or an ARN, or one with edits but no diff because GitHub left the diff out for size. A sweeper-style tool is a file under devtools/, .github/scripts/, .github/workflows/ or infra/scripts/ whose name contains sweep, reap, retire, purge, destroy, delete or prune; a tool with another name for the same job is left to the model check.
The model check
When the rule above gives risk:low, a small model reads the PR title, the file list with change types and line counts, and the body. It answers three yes/no questions: does the PR change who can read or write what, is it safe only if a deploy-time condition holds, and does it delete or overwrite live resources or data. A yes to any of them makes the label risk:moderate. The model never sets risk:high. If the call fails, a PR that already has a risk: label keeps it, and a PR with none gets the label the rule gave it.
Try it
* Illustrative threshold for this widget only. The live value is source_file_modified_threshold in code-review-heuristic.yml.
Worked examples
Read these as the rule applied to diff shapes — the ✓/✗ columns are what matters, not exact counts.
| Diff shape | cond_path | cond_size | Label |
|---|---|---|---|
| New feature — all brand-new files (even 80 of them), anywhere | ✗ | ✗ | risk:low |
| New files added under a risky dir, no existing file modified | ✓ | ✗ | risk:moderate |
| A few-line edit to a high-risk file (e.g. an authz check) | ✓ | ✗ | risk:moderate |
| Large refactor of existing code, no high-risk path touched | ✗ | ✓ | risk:moderate |
| Large refactor that modifies a high-risk file | ✓ | ✓ | risk:high |
| Pure rename of an existing module (4 or fewer changed lines per file) | ✓ if under a risky path, else ✗ | not counted | risk:low, or risk:moderate under a risky path |
| A new migration, route table, or IAM change | ✓ | ✗ | risk:moderate |
| Docs/config-only PR, or test-only edits | ✗ | ✗ | risk:low |
What to do with the label
| Label | Suggested action |
|---|---|
risk:low | Nothing — the automated SDK review is enough. |
risk:moderate | Consider running /code-review ultra <PR#> locally for a deeper pass, especially if you touched the risky area's behavior. |
risk:high | Strongly worth /code-review ultra <PR#> before merge — this PR both modifies a risky surface and is large. |
The separate bot-approved label is the bot's read that it is asking for nothing more, not a human approval — a human review is still required to merge. REVIEW.md owns exactly when it is granted; don't restate the condition here, it has drifted before.
Where this lives
- Heuristic config (paths + threshold):
.github/code-review-heuristic.yml - Classifier:
.github/scripts/classify-risk.ts - Workflow:
.github/workflows/ai-review.yml - Label application:
devtools/review-swarm/github/labels.ts - Security reviewer's path list (same
high_risk_paths, viapatternsFrom):devtools/review-swarm/reviewers/security/config.yml - Review rubric (what the reviewers flag):
REVIEW.md