Flipping ai-task-builder

Turn AI subtask authoring on for one vendor without a human's next "Save Playbook" starting to fail.

Owner: MauricioWhen to use: enabling AI subtask authoring per vendorScope: one vendor_features row

Canonical sources. The gate itself is app/api/playbooks/services/task_step_draft_service.py (AI_TASK_BUILDER_FEATURE_CODE, enforce_form_config_gate, and reconcile's strict_form_validation parameter). The creatable set is AI_CREATABLE_STEP_TYPE_CODES in app/api/utils/constants/step_types.py. Agent-side registration is task_builder_enabled in onramp-agents/src/shared/feature_codes.py. This runbook is the HOWTO; those are the truth.

TL;DR

🅰 One flag, two jobs

It is not only a capability switch

It registers the agent's authoring tools and flips server-side FORM/step-type validation from log-and-persist to hard rejection. The second job is the one that can break a surface nobody was testing.

🅱 The blast radius is the human save

Not the agent, and not library content

Both agent flush endpoints are already strict, unconditionally. What the flip changes is the whole-playbook update_from_draft save — the button a human presses — plus AI playbook generation.

🅲 Audit before, not after

Two classes of pre-existing content

Draft playbooks can hold FORM documents and step types that only ever persisted because validation was lenient. Find them first, because they surface as a 4xx on a save the user did not think was risky.

What the flag actually does

ai-task-builder is one features row with per-vendor vendor_features, default off, seeded by migrations/versions/2026_07_21_d84f2a1c9b3e_seed_ai_task_builder_feature.py. It is read in two very different places.

Job 1 — agent tool registration (fails closed, low risk)

task_builder_enabled(agent_state) reads enabled_feature_codes and fails closed. Off, the playbook orchestrator does not register build_task_steps at all and the task-builder domain resolves TargetStatus.CAPABILITY_OFF. The same flag gates the prompt block that tells the model authoring is unavailable — registration and prompt must never disagree, or the model advertises a tool it does not have.

This half is safe to flip. Nothing pre-existing changes; a capability appears.

Job 2 — server-side strict validation (this is the risk)

TaskStepDraftService.reconcile's strict_form_validation flips two gates from warn-and-persist to FormValidationRejectedException (a 4xx carrying structured findings plus a step_ref):

• FORM config — a new or changed FORM step_configuration runs through form_schema.validator.validate_document. Any finding rejects the whole request before any further step is processed.
• Step type — a new step, or an existing step whose type is changing, must resolve to a StepType row that is active, in AI_CREATABLE_STEP_TYPE_CODES, and not a bare FORM_* element palette code.

⚠ Read the call sites before assuming what changes. Only two of the four are flag-gated:

Path	Strict?
update_from_draft — whole-playbook human save	flag-gated ← the flip changes this
AI playbook generation (ai_playbook_service)	flag-gated ← and this
update_steps_from_draft — task-scoped, playbook surface	always True
update_library_steps — both library surfaces	always True

The two task-scoped endpoints are the agent's own flush targets and were built with no legacy callers to protect, so they are strict today regardless of the flag. Library content is therefore not what the audit is for — it already cannot persist an invalid FORM document. The audit target is the existing content of draft playbooks, which a human "Save Playbook" will start hard-rejecting.

The procedure

1

Audit

read-only

2

Remediate

if anything found

3

Flip

one vendor

4

Watch

telemetry

1 · Pre-flip audit

Read-only, against the production read replica — never the primary. The prod-db-investigation skill carries the only approved connection recipe (reader-host pin, forced read-only transaction, statement timeout, and the pg_is_in_recovery() guard). Scope every query to the one vendor_id being flipped.

Three questions, in increasing cost:

(a) Step types outside the creatable set. Every step in the vendor's draft playbooks whose type code is not in AI_CREATABLE_STEP_TYPE_CODES — YES_NO, SINGLE_SELECTION, DROPDOWN_SELECTION, FORM, EMBEDDABLE, PRESENTATION. Note MULTI_SELECTION is deliberately absent: multi-select asks become a checkbox element inside a FORM step.

This is the question most likely to be over-read. An update that keeps its existing type is never subject to the type gate, so a legacy tree full of SHORT_ANSWER steps keeps saving after the flip. A hit here only matters if a human is going to change that step's type or add a new step of that type. Count them, do not panic about them.

(b) Bare FORM_* palette codes used as step types. These are form-builder elements, not step types. This seam never runs the element-to-FORM rebind, so one persisted directly is a broken step and will raise FORM_ELEMENT_STEP_TYPE. Any hit here is a real defect regardless of the flip.

(c) FORM documents that fail strict validation. The expensive one, and the only one that is not a SQL query — validate_document is Python. Pull the step_configuration of every FORM step in the vendor's draft playbooks and run it through app/api/utils/form_schema/validator.py, collecting findings by code (CYCLE, DANGLING_REF, UNKNOWN_TYPE, unknown element keys). A document that only ever persisted under the lenient path is exactly what a post-flip save rejects.

Remember the trigger condition: only a new or changed config is gated. A FORM step nobody edits keeps saving. The risk is a user opening a legacy form, changing one label, and getting a rejection on a document that was already invalid before they touched it.

2 · Remediate what you found

• (b) bare FORM_* step types — fix before flipping, unconditionally. These are broken rows, not a gate artifact.
• (c) invalid FORM documents — fix before flipping. Each one is a save the user will be unable to complete, and the findings are per-step, so the fix is targeted.
• (a) non-creatable step types — usually leave alone. Only act if the vendor is actively converting those steps to another type. Do not widen AI_CREATABLE_STEP_TYPE_CODES to make an audit hit go away: it is an explicit allowlist that mirrors the human drag palette, and widening it changes what the agent is allowed to mint everywhere.

3 · Flip it

Enable the ai-task-builder feature for the single vendor_id. One vendor at a time — the audit is per-vendor, so the confidence is per-vendor too.

🚫 A fresh chat session is required. Tool registration is cached per agent session, so a user with an open Aero conversation keeps the pre-flip tool set until they start a new one. Do not conclude the flip failed because an existing session still refuses.

4 · Watch

The precedent to copy is form-server-validation-authority, whose own gate documents the pattern: off is the pre-flip state that logs each finding as a shadow-mode warning keyed by vendor and step, and the flag flips to hard-enforce once shadow telemetry shows ~0 mismatches (enforce_validation in app/api/utils/form_validation/compiler.py).

Per-vendor OpenSearch queries for those shadow signals already exist — use them rather than re-deriving: .agents/skills/opensearch-investigation/references/form-telemetry-signals.md. Load the opensearch-investigation skill first — it carries the tenant and index map those queries assume.

Read form_validation_shadow_mismatch sliced by surface: only portal_submit counts toward the gate. A customer-API write has no client in the loop, so its findings are invalid payloads hard-enforce is meant to reject — they never soak to zero, and a bare per-vendor count that sums them makes the gate unreachable. These signals are filterable only by match_phrase on message; the reference above records which dimensions each token puts there, and carries the probe that proves a per-vendor read is available on your window before you trust a zero.

After the flip, watch for FormValidationRejectedException on update_from_draft for that vendor. A rejection there is a human save that failed, so it is user-visible and worth reacting to rather than aggregating.

Verify

• A user on the vendor, in a new chat session, with a task's subtask builder open, can have Aero add a subtask — and the step appears on the canvas.
• From a module/task list with nothing open, "build out subtasks for Kickoff" works and is additive: it builds on a task that already has steps rather than refusing.
• A human "Save Playbook" on an existing draft playbook still succeeds. This is the regression the audit exists to prevent, so test it on the vendor's oldest draft playbook, not a fresh one.
• Deleting a subtask via Aero, with a builder open, raises a confirmation card.

Troubleshooting

🚫 Aero still says authoring is unavailable

The session predates the flip. Tool registration is cached per session — start a new chat. If a fresh session still refuses, check enabled_feature_codes is actually reaching agent_state: task_builder_enabled fails closed, so an absent key looks identical to a disabled feature.

⚠ A human save now 4xx's with `findings`

Expected shape when the audit missed something. The response carries step_ref and per-finding codes, so the offending step is identified. Fix the document; do not turn the flag back off unless the count is large — flipping back leaves the bad document in place and only defers the same failure.

🚫 `FORM_ELEMENT_STEP_TYPE` on save

A bare FORM_* element code is persisted as a step type — audit question (b). This is a broken row that predates the flip. Repair the row; the gate is correct to reject it.

⚠ Agent authored a step type the save rejects

The palette the agent sees and the set the gate permits are supposed to be the same authority — get_agent_creatable_step_types feeds the builder-context payload from AI_CREATABLE_STEP_TYPE_CODES. A divergence here is a defect in that pairing, not something to fix by widening the allowlist.

Related

• Subtask Authoring (Dual-Mode) — what the capability half of the flag turns on
• Form Builder Architecture — the FORM document and its meta-schema
• Form-Document Schema — the generated field reference the validator enforces