Preview Database Lifecycle
A PR preview's Neon branch is created once, on first deploy, and never reset without someone explicitly asking for it. This is the contract that governs schema freshness, seed freshness, and the one opt-in path that wipes data.
The contract
Every open PR that gets a preview also gets a Neon branch named pr-<N>, forked from the shared golden-seed parent. preview-branch.ts ensure creates that branch on the PR's first deploy and never resets it again on any later deploy — see the header comment in devtools/db/preview-branch.ts for the full rationale. Concretely:
- Data authored inside a live preview is durable for the preview's lifetime. A record created, edited, or deleted through the app persists across every subsequent redeploy of that PR.
- The only implicit way a preview's data goes away is the preview itself going away — the PR closes, someone adds the
no-previewlabel, or the nightly sweeper (preview-sweeper.yml) reaps it. All three delete the branch outright (preview-branch.ts delete), which is a different thing from a reset. - The only explicit way to wipe a live preview's data without closing it is the refresh checkbox described below.
This replaced an earlier design where ensure would silently reset a branch it judged stale. That self-heal was the actual bug reports were describing: an unrelated PR touching devtools/db/** could wipe a completely different PR's week-old preview data on its next redeploy. The create-once contract removes that failure mode by construction — ensure only ever creates a branch or, for a zombie (below), deletes and recreates one.
How freshness actually works
Never resetting a branch splits "is this preview up to date" into two independent questions with two different answers.
Schema is almost never an issue. preview-deploy.yml's deploy-core job runs preview-branch.ts migrate on every deploy, unconditionally — not gated on a migrations-changed file filter. That command runs flask db upgrade, then flask verify-schema, then flask reconcile-authz against the branch's unpooled connection. flask db upgrade is a fast no-op once the branch is already at head, and reconcile-authz has to run regardless of whether a migration shipped, because the permission catalog it reconciles can change without one. Previews serve Lambda images that never run docker/flask-entrypoint.sh, so this step is the only place a branch's authorization state reaches its database. A branch that has been sitting untouched for weeks forward-migrates cleanly to whatever head its own PR expects, the same way flask db upgrade always has.
The one schema case that does not self-heal is a PR whose own migration gets re-parented. A preview branch is stamped with the PR's revision the first time it deploys; when the PR is later updated, its migration's down_revision moves onto whatever main has landed since. Those newly-landed migrations are now ancestors of a revision the branch already records, so flask db upgrade sees a database at head, applies nothing, and reports success — while the image about to be deployed selects columns the branch never received. The revision pointer cannot tell this apart from a healthy database, so flask verify-schema compares the ORM's metadata against the live schema and fails the deploy with the missing tables and columns named (app/api/base/schema_drift.py). The remedy is the refresh checkbox below: re-running every migration from scratch against a fresh fork is the only way a skipped one gets applied.
Seed rows are the accepted gap. New fixture rows land in the golden parent when reseed-preview-db.yml runs reseed-neon.ts on a devtools/db/** change to main (see devtools/db/reseed-neon.ts's header for exactly what that rebuild does). That rebuild only ever touches the golden parent — it does not touch any pr-* branch. A branch that already existed before the rebuild keeps whatever rows it forked with; it does not retroactively gain the new ones. A branch created after the rebuild forks from the parent's new state and has them from the start. This is the seed freshness gap the refresh checkbox exists to close on demand.
The refresh checkbox
The sticky PR-preview comment (marker pr-preview-status, rendered by .github/scripts/render-preview-comment.sh and upserted by upsert-pr-comment.sh) carries a <!-- preview-db-state --> block with:
- [ ] 🔄 **Refresh preview DB from golden seed** — wipes any data added inside this preview.Checking it wipes the preview back to the current golden seed. Mechanically, checking the box (or dispatching the workflow — see below) drives preview-branch.ts refresh pr-<N>, which:
- Calls Neon's
reset_to_parenton this one branch, pointing it at the current golden parent (picking up any fixture rows the parent gained since this branch forked). - Waits for that reset to actually land via
waitForOperations(devtools/db/neon-operations.ts) before doing anything else — a refresh that cannot prove the reset settled throws rather than proceeding against a half-reset database. - Runs the exact same migrate step
deploy-coreruns on every deploy (flask db upgrade+flask reconcile-authz), so the PR's own in-flight migrations reapply on top of the freshly reset seed instead of leaving the branch pinned to the parent's schema.
What it wipes: everything authored inside the preview since it was created — every row the branch accumulated beyond what it inherited from the golden parent at fork (or last refresh) time.
What survives: the branch's endpoints and pooler connection strings. Neon preserves a branch's endpoint identity across reset_to_parent, so the already-running preview Lambda's DATABASE_URL does not change — no redeploy, no Pulumi run, no Lambda config update is needed for the refresh to take effect. The next request the Lambda serves just sees the reset data.
Who may check it: the box is only honored for someone with write, maintain, or admin on the repo, verified against the GitHub API (the same repos/<owner>/<repo>/collaborators/<user>/permission → role_name check other write-gated dispatch workflows in this repo use — see .github/workflows/preview-shared-apply.yml for the pattern). A checkbox edit from anyone else, or from a bot, is ignored.
Fallback: workflow_dispatch on the refresh workflow with a pr input covers the case where editing the comment isn't convenient — a rerun after a transient failure, or triggering a refresh from the Actions tab directly. It clears the same authorization bar, plus one more: the PR's sticky comment must already carry the preview-db-state block, because that block is the only place the refresh reports its result. A dispatch against a comment that predates this tooling is refused before the wipe rather than wiping a database it cannot then report on — redeploy the preview to re-render the comment, then refresh.
Concurrency: the refresh shares preview-deploy's pr-preview-<N> concurrency group, so it can never run against the branch a deploy is migrating. The cost is that GitHub keeps only one pending run per group: a refresh queued behind a running deploy is superseded if a push queues a second deploy first. That resolves itself visibly — the superseding deploy re-emits db_stale=true, so the ⚠️ hint and the unchecked box come right back.
The default-branch caveat
The refresh workflow triggers on issue_comment (types: [edited]) — checking the box is editing the sticky comment. GitHub always runs an issue_comment-triggered workflow from the default branch's copy of the workflow file, never from the PR's branch. Until this tooling is merged to main, checking the box does nothing on any open PR, including PRs opened before this one that already carry a preview comment. Their comments already render the checkbox once their next deploy re-renders it (the comment is re-rendered on every deploy transition), but checking it is a no-op until main has the workflow. After the merge, every open PR's next comment edit picks it up automatically — no redeploy required for the checkbox to start working, only for the box to have appeared in the first place.
The staleness warning
preview-branch.ts ensure compares two timestamps on every deploy: the branch's fork/last-refresh vintage (parent_timestamp, falling back to created_at) against the golden parent's updated_at. When the parent is newer, isBranchStale (devtools/db/branch-health.ts) returns true and ensure emits db_stale=true, which the comment renders as:
⚠️ Golden seed has moved since this branch forked (seed forked …, golden rebuilt …) — check the box below to pick up new fixture rows.
The same vintage line — "Seed forked … · golden rebuilt …" — always renders, stale or not, so the state is visible even before it's a problem. This warning is purely informational: it does not gate the deploy, retry anything, or queue a refresh on its own. Nothing in the pipeline reacts to db_stale=true except printing it (to the job log via ::warning::, to the step summary, and to the comment). The only response to it is a human deciding to check the refresh box.
The accepted trade-off
A long-lived preview misses new golden fixture rows until its owner refreshes it. This is a trade-off, not a bug: the alternative is exactly the self-heal-on-every-deploy behavior the create-once contract exists to remove, which wiped unrelated PRs' data on a schedule nobody controlled.
Concretely: a fixture landing in devtools/db/ — new library-module and library-task fixtures, or a pinned CRM fixture in devtools/db/crm-fixtures.ts — reaches every new preview from the moment reseed-neon.ts rebuilds the golden parent with it. It reaches an existing preview only when that preview's owner checks the refresh box. There is no background process that reconciles it for them.
Troubleshooting
Check, in order: (1) has this refresh tooling merged to main yet — see the default-branch caveat above; (2) does the account that checked it hold write/maintain/admin on the repo; (3) was the edit made by a bot (bot edits are ignored, since the sticky comment is itself bot-authored and re-rendered by every deploy); (4) is the PR from a fork — fork PRs can't mint the scoped credentials the refresh needs, same restriction as the deploy workflow.
The comment's db-state block carries a DB_REFRESH_NOTE line with a ❌ marker and a link to the failed run when a refresh errors. Open that run's log — preview-branch.ts refresh fails loudly (ON_ERROR_STOP semantics throughout) rather than leaving a half-applied state. Rerun via workflow_dispatch once the underlying cause (typically a transient Neon API error) is addressed.
If the branch's parent_id no longer points at the live golden parent (the parent itself was deleted and recreated), reset_to_parent 404s forever — preview-branch.ts refresh detects this and refuses outright with a message pointing back at ensure. Redeploy the preview (push a commit, or re-run the deploy workflow); ensure deletes and recreates the zombie from the current parent. Refresh again afterward if the recreated branch also needs the latest seed rows (it won't, immediately after ensure recreates it).
flask verify-schemaThe branch's schema is behind the image being deployed, and flask db upgrade cannot repair it — the step names the missing tables and columns. Effectively always a re-parented migration (see "How freshness actually works"): the branch recorded this PR's revision before the migrations now beneath it landed on main. Tick the refresh checkbox, which re-forks from the golden parent and re-runs every migration from scratch, then redeploy. If the missing objects come from this PR's own migration rather than main's, the migration itself did not apply — read the flask db upgrade output directly above the failure.
Multiple head revisionsNot a preview-infra defect: flask db upgrade refuses because the PR's own migration branches off a parent that main has since moved past, the same divergent-graph condition the migration/single-head required status check (scripts/check-pr-single-head.ts) already evaluates for every PR. Check that status on the PR first — if it's red, this deploy failure is downstream of it, not a separate bug; rebase onto the base branch and add a merge migration (uv run flask db merge heads), then redeploy. Only treat it as a preview bug if migration/single-head is green on the head commit that failed.
This is the half-reset class waitForOperations (devtools/db/neon-operations.ts) exists to prevent — every mutation preview-branch.ts makes waits for Neon to confirm the async operation actually landed before doing anything else, specifically so a caller never gets handed a connection to a branch still mid-reset (which reads as a seed-less database: user.id = 0 does not exist). If you see it anyway, report it with the run link — that link's settle(...) timeout output is what makes the failure class root-cause-able.