RFC: Environment IaC — One Tier-Composed Pulumi Program

Every environment — PR previews, dev, stage, prod — runs the same Pulumi program; what an environment is becomes a config entry, not a separate codebase.

Owner: Ross RasmussenCreated: 2026-06-30

💬 Feedback wanted — this is a draft. A GitHub Discussion (Ideas category) will be linked here before it's circulated. Open questions are flagged in Risks & open questions.

TL;DR

🅰 One program, composed by tier

Previews and prod run the same code.

An environment is a config entry that selects which component functions to compose — not a forked program. Adding an environment is adding a row.

🅱 Variation is data, not forks

A presence-based tier resolver decides what each env composes.

No per-env copy-pasta, no if (env === "prod") branches threaded through logic. A resource is omitted from an env by not listing it in that tier.

🅲 Convergence, not parity

The preview-only set is a visible, shrinking list.

Prod compute isn't in Pulumi yet, so previews can't be byte-identical to prod today. The model makes them converge as prod migrates in, instead of diverging.

Motivation

PR previews exist to exercise the same infrastructure as production, with a small, explicit set of deviations: expensive background pipelines and alert noise turned off, and a few preview-only essentials (edge/DNS/cert, a stand-in backend) turned on. The current shape works against that goal.

🧱 Divergence is baked in

Previews are a standalone Pulumi program (infra/pr-previews/). It shares nothing structural with the (future) prod stacks, so the two will evolve as separate codebases — permanent reconciliation and copy-pasta as prod infra lands.

🧬 Project proliferation

Every new concern becomes a new top-level project: pr-previews, ci-identity, datawarehouse — and the count grows with each one. The shared infra/components/ directory is nearly empty, so almost nothing is actually reused across them.

💸 No safe per-env "off" switch

Expensive infra (the DMS-based CDC pipeline) and noisy infra (alerting) are kept out of previews only by never wiring them in — an implicit, fragile guarantee. There is no first-class "this env omits X" mechanism, so the next contributor can wire one in by accident.

Current state

Grounding facts this RFC builds on:

• Backend. Self-managed Pulumi S3 backend; one bucket holds many projects via the project-scoped stack layout. A per-environment state-bucket split (seed / previews / prod) is emerging from the ABAC deploy-role model in infra/ci-identity/ — the boundary is per-env, not per-project.
• Deploy roles. The env-scoped deploy-role model in infra/ci-identity/ is the mechanism for promoting IaC to an environment from CI; each environment's role derives from the same presence-based registry below. Onboarding an environment is adding a registry entry.
• Tier resolver already exists. infra/ci-identity/envs.ts resolves an environment from a presence-based ENVS[] registry (resolveTier) — currently carrying the previews environment; a new environment is one entry and the rest derives. This RFC extends that pattern to resource composition.
• Prod compute is not in Pulumi. Production RDS and application compute are DuploCloud-owned. So "previews = prod" is directional: prod migrates into Pulumi over time; until then previews necessarily carry a stand-in backend prod doesn't.

Proposal

A three-layer model, with environment variation living in exactly one place.

flowchart TD
    subgraph L3["Tier composition — envs.ts (the ONLY place env varies)"]
        P["previews tier"]
        D["dev tier"]
        S["stage tier"]
        PR["prod tier"]
    end
    subgraph L2["bundles/ — promoted on 2nd env, not pre-built"]
        CORE["core (all envs, never branches)"]
        OBS["observability (optional)"]
    end
    subgraph L1["components/ — one resource cluster each"]
        EDGE["createSpaEdge"]
        BE["createLambdalithBackend"]
        CDC["createCdcPipeline"]
        ALARM["createAlarms"]
    end
    P --> CORE
    PR --> CORE
    PR --> OBS
    PR --> CDC
    CORE --> EDGE
    OBS --> ALARM
    style L3 fill:#efe7ff,color:#000
    style L2 fill:#e6f3ff,color:#000
    style L1 fill:#eafbea,color:#000

Layer 1 — components/. Each resource cluster is a ComponentResource factory: createSpaEdge(cfg), createLambdalithBackend(cfg), createCdcPipeline(cfg), createAlarms(cfg). This is the existing repo convention (infra/components/agent-runtime.ts), applied consistently.

Layer 2 — bundles/. A bundle composes a fixed set of components. Two legitimate shapes:

• core — composed by every environment, never branches. If it's in core, every env gets it.
• optional bundle (observability, data) — a tier composes it or omits it.

Bundles are promoted from real duplication, not pre-built: extract one the moment a second environment repeats the same component calls, so it has two real callers validating its contract.

Layer 3 — environment composition (envs.ts). One env-agnostic program composes bundles + components, driven by the presence-based environment registry (extending the existing resolveTier). This is the only place environment variation lives. Inclusion is presence, not a flag — an environment that omits cdc simply doesn't list it.

The variation is symmetric

The same one knob — does this tier compose function X? — handles both directions:

Direction	Examples	Mechanism
Off for previews/dev	CDC pipeline, alerting / monitoring	prod tier composes it; previews tier omits it
Only for previews	SPA edge (CloudFront + zone + ACM), stand-in lambdalith backend, per-PR ephemeral wrapper	previews tier composes it; prod omits it

The rule that keeps bundles honest

A bundle is a fixed set of components and never asks "what environment am I?" The moment a bundle wants if (env === "prod") inside it, split it or hoist the choice up to the tier. Otherwise a bundle silently re-hides the variation that tier composition exists to make visible — the monolith with extra indentation.

Litmus: observability is not a core member — previews deliberately omit alerting — so it's an optional bundle. If something can't go in core without a branch, it's optional by definition.

Registry ergonomics & typing

The registry is engineered for scale from day one: as environments and components grow it stays a typed, declarative surface, not a copy-pasted config blob. The shape is two factory levels so no environment ever repeats a { ...base } spread.

Component-owned types, composed into EnvArgs. Each component exports its own args interface. Two kinds of member: optional members are presence-toggled (if (env.cdc) createCdcPipeline(env.cdc) — presence means enabled, no enabled: true booleans, a flag identical on every instance is a default in disguise); required members are always composed — every env has them, only their args vary, so they're called unconditionally and never guarded.

interface EnvArgs {
  edge?: EdgeArgs                    // optional — SPA + CloudFront + cert (preview-only today)
  backend?: BackendArgs              // optional — stand-in lambdalith (preview-only today)
  cdc?: CdcArgs                      // optional — data-warehouse pipeline (prod only)
  alarms?: AlarmArgs                 // optional — paging / SNS (prod only)
  observability: ObservabilityArgs   // REQUIRED — every env; only retention varies, never toggled off
}

Two factory levels, shorthand by default. Each component exports a default value (used by shorthand) and a *With factory (used only when an environment deviates). The env-level buildEnv folds in the shared base once.

// component level — default value + override factory (edge / backend / observability follow the same pair)
export const cdc: CdcArgs = cdcBase
export const cdcWith = (o: Partial<CdcArgs>): CdcArgs => ({ ...cdcBase, ...o })

// env level — the base every environment shares, merged here once
const buildEnv = (overrides: Partial<EnvArgs> = {}): EnvArgs => ({ ...baseEnv, ...overrides })

Usage stays declarative — shorthand for the default, factory only where an env deviates — and satisfies locks every entry to EnvArgs without widening, so a typo or wrong-shaped args fails at compile time, not at deploy. The intended matrix:

Env	edge / backend	cdc	alarms	observability
previews	✓ (preview-only)	—	—	base (short retention)
dev / stage	—	—	—	base (short retention)
prod	—	✓	✓	full (long retention)

const ENVS = {
  previews: buildEnv({ edge, backend }),            // + preview-only edge/compute
  dev:      buildEnv(),                             // base only
  stage:    buildEnv(),                             // base only
  prod:     buildEnv({ cdc, alarms, observability: observabilityWith({ retentionDays: 365 }) }),
} satisfies Record<EnvName, EnvArgs>

dev/stage are literally buildEnv() — the common case is empty; only prod's deltas are spelled out. At most one factory deep at every site: buildEnv is the container, observabilityWith({ … }) a leaf value — composition, not a wrapper chain. Avoid chained same-kind wrappers (the HOC withA(withB(…)) smell); a container factory taking leaf-factory values is fine.

The spreads are shallow on purpose and safe: every value in an environment entry is a whole component object — a default or a *With result — never a partial patch into a nested object, so { ...base, ...overrides } can't clobber half a nested config (the classic shallow-merge footgun). If a genuine deep merge ever becomes unavoidable, reach for a small typed helper or a types-preserving lib (e.g. ts-deepmerge) — never an any-returning merge, and never a config framework; the repo keeps dependencies minimal.

Prototyping in a preview (creds-free)

Previews are the proving ground for prod-grade infra — but devs hold no AWS credentials, and deploys run only in CI under GitHub OIDC. So the only way to try new infra is a preview deploy. There is no special prototyping mechanism — you use the same registry:

To prototype component X, add x: x() to the previews entry in envs.ts (and its if (env.x) createX(env.x) compose site in index.ts's pr-<N> branch). Your PR's preview builds it under the previews OIDC role — no dev creds, no separate stack. When it's proven, move it to the target env's ENVS entry (or buildEnv's base if every env wants it) and drop it from previews if it was only there to test.

// compose — required members unconditionally; optional members presence-toggled
createObservability(env.observability)            // REQUIRED — always present, only args vary
if (env.edge)    createSpaEdge(env.edge)          // optional — presence = enabled
if (env.backend) createBackend(env.backend)
if (env.cdc)     createCdcPipeline(env.cdc)       // identical call from any env's entry
if (env.alarms)  createAlarms(env.alarms)

The guards test plain config objects (synchronous presence), not Pulumi Outputs — never if on an Output, it's always truthy; resolve those with .apply(). Because the same compose loop builds every env, what you prove in a preview is the same component that later ships to prod — moving x into prod's ENVS entry runs identical code.

Discipline (no mechanism, deliberately): a prototype left in the previews entry deploys to every preview once merged, so keep it on your branch until you migrate it. The previews entry is reviewed in the PR like any other change — the same don't-merge-WIP rule as feature code. (An earlier draft added a label-gated POC_OVERLAY quarantine + a CI gate to enforce this; it was removed as over-engineering — plain review handles a cheap, ephemeral, auto-swept preview resource.)

• Auto-torn-down with the preview on PR close (existing sweeper); cost bounded to one ephemeral preview.

Current vs proposed

A standalone infra/pr-previews/ program builds preview infra in isolation. Each additional concern (e.g. the data warehouse) is its own sibling project with its own per-env stacks. An environment's footprint is spread across N projects; components/ holds almost nothing reused. Keeping expensive/noisy infra out of previews is an implicit "never wired in" guarantee.

One env-agnostic program composes components/ (and, once earned, bundles/). A presence-based tier registry (envs.ts) decides what each environment composes. Previews omit CDC and alerting by not listing them; previews include edge/cert/backend by listing them. Adding an environment is adding a tier entry; enhancing a resource is editing one component every environment inherits.

Migration

Two phases, so the reorg is decoupled from the data-warehouse work.

Phase 1 — collapse previews into the env-composed program

• Promote the resource clusters in infra/pr-previews/{shared,preview}.ts into components/ factories (SPA edge, stand-in backend, Neon branch, agent runtimes — the last already a component).
• Stand up the env-agnostic program at infra/app/, composing those components via the envs.ts registry. The previews environment composes the preview-only set and explicitly omits cdc and observability; the ephemeral pr-<N> deploy is a sub-mode of that environment (a PR-number parameter), not a separate program. dev/stage/prod are declared but compose only what's buildable in Pulumi today.
• Build beside, then cut over. infra/app/ stands up alongside the untouched infra/pr-previews/; the new app/shared stack adopts the existing live CloudFront, cert, and buckets via pulumi import — no recreate, no DNS flip. The deploy workflow's project target flips to app only once the imported stack verifies clean, then infra/pr-previews/ is deleted.
• The pr-<N> stacks are ephemeral — no migration; the next deploy recreates them under app. Renaming the project to app updates the previews environment's stateStackPrefix (the deploy-role's state-bucket grant) in the same change.
• CDC is untouched this phase → clean decoupling.

Phase 2 — the turnkey multi-env pipeline (the north star)

Phase 1 makes previews run on the tier-composed program; the goal is that any dev ships infra like app code — draft it, a PR deploys it to a prototype, validate in the PR, merge → dev, promote → stage → prod. That needs three things on top of Phase 1's foundation (the program + envs.ts + the per-env deploy roles):

1. Deploy infra/app across envs, mirroring AgentCore. The normal AgentCore prod deploy is a job inside the release workflow (deploy-agents in build-and-deploy.yml, called from deploy-release.yml) — not a standalone workflow. Add a sibling deploy-infra job the same way: PR → preview (Phase 1), merge-main → dev, promote-stage → stage, promote-prod → prod — each a full pulumi up under that env's deploy role. A deploy-infra-reusable.yml (workflow_call, environment input) + workflow_dispatch escape hatches complete the 1:1 with AgentCore.
2. dev/stage/prod tiers compose real infra, not buildEnv() stubs.
3. Turnkey DX — a dev adds a components/ factory + one line in a tier and ships it through the same promote flow; a skill + templates keep Pulumi internals out of sight (and steer hard away from new stacks — see the guard).

Infra deploys in parallel and decoupled — release wall-clock stays flat. Serializing infra before the app deploy (a needs:) would tax every release by ~3-5 min, which is exactly what we won't pay. Instead deploy-infra runs in parallel with the app deploy (no needs: on the release critical path), exactly like AgentCore's deploy-agents. "Source code needs its IaC" is handled by app resilience, not serialization: the app degrades gracefully while new infra converges (the AgentCore 503/SSM pattern). A release whose app genuinely hard-requires new infra either rides that graceful-degrade window or lands the IaC in an earlier change — we don't serialize the common case to cover the rare one. (Revisit only if a hard-dependency case proves resilience insufficient.)

Decoupled prod stacks (CDC, and future ones) ride this pipeline as their own stacks — no fold-in to the env's core stack. The CDC data-warehouse pipeline is decoupled infra (zero StackReferences; reads its RDS source from config; nothing consumes its outputs synchronously), so it deploys as its own stack — per the tier refinement below, a Tier-2 stack-in-app (organization/app/<env>-<name>, absorbing infra/datawarehouse once its prod state is imported), not a separate project. The mechanism is deploy-infra-reusable.yml's optional stack input: a thin wrapper passes { environment, stack } and the stack deploys under the env's role / state plane / KMS — same pipeline, separate state (infra/README.md "Adding a decoupled DURABLE stack"). Previews/dev never auto-deploy CDC (expensive/prod-oriented); it reaches a preview only if someone explicitly adds it to the previews entry to prototype it (and removes it before merge).

Sequencing / scope. Phase 1 (previews on the tier-composed program) shipped first. The multi-env deploy-infra pipeline for the app program (Phase 2, item 1 — parallel/decoupled, wall-clock-flat) has shipped too, including the decoupled-stack path (deploy-infra-reusable.yml's optional stack input — no consumer yet). Still later, separately-shipped steps: absorbing CDC as its own Tier-2 stack (lower urgency: it already runs standalone today) and the anti-new-stack IaC-creation skill.

Stack boundaries follow coupling, not convenience

Default: one stack per environment — tightly-coupled infra (synchronous Pulumi-output dependencies; hard-fails without its peers) lives together. A resource cluster earns its own stack ONLY when it clears a high bar — four named, defensible reasons, nothing else:

Split reason	Test	Example
Bootstrap chicken-egg	mints what the others deploy under	ci-identity
Cross-env regional singleton	fires once per account-region, not per env	account-wide alerter
Loose-coupling seam	both sides deploy and recover independently — late-bound/config-driven binding, graceful degradation, no synchronous cross-dependency	AgentCore runtimes; CDC pipeline
Migration / replacement	replaces an existing stack/project being retired — net-zero new surface	infra/app (retires pr-previews)

The loose-coupling test is strict: AgentCore qualifies because the backend discovers runtime URLs via SSM and returns 503 RUNTIME_NOT_READY (re-polling) until they appear, while the runtimes only curl the backend's live URL — neither hard-depends on the other's synchronous deploy outputs. CDC qualifies differently — zero StackReferences, its RDS source read from config (RDS is externally owned), nothing consuming its outputs synchronously — so it deploys and recovers entirely on its own. Almost nothing else clears this bar; when in doubt, same stack.

A lightweight CI guard enforces this (no dedicated workflow): the Infra CI workflow fails a PR that adds a new infra/** Pulumi stack/project unless it carries the reviewed infra-new-stack-approved label, and the PR-review bot (REVIEW.md rule 10) flags any new stack whose justification against the four reasons doesn't hold. The label is the human gate; the review bot is the judgment.

The label is applied by an independent reviewer, NEVER self-applied by the author. The gate only works if the human approving a new stack isn't the person who wants to ship it. Self-applying infra-new-stack-approved to bypass the gate is exactly how the bespoke infra/model-eol/ standalone stack shipped (ONRAMP-5462 / issue #9521) before being reworked into an infra/app alarms member — the incident this whole rule exists to prevent. Terse restatement for always-on authoring: .claude/rules/infra-composition.md; mechanics: infra/README.md.

Refinement (post-model-eol): PROJECT vs STACK are different gates

The four reasons above conflate two decisions that deserve separate, sharper gates — a new Pulumi project and a new stack are not the same escalation. Hardened model, three tiers:

• Tier 1 — member (default): a components/* component + a presence-toggled envs.ts member. Model-EOL, budget, observability.
• Tier 2 — stack-in-app (guarded): a new stack shape in the app project — own pulumi up/state/loose-coupling seam, same project. AgentCore (pr-<N>-agents), the CRM broker/integrations. Add one only when it clears the loose-coupling seam above; otherwise it is a Tier-1 member (no CI sprawl).
• Tier 3 — new project (FROZEN): the sanctioned projects are app + ci-identity — the complete set going forward (pr-previews retiring; datawarehouse to be absorbed into app as a Tier-2 stack once its prod state is imported). The only thing that can ever justify a new project is a bootstrap / root-of-trust layer, by a one-question bright-line: does it create the deploy identity / state plane / KMS other stacks consume, AND must it run under a higher principal than any deploy role? Yes → bootstrap (a ci-identity-per-account is the sole future case, if accounts ever split). No → Tier 1 or 2. A feature / service / alerter / broker / pipeline is never a project. This replaces the looser "account-global singleton" phrasing — that criterion was mis-readable (an account-wide alerter sounds account-global but is a member/stack).

How the gates map to the tiers. The infra-ci file-gate matches any newly-added infra/<x>/Pulumi.yaml or Pulumi.<stack>.yaml. It therefore catches every new project (a project always adds a Pulumi.yaml) — that's the airtight project freeze — and also any stack that commits a per-stack config file (e.g. infra/app/Pulumi.shared.yaml). A Tier-2 stack whose config is generated at deploy time (the runtime secrets-provider yaml) evades the file-gate, so stacks are guarded primarily by the loose-coupling-seam bar in review + the PR-review bot, with the file-gate as a partial backstop. The airtight stack-guard is the program itself: infra/app/index.ts dispatches on the stack name and throws Unknown stack for any shape it doesn't recognize, so a stack that isn't dispatched doesn't deploy and adding one is always a reviewable index.ts diff. (A separate STACK_SHAPES registry table was considered and deliberately not built — the dispatch is the registry; a parallel table would add a drift surface without adding enforcement.)

Deploy phasing: partition, never --target

A fast-URL-first / dependents-later phasing is fine, but the phase boundary must be a real resource partition (separate stacks), never a pulumi up --target slice of one program. A --target partial apply re-evaluates the whole program; any non-targeted resource that develops a cross-pass diff is planned for destroy, refused, and hard-fails the deploy. Per-PR topology:

app/shared            long-lived singletons
app/pr-<N>            core (Lambda / SPA / edge / KVS)   full `pulumi up`  → refs shared
app/pr-<N>-agents     AgentCore runtimes + readiness gate full `pulumi up`  → refs shared + core URL

Each stack is a full apply; the AgentCore split (the loose-coupling seam above) is what keeps core fast-to-URL without a --target slice — and retires the empty-zip placeholder hack the slice required.

Alternatives considered

• Monolith env stack (one stack holds an entire environment). Rejected: it forces sharply divergent lifecycles into one blast radius — ephemeral per-PR previews, bootstrap identity, and a rarely-changed DMS pipeline. Every preview pulumi up would diff/refresh the DMS instance; a data-warehouse error would block preview deploys.
• Project-per-component (status quo). Rejected: bakes in preview/prod divergence and proliferates top-level projects, as in Motivation.
• Pre-build the bundle layer now. Deferred, not rejected: extracting a shared abstraction before a second real consumer can't validate its contract. Bundles are promoted when a second environment repeats the same composition.

Risks & open questions

HighMigrating the live preview state

Renaming the previews project relocates its backend state; a mishandled move can orphan or duplicate live preview resources.

Mitigation: pr-<N> stacks are ephemeral and self-heal on the next deploy — only the single shared stack needs care, via pulumi state mv or a bounded recreate. Do it now, while there is one durable component to move rather than several.

MedConvergence depends on prod entering Pulumi

Prod compute/RDS are DuploCloud-owned, so previews carry a stand-in backend prod lacks until that migrates.

Mitigation: the preview-only set is explicit and shrinks as prod migrates in; Phase 1 ships without requiring prod-in-Pulumi.

MedBundles drifting into env-branching

A bundle that grows an internal if (env === …) re-hides the variation tier composition exposes.

Mitigation: the no-branch rule — bundles are fixed component sets; environment choice lives only at the tier site. Enforced in review.

Open questions

How are cross-env regional singletons handled? A resource that must fire once per account-region (not once per env) — e.g. an account-wide alerter — doesn't fit per-tier composition. Decide whether such cases stay standalone regional stacks (like ci-identity) or get a dedicated regional composition path.

Resolved (ONRAMP-5462). The first such case — the Bedrock model-EOL alerter, which fires once per account-region (us-west-2 + eu-west-1), not once per env — did not become a standalone regional stack. It composes as a per-region member of the single prod stack: the alarms member (infra/components/model-eol-alerter.ts), which index.ts builds twice — once on the stack's default us-west-2 provider and once on an in-program new aws.Provider("model-eol-eu", { region: "eu-west-1" }) — so one pulumi up owns both regions. The prod deploy role reaches both regions (infra/ci-identity/envs.ts prod regions: ["us-west-2", "eu-west-1"]), and the component region-prefixes every Pulumi logical name + region-suffixes the global IAM role names so the two instances never collide. This is the pattern for future account-region singletons: an in-program per-region provider inside the owning tier's stack, not a new standalone stack.

Runtime validation at the composition boundary? A lightweight per-component schema (e.g. zod) could validate a tier's args where the program composes them — defense-in-depth beyond compile-time types. Optional and light if adopted; weigh against the repo's dependency-minimalism ethos.