RFC: Production Compute — Domain-Driven Lambdaliths
Evolve the backend from one container Lambdalith into a small set of domain-scoped lambdaliths, graduating each onto the AWS Lambda managed runtime + Python SnapStart as it slims — so cold starts approach sub-second while we keep pay-per-use economics.
💬 Feedback wanted — this is a draft of a direction, not a committed plan. Weigh in on the GitHub Discussion before we commit. Open questions are flagged in Risks & open questions.
TL;DR
Domain-driven lambdaliths
Split the monolith by bounded context into ≈5–8 LWA services behind a CloudFront path-router. Each domain is its own deploy unit with its own (smaller) init.
Managed runtime + SnapStart
As a domain slims under the 250 MB cap and sheds native deps, it graduates off the container image onto the managed runtime, where SnapStart restores a warm snapshot in sub-second — killing the cold start at its root.
Pay-per-use, no standing fleet
Our traffic is spiky/modest. With sub-second init, spikes are cheap and we keep scale-to-zero. Provisioned Concurrency is a stop-gap only, retired once SnapStart lands.
🎯 Bets this direction is making. (1) AWS treats Lambda cold start as a first-class pain point and keeps closing it from the microVM side — SnapStart for Python (GA late 2024) is the trajectory, not the endpoint. (2) Managed runtimes are the destination; self-managed OS/orchestration trends toward "a thing of the past" on a 5–10 year horizon — the same way hand-rolling a compiler became unnecessary. (3) No-regrets across isolation models — every fast-start future, whether microVM snapshotting (SnapStart), WASM/WASI, or V8-style isolates, rewards the same shape: small, native-dep-free, managed-runtime services. The slimming work is the prerequisite for all of them; a container is the one shape that dead-ends across all of them. We optimize for where the platform is going and keep a container only as a break-glass escape hatch — see Containers are an escape hatch.
Motivation
The serverless PR-preview stack is our production proving ground: Flask on Lambda via the Lambda Web Adapter (LWA), fronted by CloudFront, backed by Neon. It works — but exercising it surfaced three structural limits that block it from being a production substrate as-is.
Warm-path navigation is fine, but the Executive Dashboard fan-outs 8 parallel API calls; with one warm env, the overflow spins up fresh cold envs that each pay a 9–11s init on the user's request. App init exceeds Lambda's 10s init window, so it runs inside the first invocation at provisioned CPU.
One image, one import graph, one cold-start cost — and all three grow with every model, blueprint, and dependency added. An arbitrary 2-way split only defers the problem; the growth curve is the issue.
site-packages is 662 MB — 2.6× the 250 MB managed-runtime cap — and carries native C extensions (xmlsec, PyMuPDF, psycopg2). That forces a container image, which is SnapStart-ineligible. The fast-cold-start tool is off the table until we slim.
Proposal
A small number of domain lambdaliths — each a bounded context, each the same LWA artifact — fronted by CloudFront path-routing (the api-origin-dispatch pattern previews already use). Light domains run on the managed runtime with SnapStart; heavy domains stay on a container image. Substrate is a per-domain deploy decision, not a global commitment.
flowchart TB
CF["CloudFront — path router"]
CF -->|/auth/*| AUTH["Auth lambdalith"]
CF -->|/org/*| ORG["Org lambdalith"]
CF -->|/portal/*| PORTAL["Portal lambdalith"]
CF -->|/core/*| CORE["Core / shared lambdalith"]
CF -->|/docs/*| DOCS["Docs + PDF lambdalith"]
AUTH --> NEON[("Neon Postgres")]
ORG --> NEON
PORTAL --> NEON
CORE --> NEON
DOCS --> NEON
style AUTH fill:#e6f3ff,color:#000
style ORG fill:#e6f3ff,color:#000
style PORTAL fill:#e6f3ff,color:#000
style CORE fill:#e6f3ff,color:#000
style DOCS fill:#e6f3ff,color:#000Every domain targets the managed runtime + SnapStart. Native deps are eliminated by offloading to managed services (SAML → Cognito, analytics → the data warehouse), not parked on a container. A container survives only as a break-glass escape hatch — see below.
The substrate ladder
Each domain climbs the same ladder independently. We don't boil the ocean — we break out sizeable domains first (Org, Portal), then Auth, then finer cuts, as time allows. No step requires the next one to be valuable.
The existing LWA container on Lambda. Single deploy unit. Works now; cold start on the request path. No re-platform — this is the baseline.
Split by bounded context into separate container lambdaliths behind the CF router. Each domain's init shrinks (smaller import graph). No SnapStart or managed-runtime dependency — pure decomposition win.
Flask → FastAPI (ASGI) per domain. Prerequisite for slimming and better in-env concurrency; also the forward-compatible substrate for any future edge/ASGI move.
Offload or swap native deps: SAML → Cognito, analytics → the data warehouse, PDF → pypdfium2 (managed-runtime-safe, permissive license). Goal: zero native blockers, unzipped deps under 250 MB.
Qualifying domains move off the container onto the managed runtime; enable SnapStart → sub-second restore on scale-up, AWS owns OS/CVE patching, no base image to maintain. Retire any stop-gap PC.
Managed-runtime bar (Stage 3 → 4)
The bar every domain clears to reach the managed runtime. The work is eliminating native deps (above); the checklist is:
| Criterion | Why |
|---|---|
| Unzipped deps < 250 MB (function + layers) | Hard managed-runtime cap. |
| No non-portable native C extensions | Managed runtime can't carry libxmlsec1-class system libs. |
| DB connections opened lazily (not at import) | SnapStart snapshots the init phase — a connection opened then is dead on restore. |
| Function URL points at a published alias | SnapStart runs on versions/aliases, never $LATEST. |
| Request handler resilient to snapshot uniqueness | Regenerate secrets/entropy after restore, not at init. |
Containers are an escape hatch, not a tier
The target is managed runtime for every domain. Of today's native deps, only xmlsec genuinely can't run on the managed runtime — it needs a libxmlsec1system library that isn't present. The rest (psycopg2, lxml, PyMuPDF, pandas) ship self-contained manylinux wheels that run fine there; their only constraint is the 250 MB bundle cap, which decomposition (slim per-domain bundles) handles. So "native deps" splits into one real blocker plus a size question:
- SAML/SSO → Cognito (we already federate through it) — drops xmlsec, the one true system-lib blocker. This is the only escape-hatch driver, and only until Cognito lands.
- Analytics → the Data Warehouse — moves pandas off the request path; aggregation belongs in the warehouse anyway.
- PDF thumbnails →
pypdfium2— PyMuPDF is used in exactly one spot (thumbnail_service.py: rasterize page 1 → preview image, graceful fallback). Swap for pypdfium2 (Google's PDFium): near drop-in render-to-PIL, a self-contained manylinux wheel that runs on the managed runtime (AWS uses it on Lambda), and Apache-2.0/BSD vs PyMuPDF's AGPL — retiring a copyleft liability for a proprietary app at the same time. (Worth doing now, independent of this RFC.)
A container is a break-glass escape hatch for a genuinely irreducible native dep — realistically only xmlsec, pending Cognito. This work is non-urgent, so by execution time we expect Cognito to have removed even that and the escape-hatch set to be empty. If one dep holds out, it falls back to a single container Lambda — the exception that proves the rule, not the architecture.
Current vs proposed
Domain lambdaliths behind a CF router, all on the managed runtime + SnapStart — restore is sub-second, so a fan-out spike is cheap and scale-to-zero survives. Native deps are offloaded to managed services rather than parked on a container; a container is reserved as a break-glass escape hatch only.
Alternatives considered
- AWS Fargate / ECS (the container escape hatch) — viable, and it wins at sustained-high concurrency (an async task amortizes many concurrent requests, no per-request cold start). It is not the plan: our traffic is spiky/modest, so pay-per-use + scale-to-zero is cheaper, and the managed runtime offloads OS/CVE patching. It's reversibility insurance — because the LWA artifact runs on Fargate unchanged, a domain that proves sustained-high (or carries an irreducible native dep) can be re-pointed without a rewrite. Insurance, not destination.
- Cloudflare Workers (isolates) — the isolate model is the elegant long-term answer to the cold-start class (~5 ms starts). But a hard fit-wall today: 128 MB/isolate (we run at 3538 MB), Pyodide/WASM (no psycopg2, lxml, xmlsec, PyMuPDF), and cross-cloud data gravity (Neon lives on AWS, so every chatty query round-trips clouds). Greenfield-only for the Python core; revisit only for stateless edge concerns (routing, auth-token validation, edge render) in TypeScript.
- Managed runtime now (skip decompose) — impossible: 662 MB » 250 MB. Decompose
- slim is the prerequisite, not an alternative.
- Heavy Provisioned Concurrency as steady state — rejected. It discards Lambda's scale-to-zero economics and bills per-env at sustained concurrency (1 request : 1 env). Stop-gap during the journey only.
Risks & open questions
A 221-dependency monolith → FastAPI is months: every route, every model, async conversion, re-test.
Mitigation: migrate per-domain during decomposition, not big-bang. It's also the forward-compatible ASGI substrate, so the work isn't throwaway.
Snapshotted secrets/entropy/connections are shared across restored envs — a correctness and security hazard.
Mitigation: lazy DB connect + restore hooks that regenerate uniqueness after restore (a Stage-4 graduation criterion).
N services = N deploy pipelines, CF routing rules, fragmented warm capacity, distributed tracing.
Mitigation: keep N small (5–8), shared base image + one IaC module, tracing from day one.
A system-lib dep — realistically only xmlsec, pending Cognito — blocks its domain from the managed runtime. PDF is not a blocker: pypdfium2 runs on the managed runtime and is a near drop-in.
Mitigation: the break-glass container escape hatch for that single service — expected empty by execution, since Cognito removes the lone true blocker.
Open questions
What's our real duty cycle? Peak per-domain concurrency and idle ratio decide the Lambda-vs-Fargate crossover per domain. We need the prod traffic shape to confirm Lambda is cheaper for us, not assume it.
Where do the domain seams fall? Org / Portal / Auth are the obvious first cuts. What lands in a "core / shared" bucket, and how do we keep it from becoming a second monolith?
Which domains actually clear 250 MB after slimming? The baseline (boto3 + cryptography + SQLAlchemy + framework) eats a chunk before domain code. Needs a real per-domain bundle measurement.
SnapStart cost at our scale? Snapshot cache storage + restore charges are modest but non-zero — measure against the cold-start latency we're buying back.