RFC: CRM Token Broker (M1)
OnRamp stores and refreshes CRM access tokens itself, so CRM reads and writes go directly to HubSpot/Salesforce instead of through a Prismatic flow.
💬 Feedback — this is the M1 build under Prismatic Decoupling v1. Weigh in there; open items are in Risks & open questions.
TL;DR
Store + refresh in OnRamp
A FastAPI lambda holds CRM tokens in DynamoDB and refreshes them directly against HubSpot/Salesforce. CRM calls go direct.
Prismatic supplies the refresh token
When a customer connects their CRM, Prismatic pushes the resulting refresh token to an OnRamp ingest endpoint, which stores it for the lambda to use.
Storage + refresh are permanent
This is the first step of owning the OAuth lifecycle. When OnRamp later runs consent itself, it writes the same store — no rework.
Motivation
Today, OnRamp reaches a customer's CRM by executing a Prismatic flow: Prismatic holds the customer's OAuth connection, runs the flow, calls the CRM, and returns the result. That flow execution is multi-second and async-polled, so it sits in the latency path of every CRM read and write.
Every CRM read/write pays Prismatic's flow-execution overhead. The win is calling the CRM directly — which requires OnRamp to hold a usable access token.
CRM OAuth access tokens are short-lived (HubSpot ~30 min). To call the CRM directly, OnRamp must cache the access token and refresh it when it expires, so a refresh happens at most once per token lifetime — not per call.
OnRamp owning token storage + refresh is the first concrete step of Prismatic Decoupling toward owning the OAuth lifecycle outright.
Proposal
A stateless FastAPI lambda backed by a DynamoDB token store. Clients call the lambda directly — the app's reverse proxy routes /api/crm/* straight to it, same origin, no Flask-app hop. The lambda authenticates the caller's principal and derives the vendor from it (never a request parameter), resolves a token from the store (refreshing on miss), and hits the CRM directly. Prismatic populates the store with each customer's refresh token at connect time.
flowchart TD
UI["OnRamp UI / service"] --> RP["Caddy reverse-proxy /api/crm/*"]
RP --> LB[CRM lambda FastAPI]
LB --> DDB{Token in DynamoDB valid?}
DDB -->|valid| CRM[Direct CRM call]
DDB -->|missing or expired| RF[Refresh single-flight]
RF -->|refresh_token + app client creds from SSM| CRMTOK[CRM token endpoint]
CRMTOK --> DDBW[Store token]
DDBW --> CRM
Consent["Customer connects CRM via Prismatic"] --> PID[Prismatic post-deploy flow]
PID -->|push token + refresh_token| INGEST["Flask ingest route, validates vendor api key"]
INGEST --> DDB
style RF fill:#e6f3ff,color:#000
style PID fill:#fff0e6,color:#000The two paths are independent. Reads go UI → lambda directly (Caddy reverse-proxy, same origin).* Ingest is the only Flask hop — it lives in the Flask backend because validating the vendor's api key resolves it against Postgres, which the DynamoDB-only lambda can't reach.
* /api/crm/* is a Caddy reverse-proxy rule today. Fronting it with CloudFront is a separate migration, out of scope here.
What this builds
Key (vendor_id, crm) → { access_token, refresh_token, expires_at, instance_url? }. KMS-encrypted at rest, tight IAM read scope. Tokens are credentials — never logged, never returned to the browser.
getAccessToken(vendor, crm): strongly-consistent read of the store → if the access token is valid, return it; else refresh. Mangum handler, lazy-initialized clients (SnapStart-clean, per domain-lambdaliths).
POST {crm token endpoint} with grant_type=refresh_token, the stored refresh_token, and the app's client_id/client_secret. HubSpot → https://api.hubapi.com/oauth/v1/token. Store the new token back.
Coalesce concurrent refreshes (see Concurrency). In-process lock plus a cross-instance DynamoDB conditional-write lock or optimistic CAS.
A narrow Flask route — validates the vendor api key, resolves the vendor against Postgres, upserts one DynamoDB row. It lives in Flask (not the lambda) precisely because that api-key → vendor resolution needs Postgres.
The CRM app's client_id + client_secret, per region, in SSM. Required for the refresh call in (3).
Concurrency
The lambda owns the refresh, so it owns the refresh race. Both controls are required from the start:
- Concurrent refreshes → single-flight. When several requests hit an expired token at once, only one refresh should run; the rest reuse its result. Coalesce in-process, then guard across lambda instances with a DynamoDB conditional-write lock or optimistic CAS. This is mandatory for Salesforce: a connected app caps the number of active access tokens and evicts the oldest past that cap, so uncoordinated refreshes can evict a token another request is mid-flight on. (HubSpot allows concurrent access tokens, so there an uncoordinated herd is merely wasteful.)
- DynamoDB read consistency → DynamoDB default reads are eventually consistent. Use strongly-consistent reads for the token lookup; conditional writes are linearizable.
Population
Prismatic facilitates the customer's OAuth consent and holds the resulting connection (it needs this to run the event/trigger flows that remain on Prismatic). At flow runtime, a flow can read that connection's refresh_token. So population is a push: Prismatic's post-deploy flow, which runs after a connection is established, sends { vendor_id, crm, refresh_token, expires_at, instance_url? } to the ingest endpoint (5). Already-connected customers are seeded by a one-time re-run of that flow across instances.
That flow must be authored low-code, matching the existing HubSpot/Salesforce integrations, so it ports into them directly.
How a CRM call resolves
CRM read/write → OnRamp lambda → DynamoDB token (refresh directly on miss) → CRM. Prismatic only pushes the refresh token at connect time.
Scope boundary
OnRamp owns token storage + refresh. Prismatic still facilitates OAuth consent and runs the CRM event/trigger pipeline (webhooks → OnRamp workflows). Both sides hold the same refresh token during this period; that is safe because HubSpot and Salesforce refresh tokens are non-rotating and both providers allow concurrent access tokens, so neither side invalidates the other on refresh.
Alternatives considered
- Mint a token per call from Prismatic. Keeps Prismatic's request in the latency path of every CRM call. Caching the token is what removes that — and once the token is cached, refreshing it directly is simpler than routing each refresh back through Prismatic.
- Fetch tokens from Prismatic at request time instead of storing them. Keeps Prismatic in the hot path and requires Prismatic management-API credentials in the lambda. Storing the refresh token and refreshing directly avoids both.
- Run OAuth consent in OnRamp now. Not viable while the event pipeline is on Prismatic: those flows need Prismatic to hold the OAuth connection, and Prismatic cannot be handed a token minted elsewhere. Consent moves to OnRamp when the event pipeline does.
Risks & open questions
A connected app caps concurrent access tokens and evicts the oldest, so uncoordinated refreshes can evict an in-flight token.
Mitigation: single-flight is mandatory for Salesforce; confirm the exact cap when building the SF path.
refresh_token + access_token live in DynamoDB for the cache window.
Mitigation: KMS encryption, tight IAM read scope, never logged, never sent to the browser.
Already-connected customers have no future consent event to trigger the push.
Mitigation: one-time re-run of the push flow across existing instances.
Open questions
Client-cred provisioning. Where the per-region CRM app client_id/client_secret live in SSM, and the lambda's IAM path to read them.
Salesforce refresh specifics. Confirm the token host (login/test/My Domain) and that the response returns a storable instance_url. (HubSpot's path is settled: POST /oauth/v1/token, form-encoded, non-rotating, expires_in 1800.)
Caller authentication. The lambda derives the vendor from a verified principal, so the caller must present a token the lambda can verify (JWT/JWKS) — not an opaque server-side session. Confirm what the UI presents on /api/crm/*, and bridge to a verifiable token if needed.
Out of scope for M1: OnRamp running OAuth consent itself, and migrating the CRM event/trigger pipeline off Prismatic. Those move together, per integration, because consent and events share the OAuth connection.