OnRamp Integrations North Star
The durable target for OnRamp's CRM/integration fabric, and the principles that keep us iterating toward it instead of rebuilding. This is the why and the shape — the Prismatic Decoupling — First Step (v1) doc is the scoped slice we build now.
💬 Feedback wanted — draft. A GitHub Discussion (Ideas) will be linked here for comments before we commit. Concrete open questions live in the first-step v1 doc.
Read this for direction, not scope. v1 implements a slice of this, never all of it. The one rule that lets us chip away instead of rewriting: build the seams now, defer the implementations. Everything below is in service of that.
TL;DR
Never replicate the CRM
Salesforce / HubSpot stay the system of record. OnRamp holds a thin, demand-driven projection of only the slice each tenant actually touches.
Canonical envelope + adapter
Each source gets an adapter that translates its native shape into a canonical entity in a versioned envelope. Downstream code never sees provider-specific JSON. Adding a CRM = adding one adapter.
Own what creates lock-in
AI collapses the per-source adapter from weeks to hours. It does not remove token lifecycle, rate-limit choreography, idempotency, or replay. Build those ourselves; buy only undifferentiated heavy lifting.
The shape of the target
System of record vs. system of reference. The CRM owns the data. OnRamp holds a system of reference: a sparse, tenant-scoped projection of only the objects and fields a tenant's workflows reference. We never mirror a CRM wholesale.
This already matches how OnRamp behaves today — inbound events are evaluated on-demand and arbitrary fields are hydrated by name only when a predicate needs them (fetchObject in onramp-prismatic-api-handler/shared/hubspot-inbound/workflow-trigger-helper.js). Keep that posture; formalize it.
The decoupling seam — must exist day one of any phase. Every source gets an adapter (an anti-corruption layer) that maps its native shape to a canonical entity wrapped in a versioned envelope. Routing happens on a structured source + domain, not by string-parsing a provider type. Point-to-point integration grows quadratically; a canonical seam grows linearly.
The envelope has two faces — both matter:
The inbound change-event contract: { source, objectType, objectId, objectDetails: [{name, value, type}] }, evolved toward CloudEvents context attributes + a canonical type vocabulary. A de-facto version already exists and is consumed CRM-agnostically — the work is to formalize and version it, not invent it.
A normalized field-schema descriptor + canonical operator vocabulary that each adapter produces, so one trigger-config UI renders any CRM. This is the half that kills the per-CRM UI sprawl (today: two ~450-line near-duplicate Vue trigger components with divergent operator vocab).
The trigger substrate (the core bet)
The single most important architectural decision, and the one that unlocks adding CRMs cheaply:
Evaluate triggers in OnRamp — poll + evaluate-in-OnRamp + filter-in-our-DB — not in the customer's CRM.
Today HubSpot triggers depend on app-global webhook subscriptions (only 6 reserved property names: onramp_condition_1..5, send_to_onramp) and Salesforce triggers depend on per-customer in-CRM Flows. Both couple us to the CRM's filtering, burn the customer's automation/subscription quota, and depend on the customer's plan tier.
Ross's three acceptance principles — any trigger solution must pass all three:
- No dependence on the customer's subscription tier or being within their rate/automation-count limits.
- No dependence on non-ubiquitous CRM capabilities (e.g. HubSpot Workflows is new / Pro+).
- No customer-tweakable artifact living in their CRM account (a filter/automation they can break to their own detriment).
Poll + evaluate-in-OnRamp passes all three. It is symmetric across CRMs, which is what makes "add a new CRM = add one adapter" real.
What AI changes — and what it doesn't
AI collapses the per-source adapter (typed client from an OpenAPI spec, query translation, schema→canonical mapping) from weeks to hours. That is the build-vs-buy tipping point: the historical reason to rent an iPaaS was adapter cost, and that cost is now small.
AI does not eliminate the durable hard parts — multi-tenant token lifecycle, rate-limit choreography, change-event durability/replay, idempotency, conflict resolution, observability. These are exactly where Prismatic added friction without removing difficulty. Build the thin fabric ourselves; buy only undifferentiated lifting, and never anything that takes custody of tokens or data in a way that creates a re-auth migration tax later.
The long arc
v1 is the first slice. Later phases are additive behind the seam, not rewrites.
flowchart LR
V1[v1 — own eval + adapter seam<br/>Prismatic reduced to auth broker] --> P2[Salesforce read path<br/>same adapter interface]
P2 --> P3[Write-back<br/>outbox, idempotent upserts, echo suppression]
P3 --> P4[Agent surface<br/>canonical read/write as MCP tools]
V1 --> P5[Sunset Prismatic<br/>drain refresh tokens to own auth server]
style V1 fill:#e6f3ff,color:#000- v1 — HubSpot trigger decoupling (the first-step doc). Evaluate and execute triggers in-house behind one adapter + the envelope seam, and reduce Prismatic to an auth broker only — OAuth consent, refresh-token storage, and a mint-access-token endpoint OnRamp calls. Prismatic no longer executes inbound flows. (Custom-field polling is a v1 stretch.)
- Phase 2 — Salesforce read path. SF adapter on the same interface: Pub/Sub API (gRPC CDC) + persisted replay cursor, JWT bearer via External Client App, watermark-polling fallback. This is the real test of v1's seam — it relocates SF predicate evaluation (today in-CRM via Flows) into the shared evaluator.
- Phase 3 — Write-back. Transactional outbox, idempotent external-ID-keyed upserts, echo suppression (SF
ChangeEventHeader.commitUser; HSchangeSource+ short-lived write-marker), declarative per-field conflict policy. One object, one direction, behind a flag. - Phase 4 — Agent surface. Canonical read/write exposed as MCP tools (Strands/AgentCore), with prompt-injection guardrails on write tools.
- Phase 5 — Sunset Prismatic. A Prismatic flow drains the stored refresh tokens into OnRamp's own auth server (custom or Nango); OnRamp takes over OAuth consent + token custody; Prismatic is removed entirely. The drain exports refresh tokens rather than re-consenting, which is what makes the cutover clean (see settled decision 5). Independent of Phases 2–4 — pull it whenever the broker is the last thing Prismatic does.
Repo topology — toward one monorepo
Today the integration runtime lives in a separate crm-integrations meta-repo — a bun / VS Code workspace stitching ~9 sibling JS/TS repos (the Prismatic flows; onramp-prismatic-api-handler, which hosts the acker, processor, and shared evaluator; onramp-auth-handler; the onramp-database-js Sequelize models; onramp-integration-js; aws-helpers-js; async-automation-handler; local AWS emulation). The two-repo span is real friction — split toolchains, cross-repo PRs, and the standing question of which repo new integration code belongs in.
Directional goal: collapse the integration fabric into the main-web-application monorepo (it already runs a bun workspace alongside Python/uv) and retire the meta-repo. Opportunistic and behind the seam — new code lands in the monorepo first; existing pieces migrate as they're touched, not in a big bang. This is what makes "the fabric is one thing" true at the repo level, not just the architecture level.
Runtime direction — serverless via Lambda Web Adapter
Runtime/compute direction is owned by the Domain Lambdaliths RFC (docs/proposals/domain-lambdaliths.md): the backend becomes a few domain-scoped Lambda Web Adapter lambdaliths behind a CloudFront path-router, each climbing a ladder — container LWA today → per-domain FastAPI → slim / kill native deps → managed runtime + SnapStart (sub-second cold start). The integration/CRM domain is one of those lambdaliths. v1's processing slice starts on the bottom rung — reuse the Flask app + ORM unchanged on container LWA, the way the PR-preview stack already runs — and rides the ladder up. Build it SnapStart-ready from day one: lazy DB connections (not at import), no native deps imported at init, served from a published alias — so the domain graduates cleanly. (FastAPI is a planned per-domain stage, not a rejected alternative; same code, same ORM, no parallel executor to keep in sync.)
Settled decisions (do not re-litigate)
- Trigger substrate = poll + evaluate-in-OnRamp + filter-in-our-DB. Ubiquitous, no tier dependence, no customer artifact, symmetric across CRMs.
- HubSpot near-term = hybrid partition. Poll only triggers whose predicate references a non-subscribed custom field; leave standard/creation triggers on webhooks. Partition routing prevents double-fire; execution dedup backstops. Retires
onramp_condition_*. - Salesforce long-term = idempotent dual-run → shadow-compare → cutover → delete Flows.
- Envelope is the spine, with two faces (runtime event + design-time config/schema).
- Prismatic = auth broker now; own it later. This effort reduces Prismatic to OAuth consent + refresh-token storage + a mint-access-token endpoint OnRamp calls — nothing else. A later phase drains those refresh tokens into our own auth server (custom or Nango) and sunsets Prismatic. The drain exports tokens (no customer re-consent), which removes the re-auth migration tax — and is exactly why Nango stays a viable option rather than being ruled out.
Rejected directions (so we don't resurrect them)
- HubSpot Workflows / per-ramp in-account automations (
crm-integrations/prismatic/.docs/hubspot-workflows-prefilter-plan.md; the live-but-gatedcreateTrigger.tsflow). Fails all three acceptance principles — depends on a Pro+ feature and plants a customer-tweakable artifact in their portal. Note:createTrigger.tsis registered live in the flow index but its rollout is gated; treat the direction as rejected, not the code as dead. - Big-bang backend rewrite — migrating the whole backend (or all Flask routes to FastAPI) in one shot. Out. The path is incremental and per-domain — the lambdalith ladder in Runtime direction above.
Ground truth & reuse inventory
What already exists (so phases scope against reality), and the current constraints to design around.
- OnRamp can already self-refresh CRM tokens (
vendor_platform_integrations,app/api/integrations/dataloaders.py:59-90) — proof the eventual own-the-tokens auth server (the Phase 5 sunset) is cheap. v1 still mints via the Prismatic broker; this is the migration asset, not the v1 token path. - Direct CRM REST clients —
HubspotDataLoader/SalesforceDataloadercall CRM APIs directly, no Prismatic (dataloaders.py). - CRM-agnostic predicate evaluator —
meetsCondition/checkObjectMeetsConditionsoperate on the normalized{name,value,type}shape (workflow-trigger-helper.js). - Postgres schema cache — objects/fields/related cached in
or_integration.schema_definition. - Inbound pipeline — acker → SQS FIFO → processor, with DLQ.
- IaC for CRM lambdas is DuploCloud (
duploctl) + Serverless Framework + a bash provisioning script — not Pulumi. (Pulumi is the agents infra.) - Tokens are plaintext in Postgres — "encrypted at rest" is aspirational, not done.
async-automation-handleris load-bearing, not "dead weight" — it creates execution records and invokes Flask.getActiveRampshardcodestrigger.code:'HUBSPOT'— the evaluator's discovery is not CRM-agnostic even though its predicate logic is.- Mode enum is
MANY_TIMESbut code switches on'MANY_TIME'(typo, dead case in two files) — onlyONE_TIME/UNTIL_SUCCESSFULactually dedup today. crm-integrationsis a separate repo — this work spans two repos.
Guiding principle: build seams, defer implementations
- A seam is cheap and must exist in v1: the envelope contract (both faces), the adapter interface,
source+domainrouting, canonical entities for objects in use today. These make every later phase additive. - An implementation behind a seam is deferred: the Salesforce adapter, write-back, the conflict engine, the MCP surface.
YAGNI guardrails so the seam doesn't become the ocean:
- Model only the canonical entities/fields in use today — not a full CRM model.
- One adapter in v1 (HubSpot). Build one adapter conforming to the interface — not a generic "any-source" connector framework. The interface is the abstraction; a framework is premature.
- JSON Schema + CloudEvents only. No schema registry, no Avro/Protobuf in v1.
References
- First step: Prismatic Decoupling — First Step (v1)
- HubSpot subscription proof:
crm-integrations/prismatic/hubspot/scripts/clone-app-subscriptions.ts - CRM-agnostic evaluator + fast-path filter:
crm-integrations/onramp-prismatic-api-handler/shared/hubspot-inbound/workflow-trigger-helper.js - Token custody + direct clients:
app/api/integrations/dataloaders.py;app/api/platform_integrations/or_models.py - Prismatic token-export runbook:
crm-integrations/prismatic/.docs/oauth-credentials-and-migration.md - Per-CRM UI duplication:
app/ui-organization/pages/workflows/components/rampBuilder/rampsSteps/{Hubspot,Salesforce}TriggerStep.vue - External: CloudEvents spec (
github.com/cloudevents/spec); Salesforce Pub/Sub API; HubSpot Webhooks v3.