Salesforce Restricted Picklist Rejections — Runbook
Why updateSalesforceField automations fail with bad value for restricted picklist field, and how to tell the one code defect apart from the customer-configuration majority so this signature stops re-opening in triage.
The signature. Emitted from app/api/project_automation/services/project_automation_service.py — the datafield runner (_process_datafield_value_updated_rules) and the task runner (_dispatch_task_rule_action) both raise it:
Failed to process datafield automation rule with action: updateSalesforceField:
Error updating Salesforce object: There were custom validation error(s) ...
The first validation error encountered was "Step: bad value for restricted
picklist field: 01.Kick-off".TL;DR
The value simply is not in the target picklist. Nothing to fix in code — the customer's restricted picklist has to gain the value, or the mapping has to change.
Values containing & were unmatchable. Rich-text answers reach the CRM write already bleached (& → &). Fixed by decoding entities in format_field_value_by_type.
💡 in the message is Salesforce's own escaping, not corruption we sent. Do not chase it as an encoding bug.
The 💡 trap
Most rejected values in the logs carry HTML numeric character references — 💡 Clever, 🔗 Classlink, 👣 Free: Learning Paths. These look exactly like an outbound encoding bug and are not one.
Salesforce escapes its own error strings, and only outside the BMP. In the same 8-day window, ▶️ Free: On Demand Webinars (U+25B6, inside the BMP) appeared literally, while every astral emoji (U+1F4A1 and friends) appeared as a charref. That split along the surrogate-pair boundary is the fingerprint of the vendor's XML serializer, not of anything OnRamp does.
Three independent checks confirm it:
sanitize_rich_text(and barebleach.clean) leave emoji untouched — only& < > " 'are escaped.- Nothing in the repo does charref or ASCII-with-
xmlcharrefreplaceencoding. - In prod logs, charrefs occur only inside Salesforce error strings — never in a request path, a payload, or any value OnRamp logged on the way out.
So a charref-bearing rejection means we sent the correct emoji value and Salesforce rejected it anyway: the picklist does not contain it. That is direction 🅰.
Direction A · Customer configuration
The value OnRamp sent is correct and simply is not an allowed entry on the target restricted picklist. Observed examples beyond the emoji set: 01.Kick-off, In Progress, Complete, Green, SF, 21286.
This is not a code bug, and it is already surfaced in the product — the failure is not swallowed:
log_project_errorincrementsORProject.automation_errors_count.ProjectRuleHistoryHelper().rule_processed(..., False, json_data, error)stores the attempted payload and the error detail.- The org UI renders both:
ProjectAutomationLogs.vue, and theautomation_errors_countbadge column on the projects list.
What is genuinely missing is operator-side alerting — the log line is a WARNING with no Sentry event, so volume changes are invisible until someone reads the logs. Treat that as the open follow-up, not the picklist values themselves.
Remediation
Either the customer adds the value to the restricted picklist, or the automation's field mapping is pointed at a field whose allowed values match. A durable product fix is to validate the mapping at rule-save time against the Salesforce describe API, so the rule cannot be saved to send a value the picklist will reject.
Direction B · The & defect (fixed)
A restricted-picklist label containing & could never match, no matter how the customer configured it.
Rich-text and form answers are bleached on every write that reaches ORTaskStep.action_response, which escapes & → &. When an automation maps such an answer into a CRM field (FieldValueSource.step), the escaped form went out on the wire, so Program Strat & UCD was sent as Program Strat & UCD.
Reading the double escape. The log showed &amp; — two layers. One is ours (bleach), one is Salesforce escaping its own error string. The wire value carried a single &.
The fix decodes entities at the single marshalling chokepoint for both Salesforce and HubSpot writes, format_field_value_by_type in app/api/utils/models_as_fields.py, using decode_html_entities from app/api/utils/html_sanitization.py — the inverse lives next to the escaping it undoes.
Never reach for bare html.unescape here. The legacy entities match as a prefix, so it decodes ¶m2= into ¶m2= and corrupts every URL field with an ¶m, ×, or ¬in segment. decode_html_entities instead matches semicolon-terminated entities and resolves named ones by exact lookup in html.entities.html5, so an unknown name is left intact whether or not it is terminated — note that html.unescape("¶m2;") also yields a pilcrow, so termination alone is not a sufficient guard. Numeric character references delegate to html.unescape, where the token is unambiguous. Single pass: a doubly-escaped value keeps one layer.
Decode list-shaped values too. A value only gets joined into a string for the string / textarea / url types, so for multipicklist and enumeration a multi-select answer stays a list (get_form_element_answer returns a list of labels for an array element). An early version of this fix decoded only scalars and left exactly half the class — every multi-select write — unfixed.
Triage checklist
- Pull the rejected values, not just the count — the value is what classifies the failure.
- Strip charrefs from consideration; they are vendor-side escaping (🅲).
- Does the value contain
&? If so, and it predates theformat_field_value_by_typefix, it is direction 🅱 and already resolved. - Otherwise it is direction 🅰: customer configuration. Route to the CSM with the project's automation-error log; do not open a code ticket.
OpenSearch recipe (prod-usw2 flask, tenant + app filters per the opensearch-investigation skill) — aggregate on match_phrase of bad value for restricted picklist field, then read the message of the hits to extract field name and value. message has no .keyword subfield, so classify by reading the hits rather than by terms-aggregating them.