Joy Loyalty · Integrations

Integrations
roadmap

What is breaking, why, and the work to make every integration deliver, recover and tell the merchant.

The problem in numbers · prod logs 02/11 → 19/11

Capacity is fine. Reliability is not.

205k
deliveries dropped after a partner rate limit: Omnisend contacts 172,229, Klaviyo profiles 22,479, Omnisend events 10,404
27,166
failed Klaviyo token refreshes on 44 shops. Dead credentials are called for weeks; the merchant still sees "Connected"
40.7k
self-inflicted error lines: the admin customer page probes Klaviyo and Omnisend for shops that never connected them
344
order webhooks not queued while Shopify was told "success" (303 "Task size too large"). Those orders may never have earned points
21,382
log lines carrying a merchant's Mailchimp authorization header. Recharge addresses and Apple Wallet auth headers leak too
10,491
outbound merchant webhooks counted as failed only because the endpoint answered a plain ok

Counts are log lines, not unique customers. Shop ids and emails stay out of this deck.

One cause behind all of it

There is no integration module. Each app is hand-wired across ~30 files, on a path that cannot retry, with no delivery id, no dead-letter queue, no health state and no success metric.

Compute, Pub/Sub and Cloud Tasks scale well past 5 to 8 more partners. What is missing is everything that makes a delivery safe.

Why rebuild instead of patching

What changes for each side when integrations run on one platform

Today
After the rebuild
Merchant
Sees "Connected" while the credential is dead for weeks. Lost syncs are invisible: 205k drops in 18 days, nobody told.
A badge and a Home banner say which app broke and what stopped (reminders, birthday events, profile sync), with Reconnect and Re-sync buttons and one email.
End customer
Order points can silently never arrive (344 orders); Klaviyo/Omnisend profiles go stale, so loyalty emails show wrong balances.
Orders are accepted durably before Shopify gets 200; every profile update is retried until delivered or quarantined for review, never dropped.
Support
Digs through Cloud Logging by hand; 18-day retention; no per-shop view of what failed.
DevZone diagnostics per shop: last success, 7-day outcomes, error in plain words, replay a failed delivery. Tickets labelled per integration.
Developer
A new app = ~32 files, 45 edit sites, 17 switch branches; no test catches a missed site; every app reinvents retry and rate limit.
A new app = one registry entry + one adapter. Retry, rate limit, dedupe, health and metrics come from the platform; contract tests catch drift.
Business
Postscript and other SMS partners cannot be added safely (a duplicate is a second paid SMS). No partner API, so partners cannot build on Joy.
Partner API F1 lets Loop, Tapcart, Okendo build on Joy; Postscript ships on the delivery contract; Joy can lead where no competitor is (Loop exchange points).

The core change, on one delivery

A partner answers 429. What happens next?

Today

Points change
publish to Pub/Sub, retry:false
Omnisend 429
handler sleeps, tries again in the same request
Still 429
returns undefined; dispatcher returns 'ok'
Dropped
one log line. No retry, no replay, no metric, merchant not told
172,229 times in 18 days

After

Points change
producer writes a pending delivery with a stable deliveryId, then acks
Admission
worker takes a slot in the shop's budget at Omnisend's real limit
429 → reschedule
wait as long as Omnisend allows (a body field, no Retry-After header); a deferral is not an attempt
Delivered
counted as success. Klaviyo and LINE dedupe on the delivery id; Omnisend does not, so Joy keeps the id itself
Or quarantined
timeout after send = unknown whether accepted → held for review, never blindly resent (an automation would fire twice)

Same pattern for auth: a 401 becomes auth_broken (calls stop, tokens kept, merchant told), not weeks of retries.

1 · Integrations in use · prod Firestore, read-only count, 20/11

Who uses what: merchants connected per integration

Joy still installed (integration doc) Joy still installed (shop flag) incl. shops that uninstalled Joy Base: 20,835 installed shops

What the count itself shows

Joy does not agree with itself on "connected?"

Klaviyo

790 vs 761

integration doc vs shop flag

Judge.me

1,114 vs 1,216

integration doc vs shop flag

Judge.me

830

connections on shops that uninstalled Joy

Klaviyo

239

connections on shops that uninstalled Joy, matching the dead-refresh calls

  • "Is this app connected?" has four answers: shop flag, integration doc, an admin fallback map and the AI tool's map. Seven admin flags have no writer.
  • At least seven code paths write connection state. One of them, the Klaviyo auto-disconnect, also revokes the grant at Klaviyo.
  • Not countable today: Smax, Chatty, Joy Wishlist, Klaviyo Reviews, Facebook Page, Joy APIs. No connection record exists.
Fixed by the registry (3.2) and the per-app health service (3.6).
02

Today

Current architecture
& infrastructure

Read from the code at HEAD 2ff18f82dc.

2.1 · How an integration is wired today

Adding one app touches ~32 files and 45 edit sites

Constants

App id with the loyalty-program constants; events, message and job types elsewhere. Ids use four casings.

const/programs/integrationPrograms.js

17 switch branches

Repository connect/disconnect, controller paths, fan-out lists, the Cloud Task handler map, and admin cards, routes, status map.

subscribeBackgroundHandling.js: 212 cases, 9,933 lines

4 "connected" answers

Shop statusConnected* (19 names), doc isConnected, admin fallback map, AI tool map. They disagree, as measured.

7 writers of state

Connect callbacks, disconnect, repo side effects, Klaviyo auto-disconnect (revokes!), Intercom auth failure, Drip bulk failure, generic update.

No test fails when one of these sites is missed. Credentials are copied into Pub/Sub and Cloud Task payloads, and a reconnect overwrites the same doc, so old queued work can run under a new installation's credentials.

2.2 · Outbound path today, and where work is lost

Every drop path logs one line and returns

01
Loyalty event or customer change
Fan-out to every connected app in triggerIntegrationAppService
publish fails → work dropped
02
Pub/Sub backgroundHandling*
retry: false. One dispatcher with 212 cases
dispatcher returns 'ok' on any error
03
Per-app service
Each app invents its own rate limit and retry. Omnisend sleeps in the handler
429 → 3 tries → drop
04
Re-enqueue on enqueueSubscriberv2
One shared Cloud Tasks queue, maxAttempts: 1, no rateLimits
deferrals count as attempts
05
Partner API
Klaviyo, Omnisend, Mailchimp, Attentive, Gorgias…
401 → retried for weeks

Inbound is no better: four review sources are unsigned, two verifiers fail open, Recharge is unauthenticated, and an HMAC failure on a Shopify webhook returns 200 with no log.

3.2 · Infrastructure per integration type

What runs each type, and what happens on failure

TypeAppsRuns on todayOn 401 / 429 / 5xx today
Email / SMS marketing (out)Klaviyo, Omnisend, Mailchimp, Attentive, Drip, Sendlane, PushOwlPub/Sub Light/Small (retry:false) → service; Klaviyo re-enqueue on the shared queue; Klaviyo bulk collectorsThrottle gate, 3 tries, drop; only Klaviyo's private key disconnects on 401
Helpdesk / messagingGorgias, Intercom, LINE, SmaxSame Pub/Sub path; Gorgias limiter in RAMGorgias refresh failure → TypeError loop; LINE retries without an idempotency key
Reviews (in)Judge.me, Yotpo, Loox, Stamped, Fera, Review Rocket, Air Reviews, Klaviyo ReviewsHTTP webhooks on apiHookV2, inline; generic dedupe4 unsigned, 2 fail-open
Subscription / wishlistRecharge, Joy Subscription, Joy WishlistInbound HTTP + outbound callsRecharge unauthenticated, tenant from the query, PII in logs
SocialInstagram, Facebook PageSigned Meta webhooks → Pub/Sub; token refreshed on demand, no cronRevoked token retried on every use
WalletApple, Google WalletHTTP + APNs / Google APIAuth headers and push tokens still logged
Joy emailMailgun custom domain, merchant SMTPsentEmailNotification topic, no retryThe only full breaker in Joy, but custom domain tried on every send (~18k/day)
Shopify (integration paths)Order webhooks, Admin API, FlowapiHookV2 + Cloud Tasks for ordersEnqueue failure answers 200; ~10.7k throttled Admin calls/day; Flow 200-on-failure
Joy-provided APIsREST v2, outbound webhooks, headless, MCPHTTP functions; webhooks via webhookSend, no retryNon-JSON 2xx counted as failure; merchant 5xx never retried

3.3 · Is today's infrastructure enough?

For scale, yes. For reliability, no.

Exists, unused

Durable queue with retry

Integration topics are retry:false; the shared queue allows one attempt

Partial

Rate limit honouring Retry-After

3 apps share integrationRateLimits; 30 s per-instance cache

Missing

Idempotency per delivery

Klaviyo unique_id falls back to a random id; no task has a name

Missing

Dead-letter queue + replay

Only the ordered topic has one, via a manual script not in CI

Missing

Auth breaker that tells the merchant

Only SMTP has one. 61,240 Klaviyo refresh-failure lines

Missing

Success metric per app

Apps log failures as free text and never log successes

Partial

Inbound verification

4 unsigned sources, 2 fail-open, 1 unauthenticated

Partial

Secrets handling

Tokens in message payloads and logs; one constant key in source

04

Evidence

What the prod
logs show

Cloud Logging, 02/11 → 19/11 (the bucket keeps 18 days), read-only, paginated to exhaustion. High-volume fingerprints counted exactly.

4.2 · Root-cause rollup · 336,363 counted lines

Infrastructure causes most of the loss; code causes most of the noise; nothing tells the merchant

INFRA Work dropped after a rate limit: no durable retry, no bulk pathOmnisend, Klaviyo · 205,112 lines
61.0%
CODE + INFRA Dead credential retried, no breakerKlaviyo · 61,240 lines = 27,166 failed refreshes
18.2%
CODE Self-inflicted probe from the adminKlaviyo, Omnisend · 40,714 lines
12.1%
SEC Secrets and PII in logsMailchimp sample · 21,382 lines
6.4%
INFRA Klaviyo 429 lines themselves6,241 lines, 5,466 of them on 13/11 (one shop's tier demotion)
1.9%
CONFIG → UX Misconfiguration nobody seesKlaviyo missing scope, Flow empty email · 1,674 lines
0.5%

Not in the shares (different sample windows): false webhook failures 10,491 · custom-domain fallback ~18k/day · Shopify Admin throttling ~10.7k/day · order webhooks not queued 344.

4.1 · Findings by integration

Trigger → why it keeps happening

Omnisend

182,633

contacts + events dropped after 429. Retry sleeps in the handler, then returns

Klaviyo

22,479

profile syncs dropped; deferrals count toward 3 tries. 98% from the 13/11 burst

Klaviyo OAuth

44 shops

dead grants refreshed for weeks; the invalid_grant branch is unreachable

Gorgias

~3,000/day

invalid_client since 05/11 on 8 accounts, then a TypeError loop

Mailchimp

17.4k

401/403 in 10 days, every line with the merchant's auth header

Instagram

~6,300

revoked session retried on every use; no cron, no reconnect prompt

Order webhooks

344

not queued, 200 returned. Whole ctx.state put in the task

Shopify Admin

~10.7k/day

throttled calls inline in webhooks; customer reads return empty, tags dropped

Merchant webhooks

10,491

false failures: 2xx with a non-JSON body becomes 500

Joy email

~18k/day

custom domain tried for shops with no verified domain

05

Target

One integration
platform

Built once, proven on one app first (Omnisend events for delivery, Klaviyo for health), then each app moves behind a flag.

5.1 · Target architecture

Accept durably, deliver once, know the health, tell the merchant

Target integration architectureThree lanes: Joy sends to partners through a durable delivery worker; partners send webhooks through a verifier and durable acceptance; partners call Joy through the F1 API. All lanes share a registry, credentials by reference, a health service that drives the merchant UI, and outcome metrics. 1 · SEND OUT · Joy → partner 2 · RECEIVE · partner webhook → Joy 3 · PARTNER CALLS JOY SHARED FOUNDATION · every app uses it Event in Joy points, tier, order, profile Pending record stable deliveryId, saved first Delivery worker own Cloud Tasks queue · per-shop slot at the partner's real limit Adapter per app Klaviyo, Omnisend, Postscript… dedupe where supported Partner API Klaviyo, Omnisend, … Outcome success · retry later ambiguous → quarantine Sweeper repairs failed handoffs Partner webhook reviews, Loop, Recharge Verifier signature + right shop Save, then answer 200 durable acceptance, dedupe Loyalty action award points, log review Partner apps Loop, Tapcart, Okendo Partner API F1 scoped key default-deny routes REST v2 customers, points, tiers, rewards Integration registry one module per app Credentials loaded by reference never inside a message Health service connected · degraded · auth_broken Merchant sees it badge, banner, re-sync, email Metrics + alerts success rate per app results + auth errors

5.1b · Every partner is different · Joy sends out

One shared platform, but every partner has its own rules

The differences live in each app's adapter; the delivery worker only asks it "when may I retry?" and "does the partner dedupe?".

PartnerAuthLimitWhen busy (429)DedupeSpecific
KlaviyoOAuth (or private key)Per account + app, by endpoint tier (events 350/s, bulk 10/s)Retry-After headerYes: unique_idToken refresh max 10/min; bulk cannot clear a property
OmnisendAPI key / OAuth400/min per brand, shared with other appsretryAfter in body (some endpoints)No dedupe for automation eventsMoved to a date-versioned API; Joy still on /v5
MailchimpOAuth10 concurrent connections (not a rate)NoneUpsert by email hash, safe to repeatBatches are unordered
AttentiveOAuthSubscribe 10/sx-ratelimit-* headersUndocumentedRe-subscribing texts the customer
GorgiasOAuth80 requests / 20 s——Joy limits itself to 35/20 s in memory; invalid_client since 05/11
InstagramMeta OAuth———Token lives 60 days; refresh before expiry
LINEChannel tokenMonthly quotaNo headerYes: X-Line-Retry-Key (24 h)A 429 may mean the monthly quota is gone
Postscript (new)Partner key + shop key15/s per partner–shop pairError code 3002, no headerexternal_id dedupe undocumentedMax 50 properties; never subscribe customers

5.1c · Every partner is different · partners send to Joy

Each webhook source signs and retries its own way

The verifier has one contract, but how a signature is checked is specific to each provider.

SourceHow to tell real from fakeWhen Joy answers an errorDedupe / note
Shopify webhookHMAC on the raw body5 s timeout; 8 retries over 4 h, then the subscription is deletedX-Shopify-Webhook-Id (Joy uses Event-Id)
Shopify Flow actionHMACRetried up to 36 h, incl. timeouts and 429action_run_id
Judge.meJUDGEME-HMAC-SHA256, Joy's OAuth app secret——
LooxHMAC only if the merchant sets a key (Convert/Unlimited)——
FeraJWT in the URL, signed with the store secret Joy holds—Verifiable today
YotpoUnsigned → secret token in the URLAt least 10 retriesWebhookMessageId
StampedSecret in a header, no body signature——
RechargeX-Recharge-Hmac-Sha256 = SHA-256(secret + body); Joy lacks the secretRetries, then deletes the webhook—
Loop (new)X-Loop-Signature, merchant's secret10 s timeout; 5 retries; at-least-onceTopic return, trigger return.closed; exchange must be derived

Source: each partner's official API docs, checked on 20/11 (verify-partner-api-A/B.md).

5.3 · Seven design principles

Seven principles, in plain words

1

Each app is its own module

A new app means writing that app's module, not editing 30 files.

Not chosen: app settings stored in the database (still needs code, and is hard to test).

2

Every send goes through one queue

Each send has its own id. Partners that dedupe (Klaviyo, LINE) receive it once; for the others, a send with an unknown result is held for review.

Not chosen: turning on retry for the shared queue (it would re-run hundreds of unrelated jobs).

3

One place decides "connected"

App by app, starting with Klaviyo.

Not chosen: switching everything at once while 7 other places still write it.

4

Tokens never travel in messages

The sender loads the token when it runs, so old work never uses a newer install's token.

Not chosen: moving every token to Secret Manager now (too big, blocks other work).

5

Partners get their own key

The merchant issues a key per partner, and it opens only the APIs that partner needs.

Not chosen: sharing the shop's key (full access).

6

Move one app at a time

Each app has a switch; if something breaks, turn it off and the old path runs again.

Not chosen: one switch for everything, changed in one go.

7

Lost connection = pause and tell

No token deletion, no automatic disconnect; once the merchant reconnects, it carries on.

Not chosen: disconnecting and revoking access, as Klaviyo does today.

In short

Send reliably, avoid duplicates, know which app is broken, and keep a way back for every change.

06

The work

62 items,
five groups

1 · Stop data loss now

5 days, before Black Friday

2 · Fix each app

Klaviyo, Omnisend, Gorgias…

3 · Build the shared platform

send queue, connection health

4 · New apps

Flow, Tapcart, Loop, Postscript

5 · Partner API

and removing dead code

Each item's detail (what, where in the code, when it is done) is in roadmap.md.

Group 1

20/11 → 25/11

Do now, within 5 days: small, reversible, mostly no deploy

Turn on 3 alertsdropped data, Klaviyo losing access, orders not queued → know at once, without waiting for a merchant to report it
Remove sensitive data from logsMailchimp keys, Apple Wallet headers, customer addresses and recipient emails are being logged
Customer page stops asking unconnected appsremoves more than 40,000 error lines Joy causes itself
Check review-webhook signaturesrequired wherever the shop already has a key → blocks fake reviews
Flag dead connectionsKlaviyo, Gorgias: only record which shops, change nothing yet
Test Omnisend in a sandboxdoes Joy accidentally unsubscribe customers? If yes, stop all bulk syncs
Find orders that missed points344 webhooks were lost; read-only count, no automatic re-award
Support calls shops with dead connectionsinvite them to reconnect, never to disconnect
Take down the Tapcart guidethe public guide exposes a key
Label tickets by integrationto compare ticket counts before and after the fixes

Group 2

Waves 1 – 3

Fix each app: every error must retry by itself, stop, or tell the merchant

Klaviyo

  • Lost access → stop calling and tell the merchant, instead of retrying for weeks
  • Fix 2 bugs in the retry code
  • Send points expiring soon and points to next tier

Omnisend · Mailchimp · Drip · Attentive

  • Omnisend: reconnecting does not switch events off
  • Mailchimp: one failing shop does not stop the whole cron
  • Drip: a short glitch is not counted as disconnected
  • Attentive: subscribe only consented, not-yet-subscribed customers (a repeat subscribe texts them)

Gorgias · Instagram · Chatty

  • Gorgias: end the error loop, find why it broke on 05/11
  • Instagram: expired session → ask the merchant to reconnect; rotate the app secret
  • Chatty: drop the key shared by every shop

Shopify order webhooks

  • Save the order before answering Shopify, so large orders are not lost
  • If it cannot be saved, answer an error so Shopify resends it
  • Throttled Shopify calls are queued and retried, not dropped

Flow · Merchant webhooks · Email

  • Flow errors are reported clearly, not shown as success
  • A merchant server answering "ok" counts as success; ends 10,000 false errors
  • Block webhook URLs / SMTP servers pointing into the internal network
  • Email: use a custom domain only once verified; re-encrypt SMTP passwords

Reviews · Joy Subscription · clean-up

  • Loox: create signing keys for shops that lack one
  • Joy Subscription: Shopify resends are no longer treated as "already processed"
  • Re-sync shops that lost data (shop by shop, approved)

How it is fixed · 1 / 2 · sending data and connections

Problem → fix → result

Busy partner (429) = Joy drops data

Joy retries a few times within seconds, then gives up. 205,000 times in 18 days.

Every send goes through one queue. A busy partner means "send later" by that partner's own rule, never "drop". Budgets sit below the published limit because quota is shared with the merchant's other apps.

No data lost; unclear results held for review

Bulk actions flood Klaviyo

One shop's mass tier demotion caused tens of thousands of calls, one per customer.

Group them into one bulk sync job that Klaviyo processes once.

No more floods like 13/11

Dead connections still called

Klaviyo, Gorgias, Instagram, Mailchimp keep being called for weeks without access; the merchant still sees "Connected".

On lost access, stop calling, keep the token, show the merchant a banner. On reconnect, sync again automatically.

The merchant knows and can fix it

Admin causes its own errors

Every customer page view asks Klaviyo/Omnisend, even when the shop never connected them.

Ask only the apps the shop has connected.

80% fewer Klaviyo errors in the logs

Gorgias error loop

Since 05/11 the token refresh fails and triggers another error, ~8,900 lines a day.

A failed refresh marks the connection as lost; separately, check Joy's own Gorgias OAuth account.

Loop ends; the 8 failing accounts are told

Mailchimp cron stops for everyone

One failing shop stops the whole cron; jobs stay "syncing" forever.

Each shop runs on its own; a job that fails too often is marked failed.

Other shops are not dragged down

How it is fixed · 2 / 2 · Shopify, webhooks, email, security

Problem → fix → result

Orders missing points

Large orders could not enter the queue, yet Joy told Shopify "received". 344 orders.

Save the order first, then answer Shopify. The queue holds only the order id, so it is never too large. If saving fails, answer an error so Shopify resends.

No order lost

Throttled Shopify calls

When Shopify throttles, Joy treats it as "no data" and skips tagging. ~10,700 times a day.

Throttled calls are queued and retried; large tag runs use bulk operations.

No missing tags or customer data

False errors on merchant webhooks

A merchant server answering "ok" counts as an error, while real errors are never resent.

A success answer is a success. Real errors are resent later.

10,491 false errors gone

Sensitive data in logs

Logs contain keys, headers, customer addresses and recipient emails.

Logs keep only the error type, code and ids.

No new log lines exposing secrets

Review webhooks not checked

Four review apps send unsigned webhooks; Recharge is unauthenticated. Anyone with the URL can send fake reviews.

One method per provider: Fera already signs (check now); Recharge needs the Recharge app secret; Yotpo cannot sign, so a secret in the URL. A transition period avoids blocking real data.

Fake reviews and orders blocked

Flow, email, Joy Subscription

Flow reports success on failure; email tries a custom domain the shop does not have (~18,000 times a day); Joy Subscription ignores Shopify resends.

Flow reports errors clearly; email checks for a verified domain first; Joy Subscription marks "processed" only after saving.

Errors become visible; 95% less email noise

Hidden issues · from partner docs, not yet in logs

Not errors in the logs yet, but they will be if ignored

Found by checking each fix against the partner's official API docs. Each card: what can go wrong → what to do.

Recharge may be deleting Joy's webhooks

If Joy's handler answers non-2xx, Recharge retries and then deletes the webhook. Joy does not store the secret needed to verify Recharge.

Check the status on staging; choose the secret source (ask shops, or a URL secret) when building the verifier.

Attentive: re-subscribe texts the customer

Each bulk sync calls subscribe for people already subscribed; Attentive sends them an SMS.

Subscribe only consented, not-yet-subscribed customers; cap 10 calls/s.

Omnisend replaced /v5

Joy still calls /v5; Omnisend moved to a date-versioned API (2026-03-15), no retirement date yet.

Build new adapters on the new API; ask Omnisend for the /v5 date.

Omnisend does not dedupe events

A resent event can fire a merchant automation twice.

Ambiguous sends are quarantined, never blindly resent.

Mailchimp auto-subscribes new members

New list members are added as "subscribed" with no consent check.

Review together with the Attentive consent fix.

Shopify: wrong dedupe key

Shopify documents X-Shopify-Webhook-Id; Joy dedupes on X-Shopify-Event-Id across topics.

Switch the key when reworking order webhooks.

Klaviyo: token endpoint allows 10 refreshes/min

Joy refreshes on every 401 with no lock; a 429 there can be misread as a dead grant.

Serialise refreshes per shop; a token-endpoint 429 is transient, never auth_broken.

Is Smax still used?

Stored batches that nothing processes.

Winn confirms; drain them or remove the code.

Loop, Postscript: design not yet proven

Loop has no "customer chose exchange" field; Postscript does not document dedupe on external_id.

Prove on real payloads / a test shop in the spike first.

Group 3

proven on one app first

Build the shared platform: build once, every app uses it

One send queue

Every app sends through one place. A busy partner gets the send later, by its own rules; nothing is silently dropped. Omnisend first.

Connection health

Know which app is fine, flaky or disconnected. When disconnected, stop calling and keep the token. Klaviyo first.

Merchant screen

A banner such as "Klaviyo is disconnected, birthday emails are paused", with Reconnect and Re-sync buttons.

Check partner webhooks

Accept only real data, from the right partner, for the right shop.

Bulk sync

Large actions run as one job, not tens of thousands of calls.

Support tool

See a shop's failed sends and press resend.

App registry

Each app declared in one place, with a test that catches a forgotten edit.

Metrics and alerts

Success rate per app, with an alert when it drops.

Group 4

New apps

New apps: good apps that fit Joy, which competitors already have or Joy can do first

Wave 1

Flow for review apps

Okendo, REVIEWS.io, Junip, Growave. Quick to build; Rivo does it this way too.

Wave 2

Tapcart (mobile app)

5 of 9 competitors have it. Fix the exposed key first.

Waves 2 – 3

Loop (returns)

Listing first. Joy's edge: no competitor gives points when a customer picks an exchange over a refund.

Wave 3

Postscript (SMS)

Top pick, 6 of 9 competitors. The send queue must be done first so no SMS is sent twice.

research

Subscription apps

Skio, Loop Subscriptions, Stay AI: six other loyalty apps are listed; Joy is not.

Now: apply to the Postscript, Loop and Tapcart partner programs; measure which apps Joy's paying shops use.
After 17/02: returns apps (Redo, AfterShip), Gladly, Zendesk (agents adjust points inside the ticket; no competitor has it).

Group 5

Partner API & clean-up

Give partners a safe API, remove what is dead

Wave 1

Partner API F1

The merchant creates one key per partner. A key opens only what is needed: customers, tier, rewards, add/remove points. Each partner has its own call limit, and the key can be revoked at any time. Loop, Tapcart and Okendo will use it.

Decide by 25/11 which plans may use the partner API

·Automated tests for the REST API
·REST v1: count who still uses it, publish the move-to-v2 table
·Announce the REST v1 shutdown now, switch it off after at least 90 days
·Versioning rules and a channel to reach developers
·Smax: process the stored data or remove it
·17 integration cards that are only links: label them as links

7 · Order and timing

One big change a week, watched for at least 3 days before the next

20/11 → 25/11

Wave 0

  • All of Group 1
  • Stop secret leaks, cut false errors, turn on alerts
25/11shops called, Omnisend result known, logs free of secrets, F1 plan rule decided
26/11 → 01/12

Black Friday

  • No deploys
  • Only code on branches and experiments
RULEonly Winn deploys, and only when merchants are losing data
02/12 → 18/12

Wave 1

  • Small fixes
  • Send queue: try with Omnisend
  • Klaviyo connection health (observe only)
  • Order webhooks no longer lost
  • Flow review templates, F1 API
18/12Omnisend sends ≥ 99% successful, no duplicates; no false webhook errors
21/12 → 18/01

Wave 2

  • Holiday weeks: observe only
  • From 04/01: Klaviyo disconnect banner
  • More apps on the send queue, webhook checks
  • Tapcart, Loop listing
18/01merchants see the banner; Tapcart live; tickets vs before
19/01 → 17/02

Wave 3

  • Move the remaining apps
  • Resend merchant webhooks
  • Alerts, support tool
  • Announce REST v1 shutdown; Postscript beta; Loop points
17/02every app has an alert when sends fail

Tết break 04/02 → 10/02. Big changes start only from the week of 04/01.

7.2 · Turning changes on safely

Measure it before turning it on; roll back at once if it breaks

ChangeMust be able to seeWhen to roll backHow to roll back
First small fixesAlerts for dropped data and Klaviyo errorsA new kind of error appearsRedeploy the old version, under 30 minutes
Send queue, app by appSuccessful / retried / failed sends per app; any duplicatesSuccess lower than the old path, or a duplicateSwitch that app off, under 15 minutes
"Disconnected" statusHow many shops are flagged by mistakeFalse flags above the limitSwitch off, run the script that restores the old status
Order webhooks answer real errorsOne week of observe-onlyMore errors than during observationRedeploy the old version, re-register webhooks
Signatures requiredWhich shops still send unsignedReal data blockedTurn on accept-both mode again
SMTP re-encryption, Tapcart key changeDecode errors, email send errors, shop listErrors spikeOld format stays readable; change one shop at a time

Every change merchants can see ships with a help article, a support reply template and one line of explanation in the app.

8 · Decisions needed · only the ones that block work

Five decisions

1

Who staffs new-app work?

Hire, borrow a developer for 8–10 weeks, or run the reliability-first fallback. Winn · before 02/12 · blocks Flow, Loop, Postscript, F1

2

F1 plan and pricing rule

Partners on Free? Shared quota? Custom actions Ultimate-only? Winn + product · 25/11 · blocks F1, Loop, Tapcart

3

Gorgias OAuth client

Still valid? Test: refresh one healthy account. What changed on 05/11 01:46 UTC? Winn · 25/11

4

Health thresholds

False-positive limit, ever auto-disconnect?, is degraded visible? Winn + product + support · before 04/01

5

Ambiguous sends

Avoid duplicates, avoid loss, or reconcile by hand; maximum replay age per partner. Omnisend events are quarantined by default meanwhile. Wave 2 · blocks Postscript

If decision 1 is "nobody": safety and ingress work keep their waves; F1 shrinks to a read-only pilot; Flow moves later; no Postscript or Loop bonus this quarter. The gates are never cut.

9 · Risks

What can go wrong, and how it is already handled

False "disconnected"

One week observe-only first; tokens are never deleted, so a false flag breaks nothing

Duplicate sends

Each send has its own id; sends with an unknown result are held for review

Omnisend unsubscribes customers

If true, bulk sync multiplies the damage → no bulk sync until the test result is in

Tapcart key change

Breaks the old block → change shop by shop, after the new block is live

Resending erased customers

Before sending, always check the customer may still be sent

Log costs grow

Sample only, with a budget, in a separate bucket

Staging missing setup

Tests fail for the wrong reason → prepare queues/indexes on staging first

Tickets rise at first

Errors that were silent now show → compare with that in mind

Outside Joy's control

No developer; slow partner approvals; slow Chatty update

Bigger than planned

Cut scope, never skip a safety step

Next 5 days

Stop the loss
before BFCM.

20/11 baseline + 3 alerts by 25/11 secrets out of logs · probes off · consent verdict · lost orders found · dead-credential shops contacted · F1 rule decided

Source: roadmap.md v7 and integration-usage-counts.md, plans/261119-2330-integrations-roadmap.