Reliable ERP order delivery: retries, idempotency, and replay
Separate order acceptance from external delivery, then handle uncertain responses with durable state, stable identity, and controlled reconciliation.
In this guide
The timeout after the order was created
The storefront accepts an order and an integration sends it to an ERP. The ERP commits successfully, but the response is lost. The sender sees a timeout. Retrying as a fresh order can create a duplicate; marking the order failed can abandon a successful transaction.
This is an unknown outcome, not proof of failure. Design for it before adding retry loops. Give each business command a stable identity and record delivery separately from storefront acceptance. The customer receipt should describe what commerce has accepted without pretending it knows every downstream result.
Establish the real trigger first
Identify which supported mechanism invokes the integration in the target Znode release. A legacy ERP touchpoint and a Data Exchanges custom processor have different contracts and lifecycles. A project method called 'order created' is not evidence that either mechanism invokes it automatically.
Trace one non-production order from the user action into the actual adapter. Capture the delivery baseline before placing the order, then observe the new delivery record. Do not manually dispatch afterward and mistake that activity for automatic platform behavior. If testing a manual dispatch, label it explicitly and keep the two exercises separate.
At the integration boundary, persist the command before acknowledging it as queued. Scope its stable key to Store, operation, and order, and compare a payload fingerprint on repeated submissions. The durable store owns atomic uniqueness and lease transitions; RetryDecision supplies only the scheduling policy for those persisted attempts.
Classify delivery outcomes before scheduling
RetryDecision.cs implements a bounded retry policy with typed outcomes and actions. Confirmed work completes; permanent rejection needs attention; an unknown outcome goes to reconciliation. Eligible temporary failures receive capped exponential delay and a bounded jitter input.
Call Choose after the provider adapter classifies the actual response. Supply attempts from the durable delivery record and persist the decision under the worker's current lease. Znode's legacy touchpoint and Data Exchange extension points can invoke project-owned delivery logic, but each mechanism needs its own configured trigger and adapter.
Delivery/RetryDecision.cs csharp
using System;
namespace Gcg.DeveloperExamples.Delivery
{
public enum Outcome { Confirmed, TemporaryRejection, PermanentRejection, Unknown }
public enum NextAction { Complete, RetryLater, NeedsAttention, Reconcile }
public sealed class Decision
{
public NextAction Action { get; }
public TimeSpan? Delay { get; }
public Decision(NextAction action, TimeSpan? delay)
{ Action = action; Delay = delay; }
}
public static class RetryDecision
{
public static Decision Choose(Outcome outcome, int attempts, int maximumAttempts,
double jitterFraction)
{
if (attempts < 1 || maximumAttempts < 1 ||
double.IsNaN(jitterFraction) || jitterFraction < 0 || jitterFraction > 1)
throw new ArgumentOutOfRangeException();
if (outcome == Outcome.Confirmed)
return new Decision(NextAction.Complete, null);
if (outcome == Outcome.Unknown)
return new Decision(NextAction.Reconcile, null);
if (outcome == Outcome.PermanentRejection || attempts >= maximumAttempts)
return new Decision(NextAction.NeedsAttention, null);
if (outcome != Outcome.TemporaryRejection)
throw new ArgumentOutOfRangeException(nameof(outcome));
double seconds = Math.Min(300, Math.Pow(2, Math.Min(attempts, 8)));
return new Decision(NextAction.RetryLater,
TimeSpan.FromSeconds(seconds * (0.75 + jitterFraction * 0.25)));
}
}
}
// Classify a timeout after submission as Unknown until provider reconciliation.
// Persist the selected action with the current delivery lease before scheduling.Retries need a provider agreement
A local ledger can prevent duplicate local commands. It cannot alone guarantee exactly one external side effect. Determine whether the provider accepts an idempotency key, exposes lookup by an immutable external reference, or needs an operator to reconcile uncertain results.
Classify temporary unavailability separately from invalid business data. Use bounded retries with backoff and jitter for eligible failures, and cap attempts. A timeout after submission may require a provider read before another write. Preserve an operational state for unresolved outcomes rather than hiding them under a generic failure count.
Exercise commit-then-error and concurrent replay
Build a mock provider that saves an order and deliberately drops the response. Verify that the next attempt uses the same business identity and resolves the existing result according to the mock's declared contract. Then run the same failure scenarios against the selected provider's sandbox contract.
Run two workers against one due delivery, expire a lease, and attempt simultaneous operator replays. Ensure claims and transitions remain controlled. Replay should preserve the original command identity, require authorization, and record why it was requested. Test an altered payload under the same key and a nonretryable validation failure as well.
- Keep raw provider bodies out of public status responses.
- Record sanitized failure categories and correlation references.
- Distinguish a prepared integration from an exercised live transport.
Define completion in operational terms
Agree what 'delivered' means: a transport response, provider acceptance, or a confirmed external order identifier. Later shipment and invoice events are separate milestones. Expose those distinctions so operators know which system owns the next action.
Reliable delivery comes from recoverable decisions rather than optimistic labels. A durable command, a stable identity, a provider-specific reconciliation rule, and a controlled replay process make failures manageable. They also give sales and support teams a more accurate explanation than simply asking a buyer to place the order again.
References and further reading
Independent guidance from GCG. Znode is a trademark of its owner. Examples use fictional data and are not official platform documentation. Suggest a correction.