SAP CPI retry mechanism: JMS vs Data Store

Why do I need a retry mechanism at all?

Some failures are transient. A target system is briefly down, the network hiccups, a call is throttled. If you fail the message on the first error, you turn a five-second blip into a manual incident and a frustrated business user. Retry lets the integration recover on its own by reprocessing the message until it succeeds or you give up deliberately.

Retry only makes sense for recoverable errors. For permanent errors, such as a schema violation or a business rejection, retrying just fails again. That distinction comes from the reliability pillar, SAP error handling in Integration Suite, and it is the first thing to get right before choosing a mechanism.

How does retry work in Cloud Integration?

Quick answer: you persist the message in a store, then reprocess from that store, either automatically (JMS) or on a schedule (Data Store).

The core idea is the same for both mechanisms: decouple receiving the message from delivering it by persisting it first, then reprocess from that persistent store. SAP Cloud Integration gives you two stores for this, JMS queues and the Data Store, and they behave differently in ways that matter operationally.

JMS queues vs Data Store: the comparison

Neither is better in the abstract. JMS is the natural choice when you want hands-off automatic retry for continuous flows. Data Store is the natural choice when you want deliberate control over when reprocessing happens, or when you want to hold payloads and replay them on your own schedule.

How do I implement JMS-based retry?

The pattern is to receive the message, place it on a JMS queue, and have a second step consume from the queue and deliver to the target. On a recoverable error during delivery, the message goes back to the queue and the adapter retries with backoff, incrementing SAPJMSRetries. When the attempt count crosses your threshold, route the message to a dead-letter queue.

The classic mistake: handling the exception so thoroughly that the runtime considers the message done. If you want JMS to retry, do not finalise the message in the exception path. See the exception subprocess guide for how to structure recoverable versus permanent handling.

How do I implement Data Store-based retry?

Here you write the payload to a Data Store on failure, then run a separate reprocessing iFlow on a timer. That iFlow reads pending entries, tries delivery again, and either removes the entry on success or increments SAP_DataStoreRetries on failure. When retries are exhausted, move the entry to a dead-letter store. This gives you explicit, auditable control and is handy when a target is only available in certain windows, or when you want an operator to review before replay.

How should I set backoff and limits?

Retry is not repeat immediately and forever. Two parameters make it safe:

• Backoff: increase the wait between attempts so a target that is briefly down is not bombarded. Exponential or stepped backoff is common.
• Maximum attempts: cap retries per interface based on how the target behaves, then dead-letter. A payment gateway and an internal reporting feed deserve different limits.

What about dead-letter handling?

Both mechanisms need an exit. Decide a maximum attempt count, and when a message exceeds it, move it to a dead-letter destination with full context: the original payload, headers, the last error, and a correlation ID. A dead-letter path is useless without an alert, so wire it into monitoring and alerting so a human is told there is something to look at, rather than discovering a silent backlog weeks later.

Why is idempotency non-negotiable?

Retry means the same message can reach the target more than once. If creating the same sales order twice creates two orders, retry has made things worse, not better. Make receivers idempotent: use a unique business key, an idempotent upsert, or Cloud Integration's idempotent-process support so repeated processing produces the same result. Without idempotency, you should not enable automatic retry at all.

What are the anti-patterns?

• Infinite retry on an error that will never succeed.
• Immediate retry with no backoff, overwhelming a recovering target.
• Retry without idempotency, creating duplicates.
• A dead-letter store nobody monitors, which is just lost data with extra steps.

How do I avoid overwhelming a recovering target?

Retry without restraint can turn a brief outage into a longer one, because a flood of retries hits the target just as it is trying to recover. Two techniques help. Exponential backoff spaces attempts further apart over time, giving the target room to recover. A circuit-breaker mindset goes further: if a target has failed repeatedly, stop sending for a cooling-off period rather than retrying each message individually. In Cloud Integration you approximate this by controlling the pace of the reprocessing iFlow (for Data Store) or the queue consumer, and by alerting early so an operator can pause a flow if a target is clearly down.

A worked example: an order interface

Quick answer: persist the order on a JMS queue, deliver with backoff, cap at a sensible attempt count, dead-letter the rest, and make order creation idempotent.

Consider orders flowing from a web shop into S/4HANA. A pragmatic design: the inbound step validates and places the order message on a JMS queue; a second step consumes and calls the S/4HANA order API. On a timeout or 5xx, the message returns to the queue and retries with backoff, incrementing SAPJMSRetries. After, say, a handful of attempts, it moves to a dead-letter queue and raises an alert. Order creation uses the shop's order number as an idempotency key, so even if a message is delivered twice, S/4HANA does not create a duplicate order. This single design applies everything in this article: persistence, automatic retry, backoff, a capped limit, dead-letter, alerting and idempotency.

How do I decide the retry limit per interface?

There is no universal number, but a simple framework helps. Ask: how long does this target typically take to recover from a blip, and how time-sensitive is the message? A payment authorisation that must be fresh deserves few, fast retries then a hard fail; an overnight master-data sync can tolerate many, widely spaced retries. Set the limit and backoff from the target's real behaviour, document it in the interface runbook, and revisit it if the operational picture changes.

How does this connect to monitoring?

Retry and monitoring are two halves of one loop. A growing RETRY backlog is your earliest signal that a target is degraded, often before anything has fully failed. Dead-letter arrivals are your signal that retry has given up and a human is needed. Both must be alerted, as described in how to monitor SAP Integration Suite. Retry that nobody watches is just a slower way to lose data.

Which JMS sender settings control retry behaviour?

Quick answer: four settings matter most: the retry interval, exponential backoff, the maximum retry interval, and dead-letter handling. Together they decide how often a failed message is retried and when the platform stops hammering a struggling target.

JMS dead-letter handling targets unexpectedly interrupted processing, such as a node crash. Ordinary processing errors continue to retry under your normal retry settings unless you explicitly model a limit, typically by checking the SAPJMSRetries header and ending the message once it passes your threshold.

How long can a message wait in a Data Store?

Quick answer: a Data Store entry lives until its expiration period, which defaults to 30 days and can be set up to 180 days. You can also raise an alert if an entry is not fetched within a retention threshold.

When you write to a Data Store, two settings shape the operational picture. The retention threshold for alerting defines how many days an entry may sit unprocessed before the platform flags it, which is an early warning that your reprocessing job is not doing its work. The expiration period defines when entries are deleted, and SAP requires it to be at least twice the alerting threshold.

When should I not use JMS or Data Store retry?

Quick answer: avoid queue-based retry for synchronous calls where a caller is waiting, for targets that cannot handle duplicates, and for flows where strict ordering matters unless you design for it.

• Synchronous request-reply: the caller is waiting for an answer, so return a clear error and let the caller decide; queueing silently defeats the contract.
• Non-idempotent targets: if the receiver cannot detect duplicates, retry can create them. Fix idempotency first.
• Strictly ordered sequences: retries on a non-exclusive queue can let later messages overtake earlier ones. Use exclusive access on the JMS sender, or carry a sequence number and let the receiver enforce order.
• Permanent errors: a mapping or validation failure will fail identically on every attempt. Classify it and stop, as described in the error-handling pillar.

A short decision guide

• Continuous, high-volume asynchronous flow that just needs to survive blips: JMS queue.
• Controlled, scheduled reprocessing, or you need to hold and replay payloads: Data Store.
• Either way: apply backoff, cap attempts, dead-letter the rest, alert on it, and make the receiver idempotent.

For the wider picture of how retry fits error handling, return to the reliability pillar, see the exception subprocess guide, or go back to the platform guide.