Compare retry policies
Test retry policies against simulated tool failures. See which workflows finish safely, which stop, and what recovery costs in time and calls.
Experiment settings
A four-step checkout with rate limits, timeouts, and malformed receipts. Payment timeouts happen after the charge commits.
Edit experiment settings
Your retry policy
Compared with blind retryTimeout and error handling
Call timeout and total time budget apply to both policies.
The same settings reproduce the same results.
35% faults · 3 attempts · 600 ms backoff · 15s budget
Checkout payments
| Measure | Blind retryBaseline | Your policyGuarded retry | DifferenceYour policy − baseline |
|---|---|---|---|
| Safe completionFinished, with no duplicate writes or accepted malformed outputs | 30.0% | 83.2% | +53.2 ppPercentage points |
| Duplicate writesExtra committed effects, including those in stopped workflows | 72 | 0 | -72 |
| p95 simulated duration95% of workflows finish or stop within this time | 4.99s | 8.78s | +3.79s |
| Tool callsFirst calls and retries across all trials | 1,045 | 1,371 | +326 |
How to read these measures
- Safe completion
- A finished workflow with no duplicate writes or accepted malformed outputs.
- Duplicate writes
- Extra committed effects, including those in workflows that later stopped.
- p95 simulated duration
- 95% of workflows finish or stop within this time. A shorter time can reflect earlier failure.
- Tool calls
- First calls and retries across all trials. Differences are your policy minus the baseline; pp means percentage points.
Blind retry uses 3 attempts and 100 ms initial backoff, without output validation or idempotency.
Compare a single trial
The same workflow under both policies. Select a call for details.
The initial example selects a trial where your policy avoids a duplicate write, then an improved completion. Browse any trial above; totals include every trial.
Workflow completed · 1 duplicate write
Workflow completed
Outcomes across all trials
250 per policySafe = completed, with no duplicate writes or accepted malformed output in this model.
208 safe. 42 stopped.
Your policy finished 208 of 250 workflows, including 0 unsafe completions. Blind retry finished 151, including 76 unsafe completions.
Recovery used 1,371 calls, compared with 1,045 for blind retry. Duration includes retries and waiting, even for stopped workflows.