Before — naive chunking
The context does not specify what HTTP header is needed to make an Adyen API request idempotent. It only mentions that the API supports idempotency for safe retry of requests.
Is your RAG chatbot confidently giving wrong answers?
A worked example, not a pitch deck: 18 real questions against Adyen's public payments API docs, measured before and after fixing retrieval. Click any question below to see exactly what changed.
-
Naive chunking
66.7%
Recall@2 · Faithfulness 98.1%
-
+ Structure-aware
83.3%
Recall@2 · Faithfulness 96.3%
-
+ Hybrid search
94.4%
Recall@2 · Faithfulness 98.1%
18 test questions — click to expand before/after
-
What HTTP header do I need to send to make an Adyen API request idempotent? 🔗 Fixed by hybrid search
Ground truth: Send the request with an idempotency-key:
header, where the key is a unique identifier up to 64 characters (UUID recommended). After — hybrid search
Based on the context, you need to send the `idempotency-key` header to make an Adyen API request idempotent.
-
How long is an idempotency key valid for? 🔗 Correct throughout
Ground truth: A minimum validity period of 7 days after first submission.
Before — naive chunking
Based on the context, an idempotency key has a minimum validity period of 7 days after first use.
After — hybrid search
Based on the context, an idempotency key has a minimum validity period of 7 days after first submission.
-
What error code do I get if I resend a request that's already been processed? 🔗 Correct throughout
Ground truth: HTTP 422 or 409, with error code 704 ("request already processed or in progress").
Before — naive chunking
Based on the context, if you resend a request that's already been processed, you would get error code 704 ("request already processed or in progress").
After — hybrid search
Based on the context, if you resend a request that's already been processed, you would get error code 704 ("request already processed or in progress"), which is returned with HTTP status codes 422 or 409.
-
What does the transient-error response header mean? 🔗 Correct throughout
Ground truth: It indicates a temporary, retryable error with no side effects.
Before — naive chunking
Based on the context, the `transient-error: true` HTTP header indicates temporary, retryable errors with no side effects.
After — hybrid search
Based on the provided context, the `transient-error: true` HTTP header indicates a temporary, retryable error with no side effects.
-
What fields appear in every Adyen payments webhook? 🔗 Correct throughout
Ground truth: live, notificationItems, eventCode, success, eventDate, additionalData.
Before — naive chunking
Based on the context, every Adyen payments webhook contains these core fields: - **live**: Indicates test or production environment - **notificationItems**: Array of `NotificationRequestItem` objects (typically one per notification)
After — hybrid search
Based on the context, every Adyen payments webhook contains these core fields: **live**, **notificationItems**, **eventCode**, **success**, **eventDate**, and **additionalData**.
-
What are the three categories of standard (non-disablable) payment webhook events? 🔗 Correct throughout
Ground truth: Transaction Events, Dispute Events, and Payout Events.
Before — naive chunking
Based on the provided context, the three categories of standard (non-disablable) payment webhook events are: Transaction Events, Payout Events, and Non-Default Event Codes.
After — hybrid search
Based on the provided context, the three categories of standard (non-disablable) payment webhook events are: 1. Transaction Events 2. Dispute Events 3. Payout Events
-
What fields do Platforms webhooks use instead of notificationItems? 🔗 Fixed by hybrid search
Ground truth: data, environment, timestamp, and type (e.g. "balancePlatform.accountHolder.updated").
Before — naive chunking
Platforms webhooks use the following fields instead of notificationItems: **data**, **environment**, **timestamp**, and **type**.
After — hybrid search
Based on the context, Platforms webhooks use the following fields instead of notificationItems: **data**, **environment**, **timestamp**, and **type**.
-
Which webhook type tells me about 3D Secure authentication results? 🔗 Fixed by hybrid search
Ground truth: Authentication Webhooks return 3D Secure authentication outcomes.
Before — naive chunking
Authentication Webhooks.
After — hybrid search
Based on the context, the **Authentication Webhooks** notify about 3D Secure authentication outcomes.
-
What webhook will I receive after submitting a referenced refund? 🔗 Correct throughout
Ground truth: A CANCEL_OR_REFUND webhook, containing the validation results and refund status.
Before — naive chunking
Based on the context, after submitting a referenced refund, you will receive a CANCEL_OR_REFUND webhook containing the validation results and refund status.
After — hybrid search
Based on the context, after submitting a referenced refund, you will receive a CANCEL_OR_REFUND webhook containing the validation results and refund status.
-
What's the difference between a REFUND_WITH_DATA webhook and a CANCEL_OR_REFUND webhook? 🔗 Correct throughout
Ground truth: CANCEL_OR_REFUND is sent for referenced refunds; REFUND_WITH_DATA is sent for unreferenced refunds (though some schemes may process these synchronously).
Before — naive chunking
Based on the provided context, the difference is that the CANCEL_OR_REFUND webhook includes essential parameters, while REFUND_WITH_DATA webhooks provide comparable data but apply to unreferenced refund scenarios.
After — hybrid search
Based on the provided context, the difference is that a CANCEL_OR_REFUND webhook is used for **referenced refunds**, while a REFUND_WITH_DATA webhook is used for **unreferenced refund scenarios**.
-
Can a payment be refunded more than once for more than its original value? 🔗 Correct throughout
Ground truth: No — payments cannot be refunded multiple times or exceed 100% of the original value.
Before — naive chunking
Based on the context, no. The context states that "Payments cannot be refunded multiple times or exceed 100% of the original value."
After — hybrid search
Based on the provided context, no. The context states under "Amount Limitations": "Payments cannot be refunded multiple times or exceed 100% of the original value."
-
Which card networks does Adyen automatically check cardholder validity with during a refund? 🔗 Correct throughout
Ground truth: American Express, Discover, Mastercard, and Visa.
Before — naive chunking
Based on the context, Adyen automatically checks cardholder validity with American Express, Discover, Mastercard, and Visa.
After — hybrid search
Based on the provided context, Adyen automatically checks cardholder validity with American Express, Discover, Mastercard, and Visa.
-
What two ways can I submit a referenced refund request? 🔗 Fixed by hybrid search
Ground truth: Terminal API reversal requests (for point-of-sale) and server-to-server requests to /payments/{paymentPspReference}/refunds.
Before — naive chunking
The context does not provide information about ways to submit a referenced refund request. It only describes REFUND_WITH_DATA webhooks for unreferenced refund scenarios and mentions success indicators for CANCEL_OR_REFUND events.
After — hybrid search
Based on the context, the two ways to submit a referenced refund request are: Terminal API reversal requests (for point-of-sale) and server-to-server requests to the `/payments/{paymentPspReference}/refunds` endpoint.
-
If my refund fails, how do I find out why? 🔗 Still failing
Ground truth: The webhook returns success=false along with a reason code explaining the validation failure (e.g. insufficient balance, expired processing window, incorrect currency).
Before — naive chunking
Based on the context, refunds may fail due to insufficient balance, expired processing windows, incorrect currencies, or inadequate merchant account funds. The webhook provides specific information about the failure reason.
After — hybrid search
Based on the provided context, there is no information about how to find out why a refund fails. The context only describes the parameters included in webhooks (such as `success` indicating validation success) but does not explain how to determine the reason for a failure.
-
Do I need to synchronize systems across sales channels to use referenced refunds? 🔗 Fixed by hybrid search
Ground truth: Yes — merchants must synchronize backend systems to access transaction information across all sales channels, including PSP references.
Before — naive chunking
No, you do not need to synchronize systems across sales channels to use referenced refunds. The context states that referenced refunds provide a unique PSP reference identifier that enables "straightforward matching of refunds to original payments regardless of sales channel."
After — hybrid search
Yes, according to the context, merchants must synchronize backend systems to access transaction information across all sales channels, including PSP references, as part of the requirements for referenced refunds.
-
What prevents fraud when a customer returns an item at a different store than they bought it? 🔗 Correct throughout
Ground truth: A referenced refund returns funds to the original payment method used, not to the card or cash presented in-store.
Before — naive chunking
The context does not provide information about what prevents fraud when a customer returns an item at a different store than they bought it.
After — hybrid search
Based on the context, the **Enhanced Fraud Protection** benefit prevents fraud in this scenario. It states that a referenced refund returns the funds to the payment method used for the original payment, not to the card the shopper is presenting in the store or as cash.
-
How many webhook types can a single endpoint receive? 🔗 Fixed by hybrid search
Ground truth: Each webhook endpoint receives only one webhook type; separate endpoints are required for each type.
Before — naive chunking
Based on the context provided, a single endpoint can receive only one webhook type.
After — hybrid search
Based on the context, a single endpoint can receive only one webhook type.
-
What should I use to prevent different API credentials from reading each other's idempotent responses? 🔗 Correct throughout
Ground truth: Use UUID version 4 (random) as the idempotency key.
Before — naive chunking
Based on the context, you should use UUID version 4 (random) to prevent different API credentials from accessing each other's responses.
After — hybrid search
Based on the context, you should use UUID version 4 (random) to prevent different API credentials from accessing each other's responses.