Deduplication of requests
Description
Repeat requests are API requests that contain the same headers and the same data as the original request. A repeated request can be sent if the connection failed at the moment of receiving the data.
The platform will not accept a repeated request if it changes the data in order to avoid duplicate events. This scheme works by default and does not require additional configuration. By default, the platform deduplicates within 1 day from the first request.
The platform performs deduplication according to the following algorithm:
- Checking for the presence of Request-ID header
- If Request-ID is not specified, the checksum of request body and headers is checked
It should be noted that not all requests support deduplication with checksum, and if no Request-ID is specified in such requests, then no duplication check will be performed. You can find a list of such requests in the table below.
Different request sending clients may add their own system headers or other information to the request. This may cause failure of deduplication.
How do you know if the request was taken as a duplicate one?
The response to the repeated request will be the same as the response to the original one. However, the repeated request will contain additional headers:
Duplicate-Key: NeOScRqBgsqj67CR5dYcqvcmPZA=
Duplicate-Request: true
The Duplicate-Key header consists of the Request-ID of the request and the sender token. If Request-ID is not passed, the header will store the hashed request data.
How to send a request so that it is not taken as a duplicate one?
It is recommended to mark each request with a unique identifier. In this way when you change the identifier, you will be able to execute requests, even if they contain the same data.
There are 2 ways to do it:
Change the data in the request body or URL, for example, &request_id=12345
Add the Request-ID header, for example Request-ID: a0fe37f2-b4e0-4e28-a328-4ec7461604fc
API requests that have deduplication
All requests that modify data or activation / launch of entities are checked for repetitions:
| Category | Request |
|---|---|
| Profiles (deprecated methods) | /v1.0/lists/:list_id/lead/unconfirm /v1.0/lists/:list_id/lead/unsubscribe /v1.0/lists/:list_id/lead/tags_add /v1.0/lists/:list_id/lead/tags_remove /v1.0/lists/:list_id/lead/subscribe /v1.0/lists/:list_id/lead/complain /v1.0/lists/:list_id/suspend_all /v1.0/lists/:list_id/unsuspend_all /v1.0/lists/:list_id/import |
| Integrations | /integrations/lpgenerator/profile_import /integrations/tilda/profile_import /integrations/any/profile_import |
| Suppression lists | /v1.1/suppresses/add /v1.1/suppresses/update /v1.1/suppresses/domain_add /v1.1/suppresses/domains_add /v1.1/suppresses/domain_remove /v1.1/suppresses/domains_remove /v1.1/suppresses/email_add /v1.1/suppresses/emails_add /v1.1/suppresses/email_remove /v1.1/suppresses/emails_remove /v1.1/suppresses/phone_add /v1.1/suppresses/phones_add /v1.1/suppresses/phone_remove /v1.1/suppresses/phones_remove |
| Subscriptions | /v1.1/subscriptions/set /v1.1/subscriptions/suspend_all /v1.1/subscriptions/unsuspend_all /v1.1/subscribers/import /v1.1/subscribers/update |
| Segments | /v1.1/segments/add /v1.1/segments/update /v1.1/segments/refresh_counters |
| Campaigns | /v1.1/campaigns/deactivate /v1.1/campaigns/activate /v1.1/campaigns/clone |
| Trigger campaigns | /v1.1/campaigns/triggers/add /v1.1/campaigns/triggers/update /v1.1/campaigns/triggers/start /v1.1/campaigns/triggers/import_and_start /v1.1/campaigns/triggers/clone /v1.1/triggers/start_batch (only Request-ID method) /v1.1/triggers/start_batch_task (only Request-ID method) /v1.1/triggers/import_and_start_batch (only Request-ID method) /v1.1/triggers/import_and_start_batch_task (only Request-ID method) |
| Broadcasts | /v1.1/campaigns/broadcasts/add /v1.1/campaigns/broadcasts/update /v1.1/campaigns/broadcasts/start |
| Regular campaigns | /v1.1/campaigns/regulars/add /v1.1/campaigns/regulars/update /v1.1/campaigns/regulars/start |
| Senders | /v1.1/admin-guide/senders/queue/hold /v1.1/admin-guide/senders/queue/unhold /v1.1/admin-guide/senders/queue/purge /v1.1/admin-guide/senders/update /v1.1/admin-guide/senders/add_to_account /v1.1/admin-guide/senders/add /v1.1/admin-guide/vsenders/add /v1.1/admin-guide/vsenders/update /v1.1/admin-guide/vsenders/clone |
| Nodes | /v1.1/admin-guide/nodes/mta/start /v1.1/admin-guide/nodes/mta/stop |
| Static segments | /v1.1/profiles/segstatic/add /v1.1/profiles/segstatic/remove |
| Profiles | /v1.1/profiles/add /v1.1/profiles/update /v1.1/profiles/import /v1.1/profiles/add_batch /v1.1/profiles/update_batch /v1.1/profiles/import_batch |
| Promocodes | /v1.1/promocodes/attach /v1.1/promocodes/activate /v1.1/promocodes/import /v1.1/promocodes/detach /v1.1/promocodes/update |
| Relations | /v1.1/relations/attach /v1.1/relations/detach /v1.1/relations/edit /v1.1/relations/strengthen |
| Profiles databases | /v1.1/databases/wipe /v1.1/databases/refresh_counters |
| Virtual senders | /v1.1/vsenders/add /v1.1/vsenders/update /v1.1/vsenders/clone |
| Scenarios | /v1.1/workflows/start /v1.1/workflows/import_and_start /v1.1/workflows/import_and_start_batch (only Request-ID method) /v1.1/workflows/import_and_start_batch_task (only Request-ID method) |
| Push event registration | /v1.1/pushes/post_events /v1.1/pushes/crypto_post_events |
| Message templates | /v1.1/templates/add /v1.1/templates/update /v1.1/templates/delete |
| Resources | /v1.1/resources/push_subscribe /v1.1/resources/refresh_counters |
| Data tables | /v1.1/datatables/query_templates/update /v1.1/datatables/query_templates/delete /v1.1/datatables/query_templates/add /v1.1/datatables/query_segments/update /v1.1/datatables/query_segments/delete /v1.1/datatables/query_segments/add |