Deduplication request settings
Repeat requests are API requests that contain the same headers and the same data as the original request. A repeated request can be sent if the connection failed at the moment of receiving the data.
The platform will not accept a repeated request if it changes the data in order to avoid duplicate events. This scheme works by default and does not require additional configuration. By default, the platform deduplicates within 1 day from the first request.
To configure deduplication, specify the following parameters in the main.json configuration file:
"API_DEDUPLICATION_ENABLE": true,
"API_DEDUPLICATION_LEVELDB_PATH": "data/api/deduplication",
"API_DEDUPLICATION_LIVE_TIME_SEC": 86400,
"API_DEDUPLICATION_PERIOD_SEC": 1800
| Parameter | Type | Default value | Required | Description |
|---|---|---|---|---|
| API_DEDUPLICATION_ENABLE | bool | true | No | Enable deduplication |
| API_DEDUPLICATION_LEVELDB_PATH | string | data/api/deduplication | No | LevelDB storage location directory |
| API_DEDUPLICATION_LIVE_TIME_SEC | int | 86400 | No | The maximum number of seconds between identical requests. If more time passes between requests, the request will not be treated as a repeated one. by default – 1 day |
| API_DEDUPLICATION_PERIOD_SEC | int | 1800 | No | Deduplication database cleanup period (in seconds) by default – 30 minutes |