Deduplication request settings

Repeat requests are API requests that contain the same headers and the same data as the original request. A repeated request can be sent if the connection failed at the moment of receiving the data.

The platform will not accept a repeated request if it changes the data in order to avoid duplicate events. This scheme works by default and does not require additional configuration. By default, the platform deduplicates within 1 day from the first request.

To configure deduplication, specify the following parameters in the main.json configuration file:

"API_DEDUPLICATION_ENABLE": true,
"API_DEDUPLICATION_LEVELDB_PATH": "data/api/deduplication",
"API_DEDUPLICATION_LIVE_TIME_SEC": 86400,
"API_DEDUPLICATION_PERIOD_SEC": 1800

Parameter	Type	Default value	Required	Description
API_DEDUPLICATION_ENABLE	bool	true	No	Enable deduplication
API_DEDUPLICATION_LEVELDB_PATH	string	data/api/deduplication	No	LevelDB storage location directory
API_DEDUPLICATION_LIVE_TIME_SEC	int	86400	No	The maximum number of seconds between identical requests. If more time passes between requests, the request will not be treated as a repeated one. by default – 1 day
API_DEDUPLICATION_PERIOD_SEC	int	1800	No	Deduplication database cleanup period (in seconds) by default – 30 minutes