Environment
- EDR Server: 7.6.x
- Site Throttling Enabled
Symptoms
- Event and binary ingest rate decreases after server restart
- Growing backlog queue
- Nginx access.log shows a growing number of 503s after restart
- Nginx submit requests show longer response times even if successful
Cause
Sensor site throttling was applied previously, but did not get applied until after restart
Resolution
- The actual behavior is expected with site throttling, the issue is that site throttling wasn't enabled sooner
- If the backlog is unacceptable, site throttling settings will need to be increased or disabled
Additional Notes
- Site throttling reduces the amount of data accepted at a specified time. At those times, it would be expected that less sensor data would be ingested and backlog would increase
- 503 responses from the server are used to indicate that the sensor should retry sending data later. This is expected behavior if site throttle limits have been reached
Related Content
