EDR: Syslog notifications are not being sent due rate limiting

[#1]

Environment

• EDR Server: 6.x and Higher

Symptoms

• Watchlist hit notifications or Event not sent to Syslog
• Watchlist hit notifications or Event sent and truncated
• Error in /var/log/messages:

  Apr 29 14:30:07 localhost rsyslogd-2177: imuxsock begins to drop messages from pid <cb-enterprise PID> due to rate-limiting

• The PID seen in /var/log/messages error is the same as the cb-enterprise PID. Verify:

  ps -ef |grep cb-enterprise

Cause

This issue is caused when rsyslog rate limiting is enabled

Resolution

Note: the changes below are external to Carbon Black and it is up to the customer to consider if this should be done or not.

1. Disable rate limiting by modifying /etc/rsyslog.conf to:

   $SystemLogRateLimitInterval 0
   
   $SystemLogRateLimitBurst 1000

2. Restart the Rsyslog service

   service rsyslog restart

Additional Notes

• Setting $SystemLogRateLimitInterval to 0 turns off rate limiting entirely
• Setting $SystemLogRateLimitBurst to 1000 increases the threshold of the number of messages for rate limiting very high

Related Content

EDR: Partial or Truncated Messages using Syslog/Event Forwarder
SystemLogRateLimitBurst Archives - rsyslog