Environment
- Carbon Black Cloud Sensor: 2.12.0
- Linux: All Supported Versions
- CentOS Linux: 7.9.2009 (Core)
Symptoms
Following behavior was observed in threat_hunter_log.txt then suddenly sensor abruptly stops sending events.
FileUploader : Send : Successfully sent archive data to server: /var/opt/carbonblack/psc/blades/E51C4A7E-2D41-4F57-99BC-6AA907CA3B40/events//psc_eventbatch_253_timestamp_1645181656_events_90_size_5775
FileUploader : Send : Successfully sent archive data to server: /var/opt/carbonblack/psc/blades/E51C4A7E-2D41-4F57-99BC-6AA907CA3B40/events//psc_eventbatch_254_timestamp_1645181931_events_146_size_10225
DatafileReputationChecker : LoadReputationData : Loading reputations from file (/var/opt/carbonblack/psc/datafile1)...
DatafileReputationChecker : LoadReputationData : Successfully loaded 73 reputation elements from file (/var/opt/carbonblack/psc/datafile1)
In .\Core file and Diagnostic Report we see the following event_collector message being logged after the kernel module was loaded
kernel: event_collector_2_3_678931: P1 queue full, moving 4096 events to P0. Will holdoff for at least 12288 events (count=1).
The event-avg file also suggests that only a few messages were passed from the kernel module to the user-mode sensor
(the large amount of dropped events):
Stat | Total | 1 min avg | 5 min avg | 15 min avg |
Queued in P0 | 24580 | 0 | 0 | 0 |
Queued in P1 | 11516 | 0 | 0 | 0 |
Queued in P2 | 0 | 0 | 0 | 0 |
Dropped | 7684982 | 26 | 21 | 22 |
All | 713633 | 0 | 0 | 0 |
Process | 58966 | 0 | 0 | 0 |
Modload | 570 | 0 | 0 | 0 |
File | 588317 | 0 | 0 | 0 |
Net | 59075 | 0 | 0 | 0 |
DNS | 3395 | 0 | 0 | 0 |
Proxy | 0 | 0 | 0 | 0 |
Blocked | 0 | 0 | 0 | 0 |
Cause
Sensor's 'DriverThread' is using up all CPU on one core, caused by an infinite loop reading some DNS events from our kernel module driver and would also lead to high CPU usage from the sensor.
Resolution
The fix for this problem is included in Sensor 2.13.2. Upgrade the sensor to the recommended version.
Related Content
