EDR: Windows Servers May Hang on Shutdown or Restart after Windows Patching

[#1]

Environment

• EDR Windows Sensors: 7.3.0-7.3.1 & 7.4.0 & 7.4.1

Symptoms

After applying Windows updates or security patches, Windows may hang during the shutdown or restart the system.

Cause

Due to the amount of file and registry event modifications, these resources become locked by the sensor.     

Resolution

• These issues are addressed in Windows Sensor 7.3.2 (CB-39524)
• This issue has also been seen in Windows Sensor 7.4.0 and is resolved in 7.4.1 (CB-42042)
• If an upgrade is not possible, and you must remain on 7.3.2, then this Resolution can be followed instead. 
• The file contention issue can be avoided by temporarily disabling the collection of certain events during the time of file contention (which can lead to the hang). 

1.  Prior to modifying Group Settings, note which Event Collections are currently being used. 
2.  After applying Windows updates and prior to restarting, in Sensor Group Settings, disable "Binary module loads", "Binaries" and "Binary info".   Disabling these settings prevents the rehashing of the files, which will avoid the locking. 

This is a article attached image

3.  Reboot the endpoints as required by Windows.
4.  Re-enable the settings back to the original settings and remember to save the Group Settings.

Additional Notes

• The hang is not wholly due to the updates, but due to changes made between EDR sensor versions 7.2.2 and 7.3.0 with regard to how the sensor locks files during the time it processes* them.
• The security updates result in overwriting core files (e.g., user32.dll) that are not usually modified, which reveals the overly-aggressive file locking.

* "processes" or "process the files": refers to copying to store directory, updating on-disk catalog, re-hashing the files (after them having been overwritten by the patch)

Related Content

VMware Carbon Black EDR Windows Sensor 7.3.2 Release Notes