Threat Report: Exposing Malware in Linux-Based Multi-Cloud Environments | Download Now

Carbon Black Cloud: How to Collect Sensor Performance Logs Manually (Windows)

Carbon Black Cloud: How to Collect Sensor Performance Logs Manually (Windows)

Environment

  • Carbon Black Cloud Sensor: All Supported Versions
  • Microsoft Windows: All supported versions

Objective

Steps to manually collect Process Monitor (Procmon) Logs, Windows Performance Recorder (WPR) ETL Trace, and Sensor Diagnostic logs for troubleshooting issues sensor performance issues

Resolution

Prerequisites
  1. RepCLI Authentication must be enabled. If RepCLI Authentication was not enabled during the initial sensor install then RepCLI Authentication can be enabled on existing sensor installations
  2. Create a folder where all logs will be saved. For the purposes of this document, this location will be referenced as c:\temp although the c:\temp file location can be replaced with whatever location you have specified for saving the log files. 
  3. Download download ProcmonLowAlt.exe.zip at the bottom of https://community.carbonblack.com/t5/Knowledge-Base/All-Products-How-to-Collect-a-low-Altitude-Procm... or download Procmon directly from Microsoft and configure as per Option 2
  4. Ensure wpr.exe exists in C:\Windows\System32\ 
NOTE: If C:\Windows\System32\wpr.exe does not exist, download Debugging Tools for Windows and at the "Select the features you want to download" install prompt deselect all other options except "Windows Performance Toolkit".  WPR.exe will download to C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit by default. Once downloaded copy wpr.exe to C:\Windows\System32\ 

 
Sensor Active:
Reproduce the behavior when Sensor is Active
  1. Open a command line prompt using "Run as Administrator".
  2. Run the following command to start the WPR Trace (ETL) Log and reset counters depending on the estimated time needed to reproduce
Less than 5 Minutes 
C:\temp>wpr -start GeneralProfile -start CPU -start Registry -start FileIO -start DiskIO -start Minifilter -start Network -filemode
More than 5 Minutes 
C:\temp>wpr -start GeneralProfile -start CPU -start Registry -start FileIO -start DiskIO -start Network -filemode
  1. Reproduce the behavior
    1. Document exact steps taken to reproduce to provide to Support
    2. Note local machine time when testing started, and timezone of machine
    3. Note local time when issue occurred
  2. Collect the wpr-active.etl, counters.txt, and psc_sensor.zip 
A. C:\Program Files\Confer>repcli counters > C:\temp\counters.txt -- Change to desired location 
B. C:\temp>wpr -stop c:\temp\wpr-active.etl
C. C:\Program Files\Confer>repcli capture c:\temp -- Change to desired location
Collecting diagnostic data (this may take a few minutes)...
....
Captured diagnostic data in c:\temp\psc_sensor.zip
  1. Rename counters.txt to wpr-active-counters.txt, and psc_sensor.zip to wpr-active-psc_sensor.zip
  2. Run the following commands to collect the Procmon, Sensor logs, Counter logs :
A. C:\WINDOWS\system32>cd c:\program files\confer
B. C:\Program Files\Confer>sc qprotection cbdefense --Result will show either ANTIMALWARE LIGHT or None. If none skip to step H.
C. C:\Program Files\Confer>repcli bypass 1
D. C:\Program Files\Confer>repcli registerProtectedSvcs 0 
E. C:\Program Files\Confer>repcli stopCbServices
F. C:\Program Files\Confer>sc start cbdefense
G. C:\Program Files\Confer>repcli bypass 0
H. C:\Program Files\Confer>repcli deletepolicy 1DED7E47-CE4C-448E-AD01-6F4AC3CE7F5D
I. C:\Program Files\Confer>repcli resetcounters
  1. Launch Procmon.exe
  2. Start collection in Procmon (CTRL+E)
  3. Reproduce the behavior
    1. Document exact steps taken to reproduce to provide to Support
    2. Note local machine time when testing started, and timezone of machine
    3. Note local time when issue occurred
  4. Stop collection in Procmon (CTRL+E) and save the log file
    1. In the Save pop-up window, Select "Events to save: All events" and "Format: PML"
    2. Change the file name to procmon-active.PML
    3. Click "Save" and save to C:\temp 
  5. Run commands to collect counters.txt and psc_sensor.zip  
A. C:\Program Files\Confer>repcli counters > C:\temp\counters.txt -- Change to desired location
B. C:\Program Files\Confer>repcli capture c:\temp -- Change to desired location
Collecting diagnostic data (this may take a few minutes)...
....
Captured diagnostic data in c:\temp\psc_sensor.zip 
  1. Rename counters.txt to procmon-active-counters.txt, and psc_sensor.zip to procmon-active-psc_sensor.zip


Sensor Bypass:
Reproduce the behavior when Sensor is in Bypass
  1. Place the sensor into bypass mode:
 C:\Program Files\Confer>repcli bypass 1
  1. Run the following command to start the WPR Trace (ETL) Log and reset counters depending on the estimated time needed to reproduce
Less than 5 Minutes 
C:\temp>wpr -start GeneralProfile -start CPU -start Registry -start FileIO -start DiskIO -start Minifilter -start Network -filemode
More than 5 Minutes 
wpr -start GeneralProfile -start CPU -start Registry -start FileIO -start DiskIO -start Network -filemode
  1. Reproduce the behavior
    1. Document exact steps taken to reproduce to provide to Support
    2. Note local machine time when testing started, and timezone of machine
    3. Note local time when issue occurred
  2. Collect the wpr-bypass.etl 
C:\temp>wpr -stop c:\temp\wpr-bypass.etl
  1. Launch Procmon.exe
  2. Start collection in Procmon (CTRL+E)
  3. Reproduce the behavior
    1. Document exact steps taken to reproduce to provide to Support
    2. Note local machine time when testing started, and timezone of machine
    3. Note local time when issue occurred
  4. Stop collection in Procmon (CTRL+E) and save the log file
    1. In the Save pop-up window, Select "Events to save: All events" and "Format: PML"
    2. Change the file name to procmon-bypass.PML
    3. Click "Save" and save to C:\temp
  5. Run commands to restore sensor:
A. C:\Program Files\Confer>repcli registerProtectedSvcs 1 -- Unnecessary if was skipped in step Sensor Active Step 6
B. C:\Program Files\Confer>repcli stopCbServices
NOTE: If using a sensor version prior to 3.6.0.1897, edit cfg.ini and remove "UnregisterProtected=True"
C. C:\Program Files\Confer>sc start cbdefense
D. C:\Program Files\Confer>repcli bypass 0
  1. Go to C:\temp, zip the files below and rename zip as perf-logs.zip
    1. wpr-active.etl
    2. wpr-active-counters.txt
    3. wpr-active-psc_sensor.zip
    4. wpr-bypass.etl
    5. procmon-active.PML
    6. procmon-active-counters.txt
    7. procmon-active-psc_sensor.zip
    8. procmon-bypass.PML
  2. Upload the zip to CB Vault
  3. Once the upload completes, please comment on the support case that the data is available for review

Additional Notes

  • Process Monitor records a large amount of information, please make sure to provide an accurate time stamp in step 7, which will help expedite troubleshooting
  • The WPR Trace cannot be collected at the same time as a Procmon Log
  • The repcli unlock <uninstall-code> command is not needed for deleting a policy, only for adding/updating a policy.
  • Both Sensor Service (repmgr stack) and File Filter Driver (ctifile) stack info are required to troubleshoot sensor performance issues. The steps above will ensure that Sensor Service (repmgr stack) info is included in Procmon Logs, but LowAltProcmon will be needed to ensure that File Filter Driver (ctifile) stack information is included in the procmon capture. 
  • If Repcli Repro (Sensor 3.8+) or Sensor Capture Script is used then the above steps are not required to capture Sensor Service (repmgr stack) info; However if Repcli Repro is used procmon.exe must be downloaded directly from Microsoft as the ProcmonLowAlt.zip attached to All Products: How to Collect a low Altitude Procmon Capture cannot be used as this version of procmon is not signed by a valid publisher. RepCLI Repro cannot only invoke procmon when it has been signed by a valid publisher. 

Related Content


Was this article helpful? Yes No
100% helpful (1/1)
Article Information
Author:
Creation Date:
‎08-25-2020
Views:
20209
Contributors