Environment
- EDR Sensor: Windows 7.3.0, 7.3.1
Symptoms
- Sensordiag reports HrError[0x80072F76] for all submits while reserves are successful.
- CBdiag nginx access.logs report HTTP 200 for the sensor reserves but no submit attempts were recorded.
Cause
The API that is failing is WinHttpAddRequestHeaders which is a preparatory step to the submit transmission. This API is getting a return code of 2F76 due to a malformed IP address after checking each IP addresses on the sensor. While recently updating the code, the default IP address was removed.
Resolution
This is sensor issue CB-39155. The fix is expected in Windows sensor version 7.3.2. Please remain on version 7.2.2 if the sensor is encountering this issue.
A) To identify an impacted sensor from a sensordiag, look for the following errors in the Sensor.LOG file:
(w): WinHttpAddRequestHeaders() failed: WinError[0x00002F76: Unknown error 0x2F76]
(e): PutRawEventLog failed HTTPCode[2147954550] HrError[0x80072F76]
(w): Unable to push event log 'eventlog_12218468.10486.log.zip' HrError[0x80072F76]
(w): Unable to properly synch with server HrError[0x80072F76]
B) To identify impacted sensors from the EDR server, or cluster:
1. On each EDR server, gather a list of Sensors that request a 'reserve':
# grep reserve /var/log/cb/nginx/access.log | cut -d '/' -f 7 | sort -u > sensors-reserve.txt
2. Gather a list of Sensors that request a 'submit':
# grep submit /var/log/cb/nginx/access.log | cut -d '/' -f 7 | sort -u > sensors-submit.txt
3. Run 'sdiff' on the 2 resulting files:
# sdiff sensors-reserve.txt sensors-submit.txt > sdiff-results.txt
4. Get your final results:
# grep "<" sdiff-results.txt > final-output.txt
5. Advanced: If many sensors are affected move the them to a new group in bulk and downgrade to win-7.2.2.
In the EDR console, create a new group.
# cut -d ' ' -f 1 final-output.txt > sensorlist.txt
Make a comma-delimited list from sensorlist.txt:
# vi sensorlist.txt (Run ":%s/$\n/,/" to create a comma-delimited list. Then save with ":wq" )
On the Primary EDR server, obtain the new group's ID.
psql cb -p 5002 -c "select id,name from sensor_groups;"
Move the list of sensors to the new group.
psql cb -p 5002 -c "update sensor_registrations set group_id=<new group id> where id in (<cut-and-paste comma-delimited sensor list>;"
Additional Notes
- Some sensors with multiple IP addresses may receive this error, while other sensors with multiple addresses work properly.
- There may be other, more efficient ways to identify impacted sensors than what is described above, but those steps should work.
- Gather sensor logs or use Live Response to manually confirm the presence of the 0x80072F76 error code.
Related Content
