Threat Report: Exposing Malware in Linux-Based Multi-Cloud Environments | Download Now

EDR: Troubleshooting Rsyslog Configuration

EDR: Troubleshooting Rsyslog Configuration

Environment

  • EDR Server:  All supported
  • Rsyslog

Objective

Confirm rsyslog settings and check connectivity.

Resolution

Steps to troubleshoot rsyslog from the EDR server to a remote rsyslog server.

1.  Confirm rsyslog port is open.
netstat -anp | grep LISTEN | grep <port>  (confirm rsyslog is listening on the port)

To test an open a port on the rsyslog server.
ncat -ul 1234 (where 1234 is the syslog udp port)

2.  Confirm firewalld or iptables allow the traffic on EDR and the syslog server.
On EDR:
     /usr/share/cb/cbcheck firewall -a
     iptables -nL | grep <port>

On rsyslog server
     iptables -nL | grep <port>
3.  Test connectivity.   Send data from EDR to the rsyslog server on the designated port.
On the rsyslog server:
     tcpdump -Xnni <interface> <port> proto 17

Send data from the EDR server:
      /usr/share/cb/cbsyslog -l  (lists the different types) 
      /usr/share/cb/cbsyslog -vfe feed.storage.hit.process
      /usr/share/cb/cbsyslog -vfe watchlist.hit.process
      or send a test file
      ncat -vu --send-only <rsyslog IP> <port> < <filename>
4.  Look for errors in rsyslogd.
rsyslogd -dn 2> /dev/null  | less
5.  Check if the logs are appearing on the EDR server.
tail -f /var/log/cb/notifications/cb-notifications-test.log

6a.  If the events are not appearing on the remote rsyslog server, add spooling to cb-coreservices.conf
if $programname startswith 'cb-notifications-' then -?
DynaFile;CbSyslogStandardFormatWithPID
$WorkDirectory /var/lib/rsyslog  # location of spoolfiles on the disk
$ActionQueueFileName cbtest   # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
& @192.168.10.252:1234;CbSyslogStandardFormatWithPID 
& ~
6b. Restart rsyslog.
systemctl restart rsyslog

7.  For testing purposes, consider adding the remote syslog server to /etc/rsyslog.conf.  Add the following line in the 'forwarding rule' section.  Restart rsyslog service.
*.* @<rsyslog IP>:<port>


 

Additional Notes

  • If rsyslog is using TCP, remember to use @@ instead of @.
  • To configure EDR to rsyslog events, refer to the EDR Integration Guide.  Quick notes below.
cd /etc/rsyslog.d/
cp cb-coreservices.conf cb-coreservices.conf-orig
vi cb-coreservices.conf
(Per the guide, add the following to the end of the first $programname == 'cb-notifications' line)
& @<remote IP address>:<UDP port>;CbLogFormatWithPID & ~
  • If possible, it is more efficient to use Event Forwarding syslog feature.  Why?  Event Forwarding listens on the RabbitMQ bus.

Related Content


Tags (1)
Was this article helpful? Yes No
No ratings
Article Information
Author:
Creation Date:
‎12-01-2023
Views:
260