Just Published! Threat Report: Exposing Malware in Linux-Based Multi-Cloud Environments | Download Now

6.1.0 CD UI Performance and Sensor Comms Issues due to OS Netfilter Connection Tracking Max Reached

6.1.0 CD UI Performance and Sensor Comms Issues due to OS Netfilter Connection Tracking Max Reached

Version

Cb Response 6.X (Large Clusters)

Issue

In 6.1, the sensor check-in pipeline was greatly improved allowing for an increase check-in interval. Many additional improvements were that also increased inter-node communications. This increase in communications (connections) required the OS to manage more connections. In certain scenarios and in larger deployments, the OS Netfilter connection tracking were getting maxed out. This is causing connection to be dropped, which is causing issues loading UI pages, resulting in the perception of slow UI performance.

Symptoms

/var/log/messages

kernel: nf_conntrack: table full, dropping packet

Solution

Workaround

Increase nf_conntrack_max setting:

  1. Check current settings
    cat /proc/sys/net/ipv4/netfilter/ip_conntrack_max
    or
    /sbin/sysctl -a|grep -i nf_conntrack_max
  2. Check current connections tracked
    /sbin/sysctl net.netfilter.nf_conntrack_count
  3. If connection max are met or close:
    1. Increase connection tracking max to 262144
      sysctl -w net.netfilter.nf_conntrack_max=262144
    2. Update hashsize proportionally with nf_conntrack_max change (new value should be nf_conntrack_max/4)
      echo 65536 > /sys/module/nf_conntrack/parameters/hashsize
  4. Save changes permanently
    1. Modify or add the following line item to /etc/sysctl.conf:
      net.netfilter.nf_conntrack_count = 262144
Labels (1)
Was this article helpful? Yes No
No ratings
Article Information
Author:
Creation Date:
‎05-23-2017
Views:
424
Contributors