Cb Respone: cb-datagrid Fails to Start after Upgrading to 6.1

[#1]

Environment

• Carbon Black Response
• Version 6.1+

Symptoms

• cb-datagrid service fails to start after upgrading to 6.1 or migrating a cluster
• The following is seen during the cluster startup
  

  node0: Starting cb-datagrid:
  node1: Waiting for cb-datagrid to initialize...
  node1: .
  node1: .
  node1: .
  node1: .
  node1: .
  node1: .
  node1: .
  node0: Waiting for cb-datagrid to initialize.................
  node0: Exec failure (1)

Cause

cb-datagrid is unable to establish a connection between nodes over port 5701. Datagrid is new to 6.1 and it requires communication between the master and minions and between minions.

Resolution

Note: You can run this command on to get the current cbdatagrid node status:
/usr/share/cb/cbdatagrid cluster -l

1. Verify all nodes are listening on port 5701. You can use the cbcheck utility to put these iptables rules in place.
   Check here for more information: Incorrect Iptables Rules Prevent Cluster From Communicating Correctly
2. Verify that /etc/cb/cluster.conf all nodes either use FQDN or IP addresses. Datagrid expects these to match.
   1. If FQDN doesn't work, try IP for all nodes instead.
   2. If IP doesn't work, try FQDN for all nodes.
3. Verify that DatabaseURL is pointing towards the master in /etc/cb/cb.conf
   

   grep DatabaseURL /etc/cb/cb.conf

   Warning: Backup cb.conf before making any changes:
   cp /etc/cb/cb.conf /etc/cb/cb.conf.6.1bak

   1. Master:
      

      DatabaseURL=postgresql+psycopg2://cb:AUTHENTICATIONHERE@localhost:5002/cb

   2. Minion
      

      DatabaseURL=postgresql+psycopg2://cb:AUTHENTICATIONHERE@MASTERIPADDRESSHERE:5002/cb

      Note that the masters FQDN will also function correctly if it is pointing to the correct location

Related Content

Cb Response: Datagrid Issues