Environment
- Carbon Black Response
- Version 6.1+
Symptoms
- cb-datagrid service fails to start after upgrading to 6.1 or migrating a cluster
- The following is seen during the cluster startup
node0: Starting cb-datagrid: node1: Waiting for cb-datagrid to initialize... node1: . node1: . node1: . node1: . node1: . node1: . node1: . node0: Waiting for cb-datagrid to initialize................. node0: Exec failure (1) |
Cause
cb-datagrid is unable to establish a connection between nodes over port 5701. Datagrid is new to 6.1 and it requires communication between the master and minions and between minions.
Resolution
Note: You can run this command on to get the current cbdatagrid node status:
/usr/share/cb/cbdatagrid cluster -l
- Verify all nodes are listening on port 5701. You can use the cbcheck utility to put these iptables rules in place.
Check here for more information: Incorrect Iptables Rules Prevent Cluster From Communicating Correctly - Verify that /etc/cb/cluster.conf all nodes either use FQDN or IP addresses. Datagrid expects these to match.
- If FQDN doesn't work, try IP for all nodes instead.
- If IP doesn't work, try FQDN for all nodes.
- Verify that DatabaseURL is pointing towards the master in /etc/cb/cb.conf
grep DatabaseURL /etc/cb/cb.conf |
Warning: Backup cb.conf before making any changes:
cp /etc/cb/cb.conf /etc/cb/cb.conf.6.1bak |
- Master:
DatabaseURL=postgresql+psycopg2://cb:AUTHENTICATIONHERE@localhost:5002/cb |
- Minion
DatabaseURL=postgresql+psycopg2://cb:AUTHENTICATIONHERE@MASTERIPADDRESSHERE:5002/cb |
Note that the masters FQDN will also function correctly if it is pointing to the correct location
Related Content
Cb Response: Datagrid Issues