How To Purge CbEvent (Process) Data in Cb Response

[#1]

Version

Cb Response 5.x, 6.1

Topic

How to purge event process data from the solr database in Cb Response

Solution

Warning: This will irrecoverably remove data from the Cb Response server. After removing this data, you will receive a 404 page when attempting to view Alerts that lead to this Process Analysis page and this data will no longer appear in the Process Search

6.1

In 6.1 cbevents data is held in /var/cb/data/solr5/cbevents/ as 3 day rollover cores (by default):

du -h /var/cb/data/solr5 --max-depth=1

du -h /var/cb/data/solr5/cbevents/ --max-depth=1

1. Determine which core you would like to remove (usually this would be the oldest core):
   

   curl "http://127.0.0.1:8080/solr/admin/cores?action=STATUS&wt=json&indexInfo=false&indent=true" | grep name

2. Manually unload the core from the command line with CORE_NAME being the core you would like to unload:
   

   Warning: Alerts generated from process data in this core will lead to 404 pages

   curl "http://localhost:8080/solr/admin/cores?action=UNLOAD&core=CORE_NAME"

   Example:
   

   curl "http://localhost:8080/solr/admin/cores?action=UNLOAD&core=cbevents_2017_05_08_1903"

3. Verify the core is unloaded:
   

   curl "http://localhost:8080/solr/admin/cores?action=STATUS&indexInfo=true&indent=true&wt=json"  | grep name

4. If the core is no longer present, it is now safe to remove the directory:
   

   rm -rf /var/cb/data/solr5/cbevents/CORE_NAME

   Example:
   

   rm -rf /var/cb/data/solr5/cbevents/cbevents_2017_05_08_1903/

   Warning: Unloading the core but leaving it in the directory will mean that solr will not purge this data. Leaving an unloaded core on disk can greatly reduce your retention

5.x

Warning: Services need to be running in order for this command to work. You can run this command for a minute to free enough space for services to start: How To Purge Storefile (Modulestore Binary) Data​

This purges data from /var/cb/data/solr/cbevents/

du -h /var/cb/data/solr --max-depth=1

du -h /var/cb/data/solr/cbevents/ --max-depth=1

1. Determine MaxEventStoreDays setting in /etc/cb/cb.conf as a starting point to purge to:
   

   grep MaxEventStoreDays /etc/cb/cb.conf

2. Determine the cbevent shards (ProcSolrShards) that are present on your local server from /etc/cb/cbcluster.conf:
   

   cat /etc/cb/cbcluster.conf

3. Run this curl command replacing XX with MaxEventStoreDays and SHARD with any shards that are present on the local server. If additional space need to be recovered, reduce this purge time:
   

   Warning: Alerts older than XX days will lead to 404 pages

   curl http://127.0.0.1:8080/solr/SHARD/update?commit=true -H "Content-Type: text/xml" -d "<delete><query>last_server_update:[* TO NOW-XXDAYS]</query></delete>"

   Example:

   curl http://127.0.0.1:8080/solr/0/update?commit=true -H "Content-Type: text/xml" -d "<delete><query>last_server_update:[* TO NOW-30DAYS]</query></delete>"
   curl http://127.0.0.1:8080/solr/1/update?commit=true -H "Content-Type: text/xml" -d "<delete><query>last_server_update:[* TO NOW-30DAYS]</query></delete>"

   Note: This query selects all data older than XX days and deletes it. This command may take a while to complete. Solr will return a response when the purge is done

4. Please open a support case if Solr is taking over an hour to return (Internal Link: Aiding Solr Purging Temporarily With Additional Memory )