Just Published! Threat Report: Exposing Malware in Linux-Based Multi-Cloud Environments | Download Now

EDR: How to manually optimize a core

EDR: How to manually optimize a core

Environment

  • EDR Server: 6.x and Higher

Objective

Improve query performance by manually optimizing a core in Solr with large segment counts. For example, segment counts above 50 might lead to significant query performance issues.

Resolution

  1. Determine number of segments to use for the manual optimization command
    1. Find the size of the core
      • ls -lh /var/cb/data/solr5/cbevents/<core_name>
    2. Divide the size in GB by 10 to get a suggested maxSegments value to try. Cores smaller than 200GB should be optimized to 10 segments.
  2. Manually optimize each core substituting <core_name> for the core name and <N> for the number of segments
    • curl 'http://localhost:8080/solr/<core_name>/update?optimize=true&maxSegments=<N>&waitFlush=false'
  3. If the optimization fails due to OutOfMemory exceptions, you can increase the maxSegments value and try step 2 again. If the optimization was successful, you can also try reducing the maxSegments and repeating step 2 again to try to further optimize the core on a second pass.

Additional Notes

  • Consult with Carbon Black Technical Support before manually optimizing cores
  • Cores will optimize quicker if the other EDR services have been stopped and only cb-solr is running
  • The default core segment count after optimization is 10
  • Cores larger than 200GB might not be able to be optimized down to 10 segments due to memory constraints
  • The "writer" core is generally not optimized and expected to have larger than 10 segments before rollover
  • Larger segment counts will affect search performance. Ideally the segments will be as small as possible; cores should never be optimized below 10 segments
  • Cores should automatically be optimized. Manual optimization should only be done if performance issues are present after optimization failed

Labels (1)
Tags (2)
Was this article helpful? Yes No
No ratings
Article Information
Author:
Creation Date:
‎09-09-2020
Views:
724
Contributors