Cb Response: Slow Performance with Binary Searches

[#1]

Environment

Cb Response 6.1.2

Symptoms

/var/log/cb/solr/debug.log:

2017-11-01 15:13:24,271 - [ERROR] - from org.apache.solr.servlet.HttpSolrCall in qtp1007251739-230006

null:org.apache.solr.common.SolrException: Exception during facet.field: hostname

at org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:721)

at org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:706)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:660)

at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:731)

at org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:294)

at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:256)

at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:272)

at com.carbonblack.cbfs.solr.handler.CbSearchRequestHandlers$Base.handleRequestBody(CbSearchRequestHan

dlers.java:433)

at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:2102)

at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)

at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)

at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)

at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)

at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)

at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)

at org.eclipse.jetty.server.Server.handle(Server.java:499)

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)

at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)

at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.IllegalStateException: Too many values for UnInvertedField faceting on field hostname

Cause

More than 10 million cbmodule documents slow down binary search performance. Cbmodules (binary search) is any information you have on that binary in your environment. A cbmodules document is represented as a binary details page in the UI. Cbmodules documents are not purged by default.

Resolution

Follow the steps provided in this guide: How To Enable Automated Cbmodule Purging​

Additional Notes

Cbevents (process search) is any information you have on the actions that binary has taken in your environment. A cbevents document is represented as a process analysis page in the UI. Cbevents takes up a majority of the space, and has around 30 days of retention depending on the activity level in an environment.

Cbmodule documents are usually much smaller and so are not purged by default, but if your environment has a high level of unique binaries this can give your instance trouble returning results.