Just Published! Threat Report: Exposing Malware in Linux-Based Multi-Cloud Environments | Download Now

Unexpected log entry or corrupt log causing Solr not to start

Unexpected log entry or corrupt log causing Solr not to start

Version

All versions of Cb Response

Issue

Solr hangs while starting due to a corrupt tlog file

Cause

Usually caused by Solr not being properly shut down while indexing.

Symptoms

A similar error message is found to the following example in /var/log/cb/solr/debug.log

2016-01-01 01:01:01 - [WARN] - from org.apache.solr.update.UpdateLog in coreLoadExecutor-3-thread-1
Unexpected log entry or corrupt log. Entry=1523java.lang.ClassCastException: null

Solution

  1. Create a corrupt tlog directory and move all tlogs to this directory
    mkdir /var/cb/data/solr/cbevents/0/data/tlogcorrupt
    mv /var/cb/data/solr/cbevents/0/data/tlog/tlog.* /var/cb/data/solr/cbevents/0/data/tlogcorrupt/
    1. Repeat the process for any remaining shards
  2. Stop any remaining services
    1. Standalone
      service cb-enterprise stop
    2. Cluster
      /usr/share/cb/cbcluster stop
  3. Remove all cb user owned processes
    killall -KILL -u cb
  4. Start the Services
    1. Standalone
      service cb-enterprise start
    2. Cluster
      /usr/share/cb/cbcluster start
  5. Remove the tlogcorrupt folder once services have started
    rm -rf /var/cb/data/solr/cbevents/0/data/tlogcorrupt
    1. Repeat the process for any remaining shards

Important Note(s)

The tlog are raw documents created for recovery purposes. Since these are now corrupt, they are no longer useful. If they are left behind, solr will take up more disk space than what is specified in /etc/cb/cb.conf.

Labels (1)
Tags (3)
Was this article helpful? Yes No
No ratings
Article Information
Author:
Creation Date:
‎06-08-2016
Views:
1049
Contributors