Differences between revisions 29 and 30
Revision 29 as of 2013-05-09 19:05:25
Size: 10336
Editor: 81
Comment: Explain how to switch to JUL
Revision 30 as of 2013-05-10 00:25:11
Size: 10443
Editor: 81
Comment: Update log4j-jul procedure
Deletions are marked like this. Additions are marked like this.
Line 31: Line 31:
 3. Replace slf4j-log4j12-1.6.6.jar with the new slf4j-jdk14-1.6.6.jar (note: you must have only one of them)
 4. Use your old logging.properties
 3. Delete these JARs from your lib folder: slf4j-log4j12-1.6.6.jar, jul-to-slf4j-1.6.6.jar, log4j-1.2.16.jar
 4. Add these JARs to your lib folder (from slf4j zip): slf4j-jdk14-1.6.6.jar, log4j-over-slf4j-1.6.6.jar
 5. Use your old logging.properties

Solr's Logging Mechanism

Starting with Solr 1.4, the Solr Code base compiles against the "SLF4J" API.

The Solr Admin console has a screen for changing the logging level globally. This is a transient setting good for doing diagnostic work, but does not persist after restart.

Solr 4.3 and above

These versions do not include any logging jars in the WAR file. They must be provided separately. The Solr example for these versions includes jars (in the jetty lib/ext directory) that set up SLF4J with a binding to the Apache log4j library. By changing the jar files, you can easily switch to an alternate logging mechanism (like logback) or use upgraded versions of the logging components.

To get the same logging setup in e.g. Tomcat as with the example Jetty server, you need to do the following

  1. Copy the jars from solr/example/lib/ext into tomcat/lib to setup SLF4J

  2. Copy the logging config from solr/example/resources/log4j.properties into some location, e.g. tomcat/lib and edit the file for you preferred log destination

  3. Optionally, if you did not place log4j.properties on the classpath, set java option -Dlog4j.configuration=file:///path/to/log4j.properties

If the system cannot find your logging configuration, you may get erros like this:

log4j:WARN No appenders could be found for logger (org.apache.solr.servlet.SolrDispatchFilter).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

It might help to start the server with -Dlog4j.debug=true to see more details.

Switching from Log4J logging back to Java-util logging

If you want to stick to the old-style logging, here's what to do (assuming solr4.3 - note that SLF4J version may be different for newer versions)

  1. Download slf4j version 1.6.6 (the version used in Solr4.3.x). http://www.slf4j.org/dist/slf4j-1.6.6.zip

  2. Unpack, and pull out the file slf4j-jdk14-1.6.6.jar (This is the JUL logger impl)
  3. Delete these JARs from your lib folder: slf4j-log4j12-1.6.6.jar, jul-to-slf4j-1.6.6.jar, log4j-1.2.16.jar
  4. Add these JARs to your lib folder (from slf4j zip): slf4j-jdk14-1.6.6.jar, log4j-over-slf4j-1.6.6.jar
  5. Use your old logging.properties

Solr 1.4 to 4.2.1

These versions include SLF4J jars in the WAR File that bind SLF4J to JDK standard logging (JUL, the java.util.logging classes). An overview of how JUL can be configured at the JVM level can be found here:

http://java.sun.com/j2se/1.5.0/docs/guide/logging/overview.html

Many servlet containers also provide alternate log configuration options in their configuration files. You should consult your servlet container documentation to see what options are available.

Users who want an alternate logging implementation (log4j, logback etc) will need to repackage the .war file to remove the existing SLF4J jar files and replace them with the correct jar files. You can do this by extracting and repacking the .war file, or by building the war file from source without the jar files.

<!> From Solr3.6 there is a build target ant dist-war-excl-slf4j which will automatically package the WAR file with only the slf4j-api JAR and no bindings. Using this war file you can place your slf4j-XXX.jar binding file of choice in your servlet container external lib folder. Note that the slf4j jar(s) that you add must match the version number of the slf4j-api jar file that is contained within the war. For Solr 3.6, that version is 1.6.1. For Solr 4.0, it is 1.6.4.

<!> From Solr4.1 there are two build targets related to choosing your own slf4j binding. These are ant dist-excl-slf4j and ant dist-war-excl-slf4j. The first target builds the usual dist files in addition to the WAR file, the second just builds the WAR file. These build targets will package the WAR without any slf4j jars at all, so you must include all relevant slf4j jars in your container external lib directory. These build targets were removed from Solr4.3 and later.

Here are the slf4j jars that are in the WAR file when you build it without one of the special targets:

  • slf4j-api-1.6.1.jar
  • slf4j-jdk14-1.6.1.jar
  • jcl-over-slf4j-1.6.1.jar
  • log4j-over-slf4j-1.6.1.jar

The first file is the primary slf4j library. The second is the slf4j binding - in this case . Some of Solr's dependent components use other logging methods - jcl and log4j. The remaining jar files above intercept calls to these other logging methods and inject them into slf4j.

When you change the slf4j binding, you must include the primary library (the -api jar), the binding for the logging method that you wish to use, the jar(s) for that logging method, and relevant "-over-slf4j" jars. Note that if you choose either jcl or log4j for your binding, you must leave out the matching "-over-slf4j" jar.

More Info: http://www.slf4j.org/

Solr 1.0 to 1.3

These versions use JDK standard logging directly, not through SLF4J.

Customizing Logging

With Example Jetty Instance

If you don't know much about JDK containers or servlet containers and want a quick recipe for modifying the logging settings for default Solr example/ setup, see LoggingInDefaultJettySetup.

Using log4j with Solr from source, 4.2.1 or earlier

Prior to Solr4.3, using a different SLF4J binding (such as log4j) meant either repackaging the war file or building it yourself from source. To build it yourself, you need to obtain a nightly version or get it from svn. Once it's downloaded, extract it, cd to the solr directory in the extracted directory, then compile it using ant dist-war-excl-slf4j or and dist-excl-slf4j depending on the version.

Here are the jars you will need to put in your container external lib folder, assuming that you get the 1.7 version of slf4j. The last one comes from the log4j website, not slf4j:

  • slf4j-api-1.7.5.jar
  • slf4j-log4j12-1.7.5.jar
  • jcl-over-slf4j-1.7.5.jar
  • log4j-1.2.17.jar

http://www.slf4j.org/download.html http://logging.apache.org/log4j/1.2/

Note that log4j must be configured before it will work. One way to do this is to pass an argument like the following to java:

-Dlog4j.configuration=file:etc/log4j.properties

http://logging.apache.org/log4j/1.2/manual.html#defaultInit

Sample log4j.properties file:

#  Logging level

log4j.rootLogger=WARN, file


#- size rotation with log cleanup.

log4j.appender.file=org.apache.log4j.RollingFileAppender

log4j.appender.file.MaxFileSize=4MB

log4j.appender.file.MaxBackupIndex=9


#- File to log to and log format

log4j.appender.file.File=logs/solr.log

log4j.appender.file.layout=org.apache.log4j.PatternLayout

log4j.appender.file.layout.ConversionPattern=%-5p - %d{yyyy-MM-dd HH:mm:ss.SSS}; %C; %m\n

Using Logback with the Solr 3.5 binary distribution

Here are some details for implementing logback (http://logback.qos.ch/). As you may know, Logback was created by the original author of Log4j. It has several enhancements including Filters and Custom Appenders.

1. Get Solr 3.5 http://www.apache.org/dyn/closer.cgi/lucene/solr/3.5.0 and extract the tar/zip file on your system.

Note: $ORIG is your top level directory for Solr.

# cd $ORIG

# tar xzvf apache-solr-3.5.0.tgz

2. Copy the solr.war file to a new location

# mkdir -p /tmp/solr

# cd ./example/webapps

# cp solr.war /tmp/solr

# cd /tmp/solr

# jar xvf solr.war

# rm -f solr.war

3. Remove the slf4j-jdk14-1.6.1.jar file

# cd /tmp/solr

# rm -f ./WEB-INF/lib/slf4j-jdk14-1.6.1.jar

4. Copy logback classic files

Get the file: http://logback.qos.ch/dist/logback-1.0.1.tar.gz

# tar xzvf logback-1.0.1.tar.gz

# cd logback-1.0.1

# cp logback-classic-1.0.1.jar /tmp/solr/WEB-INF/lib

# cp logback-core-1.0.1.jar /tmp/solr/WEB-INF/lib

Note: If you wanted to upgrade SLF4J from 1.6.1 to 1.6.4 you would do the following...

a. Get the SLF4J from http://www.slf4j.org/dist/slf4j-1.6.4.tar.gz
b. Extract the tar into a temporary directory
c. delete the old versions:

# rm /tmp/solr/WEB-INF/lib/jcl-over-slf4j-1.6.1.jar

# rm /tmp/solr/WEB-INF/lib/log4j-over-slf4j-1.6.1.jar

# rm /tmp/solr/WEB-INF/lib/slf4j-api-1.6.1.jar

d. Copy the new ones

# cp jcl-over-slf4j-1.6.4.jar /tmp/solr/WEB-INF/lib

# cp log4j-over-slf4j-1.6.4.jar /tmp/solr/WEB-INF/lib

# cp slf4j-api-1.6.4.jar /tmp/solr/WEB-INF/lib

5. Rebuild the war

# cd /tmp/solr

# jar cvf solr.war *

6. Copy the war back to your example/webapps directory

# cp solr.war $ORIG/apache-solr-3.5.0/example/webapps

7. Create resources/logback.xml file

# cd $ORIG/apache-solr-3.5.0/example

# mkdir resources

copy logback.xml into this directory.

8. Restart Solr 3.5

Example logback.xml.

<configuration debug="true">

  <appender name="FILE" class="ch.qos.logback.core.FileAppender">

    <file>/var/log/solr.log</file>

    <encoder>

        <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>

    </encoder>

  </appender>

  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">

    <encoder>

      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>

    </encoder>

  </appender>

  <logger name="org.apache.solr.handler.dataimport.DocBuilder.level" level="SEVERE" additivity="false">

    <appender-ref ref="FILE" />

  </logger>

  <logger name="org.apache.solr.handler.dataimport.ThreadedEntityProcessorWrapper.level" level="SEVERE" additivity="false">

    <appender-ref ref="FILE" />

  </logger>

  <logger name="org.apache.solr.core.SolrCore" level="INFO" additivity="false">

        <appender-ref ref="FILE" />

  </logger>

  <logger name="org.apache.solr" level="INFO" additivity="false">

    <appender-ref ref="FILE" />

  </logger>

  <root level="OFF">

    <appender-ref ref="STDOUT" />

  </root>

</configuration>

SolrLogging (last edited 2014-04-19 22:21:01 by 209)