Develop Reference

Log SQL to file with logback & hibernate

I am trying to get the SQL logged directly to a file when running the dev profile.
This is my logback.xml
<configuration>
<property name="SQL_LOG_FILE" value="${LOG_PATH:-${LOG_TEMP:-${TMPDIR:-/tmp}}}/${HIBERNATE_LOG_FILE:-hibernate.log}"/>
<springProfile name="dev">
<appender name="SQLDEBUG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${SQL_LOG_FILE}</file>
<encoder>
<charset>utf-8</charset>
<Pattern>%-5level %logger{0} - %msg%n</Pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>utf-8</charset>
<Pattern>[%highlight(%p)] [%t] %c - %m%n</Pattern>
</encoder>
</appender>
<logger name="org.hibernate.SQL" additivity="false" level="DEBUG">
<appender-ref ref="SQLDEBUG"/>
</logger>
<logger name="org.hibernate.type.descriptor.sql" additivity="false" level="TRACE">
<appender-ref ref="SQLDEBUG"/>
</logger>
</springProfile>
<root level="${logback.loglevel}">
<springProfile name="dev">
<appender-ref ref="CONSOLE"/>
</springProfile>
</root>
I have removed the prod profile settings for simplicity.
The logger for hibernate is inside the dev profile because I don't want it enabled in prod.
I have tried many combinations of these org.hibernate settings. This version generates SQL logs but only dumps them to console, not the log file. Some general startup information is added to the log file but no SQL.
If I change org.hibernate.type.descriptor.sql to org.hibernate.type there is a lot of stack trace logs that are added directly to the file, but no SQL.
Some posts recommend using org.hibernate.SQL level=TRACE but that did not seem to change anything.
I also tried putting the logger outside of the dev profile but that also did not change the results.
There is a lot of information for enabling logback & hibernate for simple console output but not for sending the SQL to its own log file.
I also tried enabling hibernate.SQL=DEBUG in IntelliJ but that makes a lot of SQL on the console, I need to not do that.
I have been try

I doubt you spring profile is being used. To get this to work rename the logback.xml file to logback-spring.xml, allowing the springProfile tag to be used. In this tag, a name can be provided that can be set via properties, environment variables, or VM options. Below is how you can set the springProfile name to dev , which has been used to represent a development environment.
To set in application.properties or as an environment variable:
spring.profiles.active=dev
Or as a VM option:
-Dspring.profiles.active=dev
Also, modify your root-level tag to be inside the spring profile tag:
<springProfile name="dev">
<root level="${logback.loglevel}">
<appender-ref ref="CONSOLE"/>
</root>
</springProfile>

Ways to configure logs other than log4j xml, properties file, and source code (mainly java)?

I'm looking at the source code of some application. It is using Spring framework, Apache Tiles, JSP, Log4j, java, javascript, jquery, jqplot, Jsch, and etc.
I know where the logs are created. (a/b/logs) However, when I look at source code, I don't know how logs are created under the folder name 'logs'. I looked at log4j.xml, web.xml , property files. I found the code for how the path 'a/b' is created, but not logs. Also that folder has 4 types of logs. And they are name in a pattern like access.20181227001 , errors.20182111. I want to know where I have to look to find how the logs are created in this manner.
Log4J.xml
<!-- Appenders -->
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %5p [%c] %m%n" />
</layout>
</appender>
<appender name="console-infolog" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %5p %m%n" />
</layout>
</appender>
<!-- Application Loggers -->
<logger name="com.dsmentoring.chakan" additivity="false">
<level value="debug" />
<appender-ref ref="console"/>
</logger>
<!-- 3rdparty Loggers -->
<logger name="org.springframework.core">
<level value="error"/>
</logger>
<!-- Bean logger -->
<logger name="org.springframework.beans">
<level value="error"/>
</logger>
<!-- Context logger -->
<logger name="org.springframework.context">
<level value="error"/>
</logger>
<!-- Web logger -->
<logger name="org.springframework.web">
<level value="error"/>
</logger>
<logger name="org.springframework.ldap" additivity="true">
<level value="error"/>
</logger>
<!-- LDAP logger -->
<logger name="com.unboundid.ldap" additivity="true">
<level value="error"/>
</logger>
<!-- Root Logger -->
<root>
<priority value="off" />
<appender-ref ref="console" />
</root>
To sum up :
1) Is there a way to configure where logs are created and how they are created (4 types of logs) Other than log4j.xml, xml files and property files? I looked at all the java, jsp, js code but can't seem to find the configuration for logs. So I want to know if there are other ways to do that or where I should look for those configuration.
2) The 'logs' folder is possibly default for log4j?
ldap.properties
#LDAP Connection Info
ldap.host=192.168.0.17
ldap.port=22389
ldap.userName=cn=directory manager
ldap.password= 9074B18A0DE2D50C068D37B60BE5DFDE
ldap.baseDN=o=sso30root
ldap.defaultLoadSize=1000
ldap.start=start-ds
ldap.stop=stop-ds
ldap.workdir=/home/KB_openDJ // logs are created under this path
// /home/KB_openDJ/logs
In other java class, they use this.
#Value("${ldap.workdir}")
private String WORK_DIR;
// I ommited many lines in between
try{
diskUsage = sigar.getFileSystemUsage(WORK_DIR);
diskIOInfo.setDiskRead((int)(diskUsage.getDiskReadBytes()));
diskIOInfo.setDiskWrite((int)(diskUsage.getDiskWriteBytes()));
}catch(SigarException sigarEx){
log.debug("Disk Usage info load Error : " + sigarEx.getMessage());
}
I used the 'Search' feature in Eclipse many times. ( logs, WORK_DIR, and 4 types of log name, and many others. I am unable to locate the code about logging configuration. :(
My log4j version :
1.2.15

Ok, it seems that there's a solution which might help you. It requires some debugging of static initalization block of org.apache.log4j.LogManager class. This class is responsible for loading logger configuration file. Here's a documention link which thoroughly describes initalization process: link.
Here's an excerpt from a LogManager source file:
String configurationOptionStr = OptionConverter.getSystemProperty(DEFAULT_CONFIGURATION_KEY, null);
String configuratorClassName = OptionConverter.getSystemProperty(CONFIGURATOR_CLASS_KEY, null);
What I'm trying to show you here, is that your logger configuration file might be specified as a JVM option supplied to your application-server. That's why you are not able to determine actual file being used.
Even if this approach fails, I'd recommend you to investigate the appenders list retrieved at run-time. Here's a stackoverflow thread which describes how to do so: link.

Disable highly verbose logging

I am getting this highly verbose logging and I want to get rid of it.
1:39:20.187 [main] INFO c.v.c.c.ConfigManager - End XML Read
11:39:20.187 [main] INFO c.v.c.c.ConfigManager - The content of ConfigCfg_pdMetering.xml is ConfigCfg [xxx=yyy]
11:39:37.335 [Thread-1] DEBUG o.a.h.c.p.RequestProxyAuthentication - Proxy auth state: UNCHALLENGED
11:39:37.335 [Thread-1] DEBUG o.a.h.i.c.DefaultHttpClient - Attempt 1 to execute request
The logging comes from a jar I have imported in the project. I modified the logback.xml and log4j.properties in the jar using "jar uf"
logback.xml now looks like:
<?xml version="1.0"?>
<configuration debug="false">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are by default assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
</pattern>
</encoder>
</appender>
<logger name="com.hccl" level="ERROR"/>
<logger name="org.apache.http.wire" level="ERROR"/>
<root level="debug">
<appender-ref ref="STDOUT"/>
</root>
</configuration>
and log4j.properties:
log4j.rootCategory=INFO, stdout
log4j.logger.org.springframework.ws.client.MessageTracing.sent=ERROR
log4j.logger.org.springframework.ws.client.MessageTracing.received=ERROR
log4j.logger.org.springframework.ws.server.MessageTracing=ERROR
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%p [%c{3}] %m%n
log4j.logger.httpclient.wire=ERROR
I imported this modified jar and imported it to my project. This has not affected the logging.
Also I am getting the following in the console:
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
I do not know why I am seeing this warning.

Muting Apache http
The packages you want to mute are c.v.c.c. and o.a.h. Those are abbreviations. Also, you specified <logger name="org.apache.http.wire" level="ERROR"/> which doesn't match the above (take the first letters of each package name).
You might have more luck with
<logger name="org.apache.http" level="ERROR"/>
to shut down all org apache http logging (which is probably o.a.h.*).
Muting c.v.c.c
To mute the first two lines beginning with c.v.c.c.ConfigManager, you need to find out the classes. You can do this either by replacing %logger{36} with just %logger to have the full package and class name printed. Or perhaps it's your own class. In this case you can just putt a corresponding logger entry into your file by yourself.
Warning for slf4j-Binding
If slf4j outputs warnings about "actual bindings", you have probably more than one slf4j binding in your classpath. As I read from your question, you try to use both logback and log4j, which won't work - slf4j decides to use logback only. Remove log4j and the log4j.properties. If they get included by maven dependencies, specify exclude tags for those dependencies.

How to configure log4j to enable/disable logging to a file?

I've written up a small console utility. Throughout the application, I have used slf4j/log4j to log debug, trace, info statements to the console as I was developing it. Now that I am done development and want to release it, I would like to disable all output to the console except for a couple of specific loggers and send all the remaining loggers to a text file.
In theory, this should be fairly easy. I have created 2 appenders (one ConsoleAppender and one FileAppender). I have assocaited the ConsoleAppender with the specific loggers want on the Console and the FileAppender at the Root level. This works fine.
My issue now, is if I want to completely disable the FileAppender, I cannot figure out how to do it. If I add a DenyAllFilter to the FileAppender, it will prevent anything from being written to the file, but the file is still created. If I remove the FileAppender from the ROOT logger, I get Log4J writting to StdErr:
log4j:WARN No appenders could be found for logger (org.springframework.context.support.ClassPathXmlApplicationContext).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
If I set the ROOT level to OFF, the individual loggers that I have set custom debug levels to still output.
If I set the LoggerRepository Level to OFF, then all loggers (those I want to go to STDOUT and the File) are all shut off.
Is there any way to do this easily? Ideally, I want to be able to disable the file appender altogether, however provide a command line switch to enable it if desired.
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p %c{5} - %X{messageId} - %m%n" />
<!-- <param name="ConversionPattern" value="%m%n" /> -->
</layout>
</appender>
<appender name="LOGTXT" class="org.apache.log4j.FileAppender">
<param name="file" value="lss-client.log" />
<param name="append" value="false" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p %c{5} - %X{messageId} - %m%n" />
</layout>
<!-- <filter class="org.apache.log4j.varia.DenyAllFilter"/> -->
</appender>
<!-- Spring Loggers -->
<logger name="org.springframework">
<level value="INFO" />
</logger>
<logger name="org.springframework.ws.client.MessageTracing.sent">
<level value="TRACE" />
</logger>
<logger name="org.springframework.ws.client.MessageTracing.received">
<level value="TRACE" />
</logger>
<logger name="com.cws.cs.Client" >
<level value="INFO" />
<appender-ref ref="STDOUT" />
</logger>
<root>
<level value="INFO" />
<appender-ref ref="LOGTXT" />
</root>
</log4j:configuration>
Thanks!
Eric

Pretty sure Slf4J/Log4J creates the log file immediately when you configure a FileAppender.
I think your best bet here is to have 2 different and complete Log4J configurations and pick which one you load at startup based on your command line parameter.
In PseudoCode:
public static void main(String[] args){
//if log to file arg = true
DOMConfigurator.configure("logToFile.log4j.xml");
//else
DOMConfigurator.configure("logToConsoleOnly.log4j.xml");
}
Edit
Furthermore, if you don't like the idea of maintaining two configuration files. You could create one configuration with the common components (probably the spring stuff and w/e else) and then based on your parameter, programatically configure the FileAppender:
public static void main(String[] args){
DOMConfigurator.configure("log4j.xml");
//if log to file arg = true{
// Define layout
PatternLayout layout = new PatternLayout();
layout.setConversionPattern("%d{ISO8601} %-5p %c{5} - %X{messageId} - %m%n");
// Create appender
FileAppender appender = new FileAppender();
appender.setFile("lss-client.log");
appender.setLayout(layout);
appender.activateOptions();
// Get root logger and add appender.
log = Logger.getRootLogger();
log.setLevel(Level.INFO);
log.addAppender(appender);
}
}

Silencing Flyway -- a log4j problem

I've written a wrapper around Flyway I call Nomad. I am well pleased with Flyway, save the incessant logging it performs outside of Maven. I created an issue here. Each user of Nomad must make their own configuration of log4j to silence Flyway. This is problematic if not done, for instance, during Spec testing. However, getting the configuration just right is a challenge and, moreover, having to do this breaks the abstraction of my library.
My question is this: how can I permanently silence flyway so that any user of Nomad is not burdened with the task? I've found that this log4j.xml sometimes works:
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %t %-5p %c{1} - %m%n"/>
</layout>
</appender>
<logger name="org.springframework" additivity="false">
<level value="error"/>
<appender-ref ref="console"/>
</logger>
<root>
<priority value="error"/>
<appender-ref ref="console"/>
</root>
</log4j:configuration>
This stifles Flyway to the point of being helpful, rather than overly chatty. The misdirection is still broken, but not often.

The way you have done it in log4j.xml seems good. I don't see anything wrong.
Since the project uses Maven, I recommend you adding a property file instead log4j.properties, that you place that in src/main/resources
The contents can be:
log4j.rootCategory=DEBUG, stderr
log4j.appender.stderr=org.apache.log4j.ConsoleAppender
log4j.appender.stderr.target=System.err
log4j.appender.stderr.layout=org.apache.log4j.PatternLayout
log4j.appender.stderr.layout.ConversionPattern=[%d] %-5p %c %x: %m%n
# Silence springframework messages.
org.springframework=ERROR
Perhaps that will work better with you. It becomes easier to manage too.

Flyway's only non-logging framework dependency is Spring. As Mohamed Mansour pointed out you can suppress all but the ERROR messages with a simple Log4J configuration setting.
For Spring (as pointed out by Mohamed Mansour):
org.springframework=ERROR
For Flyway:
com.googlecode.flyway=ERROR