Develop Reference

Log SQL to file with logback & hibernate

I am trying to get the SQL logged directly to a file when running the dev profile.
This is my logback.xml
<configuration>
<property name="SQL_LOG_FILE" value="${LOG_PATH:-${LOG_TEMP:-${TMPDIR:-/tmp}}}/${HIBERNATE_LOG_FILE:-hibernate.log}"/>
<springProfile name="dev">
<appender name="SQLDEBUG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${SQL_LOG_FILE}</file>
<encoder>
<charset>utf-8</charset>
<Pattern>%-5level %logger{0} - %msg%n</Pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>utf-8</charset>
<Pattern>[%highlight(%p)] [%t] %c - %m%n</Pattern>
</encoder>
</appender>
<logger name="org.hibernate.SQL" additivity="false" level="DEBUG">
<appender-ref ref="SQLDEBUG"/>
</logger>
<logger name="org.hibernate.type.descriptor.sql" additivity="false" level="TRACE">
<appender-ref ref="SQLDEBUG"/>
</logger>
</springProfile>
<root level="${logback.loglevel}">
<springProfile name="dev">
<appender-ref ref="CONSOLE"/>
</springProfile>
</root>
I have removed the prod profile settings for simplicity.
The logger for hibernate is inside the dev profile because I don't want it enabled in prod.
I have tried many combinations of these org.hibernate settings. This version generates SQL logs but only dumps them to console, not the log file. Some general startup information is added to the log file but no SQL.
If I change org.hibernate.type.descriptor.sql to org.hibernate.type there is a lot of stack trace logs that are added directly to the file, but no SQL.
Some posts recommend using org.hibernate.SQL level=TRACE but that did not seem to change anything.
I also tried putting the logger outside of the dev profile but that also did not change the results.
There is a lot of information for enabling logback & hibernate for simple console output but not for sending the SQL to its own log file.
I also tried enabling hibernate.SQL=DEBUG in IntelliJ but that makes a lot of SQL on the console, I need to not do that.
I have been try

I doubt you spring profile is being used. To get this to work rename the logback.xml file to logback-spring.xml, allowing the springProfile tag to be used. In this tag, a name can be provided that can be set via properties, environment variables, or VM options. Below is how you can set the springProfile name to dev , which has been used to represent a development environment.
To set in application.properties or as an environment variable:
spring.profiles.active=dev
Or as a VM option:
-Dspring.profiles.active=dev
Also, modify your root-level tag to be inside the spring profile tag:
<springProfile name="dev">
<root level="${logback.loglevel}">
<appender-ref ref="CONSOLE"/>
</root>
</springProfile>

Ways to configure logs other than log4j xml, properties file, and source code (mainly java)?

I'm looking at the source code of some application. It is using Spring framework, Apache Tiles, JSP, Log4j, java, javascript, jquery, jqplot, Jsch, and etc.
I know where the logs are created. (a/b/logs) However, when I look at source code, I don't know how logs are created under the folder name 'logs'. I looked at log4j.xml, web.xml , property files. I found the code for how the path 'a/b' is created, but not logs. Also that folder has 4 types of logs. And they are name in a pattern like access.20181227001 , errors.20182111. I want to know where I have to look to find how the logs are created in this manner.
Log4J.xml
<!-- Appenders -->
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %5p [%c] %m%n" />
</layout>
</appender>
<appender name="console-infolog" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %5p %m%n" />
</layout>
</appender>
<!-- Application Loggers -->
<logger name="com.dsmentoring.chakan" additivity="false">
<level value="debug" />
<appender-ref ref="console"/>
</logger>
<!-- 3rdparty Loggers -->
<logger name="org.springframework.core">
<level value="error"/>
</logger>
<!-- Bean logger -->
<logger name="org.springframework.beans">
<level value="error"/>
</logger>
<!-- Context logger -->
<logger name="org.springframework.context">
<level value="error"/>
</logger>
<!-- Web logger -->
<logger name="org.springframework.web">
<level value="error"/>
</logger>
<logger name="org.springframework.ldap" additivity="true">
<level value="error"/>
</logger>
<!-- LDAP logger -->
<logger name="com.unboundid.ldap" additivity="true">
<level value="error"/>
</logger>
<!-- Root Logger -->
<root>
<priority value="off" />
<appender-ref ref="console" />
</root>
To sum up :
1) Is there a way to configure where logs are created and how they are created (4 types of logs) Other than log4j.xml, xml files and property files? I looked at all the java, jsp, js code but can't seem to find the configuration for logs. So I want to know if there are other ways to do that or where I should look for those configuration.
2) The 'logs' folder is possibly default for log4j?
ldap.properties
#LDAP Connection Info
ldap.host=192.168.0.17
ldap.port=22389
ldap.userName=cn=directory manager
ldap.password= 9074B18A0DE2D50C068D37B60BE5DFDE
ldap.baseDN=o=sso30root
ldap.defaultLoadSize=1000
ldap.start=start-ds
ldap.stop=stop-ds
ldap.workdir=/home/KB_openDJ // logs are created under this path
// /home/KB_openDJ/logs
In other java class, they use this.
#Value("${ldap.workdir}")
private String WORK_DIR;
// I ommited many lines in between
try{
diskUsage = sigar.getFileSystemUsage(WORK_DIR);
diskIOInfo.setDiskRead((int)(diskUsage.getDiskReadBytes()));
diskIOInfo.setDiskWrite((int)(diskUsage.getDiskWriteBytes()));
}catch(SigarException sigarEx){
log.debug("Disk Usage info load Error : " + sigarEx.getMessage());
}
I used the 'Search' feature in Eclipse many times. ( logs, WORK_DIR, and 4 types of log name, and many others. I am unable to locate the code about logging configuration. :(
My log4j version :
1.2.15

Ok, it seems that there's a solution which might help you. It requires some debugging of static initalization block of org.apache.log4j.LogManager class. This class is responsible for loading logger configuration file. Here's a documention link which thoroughly describes initalization process: link.
Here's an excerpt from a LogManager source file:
String configurationOptionStr = OptionConverter.getSystemProperty(DEFAULT_CONFIGURATION_KEY, null);
String configuratorClassName = OptionConverter.getSystemProperty(CONFIGURATOR_CLASS_KEY, null);
What I'm trying to show you here, is that your logger configuration file might be specified as a JVM option supplied to your application-server. That's why you are not able to determine actual file being used.
Even if this approach fails, I'd recommend you to investigate the appenders list retrieved at run-time. Here's a stackoverflow thread which describes how to do so: link.

Logback: separate logging WARN / TRACE

I'm trying to configure logback to print:
- everything (level trace or debug) to the screen
- everything (level trace or debug) to the debugfile
- warnings and above to an error file
My logback.xml config is like this:
...
<logger name="be" level="TRACE">
<appender-ref ref="FILE-AUDIT" />
<appender-ref ref="STDOUT" />
</logger>
<root level="WARN">
<appender-ref ref="FILE-ERROR" />
</root>
However, the error and debug file contain exactly the same, being ALL logging (debug and error). I've already tried to play with the additivity option, but that's apparently not what I need.
The second question is that I use name "be" to have all classes under be.* but actually I want to capture everything there (com.* as well).

Found a solution by adding a filter to the appender:
<appender name="FILE-ERROR"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- deny all events with a level below WARNING -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>WARN</level>
</filter>

Connection to database in a managed thread, Debug values doesn't appear in Tomcat console or logs

I am facing this trouble for a long time now without accessing the debug logs of the managed Threads in my Spring boot application when run on Tomcat. All the logs appear when run on the Eclipse/STS.
In Tomcat logs, I can only see the main Tread Logs.
I am connection to a database through JDBC and this is happening in a separate thread. I tried to follow the log configuration documentation but none of them helps to get the debug logs of these threads. So I do not actually see the exact problem of what is causing the connection to fail.
Here is what I tried so far:
I tried with the following logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<!-- <appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<Target>System.out</Target> <encoder> <pattern>%d{yyyy-MM-dd HH:mm:ss} %-5p
%c{1}:%L - %m%n</pattern> </encoder> </appender> <logger name="com.biscoind"
additivity="false" level="TRACE"> <appender-ref ref="stdout" /> </logger>
<root level="debug"> <appender-ref ref="stdout" /> </root> -->
<include resource="org/springframework/boot/logging/logback/base.xml" />
<logger name="org.springframework.web" level="DEBUG" />
</configuration>
When that did not resolve the issue I removed this file and see if by default, if it logs all the treads. But it did not.
So, I added the following configurations to the application.properties
logging.level.org.springframework.web:TRACE
logging.level.org.hibernate:ERROR
Then It seemed to me that this is only logging out the above namespaces, I again added
debug=true
logging.level.org.springframework.web:DEBUG
logging.level.org.hibernate:DEBUG
Tried and it did not work.
I added my namespaces also and tried as follwing,
debug=true
logging.level.com.mydomain:DEBUG
logging.level.org.springframework.web:DEBUG
logging.level.org.hibernate:DEBUG
That did not work also, I am now confused on the what should I do with the config relative to logging to make the logs to appear for the tread executions.
Irrespective of the treads, because of the property spring.jpa.show-sql=true it logs the queries that are made.

It was not a problem with the threads at all. The application was working correctly in the development environment. The problem was in the deployment environment.
It turned out to be a Java version miss-match with the jar files and the JVM version. The jars were build using Java 8 and it was running on Java 7 JVM.
When the JMV was changed to Java-8. It worked fine. So Next time I will be more careful with the version mismatch.

Log file not produced until WebLogic is restarted

I'm working on the development of an application that is deployed on WebLogic 10.3. It is packaged as an EAR and includes one module. The application works fine by itself but I am faced with issue related to logging.
I am using Log4j. The library is included in the EAR file and log4j.xml is placed under JAR module. So the config location is the following:
A.ear/B.jar/log4j.xml
Log4j config is following:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="CA" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{dd-MMM-yyyy-HH:mm:ss} %p %C{1} - %m%n" />
</layout>
</appender>
<appender name="DRFA" class="org.apache.log4j.DailyRollingFileAppender">
<param name="file"
value="servers/AdminServer/logs/EJB.log" />
<param name="Append" value="true" />
<param name="DatePattern" value="'-'yyyy-MM-dd" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{dd-MMM-yyyy-HH:mm:ss} %p %C{1} - %m%n" />
</layout>
</appender>
<logger name="com.companyname.ejb" additivity="false">
<level value="DEBUG" />
<appender-ref ref="DRFA" />
<appender-ref ref="CA" />
</logger>
<logger name="com.companyname.results" additivity="false">
<level value="DEBUG" />
<appender-ref ref="DRFA" />
<appender-ref ref="CA" />
</logger>
<logger name="com.companyname.marketdata" additivity="false">
<level value="DEBUG" />
<appender-ref ref="DRFA" />
<appender-ref ref="CA" />
</logger>
<root>
<level value="DEBUG" />
<appender-ref ref="CA" />
</root>
When I build and deploy EAR (using Maven and customized WebLogic plugin) and call application no log file appears. But if I restart WebLogic everything is fine.
WebLogic is running under Windows 7 in domain mode with single node.
I'd like to know if there is some way to make the log appear without weblogic restart (since it can cause issues on production environment)?
Update: Also I'd like to know what is the reason of such behavior (i.e. why the log file is not created right after application deployment)? Is this an issue with Weblogic, log4j or with their coupling? I've tried to find the answer in the Oracle documentation, but no luck for now.

Some notes:
In prod environment, you probably want to have your log configuration outside app packages, so you can change log levels without redeployment.
You should plan for production to be able to handle restarts. We usually have hot and cold servers, so load can be balanced and servers restarted when doing the deployment.
About the issue, if you want to, you could specify a servlet that is run on startup of app and configures your log4j. Something like:
web.xml
<servlet>
<servlet-name>SomeServlet</servlet-name>
<servlet-class>YourServlet</servlet-class>
<load-on-startup>0</load-on-startup>
</servlet>
Servlet
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import org.apache.log4j.xml.DOMConfigurator;
public class YourServlet extends HttpServlet
{
#Override
public void init(final ServletConfig config)
throws ServletException
{
final java.net.URL url = Thread.currentThread().getContextClassLoader()
.getResource("Log4j.xml");
DOMConfigurator.configure(url);
}
}
There's also an example on the web about using a servlet context listener.
Edit. as to why this happens, weblogic logging mechanism is initiated by default on startup with these kind of settings:
set JAVA_OPTIONS=%JAVA_OPTIONS% -Dlog4j.configuration=file:<path>/log4j.properties
set JAVA_OPTIONS=%JAVA_OPTIONS% -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Log4JLogger
set JAVA_OPTIONS=%JAVA_OPTIONS% -Dweblogic.log.Log4jLoggingEnabled=true
so if you just redeploy your app without restarting the server, these settings will not be used -> logging will not get initiated.

I didn't found any quick configuration solution, so code change was required.
I've added Weblogic Application Lifecycle listener as suggested here. Its postStart() method initializes log4j explicitly through DOMConfigurator and now logs appear right after application deployment.
The are few alternatives how configuration can be initialized. One of them was mentioned in eis' post and another here. But I've choose listener in order to keep single module in EAR and in order to avoid issues with singleton EJB on cluster environment (i.e. I am not sure if Weblogic will create singleton on each node or just one instance per cluster).
Also in order to prevent environment changes for local and dev environment, I am using internal log4j.xml (i.e. placed within ear file) there.
For stage and prod - external config file is specified (in Maven profile).
External file is monitored by log4j so changes will be automatically applied without any redeployments or restarts.