Develop Reference

Log SQL to file with logback & hibernate

I am trying to get the SQL logged directly to a file when running the dev profile.
This is my logback.xml
<configuration>
<property name="SQL_LOG_FILE" value="${LOG_PATH:-${LOG_TEMP:-${TMPDIR:-/tmp}}}/${HIBERNATE_LOG_FILE:-hibernate.log}"/>
<springProfile name="dev">
<appender name="SQLDEBUG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${SQL_LOG_FILE}</file>
<encoder>
<charset>utf-8</charset>
<Pattern>%-5level %logger{0} - %msg%n</Pattern>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>utf-8</charset>
<Pattern>[%highlight(%p)] [%t] %c - %m%n</Pattern>
</encoder>
</appender>
<logger name="org.hibernate.SQL" additivity="false" level="DEBUG">
<appender-ref ref="SQLDEBUG"/>
</logger>
<logger name="org.hibernate.type.descriptor.sql" additivity="false" level="TRACE">
<appender-ref ref="SQLDEBUG"/>
</logger>
</springProfile>
<root level="${logback.loglevel}">
<springProfile name="dev">
<appender-ref ref="CONSOLE"/>
</springProfile>
</root>
I have removed the prod profile settings for simplicity.
The logger for hibernate is inside the dev profile because I don't want it enabled in prod.
I have tried many combinations of these org.hibernate settings. This version generates SQL logs but only dumps them to console, not the log file. Some general startup information is added to the log file but no SQL.
If I change org.hibernate.type.descriptor.sql to org.hibernate.type there is a lot of stack trace logs that are added directly to the file, but no SQL.
Some posts recommend using org.hibernate.SQL level=TRACE but that did not seem to change anything.
I also tried putting the logger outside of the dev profile but that also did not change the results.
There is a lot of information for enabling logback & hibernate for simple console output but not for sending the SQL to its own log file.
I also tried enabling hibernate.SQL=DEBUG in IntelliJ but that makes a lot of SQL on the console, I need to not do that.
I have been try

I doubt you spring profile is being used. To get this to work rename the logback.xml file to logback-spring.xml, allowing the springProfile tag to be used. In this tag, a name can be provided that can be set via properties, environment variables, or VM options. Below is how you can set the springProfile name to dev , which has been used to represent a development environment.
To set in application.properties or as an environment variable:
spring.profiles.active=dev
Or as a VM option:
-Dspring.profiles.active=dev
Also, modify your root-level tag to be inside the spring profile tag:
<springProfile name="dev">
<root level="${logback.loglevel}">
<appender-ref ref="CONSOLE"/>
</root>
</springProfile>

Logging time incorrect in my custom Log4j2 XML configuration file Java

I am new to log4j2. Previously I am using log4j. The reason I am migrating into part 2 is for Asynchronous logging. After searching Internet I am able to write a configuration file that actually creates two log files "Errors.log" and "Messages.log". Now the problem is : I would be communicating with Servers that are kept far away from me. I wrote a client that communicates with the server and sends a request and in back the Server sends me a response. In any situation it takes at least 10 milli-seconds for the request to reach the Server and get back the response from it. But in my log files it is showing that the request sent to the Server and receive from the Server is at same time (Same milli-second). I am using the Asynchronous logging. Is this causing the wrong timestamp? or else the policies which I have used are creating these issues?
Below is my Log4j2 XML CONFIG file:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
<Appenders>
<File name="my_file_appender" fileName="LOG4j_LOGS/Errors.log" immediateFlush="false" append="false">
<PatternLayout>
<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} - %msg%n</Pattern>
</PatternLayout>
</File>
<Async name="async_appender">
<AppenderRef ref="my_file_appender" />
</Async>
<!-- file appender -->
<RollingFile name="Error-log" fileName="LOG4j_LOGS/Messages.log"
filePattern="LOG4j_LOGS/Messages-%d{yyyy-MM-dd}.log">
<!-- log pattern -->
<PatternLayout>
<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} - %msg%n</Pattern>
</PatternLayout>
<!-- set file size policy -->
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="100 MB" />
</Policies>
<DefaultRolloverStrategy max="25"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="Error-log" level="info" additivity="false">
<appender-ref ref="Error-log" level="debug"/>
</Logger>
<Root level="info" includeLocation="false">
<AppenderRef ref="async_appender"/>
</Root>
</Loggers>
</Configuration>
Can anyone please check my CONFIG file. All I want is to create two separate log files, one for storing info messages and other for storing errors. And this should create a new file every time I run my application and it should not delete the previous logs. The size of the logs can be anything. If the size has exceeded it should create a new file and write the data into it. No matter how many days I run the application the daily logs needs to be stored and the entire process has to be done in Asynchronously.
I am also using the below VM options for logging asynchronously :
-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector

When logging is done asynchronously you log messages got onto a separate queue. They are timestamped at the moment of processing by background thread that writes logs to disk (since timestamp is part of you Appender pattern). So only the order is preserved. Timestamps may be slightly different.
See https://logging.apache.org/log4j/2.x/manual/async.html for more info.

Logback - dynamic configuration

Is it possible to configure logback in such way that custom logback property :
<configuration scan="true" scanPeriod="60 seconds">
<property name="logFullMessage" value="false" />
<!-- project appenders defitions -->
<!-- project loggers defititions -->
</configuration>
will influence which pattern or appender will be used ? We do have web services application which operates with large requests/responses, by default we do not wish to log request/response body, but when problem occurs we would like to have option to switch it on (logFullMessage=true) and store full response body into the log file.

To switch the used appender you could do (only relevant lines are shown):
<property name="USE_APPENDER" value="FILE1" />
<appender name="FILE1" class="ch.qos.logback.core.FileAppender">
....
</appender>
<appender name="FILE2" class="ch.qos.logback.core.FileAppender">
....
</appender>
<root level="warn">
<appender-ref ref="${USE_APPENDER}"/>
</root>
This in combination with the rescan options which you already have enabled makes it possible to switch the appenders.

Using log4j to send email reports via the SMTPAppender

I'm trying to use log4j to send emailable reports that contain the logging statements from a background process. I want one email sent for each process run, not one email for each logging statement. I've looked at the SMTPAppender, but don't see a way to manually send the report when the process completes. I believe the TriggeringEventEvaluator may be the key, but one issue I'm running into is how to get a handle to the TriggeringEventEvaluator instance. I'm stuck using log4j 1.2.14 and the SMTPAppender.getEvaluator() method was introduced in 1.2.15. Any thoughts? Am I even on the right track? Does the SMTPAppender.close() method come into play here?
I want to be able to do this:
log.info(message1);
log.info(message2);
log.info(message3);
log.sendMail();
After thinking about this some more, I think I need to clarify what I'm hoping to accomplish. I'm trying to capture the logging from running a quartz job and send the resulting log as an email. The quartz job makes a bunch of service method calls into various services. I want the to include any logging those service methods perform as well as the logging of the quartz jobs itself. I was thinking I could do something like the following for capturing all the logging, but it isn't working.
// at the beginning of quartz job
Logger logger = Logger.getRootLogger();
StringWriter sw = new StringWriter();
WriterAppender wa = new WriterAppender(new SimpleLayout(), sw);
logger.addAppender(wa);
// at the end of the quartz job
String report = sw.toString();

You shouldn't use any of log4j's methods, you should configure it properly instead.
First of all, define in your log4j.properties file your appender properly:
#CONFIGURE SMTP
log4j.appender.email=org.apache.log4j.net.SMTPAppender
log4j.appender.email.SMTPHost=mail.mydomain.com
log4j.appender.email.SMTPUsername=myuser#mydomain.com
log4j.appender.email.SMTPPassword=mypw
log4j.appender.email.From=myuser#mydomain.com
log4j.appender.email.To=myuser#mydomain.com
log4j.appender.email.Subject=Log of messages
log4j.appender.email.BufferSize=1
log4j.appender.email.EvaluatorClass=TriggerLogEvent
log4j.appender.email.layout=org.apache.log4j.PatternLayout
log4j.appender.email.layout.ConversionPattern=%m
Note: code taken from this post. More information can be obtained in SMTPAppender API.
Next, make a special class that will be used just for sending email. Example:
package com.foo.mailer;
import org.apache.log4j.Logger;
public class Mailer {
private static final Logger logger = Logger.getLogger(Mailer.class);
public void logMail(String mailString) {
logger.info(mailString);
}
}
Next, put in log4j.properties configuration for this class:
# INFO level will be logged
log4j.logger.com.foo.mailer = INFO, email
# turn off additivity
log4j.additivity.com.foo.mailer = false
Now, whenever you want to send an email using log4j, put this in your code:
new Mailer().logMail("This mail should be sent");
Disclaimer: I haven't tested any of this code.

If you are using an XML configuration file, the following should be helpful.
<appender name="ErrorEmailAppender" class="org.apache.log4j.net.SMTPAppender">
<param name="SMTPHost" value="mail.mydomain.com" />
<param name="SMTPUsername" value="myuser#mydomain.com" />
<param name="SMTPPassword" value="password" />
<param name="From" value="myuser#mydomain.com" />
<param name="To" value="myuser#mydomain.com" />
<param name="Subject" value="Log of messages" />
<param name="BufferSize" value="1" />
<param name="EvaluatorClass" value="TriggerLogEvent" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%t %m%n"/>
</layout>
</appender>
<logger name="com.foo.mailer">
<level value="INFO" />
<appender-ref ref="ErrorEmailAppender"/>
</logger>

Log4j 2.x onwards, the following configuration for log4j.xml can be used. It's pretty straightforward and can be used to send emails.
You need to edit it and enter your smtp host, username, password, port and subject.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<SMTP>
<name>Mail1</name>
<subject>SUBJECT</subject>
<to>TO_EMAIL</to>
<from>FROM_EMAIL</from>
<smtpHost>smtp.gmail.com</smtpHost>
<smtpPort>487</smtpPort>
<ignoreExceptions>false</ignoreExceptions>
<smtpUsername>username</smtpUsername>
<smtpPassword>password</smtpPassword>
<smtpProtocol>smtps</smtpProtocol>
<HtmlLayout charset="UTF-8" locationInfo="true" />
<ThresholdFilter level="ERROR"/>
</SMTP>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="Mail1"/>
</Root>
</Loggers>
</Configuration>
Reference: Log4j SMTP Appender

Run Logback in Debug

I've recently switched from log4j to logback and am wondering if there is an easy way to run logback in debug mode, similar to log4j's log4j.debug property. I need to see where it is picking up my logback.xml from.
The docs mention using a StatusPrinter to print out logback's internal status, but that would require code changes.

[EDIT]
This has been fixed in Logback 1.0.4. You can now use -Dlogback.debug=true to enable debugging of the logback setup.
-- Old Answer --
Unfortunately, there is no way to enable debugging via a System property. You have to use <configuration debug="true"> in the logback.xml. Please submit a feature request.

This is how I do it. I set a system property called 'log.level', then I reference it in logback.xml.
Edit: The downside is that you MUST have 'log.level' always set. The way I deal with this is to check in my main method and set it to INFO if not already set, be sure to do this before you first logging calls. Then I can override on the command line, and have a sensible default.
Here is how it looks in my logback.xml:
<configuration>
<logger name="com.mycompany.project" level="${log.level}" />
<logger name="httpclient" level="WARN" />
<logger name="org.apache" level="WARN" />
<logger name="org.hibernate" level="WARN" />
<logger name="org.hibernate.cfg.AnnotationBinder" level="WARN" />
<logger name="org.hibernate.cfg.annotations" level="WARN" />
<logger name="org.quartz" level="WARN" />
<logger name="org.springframework" level="WARN" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%-16thread] %-5level %-35.35logger{30} - %msg%n</pattern>
</encoder>
</appender>
<root level="${log.level:-INFO}">
<appender-ref ref="STDOUT" />
</root>
</configuration>

You can set the status listener class via system property:
java -Dlogback.statusListenerClass=ch.qos.logback.core.status.OnConsoleStatusListener ...
See: Logback manual

I could not make it work using the chosen answer. However, the following worked:
java -Dlogback.configurationFile=/path/to/config-debug.xml com.domain.Main
Just add a file (config-debug.xml in this example) somewhere on your server and leave it there when you need to debug. Like the following.
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{dd-MMM-yyyy HH:mm:ss.SSS} %-5level [%thread] %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Run your application using the afore mentioned -D parameter.
When things are back to normal, remove the -D parameter and restart your application.
Source: Chapter 3: Logback configuration

Well, It's pretty easy. Either you can use
log.level = debug
inside the application.properties of Spring boot.
or you can also set this in the configuration file of logback.xml
<root level="${log.level}">
<appender-ref ref="ANY_APPENDER" />
</root>

In eclipse you can have multiple run configurations. Open your main class. Go to Debug dropdown on eclipse toolbar and select Debug configurations. Click the New launch configuration icon at the top left. Give your launch configuration a better name. Click the Arguments tab under the name and enter -Dlog.level=debug or whatever you want. Click Close or Debug
You can do this again and specify -Dlog.level=warn for example.