Develop Reference

reactor.netty.http.server.AccessLog producing LoggingEvent not AccessEvent

I've been trying to configure spring-boot 2.1 with webflux to store access logs in JSON. Moreover I need to have information like protocol, status code as separate JSON fields (not a part of a message). Looking at the internet I found logstash-logback-encoder. Which seems to have everything I need. But during runtime I get following error:
ERROR in ch.qos.logback.core.FileAppender[accessLog] - Appender [accessLog] failed to append. java.lang.ClassCastException: ch.qos.logback.classic.spi.LoggingEvent cannot be cast to ch.qos.logback.access.spi.IAccessEvent
at java.lang.ClassCastException: ch.qos.logback.classic.spi.LoggingEvent cannot be cast to ch.qos.logback.access.spi.IAccessEvent
at at net.logstash.logback.composite.accessevent.AccessEventFormattedTimestampJsonProvider.getTimestampAsMillis(AccessEventFormattedTimestampJsonProvider.java:20)
at at net.logstash.logback.composite.FormattedTimestampJsonProvider.writeTo(FormattedTimestampJsonProvider.java:149)
at at net.logstash.logback.composite.JsonProviders.writeTo(JsonProviders.java:77)
at at net.logstash.logback.composite.CompositeJsonFormatter.writeEventToGenerator(CompositeJsonFormatter.java:189)
at at net.logstash.logback.composite.CompositeJsonFormatter.writeEventToOutputStream(CompositeJsonFormatter.java:166)
at at net.logstash.logback.encoder.CompositeJsonEncoder.encode(CompositeJsonEncoder.java:122)
at at net.logstash.logback.encoder.CompositeJsonEncoder.encode(CompositeJsonEncoder.java:34)
at at ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:230)
at at ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:102)
at at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:84)
at at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51)
at at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:270)
at at ch.qos.logback.classic.Logger.callAppenders(Logger.java:257)
at at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421)
at at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383)
at at ch.qos.logback.classic.Logger.info(Logger.java:591)
at at reactor.util.Loggers$Slf4JLogger.info(Loggers.java:255)
at at reactor.netty.http.server.AccessLog.log(AccessLog.java:104)
at at reactor.netty.http.server.AccessLogHandler.lambda$write$0(AccessLogHandler.java:77)
at at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:103)
at at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
at at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:696)
at at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:258)
at at io.netty.channel.nio.AbstractNioByteChannel.doWriteInternal(AbstractNioByteChannel.java:216)
at at io.netty.channel.nio.AbstractNioByteChannel.doWrite0(AbstractNioByteChannel.java:209)
at at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:397)
at at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:934)
at at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:360)
at at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:901)
at at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1396)
at at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
at at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
at at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:533)
at at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
at at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:358)
at at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
at at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
at at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
at at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
at at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
at at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
at at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
My configuration is fairly simple :
<appender name="accessLog" class="ch.qos.logback.core.FileAppender">
<file>${LOGS}/access_log.log</file>
<encoder class="net.logstash.logback.encoder.LogstashAccessEncoder"/>
</appender>
<logger name="reactor.netty.http.server.AccessLog" level="DEBUG" additivity="false">
<appender-ref ref="accessLog"/>
</logger>
At this point I got stuck. I've been searching internet and looking into logback code but still I have no idea what to do to have AccessEvent which contains much more information, instead of LoggingEvent

The LogstashAccessEncoder from logstash-logback-encoder requires logback-access, which provides AccessEvents. logback-access is only available for jetty and tomcat. It is not available for reactor-netty.
Reactor-netty's reactor.netty.http.server.AccessLog just uses a standard log event (not an AccessEvent). So, if you want to use logstash-logback-encoder, you would use a LogstashEncoder instead of a LogstashAccessEncoder, like this:
<appender name="accessLog" class="ch.qos.logback.core.FileAppender">
<file>${LOGS}/access_log.log</file>
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>
<logger name="reactor.netty.http.server.AccessLog" level="DEBUG" additivity="false">
<appender-ref ref="accessLog"/>
</logger>
The downside is that LogstashEncoder knows nothing about an HTTP request. It just knows about log events logged via a Logger. Therefore, you cannot configure which HTTP details are logged when using a LogstashEncoder like you can when using LogstashAccessEncoder. Instead, your log event will just show the details provided by reactor.netty.http.server.AccessLog as part of the log message. The details will not be available as separate fields in the JSON output. Although, since logstash-logback-encoder is extremely customizable, you could write a custom JsonProvider that parsed the log message from reactor.netty.http.server.AccessLog and split it out into separate JSON fields.

How to set Spring logging level while testing? [duplicate]

This question already has answers here:
Spring Boot Unit Test ignores logging.level
(6 answers)
Closed 5 years ago.
My Spring Boot testing stack is Maven + Surefire + JUnit4. I am annotating the tests with #RunWith(SpringJUnit4ClassRunner.class).
I have application.properties in my project root with this line:
logging.level.root=INFO
This controls the logging when running the Spring boot app and it works on normal runs.
However, whenever I run any JUnit4 tests, I am spammed by pages of DEBUG output like this:
....
17:43:20.500 [main] DEBUG org.springframework.beans.factory.support.DefaultListableBeanFactory - Returning cached instance of singleton bean 'autoConfigurationReport'
17:43:20.500 [main] DEBUG org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader - Registered bean definition for imported class 'org.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration$JacksonObjectMapperConfiguration'
17:43:20.501 [main] DEBUG org.springframework.beans.factory.support.DefaultListableBeanFactory - Returning cached instance of singleton bean 'org.springframework.boot.autoconfigure.condition.BeanTypeRegistry'
17:43:20.502 [main] DEBUG org.springframework.beans.factory.support.DefaultListableBeanFactory - Returning cached instance of singleton bean 'autoConfigurationReport'
....
All this spam makes it almost impossible to see the actually relevant parts. How can I apply the logging levels to test output?
I haven't set any logging explicitly, and according to the docs Logback is used by default.

From a general perspective, you can provide a seperate logback-test.xml-file at the test-resource level. In this file you can add settings regarding the log-level targeted at the output you'd like such as:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger - %msg%n</pattern>
</layout>
</appender>
<logger name="com.your.package" level="DEBUG">
<appender-ref ref="CONSOLE"/>
</logger>
<logger name="org.springframework" level="WARN">
<appender-ref ref="CONSOLE"/>
</logger>
<logger name="org.hibernate" level="WARN">
<appender-ref ref="CONSOLE"/>
</logger>
<logger name="org.eclipse" level="WARN">
<appender-ref ref="CONSOLE"/>
</logger>
<logger name="jndi" level="WARN">
<appender-ref ref="CONSOLE"/>
</logger>
<logger name="org.apache.http.wire" level="WARN">
<appender-ref ref="CONSOLE"/>
</logger>
<root level="DEBUG">
<appender-ref ref="CONSOLE"/>
</root>
</configuration>
Hope this helps you somewhat on the path to decreasing the log output. More is documented at logback's own page:
https://logback.qos.ch/manual/configuration.html
Its mentioned in the top section:
Let us begin by discussing the initialization steps that logback follows to try to configure itself:
1.Logback tries to find a file called logback-test.xml in the classpath.
2.If no such file is found, logback tries to find a file called logback.groovy in the classpath.
3.If no such file is found, it checks for the file logback.xml in the classpath..
4.If no such file is found, service-provider loading facility (introduced in JDK 1.6) is used to resolve the implementation of com.qos.logback.classic.spi.Configurator interface by looking up the file META-INF\services\ch.qos.logback.classic.spi.Configurator in the class path. Its contents should specify the fully qualified class name of the desired Configurator implementation.
5.If none of the above succeeds, logback configures itself automatically using the BasicConfigurator which will cause logging output to be directed to the console.

Weblogic12 + Log4j logging not writing to server logs

I know there are lots of Q's regarding this, but I did go through all of them, and kinda confused myself more, I am listing the steps I followed, please let me know where I messed it up.
1) I just want to use Log4j on application level, so need to copy WL_HOME/server/lib/wllog4j.jar, and the log4j.jar in Domail_Home/Lib?
2) I am using Maven, I added the Log4j dependency in my pom.xml [war].I have my WAR wrapped in EAR.
3) Since I want to write the logs in weblogic managed server logs file, I created a custom appender, to use weblogic, NonCatalogAppender as mention in the link - https://blog.desgrange.net/2010/02/15/logging-in-weblogic-console-with-log4j.html
4) I copied the log file in my war/src/main/resources and i see maven added them to classpath i.e war/target/classes, see below for my lo4j xml
http://jakarta.apache.org/log4j/'>
<!-- stdout appender settings -->
<appender name="STDOUT" class="com.xyz.logging.util.WeblogicAppender">
<param name="Threshold" value="DEBUG"/>
<layout class="org.apache.log4j.PatternLayout">
<!-- notes on patterns:
%p - priority (i.e. level) of message
%c - class that threw error
%m - message in logger's method, e.g. "Exiting application HERE..."
%n - carriage return
%d - date information
-->
<param name="ConversionPattern" value="%c{1} %m"/>
</layout>
</appender>
<!-- settings for root debugger -->
<root>
<level value="DEBUG"/>
<appender-ref ref="STDOUT"/>
</root>
5) Now I didn't change anything on the config level, but I don't see anything appending to server logs. When I initialize NonCatlogLogger manually and call the logger it works fine:
NonCatalogLogger logger =new NonCatalogLogger("XYZ");
logger.debug("This is the debug message")
6) When I debug the application in eclipse, it looks like my custom appender is never called.

Got it, just need to put below into arguments under "server start".
-Dweblogic.log.Log4jLoggingEnabled=true

Connection to database in a managed thread, Debug values doesn't appear in Tomcat console or logs

I am facing this trouble for a long time now without accessing the debug logs of the managed Threads in my Spring boot application when run on Tomcat. All the logs appear when run on the Eclipse/STS.
In Tomcat logs, I can only see the main Tread Logs.
I am connection to a database through JDBC and this is happening in a separate thread. I tried to follow the log configuration documentation but none of them helps to get the debug logs of these threads. So I do not actually see the exact problem of what is causing the connection to fail.
Here is what I tried so far:
I tried with the following logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<!-- <appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<Target>System.out</Target> <encoder> <pattern>%d{yyyy-MM-dd HH:mm:ss} %-5p
%c{1}:%L - %m%n</pattern> </encoder> </appender> <logger name="com.biscoind"
additivity="false" level="TRACE"> <appender-ref ref="stdout" /> </logger>
<root level="debug"> <appender-ref ref="stdout" /> </root> -->
<include resource="org/springframework/boot/logging/logback/base.xml" />
<logger name="org.springframework.web" level="DEBUG" />
</configuration>
When that did not resolve the issue I removed this file and see if by default, if it logs all the treads. But it did not.
So, I added the following configurations to the application.properties
logging.level.org.springframework.web:TRACE
logging.level.org.hibernate:ERROR
Then It seemed to me that this is only logging out the above namespaces, I again added
debug=true
logging.level.org.springframework.web:DEBUG
logging.level.org.hibernate:DEBUG
Tried and it did not work.
I added my namespaces also and tried as follwing,
debug=true
logging.level.com.mydomain:DEBUG
logging.level.org.springframework.web:DEBUG
logging.level.org.hibernate:DEBUG
That did not work also, I am now confused on the what should I do with the config relative to logging to make the logs to appear for the tread executions.
Irrespective of the treads, because of the property spring.jpa.show-sql=true it logs the queries that are made.

It was not a problem with the threads at all. The application was working correctly in the development environment. The problem was in the deployment environment.
It turned out to be a Java version miss-match with the jar files and the JVM version. The jars were build using Java 8 and it was running on Java 7 JVM.
When the JMV was changed to Java-8. It worked fine. So Next time I will be more careful with the version mismatch.

Run Logback in Debug

I've recently switched from log4j to logback and am wondering if there is an easy way to run logback in debug mode, similar to log4j's log4j.debug property. I need to see where it is picking up my logback.xml from.
The docs mention using a StatusPrinter to print out logback's internal status, but that would require code changes.

[EDIT]
This has been fixed in Logback 1.0.4. You can now use -Dlogback.debug=true to enable debugging of the logback setup.
-- Old Answer --
Unfortunately, there is no way to enable debugging via a System property. You have to use <configuration debug="true"> in the logback.xml. Please submit a feature request.

This is how I do it. I set a system property called 'log.level', then I reference it in logback.xml.
Edit: The downside is that you MUST have 'log.level' always set. The way I deal with this is to check in my main method and set it to INFO if not already set, be sure to do this before you first logging calls. Then I can override on the command line, and have a sensible default.
Here is how it looks in my logback.xml:
<configuration>
<logger name="com.mycompany.project" level="${log.level}" />
<logger name="httpclient" level="WARN" />
<logger name="org.apache" level="WARN" />
<logger name="org.hibernate" level="WARN" />
<logger name="org.hibernate.cfg.AnnotationBinder" level="WARN" />
<logger name="org.hibernate.cfg.annotations" level="WARN" />
<logger name="org.quartz" level="WARN" />
<logger name="org.springframework" level="WARN" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%-16thread] %-5level %-35.35logger{30} - %msg%n</pattern>
</encoder>
</appender>
<root level="${log.level:-INFO}">
<appender-ref ref="STDOUT" />
</root>
</configuration>

You can set the status listener class via system property:
java -Dlogback.statusListenerClass=ch.qos.logback.core.status.OnConsoleStatusListener ...
See: Logback manual

I could not make it work using the chosen answer. However, the following worked:
java -Dlogback.configurationFile=/path/to/config-debug.xml com.domain.Main
Just add a file (config-debug.xml in this example) somewhere on your server and leave it there when you need to debug. Like the following.
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{dd-MMM-yyyy HH:mm:ss.SSS} %-5level [%thread] %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Run your application using the afore mentioned -D parameter.
When things are back to normal, remove the -D parameter and restart your application.
Source: Chapter 3: Logback configuration

Well, It's pretty easy. Either you can use
log.level = debug
inside the application.properties of Spring boot.
or you can also set this in the configuration file of logback.xml
<root level="${log.level}">
<appender-ref ref="ANY_APPENDER" />
</root>

In eclipse you can have multiple run configurations. Open your main class. Go to Debug dropdown on eclipse toolbar and select Debug configurations. Click the New launch configuration icon at the top left. Give your launch configuration a better name. Click the Arguments tab under the name and enter -Dlog.level=debug or whatever you want. Click Close or Debug
You can do this again and specify -Dlog.level=warn for example.