Develop Reference

How many projects use Async Logger in log4j2?

Async loggers in log4j2 can improve the logging performance a lot, but are they robust enough? When programs are killed unexpectedly, will the logging messages before that time point be flushed into disk? And does anyone know how many big projects(such as apache projects) use async loggers and give some examples? Any help will be appreciated.

When any process dies you are liable to lose log events that are being buffered. Most people who use File Appenders turn buffering on because the performance without it is considerably slower. Events in the OS buffer would be lost in that case. Likewise with most network protocols, unless you are using something like Apache Flume that immediately acknowledges the receipt, but even then a few messages could be lost simply because the process died before the data was written. But Remko's answer covers the subject of losing messages better than I could.
As for who uses it I can only answer that we know that Async Loggers are being used since we do get questions from time to time but there is no way to formally track who is using any open source project, much less how.

My company uses Async Loggers for a mission-critical equity trading system, without issues.

When not to use AsyncAppender in logback by default

Logback supports using an async appender with the class
ch.qos.Logback.classic.AsyncAppender and according to the documentation, this will reduce the logging overhead on the application. So, why not just make it the default out of the box. What usecases are better served by using a sync appender. One problem I can see with the Async appender is that the log messages will not be chronological. Are there any other such limitations?

The AsyncAppender acts as a dispatcher to another appender. It buffers log events and dispatches them to, say, a FileAppender or a ConsoleAppender etc.
Why use the AsyncAppender?
The AsyncAppender buffers log events, allowing your application code to move on rather than wait for the logging subsystem to complete a write. This can improve your application's responsiveness in cases where the underlying appender is slow to respond e.g. a database or a file system that may be prone to contention.
Why not make it the default behavior?
The AsyncAppender cannot write to a file or console or a database or a socket etc. Instead, it just delegates log events to an appender which can do that. Without the underlying appender, the AsyncAppender is, effectively, a no-op.
The buffer of log events sits on your application's heap; this is a potential resource leak. If the buffer builds more quickly than it can be drained then the buffer will consume resources that your application might want to use.
The AsyncAppender's need for configuration to balance the competing demands of no-loss and resource leakage and to handle on-shutdown draining of its buffer means that it is more complicated to manage and to reason about than simply using synchronous writes. So, on the basis of preferring simplicity over complexity, it makes sense for Logback's default write strategy to be synchronous.
The AsyncAppender exposes configuration levers that you can use to address the potential resource leakage. For example:
You can increase the buffer capacity
You can instruct Logback to drop events once the buffer reaches maximum capacity
You can control what types of events are discarded; drop TRACE events before ERROR events etc
The AsyncAppender also exposes configuration levers which you can use to limit (though not eliminate) the loss of events during application shutdown.
However, it remains true that the simplest safest way of ensuring that log events are successfully written is to write them synchronously. The AsyncAppender should only be considered when you have a proven issue where writing to an appender materially affects your application responsiveness/throughput.

Do log messages have an impact on performance?

I have a big appengine-java application that uses java.util.Logging.
For debugging purposes, I put an INFO message basically on every put, delete, get or query. The application-wide logging settings filters all log messages with level lower than WARNING.
My question is: all this INFO messages, even though filtered, do slow down my app or not?

Every additional operation you perform will add to the overhead you have. I have had some REST calls time out because I had forgotten a logger in the wrong place :)
So yes, they do slow things down, but to what effect is very, very highly dependent on how much you are logging. In a normal situation, logging should not have any noticeable performance penalty. This should be easy to measure, just set your logging level higher to not log so much, and see if the application performs faster!

How to handle disk full errors while logging in logback?

I am using slf4j+logback for logging in our application. Earlier we were using jcl+log4j and moved recently.
Due to the high amount of logging in our application, there is a chance of disk being full in production environment. In such cases we need to stop logging and application should work fine. What I found from the web is that we need to poll logback StatusManager for such errors. But this will add a dependency with logback for the application.
For log4j, I found that we can create an Appender which stops logging in such scenarios. That again will cause a application dependency with log4j.
Is there a way to configure this with only slf4j or is there any other mechanism to handle this?

You do not have to do or configure anything. Logback is designed to handle this situation quite nicely. Once target disk is full, logback's FileAppender will stop writing to it for a certain short amount of time. Once that delay elapses, it will attempt to recover. If the recovery attempt fails, the waiting period is increased gradually up to a maximum of 1 hour. If the recovery attempt succeeds, FileAppender will start logging again.
The process is entirely automatic and extends seamlessly to RollingFileAppender. See also graceful recovery.
On a more personal note, graceful recovery is one my favorite logback features.

You may try extending the slf4j.Logger class, specifically the info, debug, trace and other methods and manually query for the available space (via File.getUsableSpace() ) before every call.
That way you will not need any application dependency

2 real options:
add a cron task on linux (or scheduled one on windows) to clean up your mess (incl. gzip some, if need be).
buy a larger hard disk and manually perform the maintenance
+-reduce logging
Disk full is like OOM, you can't know what fails 1st when catch it. Dealing w/ out of memory (or disk) is by preventing it. There could be a lot of cases when extra disk space could be needed and the task failed.

Is Logging using FileHandler a bottleneck?

I am considering logging business events in a J2EE web application by using Java logging and FileHandler.
I am wondering whether that could cause a performance bottleneck, since many log records will be written to one file.
What are your experiences and opinions?
Is logging a busy web application to one file with Java logging and FileHandler likely to become performance bottleneck?

It all depends on how much log statements you add. If you add logging after every line of code then performance will must certainly degrade.
Use logging for the important cases, set the correct logging level for your current purposes (testing or actual deployment) and use constructions like
if (Logger.isDebugEnabled()) {
Logger.debug("Value is " + costlyOperation()")
}
to avoid calling code that is costly to run.
You might also want to check this article

In order to avoid generalities like "it depends" or "a little" etc. you should measure the performance of your application with and without the logging overhead. Apache JMeter can help you generate the load for the test.
The information you can collect through logging is usually so essential for the integrity of the application, that you can not operate blindly. There is also a slight overhead if you use Google Analytics, but the benefits prevail.
In order to keep your log files within reasonable sizes, you can always use rotating log files.

I think that JavaRevisited blog has a pretty good post on a problem with performance: Top 10 Tips on Logging in Java

In a recent project, I log audit events to a database table and I was concerned about performance, so I added the ability to log in 'asynchronous' mode. In this mode the logger runs in a low-priority background thread and the act of logging from the main thread just puts the log events onto a queue which are lazily retrieved and written by the background logging thread.
This approach will only work, however, if there are natural 'breaks' in the processing; if your system is constantly busy then the queue will never be emptied. One way to solve this is to make the background thread more active depending on the number of the log messages in the queue (an enhancement I've yet to implement).

You should:
Define an appropriate metric of performance (e.g., responsiveness, throughput, etc.). Then you should measure this metric with all logging turned off and then on. The difference would be the cost of logging.
Then you should experiment with different logging libraries and the modes they provide and document the observed differences.
In my personal experience, for all the three projects I worked on, I found that asynchronous logging helped improve the application throughput a lot. But the same may not hold for you, so make sure you make your decision after careful measurements.
The following does not directly relate to your question.
I noticed that you specifically mentioned business logging. In this case, you may also want to keep logging relevant and clean, in case you find your log files are growing huge and difficult to understand. There is a generally accepted design pattern in this area: log as per function. This would mean that business logging (e.g., customer requested a refund) goes to a different destination, interface logging would go to another destination (e.g., user clicked the upvote button != user upvoted an answer), and a cross system call would go to another destination (e.g., Requesting clearance through payment gateway). Some people keep a master log file with all events as well just to see a timeline of the process while some design log miners/scrappers to construct timelines when required.
Hope this helps,