Can anyone throw some light on the clear usage of different levels of LOGGER viz LOGGER.info() LOGGER.trace(), LOGGER.error() and LOGGER.debug().
Pls note its not about configuration, but its about when to use info() and when not to use etc.
I tend to use them like this:
TRACE: Mark where something has executed, like the start of a method. I'm usually not interested logging any information other than "this line executed". Usually turned off in both development and production (to prevent logging large amounts of output) but turned on if I'm diagnosing a defect that is particularly difficult to locate.
DEBUG: Output detailed information about variable state to the logs. After development is complete I turn up the logging level to INFO so these are not output to the logs. If I'm debugging a production problem, I sometimes put the logging level back down to DEBUG to start seeing this output again and assist in diagnosing the problem.
INFO: Output small amounts of important information, such as when a critical method is invoked. Sometimes I leave this on in production, sometimes not.
WARN: Output information about unexpected application state or error that does not prevent the application from continuing to execute. Usually turned on in production.
ERROR: Output information about unexpected application state or error that prevents an operation from completing execution. Always turned on in production.
You said that you aren't looking for help on configuration, but this other slf4j question might be of interest to you anyway.
These are common names for logger frameworks. Usually it's something like this:
debug is for the developer and usually disabled in production use
trace is even finer than debug, logging e.g. method calls and returns
The rest should be self explanatory. Of course it is not always clear cut what event should be logged in what level.
You should have a look at the information in the documentation.
Related
Can anyone please tell me in web applications which is the better way to store log information. If we use tomcat server internally the log information will be stored in log file. What is the better approach to log the application data console for trouble shooting either by using system.out.println(" log information here") or log4j log file to store log.
which one gives better performance when we are dealing the large application?
Which one gives better performance when we are dealing the large application?
Besides the obvious reasons for using a logger (customizable output, filtering, file handling etc.), a logger normally yields better performance, beause of the following:
the logger can provide information on the log level, i.e. if you have to build some expensive message for debugging, you can skip that if debug level is not available:
if( log.isDebugEnabled() ) { log.debug( buildSomeExpensiveOutput() ); }
This would only call buildSomeExpensiveOutput() if debug level logging is enabled and would save that cost if it isn't (e.g. in most production systems).
writing to the console might slow down the application because execution might be synchronized, whereas most loggers write their logs asynchronously
Logger is much better because you can configure appenders, size of the log file, log lever, rolling log files and many other things.
Don't use System.out.println for any regular project. You can use it only if you have test project to verify if something in Java works like you think it should.
It is much better to use logging framework (logback, log4j2) because you can fine tune log level without recompilation. For exmaple you find out that you do not need some information - decrease log level from info to error. Or you face issue - increase it to debug or trace.
The performance cost of using log framework is small. You must be careful about concatenation of strings, especially if log level is disabled:
if (log.isDebugEnabled()) {
log.debug("Value = " + value);
}
Using a logger is probably the better approach for a number of reasons:
Turn off certain log levels
You can turn off logs at the debug level when developement is complete, if you used s.o.p, you would have to comment out/remove them
Log anywhere
You can easily choose to log to a file/email/sms without changing your code, you only need to change configuration.
Also, from the log4j(1.2) home page, some performance info is given:
On an AMD Duron clocked at 800Mhz running JDK 1.3.1, it costs about 5 nanoseconds to determine if a logging statement should be logged or not. Actual logging is also quite fast, ranging from 21 microseconds using the SimpleLayout, 37 microseconds using the TTCCLayout. The performance of the PatternLayout is almost as good as the dedicated layouts, except that it is much more flexible.
I have a big appengine-java application that uses java.util.Logging.
For debugging purposes, I put an INFO message basically on every put, delete, get or query. The application-wide logging settings filters all log messages with level lower than WARNING.
My question is: all this INFO messages, even though filtered, do slow down my app or not?
Every additional operation you perform will add to the overhead you have. I have had some REST calls time out because I had forgotten a logger in the wrong place :)
So yes, they do slow things down, but to what effect is very, very highly dependent on how much you are logging. In a normal situation, logging should not have any noticeable performance penalty. This should be easy to measure, just set your logging level higher to not log so much, and see if the application performs faster!
I've looking for an answer on this for a while. In the company I work we have a highly concurrent system but recently found that the logging configuration of the web server (Jboss) includes the Console appender. The application loggers are also going to the console. We started to get deadlocks on the logging actions, most of them to the console appender (I know that Log4j has a really nasty sincronization bug, but i'm almost sure that we dont have any sincronization method in the related code). Another thing founded, is that the IT Guys regularly access to the console with a putty console, put pauses to check the logs and then just close the putty window.
Is it possible that the console appender, and the use of console for logging and monitoring in a production environment, are causing deadlocks and race conditions on the system?. My understanding is that the console should be used only on development phases with an IDE, because on a highly concurrent system it will be another resource to get (slow because of unbuffered I/O) subject to race conditions.
Thanks.
From the Best practices for Performance tuning JBoss Enterprise Application Platform 5, page 9
Turn off console logging in production
Turn down logging verbosity
Use asynchronous logging.
Wrap debug log statements with If(debugEnabled())
I heavily recommend first and last considerations in production due to a bug in Log4J that calculates what to log before logging it i.e. if MyClass#toString() is a heavy operation, Log4J will first calculate this String before (yes, it will do the heavy operation) and then it will check if this String must be logged (pretty bad, indeed =).
Also, tell the IT guys to use the less command when checking the log files in order to not blocking the files, do not check the console directly =. This command works in Linux, if your server is on Unix environment, the command will be tail (based on #Toni's comment).
IMO, I think that an official JBoss performance guide is the best proof to stop using the console logging in production (even if this still doesn't prove your deadlock problem).
This may be a purely subjective question (if no organization has attempted to standardize this), but my team struggles with this more than you would think.
We use Apache Commons Logging as our logging interface and often the use of priority is not consistent across our development team. For example, some developers log any caught exception to fatal (log.fatal(message)) even though the flow is able to handle the error whereas others only log to fatal when something cause the program to necessarily cease execution for whatever reason.
I would like to know how other teams define each priority. Does anyone work at a company that explicitly tries to define best practices for this? Has Jakarta weighed in on this?
My goal would be to send a simple recommendation for each priority out to my whole team so that we can more effectively handle our unwieldy application logging in a consistent fashion.
Here's what we use (and I'd say many others as well):
FATAL: errors that endanger the consistency of the system and might require an immediate shutdown/restart - very rarely manually used
ERROR: exceptions that should not be thrown and represent a bug in the system (normally all exceptions that are not caught up to a certain point)
WARN: exceptions that might occur but are accounted for or situations that might imply a logic/usage error - we decide whether to track those or not
INFO: whatever you need to have in the log, e.g. what users currently do (in our case of web applications: what page the user is navigating to etc.)
DEBUG: debug only messages like timings etc, generally turned off in the logs
TRACE: we don't use that, you might use it for even more specific debugging info
In some cases we start with logging messages as ERROR, in order to get the notification when something we'd normally not like to happen occurs. Later, if we decide that the source for that error can not be removed, we handle it and change the log level to WARN.
Note that our test and production systems are set up to send an email for FATAL and ERROR. Thus we don't have to check the log files for those manually but only have to check one email inbox.
Hope that helps.
Edit: did you already have a look at the Apache Commons Logging best pratices?
I have always used this as a guideline; I use Log4j more than Commons Logging but this may still be helpful:
DEBUG - for genuinely debug-level info; will not be seen in production or shipped product, as INFO will be the minimum level; good for capturing timings, number of occurrences of events, etc
INFO - minimum level for production/shipped usage; record data likely to be useful in forensic investigations and to confirm successful outcomes ("stored 999 items in DB OK"); all info here must be such that you would be OK with end users/customers seeing it and sending you it, if need be (no secrets, no profanity!)
WARN - not an error level as such, but useful to know the system may be entering dodgy territory, e.g. business logic stuff like "number of ordered products < 0" which suggests a bug somewhere, but isn't a system exception; I tend not to use it that much to be honest, finding things tend to be more natural fits to INFO or ERROR
ERROR - use this for exceptions (unless there's a good reason to reduce to WARN or INFO); log full stacktraces along with important variable values without which diagnosis is impossible; use only for app/system errors, not bad business logic circumstances
FATAL - only use this for an error of such high severity that it literally prevents the app from starting / continuing
Additionally - review your logging often, both with DEBUG and INFO levels active, you want to be able to follow a good narrative through, with prominent events easy to find, and to make sure you're not doing anything too verbose which detracts from readability.
Also ensure you use a pattern which leads to neat columns for things like timestamps, source-classes and severity - again, it helps readability massively.
My take
Fatal: The program is in an state that cannot be handled and must be terminated (automatically or by the user).
Error: The operation of the program failed in a way detectable by the user (the changes were not saved / a file could not be read), but the program still can work (try to load another file).
Warning: Something did not go as planed but the user did not notice it (a server did not answer a ping; perhaps when that server is needed it will cause an error/fatal)
Info: User actions / major program actions (file loaded, automatic backup stored).
Debug: Tracing information. What part of the program is running, which are the values of the parameters
This is what my company recommends:
TRACE - Messages probably only useful during the development cycle, and possibly generated far too frequently for suitable use in production. For example, if I were logging
intermediate values in an inner loop, I’d use TRACE.
DEBUG - Messages used to log various steps in the normal running of the server. Typically these would be aimed at developers rather than operations staff.
INFO - Messages of a positive or neutral nature which we would expect to be logged in a production environment. Should be meaningful to operations staff.
WARN - Messages indicating a condition possibly endangering the ability of the server to respond to requests in an accurate and timely fashion.
ERROR - Messages indicating an unexpected behaviour or condition.
FATAL - Messages indicating an unexpected behaviour or condition which leaves the continued running of the application process impossible or dangerous.
We expect the logs in production to be set at INFO, and we expect them to be read by people other than the developers. Style of log messages is a whole other conversation...
If you are looking for a simple recommendation that is supported by the industry, why not just look at the industry simple implementation/documentation?
We use a logback/log4j API as a logging level guide => which makes it simple / documented / supported by the industry:
Level ALL has the lowest possible rank and is intended to turn on all logging.
Level DEBUG designates fine-grained informational events that are most useful to debug an application.
Level ERROR designates error events that might still allow the application to continue running.
Level FATAL designates very severe error events that will presumably lead the application to abort.
Level INFO designates informational messages that highlight the progress of the application at coarse-grained level.
Level OFF has the highest possible rank and is intended to turn off logging.
Level TRACE designates finer-grained informational events than the DEBUG
Level WARN designates potentially harmful situations.
I'm using a simpler approach:
DEBUG: Turned off in production, thus mainly used by the developer to log certain queries, timings, parameter values, etc.
INFO: Log everything so that you know in retrospect everything to explain how your results were computed and so that you can fix bugs
ERROR: Everything that needs someones (developer/operations) attention
I don't use WARN, because no-one ever filters all log files for WARN messages. If it is important, then it is ERROR (and someone has to care about it), if not, it is INFO (and no-one is interested in it until there is a problem). Same for FATAL.
I also don't use TRACE. Everything I need to know during development is DEBUG.
This answer is taken from a post that I wrote about Logging best practices including logging levels that you may find relevant:
In my opinion, it’s sufficient to have: debug, info, warn and error log levels.
Some engineers like to have additional levels like verbose, trace, alert, critical, fatal, and some creative developers even invent additional levels…
Please don’t do that. Less is more!
Let’s discuss the different levels:
ERROR - Log entries that indicate a system error that may be critical. For example, an HTTP call failed. A former colleague of mine wrote a nice piece that includes a section on logging exceptions which I recommend reading
WARN - Significant events indicate an error, but the behavior is either expected or not critical. This could be a failed charge where there’s a retry in place, for example.
INFO - Useful information related to the operation of the system. Our production systems are tuned to report this as the minimum level logged.
DEBUG - Intended to be used in pre-prod environments or locally. Use this log level when you don’t want to remove a log-print when merging (because it may prove useful), but also don’t want it to show up in production. There should be a flag/environment-variable that can turn on debug printing (even in production) but it should be used scarcely.
I am considering logging business events in a J2EE web application by using Java logging and FileHandler.
I am wondering whether that could cause a performance bottleneck, since many log records will be written to one file.
What are your experiences and opinions?
Is logging a busy web application to one file with Java logging and FileHandler likely to become performance bottleneck?
It all depends on how much log statements you add. If you add logging after every line of code then performance will must certainly degrade.
Use logging for the important cases, set the correct logging level for your current purposes (testing or actual deployment) and use constructions like
if (Logger.isDebugEnabled()) {
Logger.debug("Value is " + costlyOperation()")
}
to avoid calling code that is costly to run.
You might also want to check this article
In order to avoid generalities like "it depends" or "a little" etc. you should measure the performance of your application with and without the logging overhead. Apache JMeter can help you generate the load for the test.
The information you can collect through logging is usually so essential for the integrity of the application, that you can not operate blindly. There is also a slight overhead if you use Google Analytics, but the benefits prevail.
In order to keep your log files within reasonable sizes, you can always use rotating log files.
I think that JavaRevisited blog has a pretty good post on a problem with performance: Top 10 Tips on Logging in Java
In a recent project, I log audit events to a database table and I was concerned about performance, so I added the ability to log in 'asynchronous' mode. In this mode the logger runs in a low-priority background thread and the act of logging from the main thread just puts the log events onto a queue which are lazily retrieved and written by the background logging thread.
This approach will only work, however, if there are natural 'breaks' in the processing; if your system is constantly busy then the queue will never be emptied. One way to solve this is to make the background thread more active depending on the number of the log messages in the queue (an enhancement I've yet to implement).
You should:
Define an appropriate metric of performance (e.g., responsiveness, throughput, etc.). Then you should measure this metric with all logging turned off and then on. The difference would be the cost of logging.
Then you should experiment with different logging libraries and the modes they provide and document the observed differences.
In my personal experience, for all the three projects I worked on, I found that asynchronous logging helped improve the application throughput a lot. But the same may not hold for you, so make sure you make your decision after careful measurements.
The following does not directly relate to your question.
I noticed that you specifically mentioned business logging. In this case, you may also want to keep logging relevant and clean, in case you find your log files are growing huge and difficult to understand. There is a generally accepted design pattern in this area: log as per function. This would mean that business logging (e.g., customer requested a refund) goes to a different destination, interface logging would go to another destination (e.g., user clicked the upvote button != user upvoted an answer), and a cross system call would go to another destination (e.g., Requesting clearance through payment gateway). Some people keep a master log file with all events as well just to see a timeline of the process while some design log miners/scrappers to construct timelines when required.
Hope this helps,