Develop Reference

Log messages with different datatypes

I'm using java.util.logging to create log files for my application. That works very well so far.
Now I have the problem that the info() method only expects one string as parameter. But I have different datatypes to log, f.e. integer, double, customized objects.
I know that I can build the string by my own, I cn use String.format etc.
And I also know that I can use the log() method. But here I have to set the log level at everytime and make an Object array.
What I'm looking for is something where I can set a global log level (one time) and then call a method like this:
log.info(String message, Object... values);
Is there a framework which supports that?

Yes there is, it's called slf4j and the particular API you're looking for is this one
org.slf4j.Logger#info(java.lang.String, java.lang.Object...)
This form avoids superfluous string concatenation when the logger
is disabled for the INFO level. However, this variant incurs the hidden
(and relatively small) cost of creating an Object[] before invoking the method,
even if this logger is disabled for INFO. The variants taking
{#link #info(String, Object) one} and {#link #info(String, Object, Object) two}
arguments exist solely in order to avoid this hidden cost.
You should have a look at jcl-over-slf4j.jar in the slf4j documentation page
Our JCL over SLF4J implementation will allow you to migrate to SLF4J gradually, especially if some of the libraries your software depends on continue to use JCL for the foreseeable future. You can immediately enjoy the benefits of SLF4J's reliability and preserve backward compatibility at the same time. Just replace commons-logging.jar with jcl-over-slf4j.jar

Have a look at SLF4j/logback, there you can write something like
LOGGER.info("log output var1={} var2={}",var1,var2);
In the Log-Message the {} will get replaced by the parameters:
15:41:28.551 [main] INFO d.h.s.Main - log output var1=abc var2=123.45

Is it fine to do logs in java class with thread id?

When we add logs in to the java class (using log4j), Is it fine to add thread id with that log messages? is it a bad practice? My idea was to add this thread id; Once we examine a log file of a multithreaded application, it is difficult to find out the correct flow using logs. (As an example, say authentication flow). Is there any better approach for this rather than logging thread id?

Log4j already supports the thread name using t placeholder in its pattern layout. So this is a supported feature that you should use if you find it useful. This way you don't need to pass the thread name manually. However it doesn't make use of the thread ID. So you should give meaningful names to your threads. This should be preferred as it is more indicative to what is going on in your application than just plain thread ids.

If it is the thread id, please refer this answer
However if you only need the thread name, you can use the t pattern configuration, please refer here.

If you are using Java Logger API - LogRecord has method getThreadID() and can be configured to log.
For log4j there are no ThreadId methods available and No harm in logging it.
In cases If your class inherits from Thread, you can use methods getName and setName to name each thread. Otherwise you could just add a name field to MyTask, and initialize it in your constructor And use a more sensible Thread Name instead of ID.

Logback has a special appender called SiftingAppender which provides a very nice solution to the type of problems you describe. A SiftingAppender can be used to separate (or sift) logging according to any runtime attribute, including thread id
If your concern is about several JVMs writing to the same FileAppender, then i'd suggest two things:
using SLF4J as a logging facade
using logback as logging implementation, in prudent mode
In prudent mode, FileAppender will safely write to the specified
file, even in the presence of other FileAppender instances running in
different JVMs, potentially running on different hosts.
Refer this: http://logback.qos.ch/manual/appenders.html#SiftingAppender

Can I set the logging level within a Java method using log4j?

I've got one method in a class that is very verbose, and this makes the logs hard to read. I'd like to reduce its logging by just changing the level of that one method, leaving other methods in the same class unaltered. Is this possible?

You can change the level configuration for that class but not for specific methods in a class. The only way to do what you want would be to use a different logger inside that method and then configure it at a different level.
Hope this helps.

You cannot change the logging level on a per method basis. The "logger" is the finest level of granularity for control. Typically, there is a one-to-one relationship between logger instances and classes ... though that depends on how the application was coded.
If you can change the code, I recommend that you create special logger object for the noisy method so that you can control it independently of the other code that uses the existing logger. (Or maybe just change the method's logger calls.)
If you cannot change the code, you may have to resort to "hacks" such as post-processing the log files, or writing and configuring a custom Appender that suppresses the unwanted log messages.

I don't think it is posible directly to change logging method for only a method

It's possible. See my comment on OP.
The trick is creating a custom Level (less than TRACE) (as explained in here) and use that log level in very verbose methods.
Ofc in that method you won't user Logger shortcuts to logging methods, but you should resort to Logger.log(Level, Object) and Logger.log (Level, Object, Thowable) methods.

Modifying Java classes at runtime to make instance fields transient -- will it work?

We have an annoying log message coming from Weblogic whenever we invalidate the HTTPSession when it has objects that are not serializable (which we don't care about but which is polluting our logs). Tweaking our log4j configuration to not log this message does not appear to be an option (the message is coming from a generic Weblogic class and we dont want to supress all messages from this class and other than writing our own adapter to look at messages about to be written to the log and suppressing the deserialization error messages I dont know how we would do it through log4j configuration).
In order to temporarily fix this to clean up our logs I want to intercept calls to add objects to the HTTP Session so that the class of the object being added (and any objects in its object graph) have their instance variable declarations changed to transient. Figured I would intercept calls by adding an HTTPSesssionEventListener and then modifying the Class's instances variables using the reflection libraries.
Does anyone know if this would work?

Does anyone have a better suggestion and/or know if this will/will not work like I would want it to?
Yes. Build to spec!
In compliance with the Java Servlet specification:
The distributed servlet container must throw an IllegalArgumentException
for objects where the container cannot support the mechanism necessary for
migration of the session storing them.
So if you want to avoid this message (and write good portable and distributable code), just make the object you put in HttpSession implement the Serializable interface.
Seriously, I can't believe you are thinking to a solution to workaround the real problem instead of fixing it.

One option if would be
tail -f yourlog | grep -v "annoying line here" > cleanLogFile
Which is much less intrusive. But creates a second file.
I did this for a project where I needed to trace an specific problem and all those stupid messages where getting in the middle.

Why go to that trouble? It sounds just as easy to mark them all as Serializable, as long as all the objects in question are yours.
If that's not possible, .aAnother way is to have a class registered within your webApp that implements HttpSessionAttributeListener. In the attributeAdded method print the sessionID and then serialize the session data:
public void attributeAdded(HttpSessionBindingEvent hsbe)
{
// Handle the details yourself here.
ObjectOutputStream.writeObject(hsbe.getValue())
}

Try modifying the ObjectOutputStream, not the serialized object.
If you extend ObjectOutputStream, you can override the replaceObject method and clear the non-serializable fields or re-create the object without the non-serializable properties.
Another option is to use reflection to nullify the non-serializable fields of the session objects, but that is very risky as a session object could be loaded back and appear to work but then throw a NullPointerException hours or even months later (when the field you nullified is eventually referenced.)
Modifying the classes will probably not work.
It is possible to extend ClassLoader and tweak flags on instance fields but:
If it's your class that contains the non-serializable field then it's simpler to change the code yourself to make it transient
If it's not your class then it will be loaded by a different ClassLoader from your application classes so by the time you see the object it will be too late (because the class has already been loaded.)
You cannot set the transient flag on an individual object, only on a class.

Logging activities in multithreaded applications

I have a layered application in Java which has a multi thread data access layer which is invoked from different points. A single call to this layer is likely to spawn several threads to parallelize requests to the DB.
What I'm looking for is a logging tool that would allow me to define "activities" that are composed by various threads. Therefore, the same method in the data access layer should log different outputs depending on its caller. The ability to group different outputs to summarize the total cost of an operation is also important.
Although the application is in Java, language is not a restriction; what I need are the design guidelines so to eventually implement it. We are currently using log4j, but can't get this behaviour from it.

You should also have a look at the nested diagnostic context feature of log4j. Pushing different contexts to the logger for different callers might do the trick for you.

You should be able to pass a logger around, so you create a logger based on some "common" for the task data - i.e. username, etc. Then, pass this logger as parameter to all methods you need. That way, you'll be able to set different filters and/or rules in your log4j config file. Or to scrape the output file based on the logger name.
EDIT: Also check MDC and NDC classes in log4j. You can add there context data.

In log4j you can log the thread name with the "%t" pattern. See log4j Pattern Layout.

In one of my (web) applications, i use a ThreadLocal logger that captures logging information into a StringBuilder. The logger object is initialized in the HttpServlet#service method, if a trace parameter is set (if it is not set, there is a very fast null-logger). The resulting output is either dumped as a HTML comment into the requesting page, or written to a log file in one segment.

In Java5 (and later) you can call
StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();
Inspect the stack trace to whatever depth you want and log accordingly.
In Java 1.4 you can get the same info with
StackTraceElement[] stackTrace = new Exception().getStackTrace();

You want to associate logger objects with threads I think. A ThreadLocal variable holding a log4j logger instance for each thread might help:
http://java.sun.com/javase/6/docs/api/java/lang/ThreadLocal.html

You will need to pass some structure to the data access layer that identifies the current "activity". You might already have an "Activity"-class that makes sense, you might use a Logger-instance as Sunny suggested or you might use a third structure to keep track of the activity-context.
In any case, since your "activity" is processed across multiple threads you cannot use thread-local-storage for keeping track of the current "activity", like most of the other current answers suggest. You will need to pass it around explicitly.
I would suggest making a small facade on top of log4j that expands the interface with methods like
void debug(Activity activity, String message);
and passing the activity-context into this from the data access layer.
You will need to make some modification to the data access layer to allow you to pass the current activity to it, but how best to do that depends strongly on the current interface.
If you use the Workspace-pattern, you might just need to add a setActivity() method on the Workspace-class, but other interface-pattern might require you to add an Activity parameter to all methods.
If you for some reason is unable or unwilling to change the data access layer, you might of course store the activity-context in thread-local-storage before invoking the data access layer and retrieve it just before spawning the sub-threads or enqueing the jobs in the data access layer. That is a workable solution, but is it a bit dangerous to pass information around in that way.

You can use MDC or NDC for your scenario, NDC works on principle of stack while MDC works on Map, here is official documentation for both
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/MDC.html
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/NDC.html