I have a scenario wherein I invoke a third party REST URL from inside a FOR loop for different values of the field JOBNAME. I wish to log the logging statements for each JOBNAME inside a separate log file i.e. the request sent to the REST URL, all the intermediate business logic logging and the response received would need to be part of different log file for each JOBNAME.
The catch here is that the processing involves no threads. Is the MDC approach for log4j feasible for non-threaded scenario's as well. If not, how can I achieve this?
Having a single thread you can still put a variable into MDC context and let the calls propagate and log whatever they do. Just make sure that you do MDC.put(..) before interesting staff gets called and remove it when you are done so it doesn't linger. For example, you can do MDC.put(..) within the FOR loop and each iteration will be with different context even though it's the same thread. Not sure about elsewhere but in Java it works fine.
Related
My requirement is that the user will see localized messages at the end of the process. If the process failed, there will be at least one "external" message of type error. If the process was successfully executed, the user might see multiple "external" information messages.
Additionally, all messages (including internal) shall be logged by the application.
Now here comes the tricky part. Assume the following callstack:
controller.uploadImageHandler
imageService.createImageContainer
galleryService.loadForImage
galleryService.validateGalleryAttributes
Now validateGalleryAttributes will validate 3 attributes. 2 aren't matching, so the messages Attribute A must be X and Attribute B must not be empty of type External are generated and logged to the application log. The calls before added multiple information messages.
Now I need have to
abort processing
bubble the Messages up
Is the right approach to pass along a Message Collection (or add a collection to the logger) into every method, throw an Exception upon failure (even simple ones such as invalid attributes), extract the messages at the controller level and return them to the user? It's not enough to add them to the exception class, since info messages must also be possible and multiple methods might produce just messages.
But honestly it seems a bit weird to have a collection that is inspected upon success and exception. Is there a better language construct?
Theres two alternative solutions that I could come up with.
If you are into monads at all you could move away from using exceptions to a custom LoggingEither<L, R> or LoggingResult<S, E> return type. They would carry the result/exception as well as all accumulated messages so far. You would just have to make sure to merge the messages of all methods you call into the object you return in the current method. For an idea of how these monad types could be implemented take a look at vavr.
If everything happens in the same Thread you could consider using ThreadLocal to keep track of all the log messages. This eliminates the need of passing a collection down the stacktrace. You will still need to take care of bundling messages and resetting the ThreadLocal, though.
I'm looking to log the request and response to a web service. I'm using slf4j with underlying log4j2 implementation. My logger statement looks like the below.
LOGGER.info("{}",new CustomObject(request,response,param1,param2));
I have implemented the toString methods in all the necessary objects and in the CustomObject class to log all attributes of that object.
I see that the toString method of the CustomObject is called before it passes the log message to the Asynch logger.
Is there anyway that the serialization / toString method call of the custom object be deferred to when the actual logging takes place?
This is by design: if an Object is logged it's possible that it is mutated before the background thread can log it, and you would end up with a log entry different from what you intended.
In your example your application doesn't hold a reference to the CustomObject, so it cannot change, but Log4j2 can't know that, so it takes the conservative approach.
There is a system property to switch this off, but using it means all objects logged must be effectively immutable or you will find your logs are lying to you... (I'm also not 100% sure that the system property still works since Log4j2 became garbage free in version 2.6.)
Update (2017-12-09): the system property still works in post 2.6 versions. (But I would not recommend rendering the message in the background thread unless you are very confident that the application doesn’t modify the objects that were logged.)
When we add logs in to the java class (using log4j), Is it fine to add thread id with that log messages? is it a bad practice? My idea was to add this thread id; Once we examine a log file of a multithreaded application, it is difficult to find out the correct flow using logs. (As an example, say authentication flow). Is there any better approach for this rather than logging thread id?
Log4j already supports the thread name using t placeholder in its pattern layout. So this is a supported feature that you should use if you find it useful. This way you don't need to pass the thread name manually. However it doesn't make use of the thread ID. So you should give meaningful names to your threads. This should be preferred as it is more indicative to what is going on in your application than just plain thread ids.
If it is the thread id, please refer this answer
However if you only need the thread name, you can use the t pattern configuration, please refer here.
If you are using Java Logger API - LogRecord has method getThreadID() and can be configured to log.
For log4j there are no ThreadId methods available and No harm in logging it.
In cases If your class inherits from Thread, you can use methods getName and setName to name each thread. Otherwise you could just add a name field to MyTask, and initialize it in your constructor And use a more sensible Thread Name instead of ID.
Logback has a special appender called SiftingAppender which provides a very nice solution to the type of problems you describe. A SiftingAppender can be used to separate (or sift) logging according to any runtime attribute, including thread id
If your concern is about several JVMs writing to the same FileAppender, then i'd suggest two things:
using SLF4J as a logging facade
using logback as logging implementation, in prudent mode
In prudent mode, FileAppender will safely write to the specified
file, even in the presence of other FileAppender instances running in
different JVMs, potentially running on different hosts.
Refer this: http://logback.qos.ch/manual/appenders.html#SiftingAppender
I have a servlet, and that servlet uses a .net web service to perform some function. To do this, I created the webservice client in Netbeans using the "jax-rpc" style client.
Let's say that my service name is "Tester". Then two of the generated classes are called "Tester", and "TesterSoap".
To get a reference to the web service, I need to do this:
Tester t = new Tester_Impl();
TesterSoap tsoap = t.getTesterSoap();
To use the webservice, I can then do this:
tsoap.runTest();
My question is, since this is a servlet which gets executed many times, should I store the first two lines in static variables (so they only ever get executed once), or store them locally so that they execute everytime the servlet is executed?
Another way of asking the same question: is there a performance hit everytime the first two lines are called? (I'm testing everything locally so it's hard to measure).
Thanks...
If the default constructor and any of the initialization blocks of the Tester_Impl() class and the method getTesterSoap() doesn't do anything expensive (e.g. reading file from disk, loading data from DB, connecting a socket, etc, I however suppose it doesn't) then you don't need to worry about it.
You can consider declaring them as an instance variable of the class extending from HttpServlet. But, a big but, it is going to be shared among all HTTP requests, because there will be only one instance of the particular servlet class during whole application's lifetime. So if the Tester_Impl class is supposed to have a state, then it is a very bad idea to declare it as an instance variable. It would then be shared among all requests. With other words, it's not threadsafe. If you want to ensure threadsafety in servlets, then declare everything in the very same method block.
I would not optimize prematurely here. Test this out in as close to a production environment as you can (i. e. not on your local box) and see what the performance hit is. What I've done in the past is write a small shell script that hits my server with wget n times with a delay of k milliseconds and then measured the latency, possibly instrumenting the code with some timing or profiling myself (or with jvisualvm or some other profiling tool).
If you want to protect your design from a possible performance hit without doing the testing, you could use a factory to provide instances of the service client and then you could swap out singleton service clients for many of them whenever you feel like it.
I have a layered application in Java which has a multi thread data access layer which is invoked from different points. A single call to this layer is likely to spawn several threads to parallelize requests to the DB.
What I'm looking for is a logging tool that would allow me to define "activities" that are composed by various threads. Therefore, the same method in the data access layer should log different outputs depending on its caller. The ability to group different outputs to summarize the total cost of an operation is also important.
Although the application is in Java, language is not a restriction; what I need are the design guidelines so to eventually implement it. We are currently using log4j, but can't get this behaviour from it.
You should also have a look at the nested diagnostic context feature of log4j. Pushing different contexts to the logger for different callers might do the trick for you.
You should be able to pass a logger around, so you create a logger based on some "common" for the task data - i.e. username, etc. Then, pass this logger as parameter to all methods you need. That way, you'll be able to set different filters and/or rules in your log4j config file. Or to scrape the output file based on the logger name.
EDIT: Also check MDC and NDC classes in log4j. You can add there context data.
In log4j you can log the thread name with the "%t" pattern. See log4j Pattern Layout.
In one of my (web) applications, i use a ThreadLocal logger that captures logging information into a StringBuilder. The logger object is initialized in the HttpServlet#service method, if a trace parameter is set (if it is not set, there is a very fast null-logger). The resulting output is either dumped as a HTML comment into the requesting page, or written to a log file in one segment.
In Java5 (and later) you can call
StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();
Inspect the stack trace to whatever depth you want and log accordingly.
In Java 1.4 you can get the same info with
StackTraceElement[] stackTrace = new Exception().getStackTrace();
You want to associate logger objects with threads I think. A ThreadLocal variable holding a log4j logger instance for each thread might help:
http://java.sun.com/javase/6/docs/api/java/lang/ThreadLocal.html
You will need to pass some structure to the data access layer that identifies the current "activity". You might already have an "Activity"-class that makes sense, you might use a Logger-instance as Sunny suggested or you might use a third structure to keep track of the activity-context.
In any case, since your "activity" is processed across multiple threads you cannot use thread-local-storage for keeping track of the current "activity", like most of the other current answers suggest. You will need to pass it around explicitly.
I would suggest making a small facade on top of log4j that expands the interface with methods like
void debug(Activity activity, String message);
and passing the activity-context into this from the data access layer.
You will need to make some modification to the data access layer to allow you to pass the current activity to it, but how best to do that depends strongly on the current interface.
If you use the Workspace-pattern, you might just need to add a setActivity() method on the Workspace-class, but other interface-pattern might require you to add an Activity parameter to all methods.
If you for some reason is unable or unwilling to change the data access layer, you might of course store the activity-context in thread-local-storage before invoking the data access layer and retrieve it just before spawning the sub-threads or enqueing the jobs in the data access layer. That is a workable solution, but is it a bit dangerous to pass information around in that way.
You can use MDC or NDC for your scenario, NDC works on principle of stack while MDC works on Map, here is official documentation for both
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/MDC.html
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/NDC.html