We implemented a small routine that in case of specific errors we get an email to let us know something happened. It's pretty easy to include some error information in the email but we would like to include what was the previous logged informations when it happened.
At first I tried to retrieve the last lines from the file where the log4j are saved. The problem is that many other threads are working at the same time and I never get relevant informations. Either the buffer doesn't have time to write the logs or something else is faster at writing something else.
Is it possible to add an appender to a specific log4j at runtime to be able to retrieve only those specific logs and only if needed ?
I would like to be able to still log with the usual commands log.error, log.warn... but I want to be able to retrieve the log content in a try/catch situation. something like a memory only buffer appender?
Would that work for all the children loggers ?
Class 1 - call Class 2
Class 2 - call Class 3
Class 3 try/catch an error
log.getLogs returns all logged informations from Class 1..3
Or am I dreaming here ?
We are using Jira and in logs in files, so I guess no query would work.
Related
I have made a nice UI with three different logs (a general log and two class specific ones).
Every log can print different lines with different colors.
I was thinking of doing this so I can show info/errors/warnings.
Now, the thing is, that I'd like to have detailed debug only when I set a variable (something like detailedDebug = true).
I'd like something like this:
Simple | Detailed
Error thrown in ... | Error thrown.. + dump of all variables related to the error
Now, with if statements I can achieve that easily, but, that seems overly complicated (complicating the code for debugging reasons too).
How could I implement this (while making it easy to use and most importantly clean)?
Should I make a method in every class that uses the logging features that automatically checks for a variable then does what asked?
You should use the the log level as the variable to control the detail. When you want more detail, turn the level down to FINEST.
Hoewever, some operations that you wish to log might require considerable resources to calculate the detail (example, you may retrieve info from the DB, etc). In this case you should use if statements because the resources will be consumed even if the log level is at ERROR level.
Example :
The following code will always execute :
logger.log(Level.FINEST, "Some detailed log info which sows the results from DB {0}",
new Object[]{ getResults() });
If you only want to execute this code when you are showing FINEST, you need to wrap the statement in an if statement :
if (logger.isLoggable(Level.FINEST)) {
// Some intensive logging
}
I'm using the commoncrawl example code from their "Mapreduce for the Masses" tutorial. I'm trying to make modifications to the mapper and I'd like to be able to log strings to some output. I'm considering setting up some noSQL db and just pushing my output to it, but it doesn't feel like a good solution. What's the standard way to do this kind of logging from java?
While there is no special solution for the logs aside of usual logger (at least one I am aware about) I can see about some solutions.
a) if logs are of debug purpose - indeed write usual debug logs. In case of the failed tasks you can find them via UI and analyze.
b) if this logs are some kind of output you want to get alongside some other output from you job - assign them some specail key and write to the context. Then in the reducer you will need some special logic to put them to the output.
c) You can create directory on HDFS and make mapper to write to there. It is not classic way for MR because it is side effect - in some cases it can be fine. Especially taking to account that after each mapper will create its own file - you can use command hadoop fs -getmerge ... to get all logs as one file.
c) If you want to be able to monitor the progress of your job, number of error etc - you can use counters.
I've got a huge number of log messages in project's log. I want to reduce that number by logging only first N of them for every line of code where log is used.
Are there any existent solutions for my task?
Thanks in advance.
This is not something i have tried myself, but if you use logback as the underlying logging mechanism of slf4j, I imagine you could write a custom filter that takes care of filtering the log output as you specify it.
http://logback.qos.ch/manual/filters.html
I decided to implement a LoggerWrapper by myself.
We can obtain the current line of a caller, count its occurrence and decide whether print a message or not. I think it's quite simple but a kind of routine.
Is there a cleaner way for me to write debug level log statements? In some ways one could say that the string literals are basically commenting the code and providing logging in one line and that it is already very clean. But after I add debug level log statements, I find the code much less easier to read up and down. Take this example (I may update to a real example if I get back to my home PC):
int i = 0;
logger.debug("Setting i to 0,"); //Just an example, would show something more complex
i++;
InputStream is = socket.getInputStream();
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
IOUtils.write(request, dos);
logger.debug("request written to output");
while (!is.read(buffer))
logger.debug("Reading into buffer");
logger.debug("Data read completely from socket");
CustomObject.doStuff(buffer);
logger.debug("Stuff has been done to buffer");
You could try using aspects, although these have the limitation that you can only put log statements "around" method calls, i.e. before entering and/or after leaving a specific method.
For more detailed logging, I am afraid there is no other way than hand-coded log messages.
I typically strive to remove the not-so-much-needed debug log statements from the code once I made sure that it works the way it should (for which unit tests are a must).
Ask yourself if I run this in a different machine/country/planet, and things go wrong and all I have is only a log file what information do I need to know what has gone wrong ?
Use debug logs in a for loop, or a while loop sparingly. For example, if you are reading 1000 records from a file, performing an op for each record. You could record before the for loop that "file exists and is readable and is going to read 1000 records" and print status after the process is done. If it is say 1000000 records then you could print something every say 100 or 1000 iterations
In your code except for the logger for setting i to 0 everything else sorta makes sense to me. Also care to use log.isDebugEnabled() if your string in the logger statmeent is hard to compute..
ex:
if(log.isDebugEnabled) {
logger.debug("Here " + obj.aMethodCallThatTakes5MinsToEvaluateToString());
}
UPDATE 1: SLF4J solves only half the problem.
if(slfLog.isDebugEnabled) {
slfLog.debug(obj.getObjectThatTakes5Mins());
}
Yes the toString is prevented but if you are logging an actual object which is result of some computation you are not prevented.
If you want very fine grained debug instructions I am not sure you can separate the actual code from the debug code.
If you want it at a higher level, maybe adding your logging using AOP could help make things easier to read, maybe use a proxy object?
But if you have debug instructions as fine grained as in the example you provided, IMHO you could gain more by replacing the loggers with unit tests. Don't write in a log that something happened, test that it did.
You will not be able to do much if you dislike the log statements. The information needs to be there somehow.
What you CAN do, is strongly considering what NEEDS to be there. You are basically writing for the log file reader who per definition does not know about how your program works, so the information needs to be concise and correct. Personally I very frequently add the method name to the log statement.
Also note that slf4j allows you to use the {}-syntax which helps somewhat
log.debug("main() date={}, args={}", new java.util.Date(), args);
Also note that having unit tests, allow you to move much stuff to there simply because you know that THAT works.
I have a legacy PSVM application which I'd like to redirect its logging output to unique files per execution. So, if I invoke it at 10:00, then have it redirect it's output to {thread-id}-10:00.log; and another thread of execution may begin an execution at 10:01, and its output would go to {thread-id}-10:01.log. I understand that this is not elegant.
My questions are:
is this possible?
does someone have an idea of how to approach?
is it possible to release/destroy an appender when it's no longer needed?
Thanks!
I would start with FileAppender and derive from that to create your own. Simply modify your version to get the current thread id and append a suitable thread-id/timestamp to the file prior to creation. You would maintain (say) a map of (buffered) FileWriters keyed on thread id.
Writing appenders is quite trivial - here's a Javaworld guide on how to do it.
In the above, is it at all likely that your program will start up twice in one minute ? Would you want to append a process id or similar to maintain uniqueness ?
It is not possible, at least not easy to do in log4j. However, if you look at SiftingAppender shipping with logback (log4j's successor), it is designed to handle the creation of appenders on runtime criteria as well as their removal when no longer needed.
If you application needs to create just one log file per application launch, you could simply name your log file based on a timestamp. Shout on the logback-user mailing list if you need further assistance.