I am using logback in our project. I have gone through the links for logback for curly braces.
logger.debug(" My class output value - {}, object toString() {}", object.value(), object.toString());
Debug is not enabled in my project. We saw the toString() is getting called in our project which impact our performance. Will the toString() be called during the code execution with debug disabled?
Can i use toString() with this approach? Because as per curly braces definition string concat will not happen. Will it be only for string concat or is that applicable for method calls too?
From the LogBack documentation:
Better alternative
There exists a convenient alternative based on message formats. Assuming entry is an object, you can write:
Object entry = new SomeObject();
logger.debug("The entry is {}.", entry);
Only after evaluating whether to log or not, and only if the decision is positive, will the logger implementation format the message and replace the '{}' pair with the string value of entry. In other words, this form does not incur the cost of parameter construction when the log statement is disabled.
So, by passing just the object, without invoking toString(), you will save the toString() overhead.
Logback doesn't change the rules of Java: When you call a method, the arguments to the method need to be evaluated in order to be passed to the method. All you save by using the curly-brace notation when the log level isn't enabled is the cost of the String concatenation, that is to say, the cost of constructing the complete String to be logged from the individual components.
If the cost of evaluating the arguments is measurably decreasing your performance to the point where it no longer meets your customers' needs, then you probably want to avoid the cost of running it if the log level isn't enabled. From the Logback Manual, Chapter 2: Architecture, "Parameterized logging":
One possible way to avoid the cost of parameter construction is by surrounding the log statement with a test. Here is an example.
if(logger.isDebugEnabled()) {
logger.debug("Entry number: " + i + " is " + String.valueOf(entry[i]));
}
This way you will not incur the cost of parameter construction if debugging is disabled for logger. On the other hand, if the logger is enabled for the DEBUG level, you will incur the cost of evaluating whether the logger is enabled or not, twice: once in debugEnabled and once in debug. In practice, this overhead is insignificant because evaluating a logger takes less than 1% of the time it takes to actually log a request.
Using the curly-brace syntax (presented shortly after in the manual) often is a good compromise, and I really prefer it just because it helps distinguish between the statement being logged and the data that goes into it. But it isn't quite the same as being skipped entirely if your log level isn't enabled, because the parameters are still evaluated and passed to the logging system before it can figure out whether it needs to log them or not.
Related
Many logging frameworks (e.g., log4j) allow you to pass lambda expressions instead of Strings to the logging API. The argument is that if the string is particularly expressive to construct, the string construction can be lazily executed via the lambda expression. That way, the string is only constructed if the system's log level matches that of the call.
But, given that modern compilers do much method inlining automatically, is there really a point to using lambda expressions in this way? I'll supply a simplified example below to demonstrate this concern.
Suppose our traditional logging method looks like this:
void log(int level, String message) {
if (level >= System.logLevel)
System.out.println(message);
}
// ....
System.logLevel = Level.CRITICAL;
log(Level.FINE, "Very expensive string to construct ..." + etc);
Let's suppose that FINE is less than CRITICAL, so, although an expensive string is constructed, it's all for not since the message is not outputted.
Lambda logging APIs help this situation so that the string is only evaluated (constructed) when necessary:
void log(int level, Supplier<String> message) {
if (level >= System.logLevel)
System.out.println(message.apply());
}
// ....
System.logLevel = Level.CRITICAL;
log(Level.FINE, () -> "Very expensive string to construct ..." + etc);
But, it's feasible that the compiler can just inline the logging method so that the net effect is as follows:
System.logLevel = Level.CRITICAL;
if (Level.FINE >= System.logLevel)
System.out.println("Very expensive string to construct..." + etc);
In this case, we don't have to evaluate the string prior to the logging API call (because there is none), and presumably, we would gain performance from just the inlining.
In summary, my question is, how do lambda expressions help us in this situation given that the compiler can possibly inline logging API calls? The only thing I can think of is that, somehow, in the lambda case, the string is not evaluate if the logging level is not a match.
Your optimization hasn't just introduced inlining - it's changed ordering. That's not generally valid.
In particular, it wouldn't be valid to change whether methods are called, unless the JIT can prove that those methods have no other effect. I'd be very surprised if a JIT compiler would inline and reorder to that extent - the cost of checking that all the operations involved in constructing the argument to the method have no side effects is probably not worth the benefit in most cases. (The JIT compiler has no way of treating logging methods differently to other methods.)
So while it's possible for a really, really smart JIT compiler to do this, I'd be very surprised to see any that actually did this. If you find yourself working with one, and write tests to prove that this approach is no more expensive than using lambda expressions, and continue to prove that over time, that's great - but it sounds like you're keener on assuming that's the case, which I definitely wouldn't.
Raffi lets look at an example on how the compiler inlining you are talking about will change the program logic and compiler needs to be very smart enough to be able to figure that out:
public String process(){
//do some important bussiness logic
return "Done processing";
}
1) Without inlining the process() will be callled regardless of logging level:
log( Level.FINE, "Very expensive string to construct ..." + process() );
2) With inlining the process() will be called only under certain logging level and our important bussiness logic wont be able to run:
if (Level.FINE >= System.logLevel)
System.out.println("Very expensive string to construct..." + process() );
The compiler in this case has to figure out how the message string is created and not inline the method if it calls any other method during its creation.
This kind of optimisation-inlining would work for only really simple examples like you have provided (when it is just String concatenation).
In fact, this API can be used in more sophisticated way:
public void log(Level level, Supplier<String> msgSupplier)
Let's say I have a dedicated supplier, which performs a quite expensive log-message producing:
Supplier<String> supplier = () -> {
// really complex stuff
};
and then I use it in several places:
LOGGER.log(Level.SEVERE, supplier);
...
LOGGER.log(Level.SEVERE, supplier);
Then, what would you inline? Unwrapping-inlining it into
System.logLevel = Level.CRITICAL;
if (Level.FINE >= System.logLevel)
System.out.println(supplier.get());
doesn't make any sense.
As it said in java.util.logging.Logger class JavaDoc:
Log a message, which is only to be constructed if the logging level is
such that the message will actually be logged.
So this is a purpose: if you can avoid construction, that you don't need to perform these calculations and pass the result as parameter.
For tracing and debugging my Java code, I am using a simple Util class rather than a full-blown logging framework:
public class Util {
public static int debugLevel;
public static void info(String msg) {
// code for logfile output
// handling of unprintable characters
// etc. omitted
System.out.println(msg);
}
public static void info4(String msg) {
if (debugLevel >= 4) {
info(msg);
}
}
}
This allows compact single-line statements like the following:
info4(String.format("%16s: ", host) + Util.toHex(sidNetto));
Using a debugLevel variable, I can control the verbosity of my program. Usually, the debug level is set globally at execution start. But it can also be adjusted locally on routine level.
Basically, I save the repeated if (debugLevel >= DEBUG_ALL) {...} brackets around my trace calls. However, the parameters of the call have to be prepared and passed at runtime regardless of the debug level.
My question:
How could I nudge the compile-time optimizer or the JVM to remove superfluous trace calls? I am thinking on the lines of C/C++ function inlining.
A related question regarding C# was discussed here. But I am not sure how to port the suggested answers to Java.
Another related post back from 2010 discussed similar approaches to mine. Im wondering if third-party tools like ProGuard are actually required do solve such a common task.
Most logging APIs that I know suggest to check if the log level is enabled before actually calling the log method in case the message has to be prepared first, e.g.:
if (logger.isTraceEnabled()) {
String msg = String.format("Name changed from %s to %s", oldName, newName);
logger.trace(msg);
}
Some logging APIs like SLF4J also provide more complex log methods that accept a format string and multiple arguments, so that the log message is only built in case the log level is enabled:
logger.trace("Name changed from {} to {}", oldName, newName);
This is sufficient in most of the cases, but sometimes your message is more complex to build, or the arguments have to be converted to strings first. In this case, checking the log level is still a good approach.
Since Java 8, you could also take advantage of lambda expressions to solve this issue. Your log method could be implemented like that:
public void log(Supplier<String> messageSupplier) {
if (isLogEnabled()) {
String msg = messageSupplier.get();
// TODO: log msg
}
}
As you can see, the message is retrieved form the messageSupplier only in case logging is enabled. Thanks to lambda expressions, implementing a Supplier<String> is very easy:
logger.log(() -> String.format("Name changed from %s to %s", oldName, newName));
Update (thanks to Joshua Taylor)
Since Java 8, the java.util.logging API already supports message suppliers, e.g. see Logger#info, so you could easily exchange your logging implementation by the 'on-board' solution from JRE.
This is how most logging frameworks do it. For lightweight arguments (this includes built-in formatters which is a good best practice) do not check the level, otherwise check the level before serialising complicated string arguments.
You could use Java 8 java.util.functions.Supplier<String> for lacy evaluation, but I think there might be no performance to gain over the explicite level test case.
The logger would look like:
void debug(String ptrn, Supplier<String> args...)
And you can use it like:
debug("Hello {0}", this::getName());
It seems weird to not use established logging framework due to their complexity, but worry about minor optimizations like method inlining, while ignoring the greater problem of formatting a log string irrespective of log level. But if you insist on reinventing the wheel:
The JVM (at least the Oracle hotspot JVM) automatically inlines short methods, and performs dead code elimination of unreachable branches. To be detected as unreachable, the log level of the message and the level threshold would have to be constant (compile-time constant, or static final). Otherwise, the JVM will compare logging levels on each call, though it is still likely to perform speculative inlining (inline the branch usually taken, guarded by a conditional branch instruction) which ensures that a branch instruction is only executed in the unusual case.
A much greater concern however is the cost of building the log message, which should only be incurred if the message must actually be logged. The old log4j approach of requiring calling code to check whether logging is enabled before preparing the message is rather verbose and easily forgotten. Instead, SLF4J defers string concatenation to the logging system by having the log methods take a format string and a variable number of objects to be inserted into placeholders. The SLF4J FAQ writes:
The following two lines will yield the exact same output. However, the second form will outperform the first form by a factor of at least 30, in case of a disabled logging statement.
logger.debug("The new entry is "+entry+".");
logger.debug("The new entry is {}.", entry);
It is worth noting that the arguments (here: entry) are of type Object, so their conversion to String only happens if the message actually has to be logged.
To be clear, there is no reliable way to skip evaluation of method arguments by redefining a method, because such elimination may only occur if the just in time compiler can prove the evaluation to be side effect free, which the hotspot jvm only detects if it has inlined the entire evaluation, which it will only to for very simple evaluations. Therefore, the API solution of moving formatting into the logging system is probabaly the best you can hope for.
I have a simple question on logging
why it is common to use this syntax for logging:
LOG.debug("invalidate {}",_clusterId);
not this:
LOG.debug("invalidate" + _clusterId);
In your example, say you have the logging level set to INFO. You'd like to ignore debug-level messages entirely. But the log method can't check the log level until the method is entered, after it gets the parameters. So if you don't know if you're going to need a parameter it's better to avoid having to evaluate it.
With your second example, even though logging is set to info, _clusterId gets toString called on it, then that resulting string is concatenated with the preceding string. Then once the method is entered the logger figures out the debug level doesn't need logging and it throws away the newly-created string and exits the method.
With the first example if debug-level logging is not enabled then _clusterId doesn't get toString called on it and the log message doesn't get built. Calling toString may be slow or create garbage, it's better to avoid it for cases where nothing is going to be logged anyway.
Here's the source code for the debug method on log4j's org.apache.log4j.Category (which is the superclass of Logger):
public void debug(Object message, Throwable t) {
if(repository.isDisabled(Level.DEBUG_INT))
return;
if(Level.DEBUG.isGreaterOrEqual(this.getEffectiveLevel()))
forcedLog(FQCN, Level.DEBUG, message, t);
}
When you have a statement with several parameters, writing the pattern as a string followed by the parameters makes the code more readable. It may also be more efficient, avoiding the needless creation of many temporary string objects, but that depends on how the logging framework implements interpolation internally.
To see the first point, compare this with the equivalent line that uses string concatenation.
LOG.debug("{}: Error {} while processing {} at stage {}", currentFile,
exception.getMessage(), operation.getName(), operation.getStage())
When there's only one parameter it doesn't really matter which one you use, apart from being consistent with the general case.
I need a lightweight solution, which would enable me to log method entry and/or exit and/or exception. I would like to be able to configure:
The entry/exit/exception log level. For instance, I might have Debug level for exit, Info level for entry and Error level for exception.
The entry/exit/exception log message. The log message should allow me to reference the method name, the method parameters and the method result.
Let us take this method, for example:
public ObjectB MyFunc(int x, string s, ObjectA y)
{
// implementation
}
Then, assuming #Log is such a magic annotation, one could have any of these #Log annotations on the MyFunc method:
#Log // should be a reasonable default
#Log(EntryLevel = LogLevel.Info, ExitLevel = LogLevel.Debug) // makes sense to have ExceptionLevel = LogLevel.Error by default
#Log(EntryText = "{method}({methodArguments}) = ?", ExitText = "{method} = {methodResult}")
#Log(EntryText = "{method}({#x}, {#s}, ...)")
Where:
{method} would be replaced by "MyFunc"
{methodResult} would be replaced by the toString of the method result
{#x} would be replaced by the toString of the x method argument
{#s} would be replaced by the toString of the s method argument
{#y} would be replaced by the toString of the y method argument
{methodArguments} would be replaced by something like "x: {#x}, s: {#s}, y: {#y}", or "{#x}, {#s}, {#y}", which could be governed by a boolean flag, like includeArgumentName.
For the record, in .NET we use a modified (by us) version of the Log4PostSharp library, which does all of the above by injecting the right bytecode during the compilation phase. Hence, I have a pretty good idea what I am looking for, though I have absolutely no idea how to do it in Java and whether something like this has already been done.
Thanks.
EDIT
I would like to address the performance issue. The code may be abundant with the logging statements. Executing these statements must not incur any visible performance penalty when no actual logging is performed (due to log level constraints, for instance).
Have you considered AspectJ? Some additional details are at http://www.christianschenk.org/blog/logging-with-aspectj/ and http://marxsoftware.blogspot.com/2008/10/logging-method-entryexit-with.html .
This sounds like a classic AOP problem.
Spring has a flexible solution that is relatively easy to use, see here.
EDIT: I would love to read reactions to Steve Reed's AOP approach. Comments to his answer are encouraged!
I'm a novice, and at some point I realized it would be helpful to know the contents of a variable during program execution. So I started doing this:
EDIT: fixed this. Used to be: var + ": " + var, which was totally wrong. Dumb typo.
System.err.println ( "var: " + var );
Later I learned that this was common practice. At least, where a debugger was unavailable or unwanted.
I use a basic text editor, and typing the print statement every time I need to debug a variable is pretty tiresome, so I thought, why not something like this:
void dbug ( Object obj )
{
String variableName = obj.somehowGetVariableName();
String variableContents = obj.toString();
System.out.println ( variableName +": " + variableContents );
}
But apparently getting the variable name is easier said than done.
java-reflection-how-to-get-the-name-of-a-variable
Am I stuck with:
System.err.println ( "var: " + var );
Or is there a popular shorthand version of this?
I wouldn't try and write any fancy methods around printing out debugging info. Just stick with either LOG.debug(...) if you are using a logger or System.err.println(...) otherwise.
You may wish to use String.format("var=%s val=%s", "VarName", val) instead of the String concatenation.
Make sure that you override the toString method in each of your classes to provide meaningful debug info.
At the end of the day it's often easier to fire up the debugger and take a look to see what's going on instead of having to trace through loads of logged debug lines.
The only time when i'd use your kind of debug method would be if my application maintained all of the state inside a map which I could easily print out the key value pairs (e.g. the session map in a web application).
I think that System.err.format is what you want:
System.err.format("var: %s\n", var);
is a shorthand for:
System.err.println(String.format("var: %s", var));
Have a look at Simple Logging Framework, it allows you to type:
class Example {
static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory.getLogger(Example.class);
void doSomething(Object obj1, Object obj2) {
LOG.debug("This is object 1: {}, and this is object 2: {}", obj1, obj2);
}
}
Personally I don't suggest to use sysout statements anywhere. You should always use a debugger (using some IDE). I cannot imagine where this wouldn't be wanted.
Otherwise I'm suggesting logging frameworks like log4j, but again, this already gets more complicated where I would again then switch to a real IDE with debugger.
Some thoughts:
I would implement toString() on objects of interest, and in that print out the members in a friendly fashion (e.g. convert timestamps to a readable format etc.). I usually choose a format like:
Object[member1=,member2=...]
Otherwise printing the object alone will give you the classname plus the identity hash code, and (as you've discovered) that's not hugely useful!
Commons has a facility to do this automatically. But here's a simple toString() tutorial which I think is more appropriate.
There are logging frameworks that you may be interested in in the future. e.g. check out Log4j. At the moment, however, I wouldn't worry about that.
If you use IntelliJ IDEA, you can use the "Live template" shortcut for printing to System.out such as soutp (and then a TAB) to debug the method parameters, soutv to trace the name of a variable along with it's value, etc.
To read the list of shortcuts\modify it, go to File->Settings->Live Templates->Output
My first advice is: stick to java.util.logging package. There is really no need for 3rd party logging libraries.
Get instance of java.util.Logger
Logger logger = Logger.getLogger("some.package.XyzClassName");
Log objects (using placeholder {N})
logger.log(Level.INFO, "logging stuff {0} and {1}", new Object[]{new String("Test"), 1222});
In case of user defined classes you need to have some sensible toString() override implementation, because this method is called when replacing placeholder in message {N}.
It's a stretch, but...
You're getting advice to use an IDE instead of a simple text editor. I'd go along with that, 100%.
You're getting advice to use a logging framework or a debugger instead of println() calls. Well, sure, but...
Better yet is unit tests. Don't ask what it is - tell it what you expect. Then, the integrated unit-testing framework (junit, typically) will verify that you're getting what you expect. The more I use unit tests, the less I need debugging and the less I need println's. And when something changes, the testing framework tells me - I don't need to manually re-evaluate every test's output, just watch the bar.
Unit tests are better than debugging, better than logging. They're not a 100% replacement for debugging and logging, but start using them and you'll find much less need for those tedious activities.
I would recommend you to properly configure and use Apache Log4J. System.out or System.err lines cause a lot of delay in execution of program. (You can confirm this by adding some time info that how much time your program takes without System.out etc and how much without these lines.)
Logging APIs use separate threads to log your logging info in log files, thats why they are more useful in realistic applications. Also logging APIs like Log4J give a lot of control over how to configure and format your logs. For example look here a few logs generated from Log4J:
2012-01-05 15:16:41,730 [main] 16 DEBUG dmfssecbuild.IOUtil -
2012-01-05 15:16:41,730 [main] 16 DEBUG dmfssecbuild.IOUtil - Application Configs Loaded
2012-01-05 15:16:41,730 [main] 16 DEBUG dmfssecbuild.Constants - Running Application between dates.
2012-01-05 15:16:41,730 [main] 16 DEBUG dmfssecbuild.Constants - Sat Jan 01 00:00:00 GMT+05:00 2011 From Date
2012-01-05 15:16:41,730 [main] 16 DEBUG dmfssecbuild.Constants - Mon Dec 31 00:00:00 GMT+05:00 2012 To Date
Have you tried DoodleDebug? It's an Eclipse plugin and meant to be as easy as System.out.println() but much more powerful.
It can be found here: http://scg.unibe.ch/wiki/projects/DoodleDebug
You can get access to the variable names with AOP and compile-time weaving. The nice thing about this is that if you don't do the weaving, you don't add the debug logging code and your runtime code is leaner and quicker as a result.
Here's an example of using AspectJ to throw an exception when a field is set to null. Note the usage of "joinPoint.getSignature()" to get access to the code metadata.
#Aspect
public class NotNullValidator {
#Pointcut(value = "set(#com.acme.NotNull * *.*) && args(valueBeingSet)")
private void setOfNonNullField(final Object valueBeingSet) { }
#Before(value = "setOfNonNullField(valueBeingSet)")
public void validate(final JoinPoint joinPoint, final Object valueBeingSet) {
if (valueBeingSet == null) {
throw new NullPointerException("Cannot set " + joinPoint.getSignature().getName() + " to null.");
}
}
}
See JoinPoint Javadoc to see what else you can get (line numbers, source and target objects, etc).
Regardless of what route you take, I'd recommend doing the following:
Make a custom Debug.out(Object) method in a "Util" or "Debug" class, so no matter how you decide to log the errors, you can change it for the whole program easily;
If you use Eclipse, VSCode, or any other IDE that has "Templates", make a "debug" template that will just write Debug.out("<method name> " + string). At least with Eclipse, you can make the template write the method name for you.