I am writing a small xml transformation layer in Java. I receive xml via web service, modify it, and then send the modified xml to another system. I then wait for a response and return the response to the original caller.
System A -> Me -> System B -> Me -> System A
I want to log the request I receive, the request I send, the response I receive, and the request I send. Basically I want to log the xml where each arrow is in my diagram.
My problem is with the RollingFileAppender. I try to roll at 10MB, sometimes it does and sometimes it doesn't roll. If it rolls a couple times, and then stops, it will continue to rename the rolled files from 3 to 4 and 4 to 5 and so on.
My best guess is that when the 10MB mark is crossed, there are multiple threads writing to the log file so the file cannot me renamed. I am hoping that Log4J has an easy solution for this, but if necessary, I am open to switching to a new logging framework. Thank you in advance for any help.
EDIT
Here is my properties file.
log4j.rootLogger=DEBUG, fileOut
log4j.appender.fileOut=org.apache.log4j.RollingFileAppender
log4j.appender.fileOut.File=/logs/log.log
log4j.appender.fileOut.layout=org.apache.log4j.PatternLayout
log4j.appender.fileOut.layout.ConversionPattern=%d %-5p %c - %m%n
log4j.appender.fileOut.MaxFileSize=10MB
log4j.appender.fileOut.MaxBackupIndex=10
log4j.appender.fileOut.append=true
EDIT 2 This is essentially a bump, as this post has a low number of views. I feel like this cannot be a unique problem. Any help is much appreciated. Thanks!
Log4J initializes itself at the classloader level. Within a certain classloader and its ancestors, Log4J can only be initialized once and the same Log4J configuration applies to all Log4J calls within the classloader.
As long as all of your logging calls are performed within the same Log4J configuration "realm", Log4J knows how to synchronize access to the physical file pointed at by the rolling appender configuration; when the time comes to roll, rolling is performed with no problem.
Things become problematic once you have two (or more) Log4J "configuration realms" using the same physical file for the rolling appender configuration. That could be:
Two different web applications on the same physical JVM
Two different web applications on two distinct JVMs
Same web application horizontally clustered on two distinct JVMs
(etc)
Log4J simply has no way of knowing who else, other than itself within the same Log4J configuration realm, uses that file. So, what ends up happening is that Log4J on System A attempts to roll the file (because it thinks that no other processes are accessing that file), and fails because someone on System B is using the file at the same time.
This is a known limitation of using file appenders, and you can't really blame Log4J for this: Log4J simply doesn't have the means of monitoring who else, other than Log4J in the same "configuration realm", is using that file.
For such usage scenario, you can use the Log4J socket appender.
If your scenario doesn't involve multiple Log4J "configuration realms", then try adding -Dlog4j.debug=true to the JVM parameters and see what exactly is going on during the file rolling operation.
For others that arrive here, check you are using RollingFileAppender NOT FileAppender!
Cut and paste errors are too easy, for me at least.
I also faced the same issue in my application.
Thanks to #Isaac found that I was doing DOMConfigurator.configure for the same log configuration in 2 web applications deployed in the application server.
I commented one of them and rolling over happened as expected.
Related
I have a 3rd-party developed big complex application full of java.util.logging.Logger#finer() calls.
Normally I use log4j and sl4j for logging, and I don't know very much of java.util.logging, even less about all the configuration details and possibilities.
Every single google about "java.util.logging" points to a full explanation about streams handlers, formatters, levels. Everyone assumes I really care about all this stuff (it is a reasonable assumption actually). But I couldn't care less.
I'm not really interested in separating logs per file, or file rotation, zipping, email, remote log etc. I'm also not concerned about log level and message formatting thrills.
All I want to do is to run this application with all available log messages spitting to stdout.
Is there an easy simple direct way to do this?
Something like jvmargs -Djava.util.logging.level=FINEST -Djava.util.logging.to=stdout.
Or maybe some simple file dropped into some location in the classpath?
All I want to do is to run this application with all available log messages spitting to stdout
Don't do this in production but it is the fast, easy, hacky way:
Edit the logging.properties file located in java/conf. E.G. /usr/lib/jvm/default-java/conf/logging.properties
That file is setup to attach a ConsoleHandler to the root logger which is what you want to do this. You just need to adjust the levels to see the output. This is a global change so be aware.
Edit that file and:
Change .level= INFO -> .level=ALL
Change java.util.logging.ConsoleHandler.level = INFO to java.util.logging.ConsoleHandler.level = ALL
Save the changes and restart your app.
The recommended way:
Most JVMs are not in your control (production/dev server) so you can instead copy that file to a location you own and use the java.util.logging.config.file system property.
E.G. -Djava.util.logging.config.file=/home/myuser/myapp/logging.properties
That way you are free to make changes that are local to your program and not global to the machine.
I'm trying to write logs of multiple services to same file, but my rolling policy given is not working, tried with both time based and size based rollings. Thing is my services are running simultanously and writting there logs to same file in my local directory. When tried to write logs by single service it is working as expected.
Please help me to solve this issue tried with different rolling policies.
Appender to log to file
${LOG_FILE}
Minimum logging level to be presented in the console logs
INFO
${LOG_PATH}/archived/log_%d{dd-MM-yyyy}_%i.log
10KB
I had an experience similar to yours with Log4j 1.x then I debugged an appender back then (~5-6 years ago) and came to the following conclusions:
I don't think you can write data from multiple services into the same file. In other words,
Logging framework usually assumes that only it can change the file. In some Operating Systems (windows) it will even stop writing into file if some other process will rename / change the current file.
Of course its just a code and you could create a more sophisticated appeneder that will probable make it work, but frankly I don't think it worth the effort.
So I suggest writing into different files, where file name can be generated in a way that it will contain a pid of the resource. The downside of this method is that if the process dies and then re-runs, on-one will take care of the old resources.
Another approach (somewhat similar) - is to create a folder with logs for each service so that they'll get different logs based on folder (even if files in these folders will be with the same name).
I have 4 java/Java EE applications-
Two are server based j2ee applications running in WebSphere.
Other two are standalone java applications.
My logging framework is log4j using log4j.properties.
Question 1: Can I have a same log file for logging from all the applications. Will it cause any file writing issues even if all the applications are writing at the same time?
Question 2: If all the applications can write without any issues, how can I pre-append the application name to each of the log statement?
Question 1:
I believe by default it will NOT work.
If you are using SLF4J, consider switching to LogBack. In LogBack's File Appender, there is a prudent mode that allow multiple FileAppenders in different JVMs to write to same log file (of course, all of them need to have prudent mode turned on)
http://logback.qos.ch/manual/appenders.html#prudent
Question 2:
You should NEVER make the log message different by manually logging your app name.
There are quite some way to do. The easiest way is: As you are having two different applications, you can have different logging config files for them. Just add corresponding app name information in the log pattern used by the appender will serve what you want.
We have a j2ee web environment. The server is configured to share session and possibly classloaders across multiple webapps. Basically, one classloader could server multiple web apps.
This seems to cause issues with log4j. Different webapps could have different log4j configurations but the logging will move to the same file.
Reading online, it looks like log4j uses singletons a lot, in terms of the appenders and other functionality.
Is there a way to completely separate the log4j configurations from one webapp from the other.
Server: websphere6+
Log4j: 1.4.2
Java: 1.5
Example log4j.properties (webapp1):
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=/usr/local/file1.log
log4j.additivity.com.app=false
Example log4j.properties (webapp2):
log4j.appender.Z=org.apache.log4j.RollingFileAppender
log4j.appender.Z.File=/usr/local/file2.log
log4j.additivity.com.app=false
Right now, logging from webapp2 may appear in the webapp1 logs and vice verse. We don't want that.
Possible Solution:
It might be possible to add a custom file appender? Would that fix the issue and what code would I add to the custom appender?
Is it possible to change the log4j initialization. E.g., could I use some startup servlet to load logj4 for each webapp.
You have two ways to solve your problem:
Configure your app. server so it doesn't share classloaders across multiple webapps. When it does do that, because of log4j's nature, only one log4j.properties file will be loaded.
If you leave the app. server so it shares classloaders, then use one "master" log4j.properties file. In it, define appenders for root of every one of your applications (example com.mycompany.webapp1, com.mycompany.webapp2)
No solution is perfect. Particularly, the second one will be problematic if your web apps share some classes that use log4j. In that case, logs from both apps will end up in the same file.
Your problem is a common one. To understand more about this topic, google for "log4j and j2ee".
Edit: since solutions 1 and 2 aren't feasible, you could try something else:
Use log4j.properties file per application. In every one of them, define an appender for their root (as explained in solution 2), and set additivity to false. This also won't be perfect if there is any class sharing between them.
Configure log4j programatically for every application. Since they're legacy applications, this could be tough. One way to do it is to use ServletContextListener for every application and configure log4j on application startup. I haven't personally tried this, so I'm not 100% sure if there will be clashes due to shared classloaders.
Log4j won't be really updated anymore. Ceki Gülcü, who created log4j, stated that he will focus his efforts on slf4j and logback, to correct some mistakes he did during development of log4j.
We are using the Windows installation of Tomcat 6. By default, the log4j output for our app goes to the ${catalina.base}/logs/stdout_.log file. This log file only rolls over when we restart Tomcat, and the file name always includes the date.
I would prefer it to behave like a DailyRollingFileAppender, where it renames the file when it rolls over... that way I can just open Notepad++ and see today's logs, since Notepad++ will remember that I opened that same file yesterday. :)
I know I can just create another appender in log4j, but I would end up with the stdout.log and another log file, and I'm afraid there would be a minor performance hit for logging to both files. I've tried adding swallowOutput=true to my context.xml but I still get all logging in stdout.log. Any ideas?
Have you tried the steps outlined in Logging in Tomcat? If you follow the steps you'll end up with log4j.properties in the lib directory that you can customize to your heart's content.
The biggest performance hit is when preparing objects that you want to log (you know, when you do logger.info(" operating on " + myObject.toString + " bla bla bla" ) then doing myObject.toString() has the biggest cost). If you already have them than loggin to file is not a problem. And log4j is really well balanced and optimized, it uses buffers to write logs, so it do not make too frequent calls to file system.
Just create another appender, you will have a differentation from tomcat logs and your appilcation logs. How many logs do you have? A 1GB a day or more that you are afraid of performance loss? Don't assume anything before testing it. Just set-up, and do some kind of performance test.