We have a bunch of applications being deployed on Webshere Application Server (WAS). Somewhere on the class path we have a very basic log4j.properties with that looks similar to below:
log4j.rootLogger=ERROR, FILE, CONSOLE
#
# CONSOLE appender is a ConsoleAppender
#
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%X{component} [%-5p] %c - %m%n
#
# FILE appender is a FileAppender
#
log4j.appender.FILE=org.apache.log4j.RollingFileAppender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=##%d{dd-MM-yyyy HH:mm:ss,SSS} %X{component} [%-5p] %c - %m%n
#
# MyApp Debug Levels.
#
log4j.logger.com.my.company.my.app=INFO
Note the missing log4j.appender.FILE.File File from our log4j.properties, This is configured art WAS level via Application Servers -> MyServer -> Logging and tracing -> JVM Logs with the file name set as below:
${SERVER_LOG_ROOT}/SystemOut.log
Changing the debug level works fine e.g. adding to the log4j.properties the following line will result in getting debug entries for MySpecialClass in the logs.
log4j.logger.com.my.company.my.app.MySpecialClass=DEBUG
That makes me conclude that my properties file is considered.
However we recently received a request that in addition to the normal log entries to output some special log entries (application error related logs) to a separate file. My initial thought was to add one more rolling file appender and leave that do the work. To my surprise it did not work, everything related to that appender was just ignored.
Then I thought this may be because the log file is configured in WAS and everything goes there regardless of what we put in the log4j.properties and I said it looks a bit odd to have multiple apps writing to the same log file. However I did not want to spend too much time on this and because the special log entries were the error ones I just said lets just use the existing WAS feature of having a separate file for System.Err logs: Application Servers -> MyServer -> Logging and tracing -> JVM Logs:
${SERVER_LOG_ROOT}/SystemErr.log
To achieve this I created my own com.my.company.my.app.CustomPatternLayout and implemented this very basic format method:
public String format(LoggingEvent event) {
String log = patternLayout.format(event);
Level logLevel = event.getLevel();
if (Level.ERROR.equals(logLevel) || Level.WARN.equals(logLevel)
|| log.contains("Exception") || log.contains("PEM_LOGGER_SEV3")) {
System.err.println(log);
}
return log;
}
I also replaced the file appender in log4j.properties to use my custom pattern layout
log4j.appender.FILE.layout.com.my.company.my.app.CustomPatternLayout
Again nothing happened in my logs and the expected log entries never got written to the SystemErr.log. More than this I set some debug breakpoints and my code never gets executed. However if I remove my custom class from the class path i got some ClassNotFoundException log4j entries in the SystemErr.out which is a sign my configuration is read.
With no many ideas left the last thing I tried was to create a file called org.apache.commons.logging.LogFactory and have its content like below:
org.apache.commons.logging.impl.Log4jFactoryImpl
Our applications are running with a parent first class loading policy and we cannot change this.
This last attempt did not work either. Now I am out of ideas and can only help someone may bring some light in what am I missing.
Thank you in advance for your inputs.
UPDATE:
Actually I was wrong. If log4j is used to produce log entries then the CustomPatternLayout is used. It was only when the log entry was written by WAS that I could not my custom pattern being used. I think this is because WAS will use JDK java.util.logging.Logger. So my question would change to:
Is there a relatively easy way of intercepting WAS calls to writing log entries and apply a logic similar to the one I put in my custom pattern.
Thanks again for your inputs.
Related
I currently have a program that uses SLF4J/LoggerFactory for capturing logs and the configuration is done through logback.xml. My logs were working as expected. I was recently instructed by the Security team at my job to update a class which uses ESAPI. I updated the class and added ESAPI.properties and validation.properties to src/main/resources/esapi. In ESAPI.properties, I updated ESAPI.Logger to ESAPI.Logger=org.owasp.esapi.logging.slf4j.Slf4JLogFactory. Based on my research online and through SO, I assumed I would be able to us my logs (which is initialized as protected final static Logger log = LoggerFactory.getLogger(this.class); and used as log.info("Example")) as normal (outputs the logs to a file on a server) but the logs just output to the CL when the java program is run.
The configuration for the LogFactory indicates which Logging Framework instance org.owasp.esapi.Logger should delegate to.
From your question, it appears you are expecting a log event to follow the flow of:
MyCode -> SLF4J -> ESAPI -> Console.
In actuality, the way it's implemented is:
MyCode -> ESAPI_LOGGER -> SLF4J -> Console
The ESAPI Logger is not an implementation of the SLF4J logging contract. It will only delegate to the logger instance configured in esapi.properties. All of the references in code will need to be updated to
Logger esapiLogger = ESAPI.getLogger(myLogger_byClass_or_byName)
There are also these relevant properties in the ESAPI.properties file that you may wish to tweak:
# ESAPI Logging
# Set the application name if these logs are combined with other applications
Logger.ApplicationName=ExampleApplication
# If you use an HTML log viewer that does not properly HTML escape log data, you can set LogEncodingRequired to true
Logger.LogEncodingRequired=false
# Determines whether ESAPI should log the application name. This might be clutter in some single-server/single-app environments.
Logger.LogApplicationName=true
# Determines whether ESAPI should log the server IP and port. This might be clutter in some single-server environments.
Logger.LogServerIP=true
# Determines whether ESAPI should log the user info.
Logger.UserInfo=true
# Determines whether ESAPI should log the session id and client IP.
Logger.ClientInfo=true
I resolved my issue after reading the README.md for the current release which references the 2.2.3.0 release notes if you are using the slf4j libraries. I needed to exclude the slf4j-simple which is pulled in because it is a dependency of AntiSamy 1.6.2.
I am new to Log4j2. I am trying to configure a logging system using slf4j and log4j2.
I have few doubts.
While configuring log4j2, I noticed configuration logs form log4j2 is printed in console. I want to print those logs in file instead of console.
Is there any way to use log4j2's appender .
PS : I noted that log4j2 uses StatusLogger to print its configuration logs. Is there any way to make it use a file.
The simplest way to do this is to redirect the process output to a file:
java -cp lib/*.jar com.mypackage.MyClass > output.txt
There are some properties you may be able to use to tell Log4j to write status logs to a file (probably org.apache.logging.logj.simplelog.logFile), but at best this will work after Log4j 2 is fully initialized. I assume you are interested in the internal status log statements that are emitted during Log4j 2 startup and initialization. For that, I'd recommend redirecting to a file.
I looked up the api about the logger class(here) and I was looking at the Logger.info method. I was confused when I saw its perimeter as a message displayed as a string public void info(String msg) which is same as System.out.println(). I am wondering what is the different between these two, and why do we use Logger instead of System.out.println when they can print out the same thing.
In Logger.
Logger.info("Hello")
Output:
[INFO ] 2015-08-07 11:18:46.140 [main] ClassName Hello
In System.out.println
`System.out.println("Hello")
Output:
Hello
Usually, because a Logger can be configured to write to a file (and the console). It might also be configured at higher (or lower) granularity as to messaging. For example, you might configure (at runtime) for level of warn. In which case, that logger would not display debug or info messages. It can include information such as the class that is writing, a line number, and a date and time (of the message).
Using a logger allows you to abstract out a lot of details and do a lot more than you could writing to stdout.
You can specify different destinations to write to. Different appenders write to a file, roll the file for given time periods, write to a queue or database, etc.
You can specify a consistent format for log messages instead of having to add it to every line you write to stdout.
You can choose an appender that buffers the output so that multiple threads can log without having the threads contend for the lock on the console object.
You can do a lot with filtering by category (typically package and classname) and log level (trace, debug, info, error, fatal), to make it easy to configure what log messages you want to see and which you want to ignore. With logging you can change the configuration in the logger properties or include a page in your application to change what gets filtered on the fly.
You can mix and match this stuff, for instance, setting up a specific smtp appender to email log messages for logging level of error or higher, in addition to writing the messages to a rolling file or whatever.
The main difference between a Logger and System.out.println is Logger: Prints the text in a file(text file)System.out.println: Prints the output in console
Logger is useful when you are going for any LIVE projects. Because if any project is developed and deployed, then you cannot check the console. At that time Logger will be useful to track the flow of your project also you can find the Error or Exception if you have given the logger in catch{...} block.
Also go through this Logger vs. System.out.println
But when we use any logging mechanism(log4j, slf4j, logback etc) we
configure appenders and corresponding target log files for each
package. By default console appender is turned off unless you
explicitly configure it to log to a destination.
The System.out.println always logs the message to a console appender.
So, it should be used only when we are sure that the console appender
is configured in the logger configuration file. Otherwise we end up
having logs logged on server console which is incorrect. Any log from
inside the application should go to the corresponding application log
and not the server log.
Let me explain with an example.
If we are building an application called Tracker using a logging mechanism to run in a tomcat container, and we configured appenders for application logging with destination log file as tracker-application.log and we did not configure the console appender.
Then if the System.out.println is encountered by the JVM then the log will go to server log of the tomcat server which is wrong because server log should only have information about the server and not the application.
But if we created a console appender with a target log file then it will be logged properly.
Hope I was clear. ;)
For best practices refer Do not use System.out.println in server side code or http://www.vipan.com/htdocs/log4jhelp.html
For differences between both of them please refer
Logger vs. System.out.println
My current project taking Log4j for logging output but when it is deployed on websphere 7 it seems the logging file is always occupied by the server hence Log4J cannot close it and open a new file. Current log file can expand to 1.5 Gb if on DEBUG level. I went through a few online tutorials but didn't get a solid solution. So I open this thread for any opinion or experience on making log4J works on WS 7.
Current log4J is on 1.2.15, and the project is a plain POJO/JSP application without other containers(like Spring, I do have another project with same issue on Spring/tomcat, will open in another thread). And here is the log4j.properties:
log4j.rootCategory=INFO, ROL
#
# Rolling File Appender
#
log4j.appender.ROL=org.apache.log4j.RollingFileAppender
log4j.appender.ROL.Encoding=UTF-8
log4j.appender.ROL.File=C\:\\Logs\\AppOut.log
log4j.appender.ROL.MaxFileSize=10000KB
log4j.appender.ROL.MaxBackupIndex=10
log4j.appender.ROL.layout=org.apache.log4j.PatternLayout
log4j.appender.ROL.layout.ConversionPattern=%d -- %p -- %c -- %m%n
But this configuration just doesn't make log file roll over.
UPDATE:
It seems like each time when I try to remove the log file after stop the application(not the server), it is always saying "the file is opened by another application" which only refers to Websphere. I can guarantee that there is no other application is opening the log file. The only way to release lock on it is stop the application server.
Thanks in advance for any ideas.
we use was 7 (with portal server 6.1 on top). (We wrap our calls to the logger using the sl4j lbrary, but I don't think that is relevant to the problem here). can't really see any major difference, other than we don't set the root category, and we explicitly set the additivty to false.
Below is part of our config (showing just one of our appenders) which works just fine and rolls the logs every 10MB as expected. Perhaps it's something configured in WAS itself - I'm not a WAS admin expert and didn't install WAS myself but could ask one of our admins on Monday if you haven't solved it.
log4j.logger.com.xxx.protection=TRACE, A1
log4j.additivity.com.xxx.protection=false
# other appenders excluded
log4j.appender.A1=org.apache.log4j.RollingFileAppender
log4j.appender.A1.File=${LOG_ROOT}/applLogs/ui_and_business_logic.log
log4j.appender.A1.MaxFileSize=10MB
log4j.appender.A1.MaxBackupIndex=10
log4j.appender.A1.Append=true
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%d %p %t %c MSG: %m%n
I take it that you've seen this article - http://www.ibm.com/developerworks/websphere/techjournal/0802_supauth/0802_supauth.html
I am using Logback in my application hosted on Websphere App Server. Logback is configured to log to System Out (and others are hesitant to change to a different file). The issue is that Websphere uses its own format for logging to System Out. Executing logger.debug("test") in my app yields:
[8/7/12 12:27:55:629 CDT] 0000003a SystemOut O DEBUG com.myapp... test
where everything up to the "O" is added by Websphere. The rest is from Logback
I have set up Logback to use the following pattern: %-5level %logger{36} - %msg%n so that I don't repeat timestamp and thread info which Websphere does on its own, but I am still annoyed that I can't fully customize the logging to System Out from within Logback.
I don't know a whole lot about logging best practices. Before, I have logged to separate files by web app, but for this project, I was told the System Out files are monitored by a third party and I should not change from using System Out. Is there any way to get around my issue given these requirements and tell Websphere not to mess with my System Out logging, or is the only solution to start logging to a different file? Thanks!
Your logback is configured to write messages to System.out. However, everything that is written to System.out is redirected by WebSphere and written to the SystemOut.log file with the same format as log messages produced by WebSphere, but with a severity indicator "O". It is not possible to change that.
Note that it is likely that the people who told you to use SystemOut.log actually meant that you should ensure that logging is done using WebSphere's log system so that messages are written with the correct category and severity and that log levels can be changed at runtime. Since WebSphere's log system is build on java.util.logging you should probably just replace logback by slf4j-jdk14 to satisfy their requirement.
I don't think you're going to be able to change the format. And if you could, it might break the current monitoring anyway.
I wonder if anyone would mind if you log to two loggers at the same time, SystemOut for the existing monitoring, and your own for a more readable format.
The problem is that Logback redirects the messages to console output instead of java.util.logging. Console output has no log levels and that's why WebSphere just writes "O".
We solved this by implementing a Logback appender that redirects logs to JUL (java.util.logging) instead. We convert Logback log levels to JUL levels (e.g. Logback "ERROR" is JUL "SEVERE").
We also wanted to use Websphere's trace options. If trace is enabled for a class/package pattern, you will see Logback DEBUG and TRACE messages in Websphere's trace.log. You can also check if the trace is enabled by calling Logback's isDebugEnabled() / isTraceEnabled().
See this answer with a full implementation:
https://stackoverflow.com/a/74386323/395879