Develop Reference

Weblogic12 + Log4j logging not writing to server logs

I know there are lots of Q's regarding this, but I did go through all of them, and kinda confused myself more, I am listing the steps I followed, please let me know where I messed it up.
1) I just want to use Log4j on application level, so need to copy WL_HOME/server/lib/wllog4j.jar, and the log4j.jar in Domail_Home/Lib?
2) I am using Maven, I added the Log4j dependency in my pom.xml [war].I have my WAR wrapped in EAR.
3) Since I want to write the logs in weblogic managed server logs file, I created a custom appender, to use weblogic, NonCatalogAppender as mention in the link - https://blog.desgrange.net/2010/02/15/logging-in-weblogic-console-with-log4j.html
4) I copied the log file in my war/src/main/resources and i see maven added them to classpath i.e war/target/classes, see below for my lo4j xml
http://jakarta.apache.org/log4j/'>
<!-- stdout appender settings -->
<appender name="STDOUT" class="com.xyz.logging.util.WeblogicAppender">
<param name="Threshold" value="DEBUG"/>
<layout class="org.apache.log4j.PatternLayout">
<!-- notes on patterns:
%p - priority (i.e. level) of message
%c - class that threw error
%m - message in logger's method, e.g. "Exiting application HERE..."
%n - carriage return
%d - date information
-->
<param name="ConversionPattern" value="%c{1} %m"/>
</layout>
</appender>
<!-- settings for root debugger -->
<root>
<level value="DEBUG"/>
<appender-ref ref="STDOUT"/>
</root>
5) Now I didn't change anything on the config level, but I don't see anything appending to server logs. When I initialize NonCatlogLogger manually and call the logger it works fine:
NonCatalogLogger logger =new NonCatalogLogger("XYZ");
logger.debug("This is the debug message")
6) When I debug the application in eclipse, it looks like my custom appender is never called.

Got it, just need to put below into arguments under "server start".
-Dweblogic.log.Log4jLoggingEnabled=true

log4j: archived files removing policy

I'm using TimeBasedRollingPolicy and RollingFileAppender to have a my logs rolling over and be archived as myLogFileName..log.gz:
<appender class="org.apache.log4j.rolling.RollingFileAppender" name="myLogFile">
<param value="/var/log/my/myLogFileName.log" name="File"/>
<rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
<param name="FileNamePattern" value="/var/log/my/myLogFileName.%d.log.gz" />
</rollingPolicy>
<layout class="org.apache.log4j.PatternLayout">
<param value="%d [%t] %-5p - %m%n" name="ConversionPattern"/>
</layout>
</appender>
What I don't understand - is where it is defined when and how really old archived files will be totally removed from the filesystem?
And, if I want to keep always only no older than one month old files - where I can set it up (using log4j (extras))?

TimeBasedRollingPolicy is defined by apache-log4j-extras. This dependency will need to be on the classpath in order to use that policy.
RollingFileAppender includes a maxBackupIndex property, which specifies the maximum number of rolled over backup files to retain. However, I don't believe this property is supported when using a TimeBasedRollingPolicy. This means that, when using RollingFileAppender with the TimeBasedRollingPolicy, old backup files will not be removed by the appender.
If you were using Logback as the logging implementation (perhaps in conjunction wiht SLF4J), then you could achieve the desired outcome using the
Logback RollingPolicy. I acknowledge that the question is seeking a log4j solution, but I'm not aware one.

Create log file with date using log4j

I'm writing my log file using below code but it stores file as QueryLog.log. Am i missing something? Check my code of log4j.properties file
log4j.logger.org.hibernate=INFO
log4j.logger.org.hibernate.SQL=DEBUG
log4j.logger.org.hibernate.type=TRACE
log4j.logger.org.hibernate.hql.ast.AST=info
log4j.logger.org.hibernate.tool.hbm2ddl=warn
log4j.logger.org.hibernate.hql=debug
log4j.logger.org.hibernate.cache=info
log4j.logger.org.hibernate.jdbc=debug
log4j.rootLogger = DEBUG, FILE
log4j.appender.FILE=org.apache.log4j.DailyRollingFileAppender
log4j.appender.FILE.DatePattern='.'yyyy-MM-dd-a
log4j.appender.FILE.File=log4j/QueryLog.log
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern= %d{HH:mm:ss} %-5p %c - %m%n
Links i used:
http://www.tutorialspoint.com/log4j/log4j_logging_files.htm
http://www.codejava.net/coding/configure-log4j-for-creating-daily-rolling-log-files

As is mentioned in this StackOverflow Q&A, the purpose of a RollingFileAppender is to automatically create a new log file at some defined interval. In the case of the DailyRollingFileAppender, that interval is 12:00 AM of each day.
What this means is that the first file created by log4j will have the file name you specified here:
log4j.appender.FILE.File=log4j/QueryLog.log
And, from then forward, each day a new log file will be created with the date appended to it.
To always name the file with the date appended, you could use DatedFileAppender by Geoff Mottram

This line sets the log file name, in your log4j properties you have:
log4j.appender.FILE.File=log4j/QueryLog.log
You can see the answer here
Setting a log file name to include current date in Log4j

The solution to log directly to a file with current active date/time such as XYZ.log.20150101.log instead of XYZ.log could be done by simply removing ActiveFileName property when using the rolling package org.apache.log4j.rolling.RollingFileAppender in the apache-log4j-extras 1.1 with log4j 1.2.x.
<appender name="defaultFileAppender" class="org.apache.log4j.rolling.RollingFileAppender">
<param name="append" value="true" />
<param name="Threshold" value="INFO" />
<rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
<param name="FileNamePattern"
value="${catalina.base}/logs/application/custom-application-logger.%d{yyyy-MM-dd_HH_mm}" />
</rollingPolicy>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5p %-10t [%-40.40c] %x - %m%n" />
</layout>
</appender>

Log4j: How to configure simplest possible file logging?

My story:
I want to make a thing which is as simple as a simplest possible log4j logger that logs rows to a file. I have found several examples with some functionality, but not a basic, general one that really works, and not one with an explanation how the each row work.
Question:
Could anybody provide one?
Prerequisites:
I already know where to put the file and I have the log4j configured and working for console logging.
Now I want to log to a file and also find the file from file system once the program has run.
Rows needed to be added to the existing log4j.properties file are the desired output.

I have one generic log4j.xml file for you:
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration debug="false">
<appender name="default.console" class="org.apache.log4j.ConsoleAppender">
<param name="target" value="System.out" />
<param name="threshold" value="debug" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p [%c{1}] - %m%n" />
</layout>
</appender>
<appender name="default.file" class="org.apache.log4j.FileAppender">
<param name="file" value="/log/mylogfile.log" />
<param name="append" value="false" />
<param name="threshold" value="debug" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p [%c{1}] - %m%n" />
</layout>
</appender>
<appender name="another.file" class="org.apache.log4j.FileAppender">
<param name="file" value="/log/anotherlogfile.log" />
<param name="append" value="false" />
<param name="threshold" value="debug" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p [%c{1}] - %m%n" />
</layout>
</appender>
<logger name="com.yourcompany.SomeClass" additivity="false">
<level value="debug" />
<appender-ref ref="another.file" />
</logger>
<root>
<priority value="info" />
<appender-ref ref="default.console" />
<appender-ref ref="default.file" />
</root>
</log4j:configuration>
with one console, two file appender and one logger poiting to the second file appender instead of the first.
EDIT
In one of the older projects I have found a simple log4j.properties file:
# For the general syntax of property based configuration files see
# the documentation of org.apache.log4j.PropertyConfigurator.
# The root category uses two appenders: default.out and default.file.
# The first one gathers all log output, the latter only starting with
# the priority INFO.
# The root priority is DEBUG, so that all classes can be logged unless
# defined otherwise in more specific properties.
log4j.rootLogger=DEBUG, default.out, default.file
# System.out.println appender for all classes
log4j.appender.default.out=org.apache.log4j.ConsoleAppender
log4j.appender.default.out.threshold=DEBUG
log4j.appender.default.out.layout=org.apache.log4j.PatternLayout
log4j.appender.default.out.layout.ConversionPattern=%-5p %c: %m%n
log4j.appender.default.file=org.apache.log4j.FileAppender
log4j.appender.default.file.append=true
log4j.appender.default.file.file=/log/mylogfile.log
log4j.appender.default.file.threshold=INFO
log4j.appender.default.file.layout=org.apache.log4j.PatternLayout
log4j.appender.default.file.layout.ConversionPattern=%-5p %c: %m%n
For the description of all the layout arguments look here: log4j PatternLayout arguments

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/" debug="false">
<appender name="fileAppender" class="org.apache.log4j.RollingFileAppender">
<param name="Threshold" value="INFO" />
<param name="File" value="sample.log"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%c{1}] %m %n" />
</layout>
</appender>
<root>
<priority value ="debug" />
<appender-ref ref="fileAppender" />
</root>
</log4j:configuration>
Log4j can be a bit confusing. So lets try to understand what is going on in this file:
In log4j you have two basic constructs appenders and loggers.
Appenders define how and where things are appended. Will it be logged to a file, to the console, to a database, etc.? In this case you are specifying that log statements directed to fileAppender will be put in the file sample.log using the pattern specified in the layout tags. You could just as easily create a appender for the console or the database. Where the console appender would specify things like the layout on the screen and the database appender would have connection details and table names.
Loggers respond to logging events as they bubble up. If an event catches the interest of a specific logger it will invoke its attached appenders. In the example below you have only one logger the root logger - which responds to all logging events by default. In addition to the root logger you can specify more specific loggers that respond to events from specific packages. These loggers can have their own appenders specified using the appender-ref tags or will otherwise inherit the appenders from the root logger. Using more specific loggers allows you to fine tune the logging level on specific packages or to direct certain packages to other appenders.
So what this file is saying is:
Create a fileAppender that logs to
file sample.log
Attach that appender to the root
logger.
The root logger will respond to any
events at least as detailed as
'debug' level
The appender is configured to only
log events that are at least as
detailed as 'info'
The net out is that if you have a logger.debug("blah blah") in your code it will get ignored. A logger.info("Blah blah"); will output to sample.log.
The snippet below could be added to the file above with the log4j tags. This logger would inherit the appenders from <root> but would limit the all logging events from the package org.springframework to those logged at level info or above.
<!-- Example Package level Logger -->
<logger name="org.springframework">
<level value="info"/>
</logger>

Here's a simple one that I often use:
# Set up logging to include a file record of the output
# Note: the file is always created, even if there is
# no actual output.
log4j.rootLogger=error, stdout, R
# Log format to standard out
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern= %5p\t[%d] [%t] (%F:%L)\n \t%m%n\n
# File based log output
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=owls_conditions.log
log4j.appender.R.MaxFileSize=10000KB
# Keep one backup file
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern= %5p\t[%d] [%t] (%F:%L)\n \t%m%n\n
The format of the log is as follows:
ERROR [2009-09-13 09:56:01,760] [main] (RDFDefaultErrorHandler.java:44)
http://www.xfront.com/owl/ontologies/camera/#(line 1 column 1): Content is not allowed in prolog.
Such a format is defined by the string %5p\t[%d] [%t] (%F:%L)\n \t%m%n\n. You can read the meaning of conversion characters in log4j javadoc for PatternLayout.
Included comments should help in understanding what it does. Further notes:
it logs both to console and to file; in this case the file is named owls_conditions.log: change it according to your needs;
files are rotated when they reach 10000KB, and one back-up file is kept

Here is a log4j.properties file that I've used with great success.
logDir=/var/log/myapp
log4j.rootLogger=INFO, stdout
#log4j.rootLogger=DEBUG, stdout
log4j.appender.stdout=org.apache.log4j.DailyRollingFileAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{MM/dd/yyyy hh:mm:ss a}|%-5p|%-30c{1}| %m%n
log4j.appender.stdout.DatePattern='.'yyyy-MM-dd
log4j.appender.stdout.File=${logDir}/myapp.log
log4j.appender.stdout.append=true
The DailyRollingFileAppender will create new files each day with file names that look like this:
myapp.log.2017-01-27
myapp.log.2017-01-28
myapp.log.2017-01-29
myapp.log <-- today's log
Each entry in the log file will will have this format:
01/30/2017 12:59:47 AM|INFO |Component1 | calling foobar(): userId=123, returning totalSent=1
01/30/2017 12:59:47 AM|INFO |Component2 | count=1 > 0, calling fooBar()
Set the location of the above file by using -Dlog4j.configuration, as mentioned in this posting:
java -Dlog4j.configuration=file:/home/myapp/config/log4j.properties com.foobar.myapp
In your Java code, be sure to set the name of each software component when you instantiate your logger object. I also like to log to both the log file and standard output, so I wrote this small function.
private static final Logger LOGGER = Logger.getLogger("Component1");
public static void log(org.apache.log4j.Logger logger, String message) {
logger.info(message);
System.out.printf("%s\n", message);
}
public static String stackTraceToString(Exception ex) {
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
ex.printStackTrace(pw);
return sw.toString();
}
And then call it like so:
LOGGER.info(String.format("Exception occurred: %s", stackTraceToString(ex)));

java logging nightmare and log4j not behaving as expected with spring + tomcat6

I have a spring application that has configured log4j (via xml) and that runs on Tomcat6 that was working fine until we add a bunch of dependencies via Maven. At some point the whole application just started logging part of what it was supposed to be declared into the log4.xml
"a small rant here" Why logging has to be that hard in java world? why suddenly an application that was just fine start behaving so weird and why it's so freaking hard to debug?
I've been reading and trying to solve this issue for days but so far no luck, hopefully some expert here can give me some insights on this
I've added log4j debug option to check whether log4j is taking reading the config file and its values and this is what part of it shows
log4j: Level value for org.springframework.web is [debug].
log4j: org.springframework.web level set to DEBUG
log4j: Retreiving an instance of org.apache.log4j.Logger.
log4j: Setting [org.compass] additivity to [true].
log4j: Level value for org.compass is [debug].
log4j: org.compass level set to DEBUG
As you can see debug is enabled for compass and spring.web but it only shows "INFO" level for both packages. My log4j config file has nothing out of extraordinary just a plain ConsoleAppender
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<!-- Appenders -->
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p: %c - %m%n" />
</layout>
</appender>
What's the trick to make this work? What it's my misunderstanding here?
Can someone point me in the right direction and explain how can I make this logging mess more bullet proof?

It might not be log4j that is doing the logging, and hence your log4j config would be ignored. Spring logs using Commons Logging, which is an api that can delegate to various logging frameworks, including log4j. To decide which implementation to use, commons logging looks into the classpath.
If you have added a dependency that dragged its own logging implementation into the classpath, commons logging might now use the other implementation.
I recommend to set a breakpoint at a call to the logging facility, and trace the execution to see which logging implementation is used.

As it was working until you loaded a number of dependencies via Maven, perhaps there's a Log4j configuration loaded inadvertently via these dependencies ?
Run mvn dependency:tree to see what's being loaded, and then see if any of these dependencies has a Log4j configuration.

I think your problem is that you're not setting the threshold param on your appenders, and (maybe) because you're not assigning those appenders to your loggers.
Try adding param name="threshold" value="debug" to your appenders and then explicitly adding them to the specific (or root) loggers like so:
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="threshold" value="debug" />
<param name="Target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p: %c - %m%n" />
</layout>
</appender>
<logger name="org.springframework.web">
<level value="debug" />
<appender-ref ref="console" />
</logger>
Also, this page says "This appender will not log any messages with priority lower than the one specified here even if the category's priority is set lower" which is probably the source of your problem.

This seems like a good thread for you to read:
http://forum.springsource.org/showthread.php?t=88250
Cutting to the chase, it seem that the poster had a problem with a Tomcat security setting. An updated Tomcat policy file fixed the issue.
Probably has something do with reading outside of the webapp for your log4j.xml file.