Develop Reference

adding microsecond level to log4j

referring to this
How can I add microsecond level logging, how to create custom pattern formatter for this?
I have also tried log4j2, but it ends up giving
12:42:05,972803200 [main] DEBUG Log4j2Test - Debug Message Logged !!!
12:42:05,975803200 [main] INFO Log4j2Test - Info Message Logged !!!
12:42:05,975803200 [main] ERROR Log4j2Test - Error Message Logged !!!
as you can see it gives 803200 microseconds and nanoseconds always.
My log4j2.xml looks like:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
<Appenders>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{ABSOLUTE_NANOS} [%t] %-5level %logger{36} - %msg%n"/>
<!-- <PatternLayout
pattern="[%-5level] %d{ABSOLUTE_NANOS} [%t] %c{1} - %msg%n" /> -->
</Console>
</Appenders>
<Loggers>
<Root level="debug" additivity="false">
<AppenderRef ref="console" />
</Root>
</Loggers>

It depends on what log4j you happen to be using. This feature is only supported if you use JDK9 and new(er) version of log4j ( 2.x).
The format you happen to be searching for is this one:
%d{HH:mm:ss,nnnnnn}
Please also notice what official documentation says, because there are lots of "catches and gotchas". I.e.:
Log4j 2.11 adds limited support for timestamps more precise than milliseconds when running on Java 9. Note that not all DateTimeFormatter formats are supported. Only timestamps in the formats mentioned in the table above may use the "nano-of-second" pattern letter n instead of the "fraction-of-second" pattern letter S.
Users may revert back to a millisecond-precision clock when running on Java 9 by setting system property log4j2.Clock to SystemMillisClock
More info on following link:Log4j 2.x layouts

Log4j2 - Why is my "Logger" element hiding content from my "Root" logger?

Before someone closes this question off as a duplicate, just hear me out... I have read through countless blog posts, tutorials, FAQs and SO questions for days now and I'm no closer to understanding why I'm getting this specific behaviour.
Configuration
My log4j2.xml config file contains the following:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="DEBUG">
<Properties>
<Property name="APP_NAME">MyCoolApp</Property>
<Property name="BASE_PACKAGE">my.cool.package</Property>
<Property name="LOG_DIR">${env:LOG_ROOT:-logs}</Property>
<Property name="LOG_PATTERN">%d [%t] %-5level %c{1.}:%L:%M | %m%n</Property>
</Properties>
<Appenders>
<RollingFile name="AppLogFile" fileName="${LOG_DIR}/${APP_NAME}.log" filePattern="${LOG_DIR}/archive/${APP_NAME}.%d{yyyy-MM-dd}.log.gz">
<PatternLayout pattern="${LOG_PATTERN}"/>
<TimeBasedTriggeringPolicy/>
</RollingFile>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${LOG_PATTERN}"/>
</Console>
</Appenders>
<Loggers>
<Logger name="${BASE_PACKAGE}" level="INFO">
<AppenderRef ref="AppLogFile"/>
</Logger>
<Root level="TRACE">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
Then each class initialises a logger with private static final Logger LOGGER = LogManager.getLogger();.
Runtime
From my understanding of log levels, loggers and appenders, this should give me the following:
All INFO and higher level logging output to file
All TRACE and higher level logging output to console
Meanwhile, if I run the app when the AppLogFile appender is enabled, I get less output to the console; counting from only after the Log4j initialisation, I get 255 lines compared to 367.
Looking through the console and file output, when the AppLogFile appender is enabled, I don't get any TRACE or DEBUG output, only INFO and higher. When I comment out just that appender (without changing anything else), then I get everything to console, including TRACE and DEBUG.
I've tried playing around with reordering the "console" and "file" related elements, I've tried explicitly enabling and disabling the logger's additivity property, I've tried using filters in the appenders and loggers, and even multiple appender references inside one logger with explicit level properties.
All I want is to get everything to go to the console and everything INFO level and higher to go to a file. What am I missing here..?

The level attribute and the <AppenderRef> element are entirely independent:
Specifying level changes the logging level of the given logger and all sub-loggers.
Specifying <AppenderRef> adds1 another appender to the given logger and all sub-loggers.
The fact that you do both at the same time, doesn't affect those independent effects.
If you want to limit log entries for the Appender, specify the level attribute on the <AppenderRef> instead.
1) If you wanted the Appender to replace, you need to specify additivity="false"
(OP edit) Just for the sake of clarity, between what I learned from this answer and the Log4j2 FAQ; I streamlined things a bit and ended up with the following <Loggers> configuration:
<Loggers>
<Root level="TRACE">
<AppenderRef ref="AppLogFile" level="INFO"/>
<AppenderRef ref="Console"/>
</Root>
</Loggers>

How can I specify a specific format for each Log4j log level

First time trying to use log4j version log4j-1.2.17.jar.
On an existing application the client has log4j in place and there is a log4j.properties file which specifies a light log output. What I want to do is depending on the log level (ERROR & WARN) output a more refined entry.
On the log4j site I came across this but I think it is to be in some .xml file. I need some assistance in understanding how I can put in place the formatting option to alter based on log level.
You don't need to declare separate loggers to achieve this. You can set the logging level on the AppenderRef element.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<File name="file" fileName="app.log">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m %ex%n</Pattern>
</PatternLayout>
</File>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%m%n"/>
</Console>
</Appenders>
<Loggers>
<Root level="trace">
<AppenderRef ref="file" level="DEBUG"/>
<AppenderRef ref="STDOUT" level="INFO"/>
</Root>
</Loggers>
</Configuration>
Would I put this xml content into the web.xml file or another file?
a) If another file what file name and where would it go?
How do I get log4j to realize that I need it to use the xml file?
Will the use of the xml ignore the log4j.properties file?
I know it is a lot of questions but there is only me on the project and the client has a production crisis that needs to be figured out today so I don't have time to go off to read and do tutorials with the client calling me every hour. I figured it may help to get this logging more useful. As the logs are right now I have a date and message output to the log but no idea where the entries are created from without doing extensive searches through the code.

You could do this by defining multiple non-additive Loggers so that the first one only logs errors, the next one warnings and the final one infos and debug.

Apppend log message at beginning of file log4j

1. I am trying to add log message at beginning of file so that I can see latest message first on log file.
2. I want to add month and year after log file but I am not getting that in current active file. eg - test_2015-04-22.log
property file is given below -
log4j.appender.APP=org.apache.log4j.DailyRollingFileAppender
log4j.appender.APP.File=${catalina.base}/logs/test.log
log4j.appender.APP.Append=true
log4j.appender.APP.Encoding=UTF-8
log4j.appender.APP.DatePattern='.'yyyy-MM
log4j.appender.APP.layout = org.apache.log4j.PatternLayout
log4j.appender.APP.layout.ConversionPattern =%d{yyyy-MM-dd HH:mm} - %m%n
log4j.appender.APP.filePattern =Test_%d{yyyy-MM-dd}.log

Your question shows a log4j-1.2 configuration, but since the question also has a log4j2 tag, I feel free to answer this by showing you how to accomplish this with log4j2.
In log4j2, you can declare a property that formats the date, then use this property to configure your appender file name.
Also, you can use the header attribute of the pattern layout to set a header that is output at the beginning of a file. For RollingFileAppender this header will be output on every rollover.
Use one of the lookups built-in to log4j2 to dynamically change the output of your header at rollover time.
Example:
<Configuration status="WARN"><!-- use TRACE to troubleshoot your config if needed-->
<Properties>
<property name="yyyyMMdd">${date:yyyyMMdd}</property>
</Properties>
<Appenders>
<RollingFile name="Application"
fileName="${sys:catalina.base}/logs/test${sys:yyyyMMdd}.log"
filePattern="logs/$${date:yyyy-MM}/app-%d{MM-dd-yyyy}-%i.log.gz">
<PatternLayout header="File: ${main:--file}">
<Pattern>%d{yyyy-MM-dd HH:mm} - %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy />
</Policies>
</RollingFile>
<Appenders>
<Loggers>
<root level="trace">
<AppenderRef ref="Application" />
</root>
</Loggers>
</Configuration>

it's not possible , think of the file as a stack which respects LAST In FIRST Out which makes perfect sense when debugging to see the last message at the end.
The log for today will have the name test.log but tomorrow it will be renamed to Test_2015-04-22.log

How to configure Log4J2

I have spent a couple hours now trying to figure this out and am at a complete loss. I find the new configuration process unnecessarily complex.
I have a Servlet with a web.xml file with the following:
<context-param>
<param-name>log4jConfiguration</param-name>
<param-value>file:///etc/myApp/log4j2.xml</param-value>
</context-param>
It doesn't seem to have any effect. I am using Tomcat 7.0.42, and I have removed all references to log4j*.jar in the catalina.properties file. The logs in my app are still being sent, but they are just being sent to System.out with none of the formatting I specified.
So, I tried to just do it by hand on my own:
InputStream in =
new FileInputStream(new File("/etc/myApp/log4j2.xml"));
ConfigurationSource source = new ConfigurationSource(in);
ConfigurationFactory factory = new XMLConfigurationFactory();
Configuration configuration = factory.getConfiguration(source);
LoggerContext context = (LoggerContext) LogManager.getContext();
context.start(configuration);
context.updateLoggers();
Logger.getLogger("Test").log(Level.INFO, "This is a logging message.");
First, this seems entirely convoluted. Clearly there exists some code that will search for different files and assume file types based on their extensions via the "log4jConfiguration" property, so why isn't there a LogManger.reconfigure(String) that has the same effect?
Second, this has no effect either. Again, the log is printed to System.out, and none of the requested formatting is being done.
Here is the contents of my log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="DEBUG">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
<File name="LogFile" fileName="/var/log/myApp/myApp.log">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %p %c{1.} [%t] %m%n"/>
</File>
</Appenders>
<Loggers>
<Root level="DEBUG">
<AppenderRef ref="Console" level="DEBUG"/>
<AppenderRef ref="LogFile" level="DEBUG"/>
</Root>
</Loggers>
</Configuration>
The output in both cases comes out something like:
Dec 03, 2013 6:06:45 PM test.Test main
INFO: This is a logging message.
Thanks in advance,
John
EDIT: This is actually working. It appears that I was missing, "log4j-jcl-2.0-beta9.jar". Remko Popma's answer works as well, if the "context-param" above is not working.

There are many possible scenarios with webapp logging, which makes configuration more complex than the ideal. Putting the log4j2 jars and the config file in your webapp's classpath should work. Did you see the manual page for log4j2 use in web apps? If the issue remains, please file a log4j2 jira ticket.

The LoggerContextalready has a method setConfigLocation that takes in a URI as parameter. It baffles me why there isn't a similar method that takes an InputStream as parameter. Instead we have to deal with the most convoluted and messy way of loading our log4j2.xml that is not on the classpath.