I have an App which has many ejb Timer methods which are fired at various intervals, in an attempt to log the performance of these scheduled tasks I have written two methods which log the start and end of every task and records some info such as name, running duration and log info each time the timer fires
The intention with log info was to intercept all logging statements raised from within these methods, output to a string and save...I am doing this by creating a new WriterAppender, appending to the Logger for all logging levels, then when the timer completes capture the string output and remove the appender
This works well but it seems to have 2 unwanted effects:
1) It is stopping any logging from the timer class appearing in our normal file appender logs, eg. the log messages are being captured but they now don't appear in our normal logs (which are configured to show everything > INFO
2) Logging statements from different timers which overlap are appearing in each other logs, I guess this is the result of the Logger working on a single thread, not sure if theres a way around this?
I am no expert in log4j so I am hoping someone can point out a simple fix for 1, 2 or both
Code below shows the 2 methods which are called at start and end of every job
private Writer w = new StringWriter();
private WriterAppender wa;
private Logger logOutput = Logger.getLogger(ScheduledActions.class);
private TimerEvent logStartTimerEvent(ScheduledTask task){
RedisActions redisActions = (RedisActions)Component.getInstance("redisActions");
TimerEvent event = new TimerEvent(task);
//Initialise a new log appender to intercept all log4j entries
w = new StringWriter();
Layout l = new PatternLayout("%m%n");
wa = new WriterAppender(l, w);
wa.setEncoding("UTF-8");
wa.setThreshold(Level.ALL);
wa.activateOptions();
// Add it to logger
logOutput = Logger.getLogger(ScheduledActions.class);
logOutput.addAppender(wa);
//Push this timer event to redis
redisActions.addToFixedLengthList(RedisItem.TIMER_HISTORY.getKey()+"_"+task.getName(), gson.toJson(event), 100L);
return event;
}
private void logEndTimerEvent(TimerEvent event){
try{
RedisActions redisActions = (RedisActions)Component.getInstance("redisActions");
event.setLogInfo(w.toString());
event.setEndTime(new Date());
redisActions.editLastMemberInList(RedisItem.TIMER_HISTORY.getKey()+"_"+event.getTask().getName(), gson.toJson(event));
w.flush();
} catch (IOException e) {
e.printStackTrace();
}finally{
//clean up the temp log appender
logOutput.removeAppender(wa);
wa.close();
}
}
I found this old question while trying to do something similar. For now, my solution is creating an independent logger/appender, with a unique loggerId, like:
var loggerId = operation.getId(); // This is a unique ID
var logger = (org.apache.logging.log4j.core.Logger) LogManager.getLogger(loggerId);
var appender = createAppender();
logger.addAppender(appender);
logger.info("using logger");
appender.stop();
logger.removeAppender(appender);
The problem that I have is that I need to pass this logger variable around. I can't use the classic "static logger".
Another solution that I am considering, is adding an AbstractAppender to the rootLogger, and from there do whatever with the logs.
I will update this answer when I develop it a bit more.
var layout = PatternLayout.newBuilder()
.withPattern("%d{ISO8601_OFFSET_DATE_TIME_HHCMM} [%t] %-5level %c{1.} %X %msg%n%throwable")
.build();
var rootLogger = (org.apache.logging.log4j.core.Logger) LogManager.getRootLogger();
var appender = new AbstractAppender("intercept-appender", null, layout, false, null) {
#Override
public void append(LogEvent event) {
System.out.println("INTERCEPTED: " + getLayout().toSerializable(event));
}
};
appender.start();
rootLogger.addAppender(appender);
log.info("Message after adding intercept appender");
My final solution is adding an Appender with ThreadContextMapFilter.
// With %X you will see the thread context values
var layout = PatternLayout.newBuilder()
.withPattern("%d{ISO8601_OFFSET_DATE_TIME_HHCMM} [%t] %-5level %c{1.} %X %msg%n%throwable")
.build();
var appender = FileAppender.newBuilder()
.setName(fileId)
.withFileName("/tmp/my-file.log")
.setLayout(layout)
// --- IMPORTANT PART ---
.setFilter(new ThreadContextMapFilter(
Map.of("my-key", List.of("my-value")), true, null, null))
.build();
// Add the appender
// https://logging.apache.org/log4j/2.x/manual/customconfig.html
appender.start();
final LoggerContext context = LoggerContext.getContext(false);
final Configuration config = context.getConfiguration();
config.addAppender(appender);
config.getRootLogger().addAppender(appender, null, null);
context.updateLoggers();
// --- IMPORTANT PART ---
try (var ignore = CloseableThreadContext.put("my-key", "my-value")) {
// the logs here will be captured by the appender
log.info("This log WILL be captured by the appender");
// Note: In case you start threads from here,
// you should pass the ThreadContext to them.
}
log.info("This log will NOT be captured by the appender");
// Wait for the logs to be flushed. Any improvement ideas?
// https://stackoverflow.com/a/71081005/1121497
try {
Thread.sleep(500);
} catch (InterruptedException ignore) {
}
// Remove the appender
final LoggerContext context = LoggerContext.getContext(false);
final Configuration config = context.getConfiguration();
config.getRootLogger().removeAppender(appender.getName());
context.updateLoggers();
appender.stop();
Related
I am developing a logger where certain transaction messages need to be logged to a file and some application logs need to be logged to the console. Currently, the transaction messages are also being logged onto the console and I want to avoid that.
for eg:
if i write logger.info("application process started");
this needs to be logged on console.
and when i write logger.info(new ObjectArrayMessage("msg1","msg2","msg3"));
this need to be logged to a csv file.
This is my current output:
19:18:42.230 [main5] INFO New.CSVlog.App - application process started
19:18:42.233 [main5] INFO New.CSVlog.App - [msg1,msg2,msg3]
I want to log only the first log to the console and the second one to the CSV file.
I have the following properties file:
appenders = csvFile, console
appender.csvFile.type = RollingFile
appender.csvFile.name = CSVFILE
appender.csvFile.fileName = csvLog.csv
appender.csvFile.filePattern= csvLog-%d{MM-dd-yyyy}-%i.csv
appender.csvFile.layout.type = CsvParameterLayout
appender.csvFile.layout.delimiter = ,
appender.csvFile.layout.header = column1,column2,column3\n
appender.csvFile.policies.type = Policies
appender.csvFile.policies.time.type = TimeBasedTriggeringPolicy
appender.csvFile.policies.time.interval = 1
appender.csvFile.policies.time.modulate = true
appender.csvFile.policies.size.type = SizeBasedTriggeringPolicy
appender.csvFile.policies.size.size=10MB
appender.csvFile.strategy.type = DefaultRolloverStrategy
appender.csvFile.strategy.max = 200
appender.console.type = Console
appender.console.name = consoleLog
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{HH:mm:ss.SSS} [%t5] %-5level %logger{36} - %msg%n
rootLogger.level =debug
rootLogger.appenderRefs = csvFile, consoleLog
rootLogger.appenderRef.csvFile.ref = CSVFILE
rootLogger.appenderRef.console.ref = consoleLog
Can anyone please help me to use the same logger for both transaction and application logging?
and what changes do I need to make in my properties file? Thanks!
You can use Filters to seperate ObjectArrayMessage messages from the rest (cf. documentation). Although there is no built-in filter that checks the class of the Message object, you can easily write one:
#Plugin(name = "ObjectArrayMessageFilter", category = Node.CATEGORY, elementType = Filter.ELEMENT_TYPE)
public class ObjectArrayMessageFilter extends AbstractFilter {
public static class Builder extends AbstractFilterBuilder<Builder>
implements org.apache.logging.log4j.core.util.Builder<ObjectArrayMessageFilter> {
#Override
public ObjectArrayMessageFilter build() {
return new ObjectArrayMessageFilter(getOnMatch(), getOnMismatch());
}
}
#PluginBuilderFactory
public static Builder newBuilder() {
return new Builder();
}
private ObjectArrayMessageFilter(Result onMatch, Result onMismatch) {
super(onMatch, onMismatch);
}
private static Result filter(Object msg) {
return msg instanceof ObjectArrayMessage ? onMatch : onMismatch;
}
#Override
public Result filter(final LogEvent event) {
return filter(event.getMessage());
}
#Override
public Result filter(final Logger logger, final Level level, final Marker marker, final Message msg,
final Throwable t) {
return filter(msg);
}
#Override
public Result filter(final Logger logger, final Level level, final Marker marker, final Object msg,
final Throwable t) {
return filter(msg);
}
}
You can then attach the filter to the appenders, so that it is always applied to a given appender:
appender.csvFile.type = RollingFile
...
appender.csvFile.filter.omaf.type = ObjectArrayMessageFilter
appender.csvFile.filter.omaf.onMatch = ACCEPT
appender.csvFile.filter.omaf.onMismatch = DENY
appender.console.type = Console
...
appender.console.filter.omaf.type = ObjectArrayMessageFilter
appender.console.filter.omaf.onMatch = DENY
appender.console.filter.omaf.onMismatch = ACCEPT
Remark: When developing custom plugins, some standard rules apply: cf. documentation. Basically the plugin's code must be compiled with the annotation processor in log4j-core.
I configure my application to use a logging class rather than the logging.properties file in the jre conf folder using -Djava.util.logging.config.class=com.jthink.songkong.logging.StandardLogging
And it works, except that I notice that information that is just mean to be going to my log file is also going to console window, this is being noticed when using --win-console with jpackage on Windows but I think it was already happening before I was using JPackage
This is my logging class:
public final class StandardLogging
{
public static int LOG_SIZE_IN_BYTES = 10000000;
//Default parent logger
public static Logger defaultLogger = Logger.getLogger("");
//jaudiotagger logger
public static Logger ioLogger = Logger.getLogger("org.jaudiotagger");
//SongKong logger
public static Logger debugLogger = Logger.getLogger("com.jthink");
//SongKong usaer Message Logger
public static Logger userInfoLogger = Logger.getLogger("com.jthink.songkong.ui.MainWindow");
//General c3p0
public static Logger c3p0Logger = Logger.getLogger("com.mchange.v2.c3p0");
//For capturing Preapred stament Cache hits
//public static Logger c3p0ConnectionLogger = Logger.getLogger("com.mchange.v2.c3p0.stmt");
//For capturing stack traces when connection lasted too long
public static Logger c3p0PooledConnectionLogger = Logger.getLogger("com.mchange.v2.resourcepool.BasicResourcePool");
//HIbernate SQL
public static Logger hibernateLogger = Logger.getLogger("org.hibernate.SQL");
//TODO not sure this even used, I think CmdLogger just does System.out
private static Logger cmdlineLogger = Logger.getLogger("cmdline");
protected void configureLoggerLevels()
{
//Default Log Level, used by any 3rd party libs we are using if not configured further
defaultLogger.setLevel(Level.WARNING);
//For Debug (songKong and jaudiotagger)
ioLogger.setLevel(Level.WARNING);
ioLogger.setUseParentHandlers(false);
try
{
//If GeneralPreferences exist and we can access set from user value
ioLogger.setLevel(Level.parse(String.valueOf(GeneralPreferences.getInstance().getIoDebugLevel())));
}
catch(Exception ex)
{
}
debugLogger.setLevel(Level.WARNING);
debugLogger.setUseParentHandlers(false);
try
{
//If GeneralPreferences exist and we cBuildBuiklan access set from user value
debugLogger.setLevel(Level.parse(String.valueOf(GeneralPreferences.getInstance().getDebugLevel())));
}
catch(Exception ex)
{
}
//C3p0 Logger
c3p0Logger.setLevel(Level.INFO);
c3p0Logger.setUseParentHandlers(false);
//Set to FINEST to see SQL
hibernateLogger.setLevel(Level.WARNING);
hibernateLogger.setUseParentHandlers(false);
//For Capturing CheckIn/Outs nad Prepared Statement Cache Hits
//c3p0ConnectionLogger.setLevel(Level.FINEST);
//c3p0ConnectionLogger.setUseParentHandlers(false);
//For capturing stacktrace from timed out connections
c3p0PooledConnectionLogger.setLevel(Level.INFO);
c3p0PooledConnectionLogger.setUseParentHandlers(false);
//For user message log
userInfoLogger.setUseParentHandlers(false);
userInfoLogger.setLevel(Level.FINEST);
userInfoLogger.setUseParentHandlers(false);
userInfoLogger.setLevel(Level.FINEST);
}
protected void configureHandlers() throws Exception
{
//Set Filehandler used for writing to debug log
String logFileName = Platform.getPlatformLogFolderInLogfileFormat() + "songkong_debug%u-%g.log";
FileHandler fe = new FileHandler(logFileName, LOG_SIZE_IN_BYTES, 10, true);
fe.setEncoding(StandardCharsets.UTF_8.name());
fe.setFormatter(new LogFormatter());
fe.setLevel(Level.FINEST);
//Set Filehandler used for writing to user log
String userLogFileName = Platform.getPlatformLogFolderInLogfileFormat() + "songkong_user%u-%g.log";
FileHandler userFe = new FileHandler(userLogFileName, LOG_SIZE_IN_BYTES, 10, true);
userFe.setFormatter(new com.jthink.songkong.logging.UserLogFormatter());
userFe.setLevel(Level.FINEST);
//Write this output to debug log file
//defaultLogger.addHandler(fe);
c3p0Logger.addHandler(fe);
c3p0PooledConnectionLogger.addHandler(fe);
//c3p0ConnectionLogger.addHandler(fe);
ioLogger.addHandler(fe);
debugLogger.addHandler(fe);
hibernateLogger.addHandler(fe);
//Write this output to user log file
userInfoLogger.addHandler(userFe);
//For cmd line output, is this still used
cmdlineLogger.setUseParentHandlers(false);
ConsoleHandler cmdLineHandler = new java.util.logging.ConsoleHandler();
cmdLineHandler.setLevel(Level.FINEST);
cmdLineHandler.setFormatter(new CmdLineFormatter());
cmdlineLogger.addHandler(cmdLineHandler);
System.out.println("debuglogfile is:" + logFileName);
System.out.println("userlogfile is:" + userLogFileName);
}
public StandardLogging()
{
try
{
configureLoggerLevels();
configureHandlers();
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
}
And this is an example of code that I would just expect to go to my songkong-debug0-0.log file, but it is also being output to the console window:
MainWindow.logger.warning("User Dir:"+ System.getProperty("user.dir"));
MainWindow.logger.warning("Java Dir:"+ System.getProperty("java.home"));
Why would that be?
Add code to com.jthink.songkong.logging.StandardLogging to print the logger tree at the end of your configuration. This will help you troubleshoot what is happening. Since you are using a configuration class you can even make your own system property to toggle printing the logger tree if you need to in the future.
If your console output looks like the format of the CmdLineFormatter you created then code is either using that logger or a child logger that is printing to the parent handlers. Assuming you control the format of the CmdLineFormatter you can include the logger name in the output to locate the logger in question.
If the output looks like the SimpleFormatter then more likely it is the console handler that is attached to the root logger. Simply remove that handler from the root logger in your configuration class.
A more complete solution is to invoke LogManager.reset at the start of your StandardLogging class constructor. This would clear out the configuration that the JRE set prior to invoking your changes. An alternative is to set the java.util.logging.config.file to point to a null device from the command line in addition to setting your java.util.logging.config.class.
Although I've configured handler to write logs to files structure, but I get the log on the console bar when I run the program, how can I masquerade logs so it will only be written on files without exposing to console.
static {
System.setProperty("java.util.logging.config.file",
"log.properties");
//must initialize loggers after setting above property
}
static Logger logger = Logger.getLogger("com.debugz");
public static void main(String[] args) {
try {
FileHandler h = new FileHandler("%h/file_%g.log",1,4);
h.setFormatter(new SimpleFormatter());
logger.addHandler(h);
logger.log(Level.INFO,"Hello");
}
catch (Exception error){
System.out.println("Error : "+error.getMessage());
}
}
You may try logger.setUseParentHandlers(false);
void java.util.logging.Logger.setUseParentHandlers(boolean useParentHandlers)
Specify whether or not this logger should send its output to its parent Logger. This means that any LogRecords willalso be written to the parent's Handlers, and potentially to its parent, recursively up the namespace.
Parameters: useParentHandlers true if output is to be sent to the logger's parent.
The above won't work for your use case if a ConsoleHandler has been explicitely added to the logger e.g. logger.addHandler(new ConsoleHandler()); in which case, you may remove that handler if needed via logger.removeHandler(<handlerToBeRemoved>)
Hope that helps! Cheers
I have made a custom formatter for the java.util logger, implemented as below
logger = Logger.getAnonymousLogger();
logger.setUseParentHandlers(false);
sb = new StringBuffer();
try {
Handler handler = new FileHandler("xx.log");
handler.setFormatter(new CustomFormatter());
logger.addHandler(handler);
} catch (IOException e) {
e.printStackTrace();
}
private class CustomFormatter extends Formatter {
#Override
public String format(LogRecord record) {
String msg = record.getMessage();
sb.append(msg);
System.out.print(sb.toString());
return sb.toString();
}
}
Running the following code:
logger.finest(msg);
logger.finer(msg);
logger.fine(msg);
logger.config(msg);
logger.info(msg);
logger.warning(msg);
logger.severe(msg);
logger.log(new LogRecord(Level.FINE, msg));
Only info, warning, and severe work. the other methods run but return nothing to console, and write nothing to the log file. I need to implement at least 4 logging levels, but only the 3 work.
What is happening to log,config,fine,finer, finest? How do I get them to use my formatter? Or how do I pass the level to the formatter?
You may set the Level of logging with setlevel(Level)
Set the log level specifying which message levels will be logged by
this logger. Message levels lower than this value will be discarded.
In your case as you are interested in all levels, it would be :
logger.setLevel(Level.ALL);
I have log4j configured to rotate the log every day.
In special situations I would like to trigger an additional log rotation manually.
Is this possible - and if so: How?
Solved like this:
void rolloverLogs() {
for(final Enumeration<?> loggers = LogManager.getCurrentLoggers(); loggers.hasMoreElements(); ) {
final Logger logger = (Logger) loggers.nextElement();
for (final Enumeration<?> appenders = logger.getAllAppenders(); appenders.hasMoreElements(); ) {
final Appender a = (Appender) appenders.nextElement();
if(!RollingFileAppender.class.isInstance(a))
continue;
((RollingFileAppender)a).rollOver();
}
}
}
The technique shown by Lahiru will only locate Appenders configured for a specific Logger which is based on the class the code resides in. The exact Appender(s) located may may vary if the configuration changes.
If you know the Appender name the best way to do this is:
org.apache.logging.log4j.core.LoggerContext context = LogManager.getContext(false);
Appender appender = context.getConfiguration().getAppender(appenderName);
if (appender instanceof RollingFileAppender) {
((RollingFileAppender) appender).getManager().rollover();
}
If you want to roll over all RollingFileAppenders you could do:
org.apache.logging.log4j.core.LoggerContext context = (org.apache.logging.log4j.core.LoggerContext) LogManager.getContext(false);
for (Appender appender : context.getConfiguration().getAppenders().values()) {
if (appender instanceof RollingFileAppender) {
((RollingFileAppender) appender).getManager().rollover();
}
}
For log4j2, you can use following.
org.apache.logging.log4j.Logger logManagerLogger = LogManager.getLogger();
Map<String, org.apache.logging.log4j.core.Appender> appenders = ((org.apache.logging.log4j.core.Logger) logManagerLogger).getAppenders();
appenders.forEach((appenderName, appender) -> {
if (appender instanceof RollingFileAppender) {
logger.info("Rolling over appender " + appenderName);
((RollingFileAppender) appender).getManager().rollover();
}
});
If you're keeping track of your appenders, you could call
https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html#rollOver()
That should do it. I guess it's also possible to iterate through all appenders you can find starting from root level - just make sure you keep track of which appenders you already rolled.