Develop Reference

Need information related to sshj library. How logging works?

I'm trying to transfer a file from one server to other server using sftp protocol. So, I'm trying to use sshj library for this. Now before transferring the file I'm performing several activities such as zipping and after transferring unzipping . So, I'm logging this details to successLog and if ay error to FailureLog.
here is my log4j properties:-
# Define the root logger
log4j.rootLogger = DEBUG, toConsole
# Define the console appender
log4j.appender.toConsole=org.apache.log4j.ConsoleAppender
log4j.appender.toConsole.layout=org.apache.log4j.PatternLayout
log4j.appender.toConsoleAppender.layout.ConversionPattern=%d{HH:mm:ss} %5p [%t] - %c.%M - %m%n
# Define the file appender
log4j.appender.success=org.apache.log4j.FileAppender
log4j.appender.success.File=logs/success.log
log4j.appender.success.Append=false
log4j.appender.success.layout=org.apache.log4j.PatternLayout
log4j.appender.success.layout.conversionPattern=%d{HH:mm:ss} %5p [%t] - %c.%M - %m%n
# Define the file appender
log4j.appender.failure=org.apache.log4j.FileAppender
log4j.appender.failure.File=logs/failure.log
log4j.appender.failure.Append=false
log4j.appender.failure.layout=org.apache.log4j.PatternLayout
log4j.appender.failure.layout.conversionPattern=%d{HH:mm:ss} %5p [%t] - %c.%M - %m%n
log4j.category.successLogger=DEBUG, success
log4j.additivity.successLogger=true
log4j.category.failureLogger=WARN, failure
log4j.additivity.failureLogger=false
and in java code I'm using this as :-
static final Logger successLog = Logger.getLogger("successLogger");
static final Logger failureLog = Logger.getLogger("failureLogger");
and example failureLog.error("exception occured due to so an so reason",e);
now I would like to store sshj library log in some file called sftp.log. How can I do that??so far I was manually logging into success and failure logs but sshj log is by default printing on console which I want to write it to some file.

Here's my example of writing the SFTP-session FINE-level log to file:
private void enableFineLogging() {
try {
fileHandler = new FileHandler(
"./logs/fine_sshj.log",
10000000, 1000, true);
fileHandler.setLevel(Level.FINER);
fileHandler.setFormatter(new SimpleFormatter());
final Logger app = Logger.getLogger("net.schmizz");
app.setLevel(Level.FINER);
app.addHandler(fileHandler);
app.setUseParentHandlers(false);
app.info(
"######################################################################## "
+ "START LOGGING NEW SSHJ SESSION "
+ "########################################################################");
} catch (Exception e) {
// Do something...
}
}
Though I'm not sure if are able to separate the log levels so that successLogger would have only the success cases... I think there will be also WARNING and SEVERE level messages.

Log4J API not printing messages in the java program

I am trying to implement Log4J API in my java project.
To do that, I have created a properties as shown in the image below (highlighted in yellow):
Project Structure Image
These are the properties I set in the file:
# TRACE < DEBUG < INFO < WARN < FATAL
log4j.rootLogger=TRACE, DEBUG, INFO, file
# Console
# log4j.appender.toConsole=org.apache.log4j.ConsoleAppender
# log4j.appender.toConsole.layout=org.apache.log4j.PatternLayout
# log4j.appender.toConsole.layout.ConversionPatter=%d{HH:mm:ss} %5p [%t] - $c.%M - %m%n
# Redirecting log messages to console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
I declared the LOGGER object in my class as following:
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
public class Credits{
Logger LOGGER = null;
public Credits(String sources) {
LOGGER = Logger.getLogger(ChunkRecon.class.getName());
LOGGER.setLevel(Level.DEBUG);
String creditNames = "select tablename, creditNumbers, credit_type from schema.table where credit_type in ("sources")";
LOGGER.debug("This is a debug message");
system.out.println("This message is from println");
}
}
In the output, I see the message from sysout: This message is from println but not the debug message.
Could anyone let me know what is the mistake I am doing here ?

Try to rename your loggerproperties to log4j.properties and check it is in classpath. Another problem with rootLogger, see explanation here
Also Logger is usually used as static variable. For example:
public class Credits {
private static final Logger logger = Logger.getLogger(Credits.class);
public Credits(String sources) {
logger.setLevel(Level.DEBUG);
logger.debug("This is a debug message");
System.out.println("This message is from println");
}
}
log4j.properties
log4j.rootLogger=debug, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

Your log4j.rootLogger declaration seems incorrect. Try as -
log4j.rootLogger=TRACE,stdout,file
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.file=org.apache.log4j.RollingFileAppender
If you only want the logs on console then remove file logging fully.
log4j.rootLogger=TRACE,stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
Refer section Configuring loggers from JavaDoc.
The syntax for configuring the root logger is:
log4j.rootLogger=[level], appenderName, appenderName, ...

Log 4j Conversion Pattern not working kafka.producer.KafkaLog4jAppender in Kafka.0.8.1.1

I am trying to use KafkaLog4jAppender to write to kafka and also a file appender to write to a file. The conversion pattern works for the file, but for the kafka appender it is not working
The output of the file appender is 2014-09-19T22:30:14.781Z INFO com.test.poc.StartProgram Message1
But the Kafka appender has output as Message1
Find below my log4j.properties file
log4j.rootCategory=INFO
log4j.appender.file=org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.file.File=logs/test.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd'T'HH:mm:ss.SSS'Z'}{UTC} %p %C %m%n
log4j.logger.com=INFO,file,KAFKA
#Kafka Appender
log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout
log4j.appender.KAFKA.layout.ConversionPattern=%d{yyyy-MM-dd'T'HH:mm:ss.SSS'Z'}{UTC} %p %C %m%n
log4j.appender.KAFKA.ProducerType=async
log4j.appender.KAFKA.BrokerList=localhost:9092
log4j.appender.KAFKA.Topic=test`enter code here`
log4j.appender.KAFKA.Serializer=kafka.test.AppenderStringSerializer

This is a known Kafka bug; the fix is in Kafka 0.8.2:
https://issues.apache.org/jira/browse/KAFKA-847

Log4j logging info messages without logging warn messages

I'm trying to log certain info messages onto a file but as soon as I run the application both warn and info messages are logged. Now, from what I've read from this site, you cannot log one without logging the other. Has anyone tried this before? If so, how did your properties file look like?
My properties file looks like this:
***** Set root logger level to INFO and its two appenders to stdout and R.
log4j.rootLogger=INFO, stdout, R
# ***** stdout is set to be a ConsoleAppender.
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
# ***** stdout uses PatternLayout.
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
# ***** Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%5p [%M has started] (%F:%L) - %m%n
/
# ***** R is set to be a RollingFileAppender.
log4j.appender.R=org.apache.log4j.DailyRollingFileAppender
log4j.appender.R.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.R.File="folder where log will be saved"
log4j.appender.R.layout.ConversionPattern=%5p [%m has started] %c{2}.[%x] (%F:%L) %d{yyyy-MM-dd HH:mm:ss} - %m%n
# ***** R uses PatternLayout.
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%5p [%m has started] %c{2}.[%x] (%F:%L) %d{yyyy-MM-dd HH:mm:ss} - %m%n

AFAIK there's no standard way to suppress higher log levels than those you are interested in.
However, you might be able to use a custom appender to do that.
It might look similar to this:
public class MyAppender extends AppenderSkeleton {
protected void append(LoggingEvent event) {
if( event.getLevel() == Level.INFO ) {
//append here, maybe call a nested appender
}
}
}

The log level WARN is higher than INFO, and the logging configuration defines the minimum threshold level to be logged by the appender. So, all messages higher than that level will also be logged.
Hence, the WARN messages are expected. And I don't think you can configure it to the way you want.

If the WARN messages that should not be printed come from a different package than the INFO messages, then you can define different log levels for these packages. The first can specify level ERROR and the second INFO
it should look something like this:
log4j.logger.com.test.something=ERROR
log4j.logger.com.other.package=INFO
cheers

Why do you want to filter the WARN level? As peshkira said, you could use two different loggers for splitting/filtering your log output, or you could use tools like grep for filtering (offline) or you could simply remove the WARN logs from your code if your don't need them anyway.

As far as I understand, advanced filters like LevelMatchFilter and LevelRangeFilter can do the trick for you.
Thing worth keeping in mind though is that using these may require xml config instead of properties: Can't set LevelRangeFilter for log4j

How to add the Process id to a LOG4J log file?

I currently have the below pattern layout in log4j. I want to add the Process id to the log file. How can I do it?
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
Pasted sample log message
2011-01-07 11:48:21,940 [main] INFO Testing1
2011-01-07 11:48:21,942 [main] INFO Test.common.ApplicationProperties - Used log4j
log4j.properties
"log4j.properties" [Read only] 26 lines, 884 characters
log4j.rootCategory=DEBUG, stdout, A1
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Threshold=WARN
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy/MM/dd HH:mm:ss} %-5p (%c) %m%n
log4j.appender.A1=org.apache.log4j.RollingFileAppender
log4j.appender.A1.Threshold=DEBUG
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
log4j.appender.A1.File=/homw/cus/logs/ccl.02.log
log4j.appender.A1.MaxFileSize=5MB
log4j.appender.A1.MaxBackupIndex=40
log4j.category.test.common.DBConnectionPool=WARN
log4j.category.test.common.DataBaseHandler=WARN
log4j.category.test.cttg.tables=WARN
log4j.category.test.middleware.tables=WARN
log4j.logger.org.apache.axis=ERROR
log4j.logger.org.apache.catalina=ERROR

You should use MDC to do it
In the config file :
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy/MM/dd HH:mm:ss} %-5p (%c) %m%n %X{PID}
%X{PID} is used to match the context value PID
And then, in the code, before the logging begins :
log4j 1.x
RuntimeMXBean rt = ManagementFactory.getRuntimeMXBean();
String pid = rt.getName();
MDC.put("PID", pid);
log4j 2.x
RuntimeMXBean rt = ManagementFactory.getRuntimeMXBean();
String pid = rt.getName();
ThreadContext.put("PID", pid);

There is no way of doing it using standard Java classes. Generally process ID is appended at file level not at the log level. And here (archived here) is an example of doing it.

With the following pattern, showing threadID and class. If you want to see the Process ID, might check here
log4j.appender.SYSLOG.layout.conversionPattern=%-5p %d{ddMMyyyy HH:mm:ss.SSS} [%t:%c] %m%n
I managed to do so, but I have several appenders, one for each part of the application, like the following:
log4j.rootCategory=ERROR, SYSLOG2
log4j.logger.com.myself.logic=DEBUG, SYSLOG
log4j.logger.com.myself.database=DEBUG, SYSLOG3
log4j.logger.com.myself.other=DEBUG, SYSLOG
Here are some examples of generated logs, to check if this is what you require:
INFO 20012015 11:56:17.318 [pool-1-thread-1:com.myself.logic.MonitoringTask] 10602: Validation of orders before UTC=2015-01-20T16:56:17
INFO 20012015 11:56:34.200 [main:com.gmv.pazgs.mus.mpu.CLibWrapper] 10402: MPU Library folder: xxxxx
INFO 20012015 11:56:34.209 [main:com.gmv.pazgs.mus.mpu.CLibWrapper] 10402: MPU Configuration folder: xxxxx
INFO 20012015 11:56:34.773 [main:com.gmv.pazgs.mus.mpu.CLibWrapper] 10402: Calling InitLeap()
INFO 20012015 11:56:34.786 [main:com.gmv.pazgs.mus.mpu.CLibWrapper] 10402: Calling InitFourier()
INFO 20012015 11:57:10.151 [CRON_Thread:com.myself.other.DTOLDispatcher] 10202: Times to send: [11:00, 23:00] - current time = Tue Jan 20 11:57:10 UTC 2015
INFO 20012015 11:58:10.165 [pool-1-thread-4:com.myself.logic.MonitoringTask] 10602: Executing Monitoring Task UTC=2015-01-20T11:58:10
INFO 20012015 11:58:10.171 [pool-1-thread-5:com.myself.logic.OrderValidationTask] 10602: Executing OrderValidationTask UTC=2015-01-20T11:58:10
INFO 20012015 11:58:10.291 [CRON_Thread:com.myself.other.DTOLDispatcher] 10202: Times to send: [11:00, 23:00] - current time = Tue Jan 20 11:58:10 UTC 2015
INFO 20012015 11:58:10.684 [pool-1-thread-4:com.myself.logic.MonitoringTask] 10602: Expiration of orders before UTC=2015-01-20T11:58:10
INFO 20012015 11:58:11.218 [pool-1-thread-4:com.myself.logic.MonitoringTask] 10602: checking orders for suppresed status before UTC=2015-01-20T11:58:11
INFO 20012015 11:58:11.244 [pool-1-thread-4:com.myself.logic.MonitoringTask] 10602: Validation of orders before UTC=2015-01-20T16:58:11