Develop Reference

Deeplearning4j Disable Logging

I have a deeplearning for java project which is producing huge amounts of logger output on STDO. I want to disable that but I cant seem to figure out how to do it.
I have a log4j.properties file in my src/main/resources folder which looks like this:
log4j.rootLogger=ERROR, Console
log4j.logger.play=WARN
log4j.appender.Console=org.apache.log4j.ConsoleAppender
log4j.appender.Console.layout=org.apache.log4j.PatternLayout
log4j.appender.Console.layout.ConversionPattern=%d{ABSOLUTE} %-5p ~ %m%n
log4j.appender.org.springframework=WARN
log4j.appender.org.nd4j=WARN
log4j.appender.org.canova=WARN
log4j.appender.org.datavec=WARN
log4j.appender.org.deeplearning4j=WARN
log4j.appender.opennlp.uima=OFF
log4j.appender.org.apache.uima=OFF
log4j.appender.org.cleartk=OFF
log4j.logger.org.springframework=WARN
log4j.logger.org.nd4j=WARN
log4j.logger.org.canova=WARN
log4j.logger.org.datavec=WARN
log4j.logger.org.deeplearning4j=WARN
log4j.logger.opennlp.uima.util=OFF
log4j.logger.org.apache.uima=OFF
log4j.logger.org.cleartk=OFF
log4j.logger.org.deeplearning4j.optimize.solvers.BaseOptimizer=OFF
slf4j.logger.org.deeplearning4j.optimize.solvers.BaseOptimizer=OFF
The specific output that is far too much is:
21:26:34.860 [main] DEBUG o.d.optimize.solvers.BaseOptimizer - Hit termination condition on iteration 0: score=1.2894165074915344E19, oldScore=1.2894191699433697E19, condition=org.deeplearning4j.optimize.terminations.EpsTermination#55f111f3
which happens multiple times a second while training.

The output of the log entry that you have provided look very much as the SLF4J output with Logback format (not LOG4J output).
Also dependencies of deeplearning4j-core advice SLF4J is used for logging.
Hence your log4j.properties have no effect on deeplearning4j logging. Try to add logback.xml configuration to the resources as well and switch to WARN or ERROR level for root logger, see https://logback.qos.ch/manual/configuration.html

There are some properties in the framework DL4J (1.0.0-beta7) that activate/deactivate the logs. I found some of them:
import org.nd4j.common.config.ND4JSystemProperties;
System.setProperty(ND4JSystemProperties.LOG_INITIALIZATION, "false");
System.setProperty(ND4JSystemProperties.ND4J_IGNORE_AVX, "true");
System.setProperty(ND4JSystemProperties.VERSION_CHECK_PROPERTY, "false");
Notice that this is an unconventional solution. On the other hand, there are some messages impossible to avoid:
MultiLayerNetwork.init()
In this method you can find a OneTimeLogger without validations:
OneTimeLogger.info(log, "Starting MultiLayerNetwork with WorkspaceModes set to [training: {}; inference: {}], cacheMode set to [{}]",
layerWiseConfigurations.getTrainingWorkspaceMode(),
layerWiseConfigurations.getInferenceWorkspaceMode(),
layerWiseConfigurations.getCacheMode());
If you find a better way to disable log messages inside DL4J please share it. There are some other ways outside the DL4J library.

How to add machine name to logfile name using log4j?

Is there a way to add machine name to logfile name in log4j?
This is my situation: I have 4 machines/servers running Weblogic and each machine/server is running 2 instances/nodes of the JVM application (provided by a vendor); this application generates logs using log4j; however logfile names are all the same for all machines/servers and that causes me too much trouble when I have to review or gather logs for troubleshooting.
I've been already able to distinguish instances/nodes among them by using the following - weblogic.Name property:
<appender name="DFe" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="servers/${weblogic.Name}/logs/dfe_${weblogic.Name}.log"/>
Now I need to distinguish logfile names by machine/server name as well. How to do that? Is there any weblogic.Machine property or something?
Example: my machine/server name is "ausplsynapp03" and instance/node names are "track60800-01" and "track60800-02"; so my logfile names would be "dfe_ausplsynapp03_track60800-01.log" and "dfe_ausplsynapp03_track60800-02.log".
Thanks in advance for any help.

By specifying ${weblogic.Name} in your file name or appender pattern you are actually using value from WebLogic JVM argument.
If you can add custom argument to JMV arguments you can use it for logging purposes.
Otherwise you can check for other WebLogic arguments here: https://docs.oracle.com/cd/E13222_01/wls/docs90/admin_ref/weblogicServer.html
Maybe weblogic.Domain will work for you.
If you are OK with having host in log message you can set MDC/NDC attribute and output it to log message using x or %X{key} (reference https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html)

Try to set environment property like below
System.setProperty(“serverHostName“, InetAddress.getLocalHost().getHostName());
then use it in your log4j.xml
as below
param name=”file” value=”${serverHostName}.log”

Maven. Nested properties filtering doesn't work

I have two property files:
#environment.properties
env = production
and second file is:
#commons.properties
production.port = 123
test.port = 567
Also,I have resource file which need be filtered by environment.properties file and commons.properties file and copied.
The resource-file contains:
${${env}.port}
So,I want to filter my resource file with first file and get:
${production.port}
and then I want to filter it with second filter file and get:
123
I use maven 3.2.5 and the resource-file isn't filtered at all.
I know that there's issue related with this problem:
https://jira.codehaus.org/browse/MRESOURCES-70 But it still unresolved.
So,my question is - is there any solution to resolve this problem? (actually,I think that resource-plugin should be modified for work with nested property filtering).
And second question - does exist any way to avoid this problem by refactoring,I mean any other architecture solution. Or, what would you do if you had same problem?

log4j file appender do not normal

My log4j.properties is:
log4j.appender.R.File=../logs/xrzgather.log
log4j.appender.R.MaxFileSize=10MB
log4j.appender.R.MaxBackupIndex=100
I have many tomcat server run the application,and all of them run normal with the right log but one of them do not normal,it generate the log file:
xrzgather.log 182M
xrzgather.log.19 254M
xrzgather.log.45 48.6M
xrzgather.log.50 400M
xrzgather.log.90 20.8M
xrzgather.log.92 66.3M
xrzgather.log.99 11.9M
Has anyone know what's wrong?

I find the reason..
I config a Context element in %tomcat_home%\conf\Server.xml like this:
<Context path="xrz-gather" docBase="xrz-gather-web-1.0-SNAPSHOT" />
if I remove this,log will be normal.

Log4J change File path dynamically

I want to change the path and file name of my log4j logfile dynamically.
I have read a lot of pages and nearly every tell me that I should use system properties like here:
how to change the log4j log file dynamically?
So my log4j.properties file looks like this:
log4j.logger.JDBC_LOGGER=INFO,jdbcTests
log4j.additivity.JDBC_LOGGER = false
log4j.appender.jdbcTests=org.apache.log4j.FileAppender
log4j.appender.jdbcTests.File=${my.log}
log4j.appender.jdbcTests.layout=org.apache.log4j.PatternLayout
log4j.appender.jdbcTests.append = false
log4j.appender.jdbcTests.layout.ConversionPattern=%d{yyyy mm dd HH:mm:ss} %5p %C:Line %L - %m%n
In my main method I am going to set my new system property:
System.setProperty("{my.log", "C:/logfile.log");
But I just get an error:
log4j:ERROR setFile(null,false) call failed.
java.io.FileNotFoundException:
at java.io.FileOutputStream.open(Native Method)....
And when I try to read my set system property with:
System.out.println(System.getProperty("my.log"));
it return null.
What do I do wrong?

I think you meant "my.log" not "{my.log"
System.setProperty("my.log", "C:/logfile.log");
I wouldn't imagine you can change this once the logging has started so you need to set this as early in your program as possible.
BTW: You can sub-class FileAppender to make it behave any way you like.

You have a misspelling: "{my.log" instead of "my.log"

Just set property before instantiating logger, once you initilize the logger setting the property will not worth. just like below:
System.setProperty("my.log", "C:\\src\\com\\web\\automation\\logs\\Application.log");
PP_LOGS = Logger.getLogger("devpinoyLogger");