I would like to know how console remove an output from a third-part library. I already removed some warnings with static block System.setProperty, but Pdfbox is printing around 100 lines of output, as I'm testing other things with the console, I'd like to remove them, they looks like:
[main] DEBUG , org.apache.pdfbox.pdfparser.PDFObjectStreamParser
and these:
[main] DEBUG org.apache.fontbox.util.autodetect.FontFileFinder - found checkFontfile
Related
I use PDFbox to read PDF files; I understand that it does not use Log4j, it uses apache's common logging (I was surprised to hear this, since its Maven configuration includes Log4j v1).
When I run my program, I get messages like:
18:13:20.093 [SwingWorker-pool-2-thread-1] DEBUG org.apache.fontbox.ttf.PostScriptTable - No PostScript name information is provided for the font CourierNewPS-BoldMT
18:13:20.102 [SwingWorker-pool-2-thread-1] DEBUG org.apache.fontbox.ttf.GlyphSubstitutionTable - Type 4 GSUB lookup table is not supported and will be ignored
18:13:20.102 [SwingWorker-pool-2-thread-1] DEBUG org.apache.fontbox.ttf.GlyphSubstitutionTable - Type 6 GSUB lookup table is not supported and will be ignored
18:13:20.102 [SwingWorker-pool-2-thread-1] DEBUG org.apache.fontbox.ttf.GlyphSubstitutionTable - Type 6 GSUB lookup table is not supported and will be ignored
(68 messages about GlyphSubstitutionTable), and
18:13:20.163 [SwingWorker-pool-2-thread-1] DEBUG org.apache.fontbox.ttf.GlyphSubstitutionTable - Type 4 GSUB lookup table is not supported and will be ignored
18:13:20.476 [Finalizer] DEBUG org.apache.pdfbox.io.ScratchFileBuffer - ScratchFileBuffer not closed!
18:13:23.327 [Finalizer] DEBUG org.apache.pdfbox.io.ScratchFileBuffer - ScratchFileBuffer not closed!
I have tried putting a commons-logging.properties file in src/main/resources containing
org.apache.commons.logging.Log=org.apache.commons.logging.impl.NoOpLog
I have tried putting the code java.util.logging.Logger.getLogger("org.apache.pdfbox").setLevel(java.util.logging.Level.OFF) into a static initializer of the class containing the static void main method of my program.
I have tried putting
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
into that static initializer, separately and together with the previous code.
When I read the PDF in my code, I still get the log messages mentioned above with any and all of the above attempted fixes. How can I eliminate them; I would prefer to just limit them to warning, error, and severe messaages instead of eliminating all messages, but at this point I'll take suggestions for either one.
public class Main {
static {
java.util.logging.Logger.getLogger(
"org.apache").setLevel(java.util.logging.Level.SEVERE);
}
// ...
has helped me to disable warnings from fontbox also.
While "org.apache.pdfbox" only disables pdfbox warnings, not fontbox ones.
And "org.apache.fontbox" somehow does not disable fontbox warnings.
I have simplified tomcat's logging properties to just this:
handlers = java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.SimpleFormatter.format=%1$tF %1$TT.%1tL [::] %4$s %3$s %5$s %n
org.springframework.aop.framework.CglibAopProxy.level = ERROR
My issue is that the last line seems to be completely ignored and I keep seeing logs like this:
2018-05-09 10:40:33.159 [::] INFO org.springframework.aop.framework.CglibAopProxy
I am absolutely sure it comes from this logger thanks to the log format I set in the logging.properties.
My issue is that the last line seems to be completely ignored...
It is ignored because ERROR fails to be parsed as valid level. Per the docs:
Valid values are integers between Integer.MIN_VALUE and Integer.MAX_VALUE, and all known level names. Known names are the levels defined by this class (e.g., FINE, FINER, FINEST), or created by this class with appropriate package access, or new levels defined or created by subclasses.
Change your logging line to one of the valid levels that is higher than INFO. Choose one of the following log lines:
org.springframework.aop.framework.CglibAopProxy.level = OFF
org.springframework.aop.framework.CglibAopProxy.level = SEVERE
org.springframework.aop.framework.CglibAopProxy.level = WARNING
I have a deeplearning for java project which is producing huge amounts of logger output on STDO. I want to disable that but I cant seem to figure out how to do it.
I have a log4j.properties file in my src/main/resources folder which looks like this:
log4j.rootLogger=ERROR, Console
log4j.logger.play=WARN
log4j.appender.Console=org.apache.log4j.ConsoleAppender
log4j.appender.Console.layout=org.apache.log4j.PatternLayout
log4j.appender.Console.layout.ConversionPattern=%d{ABSOLUTE} %-5p ~ %m%n
log4j.appender.org.springframework=WARN
log4j.appender.org.nd4j=WARN
log4j.appender.org.canova=WARN
log4j.appender.org.datavec=WARN
log4j.appender.org.deeplearning4j=WARN
log4j.appender.opennlp.uima=OFF
log4j.appender.org.apache.uima=OFF
log4j.appender.org.cleartk=OFF
log4j.logger.org.springframework=WARN
log4j.logger.org.nd4j=WARN
log4j.logger.org.canova=WARN
log4j.logger.org.datavec=WARN
log4j.logger.org.deeplearning4j=WARN
log4j.logger.opennlp.uima.util=OFF
log4j.logger.org.apache.uima=OFF
log4j.logger.org.cleartk=OFF
log4j.logger.org.deeplearning4j.optimize.solvers.BaseOptimizer=OFF
slf4j.logger.org.deeplearning4j.optimize.solvers.BaseOptimizer=OFF
The specific output that is far too much is:
21:26:34.860 [main] DEBUG o.d.optimize.solvers.BaseOptimizer - Hit termination condition on iteration 0: score=1.2894165074915344E19, oldScore=1.2894191699433697E19, condition=org.deeplearning4j.optimize.terminations.EpsTermination#55f111f3
which happens multiple times a second while training.
The output of the log entry that you have provided look very much as the SLF4J output with Logback format (not LOG4J output).
Also dependencies of deeplearning4j-core advice SLF4J is used for logging.
Hence your log4j.properties have no effect on deeplearning4j logging. Try to add logback.xml configuration to the resources as well and switch to WARN or ERROR level for root logger, see https://logback.qos.ch/manual/configuration.html
There are some properties in the framework DL4J (1.0.0-beta7) that activate/deactivate the logs. I found some of them:
import org.nd4j.common.config.ND4JSystemProperties;
System.setProperty(ND4JSystemProperties.LOG_INITIALIZATION, "false");
System.setProperty(ND4JSystemProperties.ND4J_IGNORE_AVX, "true");
System.setProperty(ND4JSystemProperties.VERSION_CHECK_PROPERTY, "false");
Notice that this is an unconventional solution. On the other hand, there are some messages impossible to avoid:
MultiLayerNetwork.init()
In this method you can find a OneTimeLogger without validations:
OneTimeLogger.info(log, "Starting MultiLayerNetwork with WorkspaceModes set to [training: {}; inference: {}], cacheMode set to [{}]",
layerWiseConfigurations.getTrainingWorkspaceMode(),
layerWiseConfigurations.getInferenceWorkspaceMode(),
layerWiseConfigurations.getCacheMode());
If you find a better way to disable log messages inside DL4J please share it. There are some other ways outside the DL4J library.
I'm trying the quickstart from here: http://datafu.incubator.apache.org/docs/datafu/getting-started.html
I tried nearly everything, but I'm sure it must be my fault somewhere. I tried already:
exporting PIG_HOME, CLASSPATH, PIG_CLASSPATH
starting pig with -cpdatafu-pig-incubating-1.3.0.jar
registering datafu-pig-incubating-1.3.0.jar locally and in hdfs => both succesful (at least no error shown)
nothing helped
Trying this on pig:
register datafu-pig-incubating-1.3.0.jar
DEFINE Median datafu.pig.stats.StreamingMedian();
data = load '/user/hduser/numbers.txt' using PigStorage() as (val:int);
data2 = FOREACH (GROUP data ALL) GENERATE Median(data);
or directly
data2 = FOREACH (GROUP data ALL) GENERATE datafu.pig.stats.StreamingMedian(data);
I get this name-resolve error:
2016-06-04 17:22:22,734 [main] ERROR org.apache.pig.tools.grunt.Grunt
- ERROR 1070: Could not resolve datafu.pig.stats.StreamingMedian using imports: [, java.lang., org.apache.pig.builtin.,
org.apache.pig.impl.builtin.] Details at logfile:
/home/hadoop/pig_1465053680252.log
When I look into the datafu-pig-incubating-1.3.0.jar it looks OK, everything in place. I also tried some Bag functions, same error then.
I think it's kind of a noob-error which I just don't see (as I did not find particular answers for datafu in SO or google), so thanks in advance for shedding some light on this.
Pig script is proper, the only thing that could break is that while registering datafu there were some class dependencies that coudn't been met.
Try to run locally (pig -x local) and see a detailed log.
Check also the version of pig - it should be newer than 0.14.0.
I am using java logging in Groovy and I wanted to modify the format string so I only have one line instead of two, but the methods I tried didn't work - I looked at this question:
How do I get java logging output to appear on a single line?
I tried passing the property to groovy, but it didn't change the format.
I passed it like this:
groovy myScript.groovy -Djava.util.logging.SimpleFormatter.format=%1$tF %1$tT
but it doesn't look like it was picked up.
Here is what I did - I added this code to my groovy:
System.setProperty("java.util.logging.SimpleFormatter.format",
'%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS %4$-6s %5$s%6$s%n');
Logger log = Logger.getLogger("")
FileHandler fh = new FileHandler("../log/replication.log")
log.addHandler(fh)
SimpleFormatter formatter = new SimpleFormatter()
fh.setFormatter(formatter)
which didn't require me to modify the java properties in command line (I didn't want to create additional script to start my groovy script).
You can use JAVA_OPTS for that. For example,
import java.util.logging.*
Logger log = Logger.getLogger('test')
log.setLevel(Level.INFO)
log.info('Test info log')
log.warning('Test warning')
log.config('Test config')
log.fine('Test fine')
and setting (i.e. below for windows):
set JAVA_OPTS="-Djava.util.logging.SimpleFormatter.format=%1$tF %1$tT [%4$s] %5$s %n"
Running the above sample script yields:
> groovy testLoggingJavaUtil.groovy
2015-03-27 17:35:48 [INFO] Test info log
2015-03-27 17:35:48 [WARNING] Test warning