This is the log4j properties file we are using, could you please explain why the limit for number of files is stuck at 13 instead of 50 as defined in "maxIndex" Property?
# Root logger option
log4j.rootLogger=INFO,file,stdout
log4j.logger.com.cisco.assurance=DEBUG, file
# Redirect log messages to console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.Threshold=DEBUG
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd'T'HH:mm:ss.SZ} %-5p %m%n
# Redirect log messages to a log file
log4j.appender.file=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.file.rollingPolicy=org.apache.log4j.rolling.FixedWindowRollingPolicy
log4j.appender.file.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.file.rollingPolicy.FileNamePattern=/var/log/smeu/AssuranceMS-%i.log.gz
log4j.appender.file.rollingPolicy.ActiveFileName=/var/log/smeu/AssuranceMS.log
log4j.appender.file.triggeringPolicy.MaxFileSize=204800000
log4j.appender.file.rollingPolicy.maxIndex=50
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d [%t] %-5p (%F:%L) - %m%n
log4j.appender.file.Threshold=DEBUG
From the API of org.apache.log4j.rolling.FixedWindowRollingPolicy, it is reduced by design to 12 backups plus the current one, so 13 maximum.
You can look at it : https://logging.apache.org/log4j/extras/apidocs/org/apache/log4j/rolling/FixedWindowRollingPolicy.html
This is happening because large window size is discouraged and the maxIndex is automatically reduced to 12. It is well documented here.
Given that this rollover algorithm requires as many file renaming operations as the window size, large window sizes are discouraged. The current implementation will automatically reduce the window size to 12 when larger values are specified by the user.
Trying to use the following pattern layout to print encoded log messages, so that \n will not be printed as a new line in the logs.
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{MM-dd-yyyy HH:mm:ss,SSS} %5p [%t] (%F:%L) - %enc{%m}%n"/>
</layout>
But it literally prints %enc{ before every message and prints } at the end of the messages, instead of encoding them. \n is also not being replaced.
Example: For this code logger.info("my \n message") following is the output log
08-01-2018 01:26:37,642 INFO [localhost-startStop-1] (ClassName1.java:53) - %enc{my
message}
instead it should be like,
08-01-2018 01:26:37,642 INFO [localhost-startStop-1] (ClassName1.java:53) - my message
Followed this document https://logging.apache.org/log4j/log4j-2.1/manual/layouts.html But I'm not sure what is wrong with the pattern.
You should determine the encoding format in curly brackets. For example %enc{%m}{HTML} encodes the message in HTML format. You could also use CRLF: enc{%m}{CRLF}
I supply a formatted java string with significant whitespace
String.format("%1$12s %2$15s", "someString", "anotherOne");
to be logged using the following conversion pattern in log4j
<param name="ConversionPattern" value="%d{ISO8601} %-5p [%t] %c{1} %m%n" />
In the logs, I receive the variable with just a single preceding whitespace.
... PatternStuff ... someString anotherOne
slf4j version 1.7.3
log4j version 1.2.14
Is it possible to show the whitespace added by String.format? Some other conversion pattern variable other than %m?
My workplace has an internal web-based logviewer that quickly gives developers a way to access the contents of log files. We have the ability to print logs out in a generated webpage or to download the log. When I view them in the browser, the spaces are collapsed (because of standard HTML output). However, when I download the log files, they appear with the given formatting! Sorry for the confusion.
I think I run into a problem using log4j. I'm trying to enable bufferedio, but the default buffer size of 8KB is too big for my current needs
<appender name="MyAppender" class="org.apache.log4j.FileAppender">
<param name="bufferedIO" value="true"/>
<param name="bufferSize" value="512"/>
<param name="Append" value="true"/>
<param name="File" value="C:/MyMonitor.log"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{dd/MM/yyyy HH:mm:ss}|%m%n"/>
</layout>
</appender>
<logger name="com.mypackage.MyMonitor">
<level value="debug"/>
<appender-ref ref="MyAppender"/>
</logger>
I've tried different sizes for the buffer but the writing to the file only happens when the buffer reaches 8KB.
I cannot find any bug in Log4J describing this problem, even when I could find another guy who had the same problem (without any solution).
Enabling the Log4J debug shows that my appender has the proper buffersize, so I think that no one is overwriting my configuration.
Any idea of what am I doing wrong? Is it really a bug or limitation in Log4J?
Thanks.
I've noticed, that you wrote bufferedIO starting from lowercase and BufferSize starting from capital. Java is case-sensitive by default, so please make all your parameters look like in the constructor (bufferSize should be lowercased).
Also as I know FileAppender is deprecated. WriterAppender is a closest replacement.
log4j 1.2.17
Dive to the source code, I find DailyRollingFileAppender use java.io.OutputStreamWriter to write characters to log file. Threre is a buffer in OutputStreamWriter. The size is 8k(8192) in jdk 1.7
Because of this second cache, you will find : If you set log4j.appender.file.bufferSize=8192, even the log data is more than 8192 ( <=8192*2 ), there is no data in log file. The reason is the second cache(OutputStreamWriter) hold the first 8192 chars and DailyRollingFileAppender cache hold the remaining.
JDK API java.io.OutputStreamWriter:
Each invocation of a write() method causes the encoding converter to be invoked on the given character(s). The resulting bytes are accumulated in a buffer before being written to the underlying output stream. The size of this buffer may be specified, but by default it is large enough for most purposes. Note that the characters passed to the write() methods are not buffered.
I have a file (more specifically, a log4j configuration file) and I want to be able to read in the file and pick out certain lines in the code and replace them. For example, within the file there is a string of text that indicates the directory it is stored in, or the level of the logger. I want to be able to replace those string of text without reading in the file, writing it to another file, and deleting the original file. Is there a more efficient way of doing find and replace texts in a file using Java?
Here is an example of the text file I'm trying to work with:
log4j.rootLogger=DEBUG, A0
log4j.appender.A0=org.apache.log4j.RollingFileAppender
log4j.appender.A0.File=C:/log.txt
log4j.appender.A0.MaxFileSize=100KB
log4j.appender.A0.MaxBackupIndex=1
log4j.appender.A0.layout=org.apache.log4j.RollingFileAppender
log4j.appender.A0.layout.ConversionPattern=%-4r [%t] %-5p: %c %x - %m%n
I want to be able to read the file and replace 'DEBUG' with another level or replace the file directory name 'C:/log.txt'. The log configuration file is also written in xml. An example of that is featured below.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration>
<appender class="org.apache.log4j.RollingFileAppender" name="A0">
<param name="append" value="false"/>
<param name="File" value="C:/log/.txt"/>
<param name="MaxBackupIndex" value="1"/>
<param name="MaxFileSize" value="100KB"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-4r [%t] %-5p: %c %x - %m%n"/>
</layout>
</appender>
<root>
<level value="DEBUG"/>
<appender-ref ref="A0"/>
</root>
</log4j:configuration>
I'm thinking it may be possible to use a hash map for this type of implementation?
Any decent text editor has a search&replace facility that supports regular expressions.
If however, you have reason to reinvent the wheel in Java, you can do:
Path path = Paths.get("test.txt");
Charset charset = StandardCharsets.UTF_8;
String content = new String(Files.readAllBytes(path), charset);
content = content.replaceAll("foo", "bar");
Files.write(path, content.getBytes(charset));
This only works for Java 7 or newer. If you are stuck on an older Java, you can do:
String content = IOUtils.toString(new FileInputStream(myfile), myencoding);
content = content.replaceAll(myPattern, myReplacement);
IOUtils.write(content, new FileOutputStream(myfile), myencoding);
In this case, you'll need to add error handling and close the streams after you are done with them.
IOUtils is documented at http://commons.apache.org/proper/commons-io/javadocs/api-release/org/apache/commons/io/IOUtils.html
After visiting this question and noting the initial concerns of the chosen solution, I figured I'd contribute this one for those not using Java 7 which uses FileUtils instead of IOUtils from Apache Commons. The advantage here is that the readFileToString and the writeStringToFile handle the issue of closing the files for you automatically. (writeStringToFile doesn't document it but you can read the source). Hopefully this recipe simplifies things for anyone new coming to this problem.
try {
String content = FileUtils.readFileToString(new File("InputFile"), "UTF-8");
content = content.replaceAll("toReplace", "replacementString");
File tempFile = new File("OutputFile");
FileUtils.writeStringToFile(tempFile, content, "UTF-8");
} catch (IOException e) {
//Simple exception handling, replace with what's necessary for your use case!
throw new RuntimeException("Generating file failed", e);
}
public static void replaceFileString(String old, String new) throws IOException {
String fileName = Settings.getValue("fileDirectory");
FileInputStream fis = new FileInputStream(fileName);
String content = IOUtils.toString(fis, Charset.defaultCharset());
content = content.replaceAll(old, new);
FileOutputStream fos = new FileOutputStream(fileName);
IOUtils.write(content, new FileOutputStream(fileName), Charset.defaultCharset());
fis.close();
fos.close();
}
above is my implementation of Meriton's example that works for me. The fileName is the directory (ie. D:\utilities\settings.txt). I'm not sure what character set should be used, but I ran this code on a Windows XP machine just now and it did the trick without doing that temporary file creation and renaming stuff.
You might want to use Scanner to parse through and find the specific sections you want to modify. There's also Split and StringTokenizer that may work, but at the level you're working at Scanner might be what's needed.
Here's some additional info on what the difference is between them:
Scanner vs. StringTokenizer vs. String.Split
This is the sort of thing I'd normally use a scripting language for. It's very useful to have the ability to perform these sorts of transformations very simply using something like Ruby/Perl/Python (insert your favorite scripting language here).
I wouldn't normally use Java for this since it's too heavyweight in terms of development cycle/typing etc.
Note that if you want to be particular in manipulating XML, it's advisable to read the file as XML and manipulate it as such (the above scripting languages have very useful and simple APIs for doing this sort of work). A simple text search/replace can invalidate your file in terms of character encoding etc. As always, it depends on the complexity of your search/replace requirements.
You can use Java's Scanner class to parse words of a file and process them in your application, and then use a BufferedWriter or FileWriter to write back to the file, applying the changes.
I think there is a more efficient way of getting the iterator's position of the scanner at some point, in order to better implement editting. But since files are either open for reading, or writing, I'm not sure regarding that.
In any case, you can use libraries already available for parsing of XML files, which have all of this implemented already and will allow you to do what you want easily.