I wrote a simple Custom Formatter for my logging that prints a DateTime in the specified format. Everything works fine, but the datetime doesn't get printed in my log file. Below is my CustomFormatter.java:
CustomFormatter.java:
public class CustomFormatter extends Formatter {
SimpleDateFormat sdf;
public CustomFormatter() {
sdf = new SimpleDateFormat("MM-dd-yyyy HH:mm:ss");
}
public String format(LogRecord rec) {
StringBuffer buf = new StringBuffer(1000);
buf.append(formatMessage(rec));
return buf.toString();
}
public String getHead(Handler h) {
return (sdf.format(new java.util.Date()) + ": \t");
}
public String getTail(Handler h) {
return "\n";
}
}
Im my main class, I initialize my logger as:
Main.java:
private static Logger logger = Logger.getLogger("org.somecompany.someproject");
public static void main(String[] args){
try{
String _pattern = "myLogger.log.%g";
int numLogFiles = 10;
int fileSize = 1000000;
boolean appendToFile = true;
FileHandler fh = new FileHandler(pattern, fileSize, numLogFiles, appendToFile);
fh.setFormatter(new CustomFormatter());
logger.addHandler(fh);
logger.setLevel(Level.ALL);
} catch(IOException i) { System.out.println("Unable to init logger");}
logger.info("Begin");
logger.info("Line 1");
logger.info("Line 2");
logger.info("End");
fh.close();
fh = null;
}
The log file should have a datetime printed at the beginning of each line and it isn't. Any help is appreciated.
I think you misunderstand the purpose of getHead() and getTail().
Those methods are not called per log entry - getHead() is called once before the Handler (FileHandler) logs the first entry and getTail() is called once before the Handler (FileHandler) closes the log stream.
This is used for example in the XMLFormatter to create a valid XML file (where there must be an XML start tag before the first log entry and an XML end tag after the last log entry to be valid XML).
If you look closely at the created log file this is exactly what happens: the log file creates a timestamp at the start and ends with a newline.
Note that adding time stamps to your log entries doesn't require writing a CustomFormatter. Properly configuring a SimpleFormatter is enough for your purpose.
What is the best way of writing a unit test for a method, such as my setProperties (see below), that uses a private configuration variable (config). I tried but failed to override it using reflection and Makito, but without success. I realize that changing the design to make the code easier to test is best, but I want to created some unit tests before I refactor the code.
public class MainClass {
private final java.lang.String config = "app.properties";
public TestClass() {
try {
setProperties();
} catch (Exception e) {
e.printStackTrace();
}
}
public void setProperties() throws Exception {
try {
InputStream input = new BufferedInputStream(new FileInputStream(config));
..
..
} catch (Exception exception) {
throw exception;
}
}
}
Do refactor a tiny bit by extracting a method with a parameter that takes an input stream. Call this new method (probably package-protected) from the old one. Write tests against the new method. Then do more refactorings.
This is an indication of a broken design; don't hard-code things like this. Better yet, determine what the appropriate responsibility for this class is, and, in decreasing order of preference:
pass in an object with the configuration properties, strongly typed
pass in a Map with the configuration properties
pass in an InputStream for the properties file
As File objects are never available from a jar, you shouldn't ever make interfaces like this more specific than InputStream or Reader, so that you can always pass in streams from your jar classpath.
So you can use Properties class in Java for this. Please have a look at this code.
public class PropertyUtil {
private static Properties prop;
private static Logger logger = Logger.getLogger(PropertyUtil.class);
private PropertyUtil() {
}
public void setProperty() {
String filePath = System.getenv("JAVA_HOME") + "/lib" + "/my_file.properties";
prop = new Properties();
try (InputStream input = new FileInputStream(filePath)) {
prop.load(input);
} catch (IOException ex) {
logger.error("Error while reading property file " + ex);
}
}
public static String getProperty(String key) {
if (prop.containsKey(key)) {
return prop.getProperty(key);
} else {
return null;
}
}
public static <T> T getProperty(String key, Class<T> claz) {
if (claz.getName().equals(Integer.class.getName())) {
return claz.cast(Integer.parseInt(prop.getProperty(key)));
}
if (claz.getName().equals(Long.class.getName())) {
return claz.cast(Long.parseLong(prop.getProperty(key)));
}
if (claz.getName().equals(Boolean.class.getName())) {
return claz.cast(Boolean.parseBoolean(prop.getProperty(key)));
}
if (claz.getName().equals(Double.class.getName())) {
return claz.cast(Double.parseDouble(prop.getProperty(key)));
}
if (claz.getName().equals(String.class.getName())) {
return claz.cast(prop.getProperty(key));
}
return null;
}
How to Log the jar file location in log file.
Say I am printing some message from LogTest.java which is in myLog.jar , is there any way to print in log file as below.
INFO 2013-10-30 15:58:07,227 [log] (LogTest.java:235:myLog.jar) - Some Log Message
Various jar in a legacy project contain similar kind of log message with the same class name,
It is taking time to investigate any issue.
Is there any way to handle this please give your suggestion.
There is no an option to include the source file with the standar org.apache.log4j.PatternLayout. However, you can create your own layout to include it the source. The following may be of use.
public class MyLayout extends org.apache.log4j.Layout {
public void activateOptions() {
// None action
}
public String format(LoggingEvent event) {
StringBuilder sb = new StringBuilder();
sb.append(event.getLevel());
sb.append(" (");
sb.append(event.getLoggerName());
sb.append(":");
sb.append(event.getLocationInformation().getLineNumber());
sb.append(":");
sb.append(getSource(event.getLoggerName()));
sb.append(") - ");
sb.append(event.getMessage());
sb.append(LINE_SEP);
return sb.toString();
}
public boolean ignoresThrowable() {
return true;
}
public static String getSource(String className) {
try {
ClassLoader loader = MyLayout.class.getClassLoader();
String name = className.replace('.', '/').concat(".class");
URL url = loader.getResource(name);
return url.getPath();
} catch (Exception e) {
return null;
}
}
}
When I'm trying to register a file instead of a directory java.nio.file.NotDirectoryException is thrown. Can I listen for a single file change, not the whole directory?
Just filter the events for the file you want in the directory:
final Path path = FileSystems.getDefault().getPath(System.getProperty("user.home"), "Desktop");
System.out.println(path);
try (final WatchService watchService = FileSystems.getDefault().newWatchService()) {
final WatchKey watchKey = path.register(watchService, StandardWatchEventKinds.ENTRY_MODIFY);
while (true) {
final WatchKey wk = watchService.take();
for (WatchEvent<?> event : wk.pollEvents()) {
//we only register "ENTRY_MODIFY" so the context is always a Path.
final Path changed = (Path) event.context();
System.out.println(changed);
if (changed.endsWith("myFile.txt")) {
System.out.println("My file has changed");
}
}
// reset the key
boolean valid = wk.reset();
if (!valid) {
System.out.println("Key has been unregisterede");
}
}
}
Here we check whether the changed file is "myFile.txt", if it is then do whatever.
Other answers are right that you must watch a directory and filter for your particular file. However, you probably want a thread running in the background. The accepted answer can block indefinitely on watchService.take(); and doesn't close the WatchService. A solution suitable for a separate thread might look like:
public class FileWatcher extends Thread {
private final File file;
private AtomicBoolean stop = new AtomicBoolean(false);
public FileWatcher(File file) {
this.file = file;
}
public boolean isStopped() { return stop.get(); }
public void stopThread() { stop.set(true); }
public void doOnChange() {
// Do whatever action you want here
}
#Override
public void run() {
try (WatchService watcher = FileSystems.getDefault().newWatchService()) {
Path path = file.toPath().getParent();
path.register(watcher, StandardWatchEventKinds.ENTRY_MODIFY);
while (!isStopped()) {
WatchKey key;
try { key = watcher.poll(25, TimeUnit.MILLISECONDS); }
catch (InterruptedException e) { return; }
if (key == null) { Thread.yield(); continue; }
for (WatchEvent<?> event : key.pollEvents()) {
WatchEvent.Kind<?> kind = event.kind();
#SuppressWarnings("unchecked")
WatchEvent<Path> ev = (WatchEvent<Path>) event;
Path filename = ev.context();
if (kind == StandardWatchEventKinds.OVERFLOW) {
Thread.yield();
continue;
} else if (kind == java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY
&& filename.toString().equals(file.getName())) {
doOnChange();
}
boolean valid = key.reset();
if (!valid) { break; }
}
Thread.yield();
}
} catch (Throwable e) {
// Log or rethrow the error
}
}
}
I tried working from the accepted answer and this article. You should be able to use this thread with new FileWatcher(new File("/home/me/myfile")).start() and stop it by calling stopThread() on the thread.
No it isn't possible to register a file, the watch service doesn't work this way. But registering a directory actually watches changes on the directory children (the files and sub-directories), not the changes on the directory itself.
If you want to watch a file, then you register the containing directory with the watch service. Path.register() documentation says:
WatchKey java.nio.file.Path.register(WatchService watcher, Kind[] events, Modifier...
modifiers) throws IOException
Registers the file located by this path with a watch service.
In this release, this path locates a directory that exists. The directory is registered with the watch service so that entries in the directory can be watched
Then you need to process events on entries, and detect those related to the file you are interested in, by checking the context value of the event. The context value represents the name of the entry (actually the path of the entry relatively to the path of its parent, which is exactly the child name). You have an example here.
Apache offers a FileWatchDog class with a doOnChange method.
private class SomeWatchFile extends FileWatchdog {
protected SomeWatchFile(String filename) {
super(filename);
}
#Override
protected void doOnChange() {
fileChanged= true;
}
}
And where ever you want you can start this thread:
SomeWatchFile someWatchFile = new SomeWatchFile (path);
someWatchFile.start();
The FileWatchDog class polls a file's lastModified() timestamp. The native WatchService from Java NIO is more efficient, since notifications are immediate.
You cannot watch an individual file directly but you can filter out what you don't need.
Here is my FileWatcher class implementation:
import java.io.File;
import java.nio.file.*;
import java.nio.file.WatchEvent.Kind;
import static java.nio.file.StandardWatchEventKinds.*;
public abstract class FileWatcher
{
private Path folderPath;
private String watchFile;
public FileWatcher(String watchFile)
{
Path filePath = Paths.get(watchFile);
boolean isRegularFile = Files.isRegularFile(filePath);
if (!isRegularFile)
{
// Do not allow this to be a folder since we want to watch files
throw new IllegalArgumentException(watchFile + " is not a regular file");
}
// This is always a folder
folderPath = filePath.getParent();
// Keep this relative to the watched folder
this.watchFile = watchFile.replace(folderPath.toString() + File.separator, "");
}
public void watchFile() throws Exception
{
// We obtain the file system of the Path
FileSystem fileSystem = folderPath.getFileSystem();
// We create the new WatchService using the try-with-resources block
try (WatchService service = fileSystem.newWatchService())
{
// We watch for modification events
folderPath.register(service, ENTRY_MODIFY);
// Start the infinite polling loop
while (true)
{
// Wait for the next event
WatchKey watchKey = service.take();
for (WatchEvent<?> watchEvent : watchKey.pollEvents())
{
// Get the type of the event
Kind<?> kind = watchEvent.kind();
if (kind == ENTRY_MODIFY)
{
Path watchEventPath = (Path) watchEvent.context();
// Call this if the right file is involved
if (watchEventPath.toString().equals(watchFile))
{
onModified();
}
}
}
if (!watchKey.reset())
{
// Exit if no longer valid
break;
}
}
}
}
public abstract void onModified();
}
To use this, you just have to extend and implement the onModified() method like so:
import java.io.File;
public class MyFileWatcher extends FileWatcher
{
public MyFileWatcher(String watchFile)
{
super(watchFile);
}
#Override
public void onModified()
{
System.out.println("Modified!");
}
}
Finally, start watching the file:
String watchFile = System.getProperty("user.home") + File.separator + "Desktop" + File.separator + "Test.txt";
FileWatcher fileWatcher = new MyFileWatcher(watchFile);
fileWatcher.watchFile();
Not sure about others, but I groan at the amount of code needed to watch a single file for changes using the basic WatchService API. It has to be simpler!
Here are a couple of alternatives using third party libraries:
Using Apache Commons Configuration
Using spring-loaded package from the Spring Framework (didn't find an example implementation for this off-hand, but it looks straight-forward to use)
I have created a wrapper around Java 1.7's WatchService that allows registering a directory and any number of glob patterns. This class will take care of the filtering and only emit events you are interested in.
try {
DirectoryWatchService watchService = new SimpleDirectoryWatchService(); // May throw
watchService.register( // May throw
new DirectoryWatchService.OnFileChangeListener() {
#Override
public void onFileCreate(String filePath) {
// File created
}
#Override
public void onFileModify(String filePath) {
// File modified
}
#Override
public void onFileDelete(String filePath) {
// File deleted
}
},
<directory>, // Directory to watch
<file-glob-pattern-1>, // E.g. "*.log"
<file-glob-pattern-2>, // E.g. "input-?.txt"
<file-glob-pattern-3>, // E.g. "config.ini"
... // As many patterns as you like
);
watchService.start(); // The actual watcher runs on a new thread
} catch (IOException e) {
LOGGER.error("Unable to register file change listener for " + fileName);
}
Complete code is in this repo.
I extended the solution by BullyWiiPlaza a bit, for integration with javafx.concurrent, e.g. javafx.concurrent.Taskand javafx.concurrent.Service.
Also I added possibility to track multiple files.
Task:
import javafx.concurrent.Task;
import lombok.extern.slf4j.Slf4j;
import java.io.File;
import java.nio.file.*;
import java.util.*;
import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
#Slf4j
public abstract class FileWatcherTask extends Task<Void> {
static class Entry {
private final Path folderPath;
private final String watchFile;
Entry(Path folderPath, String watchFile) {
this.folderPath = folderPath;
this.watchFile = watchFile;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Entry entry = (Entry) o;
return Objects.equals(folderPath, entry.folderPath) && Objects.equals(watchFile, entry.watchFile);
}
#Override
public int hashCode() {
return Objects.hash(folderPath, watchFile);
}
}
private final List<Entry> entryList;
private final Map<WatchKey, Entry> watchKeyEntryMap;
public FileWatcherTask(Iterable<String> watchFiles) {
this.entryList = new ArrayList<>();
this.watchKeyEntryMap = new LinkedHashMap<>();
for (String watchFile : watchFiles) {
Path filePath = Paths.get(watchFile);
boolean isRegularFile = Files.isRegularFile(filePath);
if (!isRegularFile) {
// Do not allow this to be a folder since we want to watch files
throw new IllegalArgumentException(watchFile + " is not a regular file");
}
// This is always a folder
Path folderPath = filePath.getParent();
// Keep this relative to the watched folder
watchFile = watchFile.replace(folderPath.toString() + File.separator, "");
Entry entry = new Entry(folderPath, watchFile);
entryList.add(entry);
log.debug("Watcher initialized for {} entries. ({})", entryList.size(), entryList.stream().map(e -> e.watchFile + "-" + e.folderPath).findFirst().orElse("<>"));
}
}
public FileWatcherTask(String... watchFiles) {
this(Arrays.asList(watchFiles));
}
public void watchFile() throws Exception {
// We obtain the file system of the Path
// FileSystem fileSystem = folderPath.getFileSystem();
// TODO: use the actual file system instead of default
FileSystem fileSystem = FileSystems.getDefault();
// We create the new WatchService using the try-with-resources block
try (WatchService service = fileSystem.newWatchService()) {
log.debug("Watching filesystem {}", fileSystem);
for (Entry e : entryList) {
// We watch for modification events
WatchKey key = e.folderPath.register(service, ENTRY_MODIFY);
watchKeyEntryMap.put(key, e);
}
// Start the infinite polling loop
while (true) {
// Wait for the next event
WatchKey watchKey = service.take();
for (Entry e : entryList) {
// Call this if the right file is involved
var hans = watchKeyEntryMap.get(watchKey);
if (hans != null) {
for (WatchEvent<?> watchEvent : watchKey.pollEvents()) {
// Get the type of the event
WatchEvent.Kind<?> kind = watchEvent.kind();
if (kind == ENTRY_MODIFY) {
Path watchEventPath = (Path) watchEvent.context();
onModified(e.watchFile);
}
if (!watchKey.reset()) {
// Exit if no longer valid
log.debug("Watch key {} was reset", watchKey);
break;
}
}
}
}
}
}
}
#Override
protected Void call() throws Exception {
watchFile();
return null;
}
public abstract void onModified(String watchFile);
}
Service:
public abstract class FileWatcherService extends Service<Void> {
private final Iterable<String> files;
public FileWatcherService(Iterable<String> files) {
this.files = files;
}
#Override
protected Task<Void> createTask() {
return new FileWatcherTask(files) {
#Override
public void onModified(String watchFile) {
FileWatcherService.this.onModified(watchFile);
}
};
}
abstract void onModified(String watchFile);
}
I want write several tests, but from a high level each of them should populate a directory structure with some files. I'd test each of these cases at least:
A single folder with a file that passes the filter.
A single folder with a file that does NOT pass the filter.
A nested folder with a file in each.
Code:
class FolderScan implements Runnable {
private String path;
private BlockingQueue<File> queue;
private CountDownLatch latch;
private File endOfWorkFile;
private List<Checker> checkers;
FolderScan(String path, BlockingQueue<File> queue, CountDownLatch latch,
File endOfWorkFile) {
this.path = path;
this.queue = queue;
this.latch = latch;
this.endOfWorkFile = endOfWorkFile;
checkers = new ArrayList<Checker>(Arrays.asList(new ExtentionsCheker(),
new ProbeContentTypeCheker(), new CharsetDetector()));
}
public FolderScan() {
}
#Override
public void run() {
findFiles(path);
queue.add(endOfWorkFile);
latch.countDown();
}
private void findFiles(String path) {
boolean checksPassed = true;
File root;
try {
root = new File(path);
File[] list = root.listFiles();
for (File currentFile : list) {
if (currentFile.isDirectory()) {
findFiles(currentFile.getAbsolutePath());
} else {
for (Checker currentChecker : checkers) {
if (!currentChecker.check(currentFile)) {
checksPassed = false;
break;
}
}
if (checksPassed)
queue.put(currentFile);
}
}
} catch (InterruptedException | RuntimeException e) {
System.out.println("Wrong input !!!");
e.printStackTrace();
}
}
}
Questions:
How to create files into each folder?
To prove that queue contains
the File objects that you expect?
The last element in queue is the
'trigger' File?
How to create files into each folder?
Extract the file IO and use a mocked repository for the tests. This means that you will have the IO somewhere else and may wish to use the below to test that.
A temp folder using the JUnit rule With a test folder you create the files to match the test.
To prove that queue contains the File objects that you expect?
.equals works well for File objects I believe.
A single folder with a file that does NOT pass the filter.
I'd pass in the blockers so I can pass in an "Always Pass" and "Always Fail" blocker.
public class TestFolderScan {
#Rule
public TemporaryFolder folder= new TemporaryFolder();
#Test
public void whenASingleFolderWithAFileThatPassesTheFilterThenItExistsInTheQueue() {
File expectedFile = folder.newFile("file.txt");
File endOfWorkFile = new File("EOW");
Queue queue = ...;
FolderScan subject = new FolderScan(folder.getRoot(), queue, new AllwaysPassesBlocker(),...);
subject.run();
expected = new Queue(expectedFile, endOfWorkFile);
assertEquals(queue, expected);
}
}