I was just curious to know the difference between normal file write and logging. Of course logging is used to record exceptions, errors, installation details and other important data. But this can be done using a normal file write also. I've seen logging use locks for resource sharing (in java). Other than that is there any particular or very important reason behind using logging?
Logging is writing data to some stream to keep a record of events that occur in an application. Note that you don't necessarily have to log to a file. You can log to a console, for example.
Some applications require an "Audit Log" of user activity in the system. This is a case where logging is fulfilling a very specific business requirement.
Note you can write to a file and NOT be logging. If you use the presence of a file to create a lock for a process, for example, you have written to a file, but you are not logging.
In general though, logging is just writing event data somewhere. "started up", "entered method x", "exception occurred" are all events. I think that really what defines a "log" vs a file with different semantics.
Writing to a file is one possibility of doing logging. Logging is a more general term for something like "save important events for later use". If you look at logging frameworks, you see that they allow you to write to a file as one option. But they provide you with more configuration options like logging levels, different logging sinks etc. One could of course implement this on its own by writing certain information to file.
Logging means appending to a file. With write you can override previous data, by appending you can't. It's just my way of thinking.
Related
All,
I am developing a feature that will on execution of an operation write the logs to a file server using ftp. Note that the write to file server will happen only if a file server is configured. If no server is configured the operation will exit and return the status. The flow is something like this:
1. Execute operation
2. If file server connected (check in DB and ping), write logs
3. return
Now I would like to know if there are design patterns for this, same feature, however the scope of the feature will vary depending on whether or not some configuration is done. I would much appreciate help on this for 2 scenarios:
Static - If the DB config is one time during boot up - as in post bootup the system can "assume" that the file server is there or not based on the read from DB
Dynamic- When the system is up and running, I might bring up a file server and configure DB. Ideally for this scenario the system should detect the file server and start writing logs to it, rather than being forced to reboot the system.
Requesting help in this regard.
Thanks
Your design looks like a breach of the Single Responsibility Principle. You are entangling two different concerns: the first concern is the operation itself, and the second is shipping the logs to a central location.
Think about separating your component into two simpler, independent components. One of them performs the business operation and writes logs, say to a local file, and that's it. The other component checks for the existence of new logs on the local file system and copies them to the central location.
You didn't mention whether or not you are using an existing logging framework, such as Log4J. If not, it would probably be a good idea - if you try to roll out your own logging framework, you can end up having to deal with additional unforeseen complexities, such as dealing with log levels (INFO, DEBUG, ERROR, etc.).
With regards to your original message - I'd consider using the Factory pattern - create a factory class that can internally check whether the file server is available, and return one of two different logger types - something like a ConsoleLogger and an FTPLogger. Make sure that both of these implement the same interface so that your calling code doesn't have to care about what type of logger it's using. Alternatively, you can also use a Decorator that can wrap the object performing your operation - and once it completes a request, have the decorator do the logging.
A final comment - try to avoid checking whether the file server is available every time that you log. A database hit on every log call could result in horrible performance, not to mention that you'll have to ensure that errors in the logging method (such as DB locks) don't result in your entire operation failing.
I am trying to use Log4J in my application, but my problem is that the machine where I want to run this app has not given me write permissions to the local drive. But Log4J is not throwing any errors, it simply skips the logging.
So what I want to do is write some code in which if Log4J cannot write to the local file system it will return some feedback/message in my app. Is there a way to tell if Log4J can write to the local filesystem, from Log4J itself? Or is there any way that I determine how many words Log4J has written into its log file in each operation? It's so then I could determine whether or not logging was actually occurring and then take appropriate action.
have you read the FAQ
Quote from the site
No. log4j is not reliable. It is a best-effort fail-stop logging
system.
By fail-stop, we mean that log4j will not throw unexpected exceptions
at run-time potentially causing your application to crash. If for any
reason, log4j throws an uncaught exception, please send an email to
the log4j-user#logging.apache.org mailing list. Uncaught exceptions
are handled as serious bugs requiring immediate attention.
I prefer my application continue even if logging fails.
What you described is possibly a function of 'application monitoring'. There are many tools to see if particular file system is getting full or a directory/file not changed for a while etc.
Having said that, you can do basic checks at the very beginning of application - like permissions to create file in the directory meant for logging.
I was just wondering if following thing exists.
I have a TCP communicator which keeps communicating with thousands of devices.
Currently, the TCP communicator logs all the events in one log file.
Now, is it possible to log communication with every device in different files. The IMEI number of every device is different. So the logger will check if a file with name equal to the IMEI number of the device exists. If the file exists, logger will start logging events of the device in that file, otherwise it will create a new file with IMEI as the file's name and start logging the events in that file.
(We are developing our application in Java.)
LogBack is the future, and it's here!
Created as a successor of log4j and fully complaint with the slf4j framework, logback might be the easy (and clean) way to fulfill your need.
I'm not an expert but I guess that SiftingAppender might be the right answer. There should be a discriminator option for you. Maybe you can build your own discriminator, extend the SiftingAppender, or get some extra help from Janino library.
Hope this helps!
If you are implementing the logger yourself, there's nothing to stop you from doing this.
For example, give the log function as a parameter the number of the device you're currently communicating with, and implement it the way you described.
If you're using Apache log4j, which I highly recommend, create a custom logging appender by extending AppenderSkeleton and writing unique files for individual connections will be as simple as doing standard file I/O with a variable filename.
If you are using java.util.logging, look at the Handler base class, if you are using log4j, look at Appender. In both cases, you need to somehow get the IMEI associated with the message, so the code writing the log message can pick the appropriate file.
There are two approaches to doing this.
First is to extend the log event class (LogRecord or LoggingEvent respectively). This would allow you to log using your event, which contains the IMEI. However, this does not account for logging performed by other libraries etc while performing the conversation with a device.
The other alternative is to use a ThreadLocal. Set the IMEI associated with a socket whenever you receive a message or are formulating message. Make sure that the logging happens in the same thread, and any queuing is done at the log handler/appender. Look for / ask questions about ThreadLocals if you are unfamiliar with this approach. I believe that Log4J's NDC and MDC implements this sort of strategy, but I've not tried to do specialized processing of context at the appender.
Finally, be aware some operating systems will run out of file handles if you are indeed thinking of keeping "thousands" of log files open. Depending on just how many files, you may want to consider writing log messages (with IMEI) to a database, or doing some sort of LRU-based file closing. In the latter, you would basically not have file handles for log files that haven't been touched in a while.
I have a Java program which runs as 3 separate processes on the same server. I would like all of the processes to share a single log file, is there a way to specify that in a logging.properties file? I am using java.util.logging to handle logging.
Currently, this is how I define my FileHandler in my logging.properties file:
java.util.logging.FileHandler.pattern=%h/log/logfile.log
This works fine for 1 instance of the program, however, if I attempt to start 3 separate instances of the program the result is:
logfile.log
logfile.log.1
logfile.log.2
Any advice on this?
Thankyou
Logback is another logger, but it supports your case.
from the docs: http://logback.qos.ch/manual/appenders.html
check out prudent mode for FileAppender
Writing to the same file from different processes (the different JVMs) is not recommended.
The only safe way to do it is to somehow lock the file, open it, write to it and then close it. This considerably slows down each writing, which is generally deemed unacceptable for a logger. If you really want to go this way, you can always write your own handler.
I would write a 2nd Java program - a logger. Have the other processes send log messages to the logging program, which would then write to the single log file. You can communicate between the programs using sockets. Here's an example of how to do that.
Paul
Elaborating on Paul's answer, you can use a SocketHandler to direct the log events from all processes to a single process, which actually writes to a file.
Most log packages provide a simple implementation of this functionality. Another widely supported option is integration with the system's logging facility (Window's Event Log or syslogd).
I want to log information to a file and want to know is there any advantage in using the Java Logger class versus my own personal method of logging?
I know using a logger I just add my own FileHandler to the logger. My personal method is just using a FileOutputStream.
Honestly your logger may be as good, it's pretty easy to put in log levels, filtering, etc.
The problem is when you start to put together code from 10 different groups each with their own logging. Some use System.out, some use some custom logger, some use log4j, ...
How do you parse or redirect the output? How do you filter all that output to just show your messages (For instance, filtering options in log4j won't prevent messages being sent straight to System.out).
It's a pretty big bulk change depending on which system you are moving from/to. Better just to use one very common one from the beginning and hope that it's the one everybody else on the project chooses.
The real question is: why would you bother writing your own logger when there are already existing libraries for doing that? Does your logger do something that the others don't? I kind of doubt it.
Standardization is another big issue - see Bill K's answer.
For most scenarios, a standard logging framework is the way to go. They are pretty flexible. But using your own implementation can also be a valid option, specially if we are not speaking of traditional logging (global debugging, problems, warning messages) but about specific informational meesages or accounting.
Among other factors, bear in mind that the standarization of logging allows third party libraries to cooperate. For example, if you have a standard web application using (say) Hibernate, and you have configured a standard Java logging lib, then you can not only log from your own code but also tell Hibernate to log debugging info to your log files (not necessarily the same files). That is very useful - almost a must.
If you code your own logging library with a plain FileOutputStream, you must decide -among other things- if you are going to keep the file opened, or reopen-append-close in each write - and you must take of synchronization and related issues. Nothing terribly complicated, though.
The logger gives to ability to define different levels of importance of the logged messages and the ability to use different sink for the output - the console, a file, etc.
Also it's easy to enable or disable only some type of message when using a logger - for example you don't want to see every debug message in production.
A logging framework allows you specify logging levels (e.g. log only critical messages, log only debug messages etc.). It also allows you to log to multiple destinations (e.g. to a file, to syslog etc.) and you can do this even after your application is fully built by just changing a config file and not changing any code. It can also format your logs easily depending on various parameters.
There are numerous other advantages since proper logging is a rather involved problem and these packages have been written to solve them. I think the important question, why would you not use them?
Well I would always prefer tested thing and approved by community over something which still need a lot of test. Logger allows you many things which will consume you some time to implement and to test the robustness of your solution. A big plus will be the know-how of the next person who will do something with your code, in case it will be your logger, normally it would take more time to learn how its working out, since there is much more examples and documentation for java.util.logger.
Like all others mention, there are more advantages to using a more common logging system over writing your own. To be honest, the Java util logging api is not nearly as extensive and configurable as other libraries you might find out there on the internet.
Bare in mind that rolling your own always has the disadvantage of being less tested and therefore more prone to break or skip some potentially crucial messages.
Personally I prefer using SLF4J since it allows me to plug in adapters for more commonly used logging frameworks and then let's me pick my own actual logging implementation, this solves most of the problems with different library writers preferring different logging frameworks. And if you consider yourself up for the task you could writer an implementation for SLF4J yourself ;)