I have a sample program which sends http post request and gets a response from server. While in the process of running the program, I am reading data from a file(txt/xml/DB) and executing the http request and the relevant response will be written back to file respectively.
Based on the above functionality, I have planned for a log file which writes exceptions, errors(paths), methods executed, classes executed and a time stamp.
Please tell me any logging programs in java, may be log4j is relevant for this kind of scenario?
log4j is more or less the defacto for logging and allows logging to files, db, jms etc. There are many resources on the net, a simple guide to configure it using a property file and a sample program to initialize it and start logging can be found here: http://www.javabeat.net/baisc-steps-to-configure-log4j-using-xml-and-properties-file/
You can read more about various logging levels and possibilities of logging in multiple files (i.e. Access logs in one, Errors in another etc)
Related
I'm trying to write logs of multiple services to same file, but my rolling policy given is not working, tried with both time based and size based rollings. Thing is my services are running simultanously and writting there logs to same file in my local directory. When tried to write logs by single service it is working as expected.
Please help me to solve this issue tried with different rolling policies.
Appender to log to file
${LOG_FILE}
Minimum logging level to be presented in the console logs
INFO
${LOG_PATH}/archived/log_%d{dd-MM-yyyy}_%i.log
10KB
I had an experience similar to yours with Log4j 1.x then I debugged an appender back then (~5-6 years ago) and came to the following conclusions:
I don't think you can write data from multiple services into the same file. In other words,
Logging framework usually assumes that only it can change the file. In some Operating Systems (windows) it will even stop writing into file if some other process will rename / change the current file.
Of course its just a code and you could create a more sophisticated appeneder that will probable make it work, but frankly I don't think it worth the effort.
So I suggest writing into different files, where file name can be generated in a way that it will contain a pid of the resource. The downside of this method is that if the process dies and then re-runs, on-one will take care of the old resources.
Another approach (somewhat similar) - is to create a folder with logs for each service so that they'll get different logs based on folder (even if files in these folders will be with the same name).
I am running a Java application on Azure Cloud Services.
I have seen this article which shows how to configure a java project to send logs to Azure insights using log4j: https://azure.microsoft.com/en-us/documentation/articles/app-insights-java-trace-logs/
However, for various reasons I do not want to do this. My java application already writes multiple log files in a log directory (application.log, error.log, etc). I want to point to this directory in Azure Insights so that it can aggregate these log files over multiple instances of my application running on Cloud Services and then present them to me. (In a similar way that AWS Cloudwatch presents logs). How can I accomplish this?
I think this is a deep question and would require a bit of custom coding to accomplish it.
The problem as I read it is, you have multiple log files writing to a location and you just want to parse those log files and send the log lines. Moreover, you don't want to add the log appenders to your Java code for various reasons.
The short answer is, no. There isn't a way to have AI monitor a directory of log files and then send them.
The next short answer is, no. AI can't do it out of the box, but Log Analytics can. This is a bit more heavy handed and I haven't read enough about it to say it would fit in this scenario. However, since you're using a cloud service you could more than likely install the agent and start collecting logs.
The next answer is the long answer, kinda. You can do this but it would require a lot of custom coding on your part. What I envision is something akin to how the Azure Diagnostics Sink works.
You would need to create an application that reads the log files and enumerates them line by line, it would then parse them based on some format and then call the TrackTrace() method to log it.
This option requires some considerable thought since you would be reading the file and then determining what to do with it.
I have seen how bugsense, sentry etc work. I like the way you can get the error/crash reports. What I want is a solution like those but for internal use. Using an external api like bugsense is out of the question.
Is there any similar open source solution that can be used internally?
If you are already logging properly, and are requiring a more granular configuration you could give logstash a try. It is basically a logshipper with various input, filter and output modules, including email as an output method.
The input can be configured to parse your existing logfiles, recieve messages from a queue and many more. For UDP/IP based logging, take a look at logstash-gelf which is basically an adapter for automated generation of well formed logging meta data. If you plan on parsing your Logfiles, look out for the "multiline codec" in regards to parsing stacktraces and "grok" as a filter for parsing the log entries. For grok, I found that the Grok debugger is a big help.
E.G.: Once you have your logs routed through the input, and your logging is configured to use a named logger for the emailed messages you can tag them in logstash input and tell the output to send an email if a message with the tag is coming through.
I think using a specific logger (slf4j, log4j...) could be used.
It can send e-mail for example for FATAL log with stack trace and what you want.
I've 2 webapplications, want to log the log messages of these two web apps into only one log file. I tried this scenario, facing the issue as "If one web app logs the message into log, the second web app is not able log the message into log file". If I stop the server, the second app is able to log.Any help?
writing to the same file from multiple independant processes is a bad idea - only one of them can get a file lock, as evident from your issues.
what you need is a centralized logging server and have all of your applications log to that server over the network. see this question
You can use Log4J's SocketAppender for that, which is much cleaner - an example can be found in this article: log4j: How to use SocketAppender.
To be honest it is a bit overkill compared to having two independent logfiles for your processes.
Btw. your approach might have problems on distributed filesystems (e.g. NFS) - don't mix logfiles.
Hope that helped a bit.
*Jost
It is not good idea to put logging of two different applications to common log. Logging module lock the log file for writing. if other application is trying to access the same log, it wont get the lock.
I would avoid such things.
I have made a java application and wants to generate log files so whenever my client would encounter some problem, he can deliver me those log files so that I can correct my code accordingly.
Kindly provide me a small sample program that writes a statement to a log file. Kindly mention the .class files you are using with their full import statements.
The application is multi-threaded so Is it better to generate separate log files for each thread or not?
Is it better to clear all previous log files before starting the program?
macleojw is correct: You should try writing the code yourself.
Here is an overview of the Java logging framework that ships with the JDK. You may wish to check out Commons Logging and Log4J.
Regarding the second part of your question (which was editted out for some reason) I would recommend having all threads log to the same file but logging the thread name along with the log message allowing you to grep the file for a specific thread if required. Also, with most logging frameworks you can configure them to maintain a rolling window of the last N log files rather than explicitly deleting old files when an application starts.
Apache Log4j does everything you require. I hope that you can figure out how to use it on your own.
Take a look at Log4j, and specifically this set of step-by-step examples. It's pretty trivial.