I have seen how bugsense, sentry etc work. I like the way you can get the error/crash reports. What I want is a solution like those but for internal use. Using an external api like bugsense is out of the question.
Is there any similar open source solution that can be used internally?
If you are already logging properly, and are requiring a more granular configuration you could give logstash a try. It is basically a logshipper with various input, filter and output modules, including email as an output method.
The input can be configured to parse your existing logfiles, recieve messages from a queue and many more. For UDP/IP based logging, take a look at logstash-gelf which is basically an adapter for automated generation of well formed logging meta data. If you plan on parsing your Logfiles, look out for the "multiline codec" in regards to parsing stacktraces and "grok" as a filter for parsing the log entries. For grok, I found that the Grok debugger is a big help.
E.G.: Once you have your logs routed through the input, and your logging is configured to use a named logger for the emailed messages you can tag them in logstash input and tell the output to send an email if a message with the tag is coming through.
I think using a specific logger (slf4j, log4j...) could be used.
It can send e-mail for example for FATAL log with stack trace and what you want.
Related
I am running a Java application on Azure Cloud Services.
I have seen this article which shows how to configure a java project to send logs to Azure insights using log4j: https://azure.microsoft.com/en-us/documentation/articles/app-insights-java-trace-logs/
However, for various reasons I do not want to do this. My java application already writes multiple log files in a log directory (application.log, error.log, etc). I want to point to this directory in Azure Insights so that it can aggregate these log files over multiple instances of my application running on Cloud Services and then present them to me. (In a similar way that AWS Cloudwatch presents logs). How can I accomplish this?
I think this is a deep question and would require a bit of custom coding to accomplish it.
The problem as I read it is, you have multiple log files writing to a location and you just want to parse those log files and send the log lines. Moreover, you don't want to add the log appenders to your Java code for various reasons.
The short answer is, no. There isn't a way to have AI monitor a directory of log files and then send them.
The next short answer is, no. AI can't do it out of the box, but Log Analytics can. This is a bit more heavy handed and I haven't read enough about it to say it would fit in this scenario. However, since you're using a cloud service you could more than likely install the agent and start collecting logs.
The next answer is the long answer, kinda. You can do this but it would require a lot of custom coding on your part. What I envision is something akin to how the Azure Diagnostics Sink works.
You would need to create an application that reads the log files and enumerates them line by line, it would then parse them based on some format and then call the TrackTrace() method to log it.
This option requires some considerable thought since you would be reading the file and then determining what to do with it.
I have a sample program which sends http post request and gets a response from server. While in the process of running the program, I am reading data from a file(txt/xml/DB) and executing the http request and the relevant response will be written back to file respectively.
Based on the above functionality, I have planned for a log file which writes exceptions, errors(paths), methods executed, classes executed and a time stamp.
Please tell me any logging programs in java, may be log4j is relevant for this kind of scenario?
log4j is more or less the defacto for logging and allows logging to files, db, jms etc. There are many resources on the net, a simple guide to configure it using a property file and a sample program to initialize it and start logging can be found here: http://www.javabeat.net/baisc-steps-to-configure-log4j-using-xml-and-properties-file/
You can read more about various logging levels and possibilities of logging in multiple files (i.e. Access logs in one, Errors in another etc)
I was just wondering if following thing exists.
I have a TCP communicator which keeps communicating with thousands of devices.
Currently, the TCP communicator logs all the events in one log file.
Now, is it possible to log communication with every device in different files. The IMEI number of every device is different. So the logger will check if a file with name equal to the IMEI number of the device exists. If the file exists, logger will start logging events of the device in that file, otherwise it will create a new file with IMEI as the file's name and start logging the events in that file.
(We are developing our application in Java.)
LogBack is the future, and it's here!
Created as a successor of log4j and fully complaint with the slf4j framework, logback might be the easy (and clean) way to fulfill your need.
I'm not an expert but I guess that SiftingAppender might be the right answer. There should be a discriminator option for you. Maybe you can build your own discriminator, extend the SiftingAppender, or get some extra help from Janino library.
Hope this helps!
If you are implementing the logger yourself, there's nothing to stop you from doing this.
For example, give the log function as a parameter the number of the device you're currently communicating with, and implement it the way you described.
If you're using Apache log4j, which I highly recommend, create a custom logging appender by extending AppenderSkeleton and writing unique files for individual connections will be as simple as doing standard file I/O with a variable filename.
If you are using java.util.logging, look at the Handler base class, if you are using log4j, look at Appender. In both cases, you need to somehow get the IMEI associated with the message, so the code writing the log message can pick the appropriate file.
There are two approaches to doing this.
First is to extend the log event class (LogRecord or LoggingEvent respectively). This would allow you to log using your event, which contains the IMEI. However, this does not account for logging performed by other libraries etc while performing the conversation with a device.
The other alternative is to use a ThreadLocal. Set the IMEI associated with a socket whenever you receive a message or are formulating message. Make sure that the logging happens in the same thread, and any queuing is done at the log handler/appender. Look for / ask questions about ThreadLocals if you are unfamiliar with this approach. I believe that Log4J's NDC and MDC implements this sort of strategy, but I've not tried to do specialized processing of context at the appender.
Finally, be aware some operating systems will run out of file handles if you are indeed thinking of keeping "thousands" of log files open. Depending on just how many files, you may want to consider writing log messages (with IMEI) to a database, or doing some sort of LRU-based file closing. In the latter, you would basically not have file handles for log files that haven't been touched in a while.
Hiho,
i'm trying to log errors with the help of java.logging
i want my logger to log everything to a log file and to a string or something(for html output(my program is a servlet))
I haven't found something like a StringHandler. Is there a possibility to do this?
greetings
The rationale behind a logging famework is to decouple your application code from logging - it sounds like you want to re-couple the two together. The biggest issue you will probably need to overcome is that all your log messages will be consolidated together, so you will see messages from separate requests in the same file.
If you want to be able to display to the user all the messages that your servlet has logged during the lifetime of their request you'll need to add an Handler as the first thing in your servlet, remove it in a finally block and then handle the messages its accumulated.
I'm not aware of any way in which you could reliably capture all relevent logging per request, as your servlet container will be executing code before you get to the point where you can intercept it, but as that sort of logging will probably deal with errors which would prevent you ever reporting anything back to the user, its probably a non-issue.
As some of the other answers intimate, logging though java.util.logging is rather basic, hence a number of other projects which provide logging, Logback being one of the best.
I think you can use the usual logger like log4j and a StringBuilder. That should do the trick.
You could try creating your own appender called StringAppender, as a subclass of WriterAppender with a StringWriter to log to.
I am running some JUnit tests in Eclipse, and my code is generating an XML log file using the java logging APIs. (java.util.logging). Is there an easy way to view this XML log output in Eclipse, other than reading the raw XML? Specifically I want to be able to easily see what threads different log messages have come from.
I've been using the SLF4J logging API coupled with the Logback logging implementation. SLF4J can be configured to map log messages in the java.util.logging, log4j, jakarta commons logging, and SLF4j APIs into a common intermediate form. On the other side, the messages can be generated via java.util.logging, log4j, or Logback. It's a flexible approach that works well, especially when you have components that use different logging APIs.
One nice thing about Logback is that you can configure it to send a copy of the log messages to a an Eclipse plug-in. The plug-in and lets you view and filter the log file in a variety of ways. Messages contain the thread that generated them, so it sounds like something you should check out.
I think you were looking for the same thing I was looking for. I found UtilLogger4E, but I started my own project (EDevTools LogViewer), because it has some missing features and is not opensource.
It has the abilities to read Java Util Logging XML logs from a file or a configurable socket, display them colored by level in a table and it can show more than one log at a time (through multiple view instances).
Java Util Logging has a built-in feature to send logs to a socket (SocketHandler), it must only be set by the configuration file.
You can take a look at LogViewer at: http://sourceforge.net/projects/edevtools/
If you want to use log viewer which is not based on eclipse, I recommend you OtrosLogViewer. I can import java.util.logging XML format and SimpleFormat.
Disclaimer: I am the author of OtrosLogViewer
I was always hoping to find something like Apache Chainsaw for the java.util.logging/Eclipse combination, i.e. a log viewer in Eclipse that listens to a port and a java.util.logging.Handler implementation writing directly to that. But I haven't found such thing yet and didn't get around writing one either. So far I just use a suitable plain-text format to log to the console. Not elegant and no handy filtering options, but it does the job. Writing your own custom formatter is not hard at all.
There is some support for analyzing log files in Eclipse's Test and Performance Tools Platform Project (the name being even worse than the acronym). I never tried that, but it might be useful for you.
Just on the side, an excellent tool for viewing log files in real-time is baretail (essentially a tail -f on Windows), but can be set up to highlight certain patterns in colour which can really help. You can find a free version here.
Depending on exactly what you're trying to do you might just need a good logging setup for your logging sub sys?
A framework I've been using recently uses slf4j to log, by providing the following log4j config I get to see which threads output what:
log4j.rootLogger = trace, default
log4j.appender.default = org.apache.log4j.ConsoleAppender
log4j.appender.default.layout = org.apache.log4j.PatternLayout
log4j.appender.default.layout.ConversionPattern = %-4r [%t] %-5p %c %x - %m%n
In my case I get output similar to the following in the Console tab in eclipse when I run my junit tests.
0 [pool-1-thread-1] INFO com.example.BaseTest - Server listening on port 9090
35 [NioProcessor-6] INFO org.apache.mina.filter.logging.LoggingFilter - CREATED
35 [NioProcessor-1] INFO org.apache.mina.filter.logging.LoggingFilter - CREATED
I don't see how Eclipse can know about your threads. If it's XML, you need to write a parser that can find your thread messages using XPath and print them out. But that's up to you - Eclipse can't read your mind.
You'll have to find an XSL-T stylesheet that can take the XML stream produced by log4j and pull out the threads that you want.