Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Unfortunately, sometimes the only way to debug a program is by going through its long log files.
I searched for a decent log viewer for a while now, and haven't found a real solution. The only program that seemed to be most appropriate was Chainsaw with its Socket connector but after a few short uses the program proved to be buggy and unresponsive at best.
For my purposes, a log viewer should at least be able to mark log levels (for example with different colors) and perform easy filtering based on packages and free-text.
Is there any other (free) log viewer? I'm looking for anything that could work well with log4j.
Just wanted to say that I've finally found a tool that I can get along with just fine...
It's called LogExpert (see http://www.log-expert.de/) and is free. Besides the usual tail function, it also has a filter and a search function - two crucial things that are missing from BareTail. And if you happen to want to customize the way it parses columns further, it's dead simple. Just implement an interface in .NET and you're done (and I'm a Java/Flex programmer...)
I'm using OtrosLogViewer. You can mark log events manually or using string/regular expression. You can filter events based on level, time thread, string or regular expression. Logs can be imported by listening on socket or connecting to Log4j SocketHubAppender
You can take a look at Youtube video or screenshots:
Disclaimer: I am the author of OtrosLogViewer
You didn't mention an OS, so I'll mention this though it is only on Windows.
Bare Metal Software makes a product called BareTail that has a nice interface and works well. They have a free version with a startup nag screen, a licensed version with no nag, and a pro version with additional features. It has configurable highlighting based on matching lines against keywords.
They also have a BareGrep product too, which provides similar grep capabilities. Both are excellent and very stable and better than anything I've seen on Windows. I liked them so much I bought the bundle with both pro versions for $50.
I am using Notepad++ with my custom log file highlighting UDL. Looks like this:
You can try logFaces, it has fantastic real-time log viewer based on eclipse-like design.
Disclosure: I am the author of this product.
Consider to use Log4j viewer eclipse plugin - that was fork of Ganemede plugin in the begging and now have more features and stability was improved significantly, and still in active development and free :)
I've always used 'tail -f | grep re' or occasionaly 'awk'.
LogSaw based on Eclipse and free. Log4j log file analyzer, simple to use with easy filtering. Supports several flavors of log4j log files: JBoss, Log4j pattern layout, Log4j XML layout, WebSphere. Works like a charm. After couple of hours googling and trying several recommended free log4j viewers, this one was pleasant surprise. Have tried Chainsaw, BareTail, Insight, LogExpert, logview4j. It is released weeks ago, and I guess still builds its way up on google.
I've rolled out Splunk (http://www.splunk.com/) for log viewing and searching with great success. The free version can be used locally and the paid version can collect all your logs into one location. We use it mostly for Log4J logs but with lots of other formats as well.
Beyond tail and grep support (without needing to know grep...) it automatically indexes logs and allows easy analysis (e.g. # of events in last xx timeframe) as well as basic charting, alerting, and event aggregation.
I won't say that the app is perfect or that the company has matured yet. But I don't hesitate at all to recommend that you try it.
I'll add that for Windows, WireShark makes for a handy syslog viewer, ironically enough. I've tried several other syslog tools, and really, Kiwi is the best for syslog out there, but the "free" version is a bit nerfed. Others I ran into were either poorly programmed (crashing on minor issues -- logview4net), had a poor interface (Star SysLog Daemon Lite), or didn't even run (nxlog)
You can use WireShark's filter language to drill down on log data. It's overkill, but until someone writes a free syslog viewer/collector for Windows and makes it decent, this is one field that will be a hard one for most people.
Example:
# Display level 6 alerts from 192.168.5.90 in WireShark
syslog.level == 6 && ip.addr == 192.168.5.90
LogMX is a crossplatform tool that parses any log format from any source, then displays log entries with many features. By default, it handles formats like Log4j, LogFactor, syslog,... and can read from local file or SFTP, FTP, HTTP... but you can write your own pluggins if your format is another one or if your logs cannot be accessed through classical protocols.
You can monitor logs in realtime like 'tail' or load a whole log file and stop monitoring it.
www.logmx.com
Depending on what platform you are running on and what other log viewing tools you have available, you can just use the appropriate log4j appender (syslog, Windows Event Logger) and just use your platform log viewing tools.
Other than that I have usually seen custom solutions developed.
Something that will drive your solution is what your overall system is like. Are you trying to aggregate logs from several computers? Or just view the logs from a single remote process?
You may want to use a custom log viewer that just works on files. I like Kiwi Log Viewer or Ganymede (an Eclipse plugin), but it's not hard to put a simple Swing app together that reads from the socket.
Take a look to http://jlogviewer.sourceforge.net/ or http://sourceforge.net/projects/jlogviewer/
Java log viewer is lightweight GUI to easily view the java application
logs generated by the "java.util.logging" package.
It's open source!!
You can use MindTree Insight, it is open source, efficient, and specific for that use case : analyze log4j files.
Another good log viewer is Lilith (http://sourceforge.net/projects/lilith/ and http://lilithapp.com/). It is open source and works well with Logback, log4j & java.util.logging.
Just published a node module for color highlighting log output log-color-highlight.
echo "this string" | lch -red.bold this -blue string
Works well on unix/linux/windows and supports config file for complex logging scenarios.
For windows I use it in combination with file-tail
I have written a custom tool for that: https://plus.google.com/u/0/102275357970232913798/posts/Fsu6qftH2ja
Alfa is a GUI tool for analyzing log files. Usually you are forced to search for data in them using editors. You open a log, press Ctrl-F and the "Next" button again and again, then reload the file as it was modified, and repeat the search. Alfa maps a log file to a database allowing you to use standard SQL queries to get data without any superfluous actions.
UPD: Google killed Google+ so please use other link: https://drive.google.com/drive/folders/0B-hYEtveqA0aN1E3Ul9NVlFlYWM
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I am in the process of designing a tool to visualize java logs. Specifically, these logs are generated by printing to the console whenever a method is invoked and whenever it is returned. These log statements are injected into the entire Android OS source code using bytecode manipulation. So far we have been able to instrument the Android OS and generate these log statements. The fields contained in these log statements are: process that invoked the method, method signature, argument types, return type and time stamp
The tool that uses these logs will have a detailed view (on zooming in) and a high-level overview (on zooming out). I am looking for efficient ways of visualizing and navigating this huge log file at high-level overview to get valuable information. Each log statement has a hierarchical relationship to another log statement. For Example, a log statement for a method invocation will be the parent of all the log statements of the methods invoked from within the parent method.
My questions are,
What would be an effective way to visualize and navigate these hierarchical log statements in a huge log file to get a good high-level overview? Sequence diagrams are useful for a detailed view but aren't suitable for a huge call trace.
Are there any existing tools in the market that have similar functionality? I have looked at log visualization tools but none have the high-level overview visualization.
As an app developer who is equipped with an instrumented VM that generates log statement for each called method and who can run an app on the said android VM, what information would be useful to you?
Any other suggestions?
Thanks in advance.
Edit: I added a few more details about hierarchical nature of the log statements.
I believe the diagram that is closest to what you describe is a Sequence diagram. There used to be a plugin for eclipse that would monitor your source code and build up a sequence diagram of all the call/return/timing/etc.
Here's a description of how the diagrams work and how to create them by hand:
http://agilemodeling.com/artifacts/sequenceDiagram.htm
They are a great way to model how the code interacts at runtime at a very high level.
Just with a quick query--this is in the eclipse marketplace:
https://marketplace.eclipse.org/content/objectaid-uml-explorer#group-metrics-tab
I haven't used this one, just what came up in the query, however I do remember creating an awesome diagram with code I was analyzing--covered an entire wall of my cube and was really helpful.
From your question, the data your using are server logs; and the structure of log files are very complex task and processing different log files for visualization is also possible under a certain limitation, you will need to create or develop your own standard tool to classify and aggregates the log files for processing then for visualization. However, some tools and platforms are already existed such as:
One of the effective ways is to process data in pipelines for big data information, since these log statements are server logs using Apache sparks, splunk, or Cisco Mars with integration in SIEM's solutions to process logs in real-time is one of the most effective ways used for processing server log files.
HPE Security ArcSight Data Platform also can deliver a high-performance, cost-effective solution that unifies big data collection, reporting and analysis across enterprise machine data.
Clock View and PeekKernelFlows use NetFloW log to monitor large IP spaces over long periods. In addition, a tool called ELVIS extensible log visualization tool, can correlate the process of server logs and provide a representation summary view of the important data.
I highly suggest you use a simple log structure ex. FTP or IIS server log files of selected fields, and visualize the output by using D3.js modules for static and interactive visualization specifically with Parallel Coordinates.
What is the best way to log data of a Java (also using JavaFX) application for later reviewing? A few other people and I are working on a project, with a client/server model, and we want to log all data in the server for reviewing it later and possibly use it for calculations (later).
So a simple text file won't do, because the data needs to be reviewed in a clear way and can be used for later calculations. You can say that we want a small database that integrates well with Java, but using one like MySQL will require all the server setup and it seems to cumbersome in this situation.
Does anyone have any suggestions?
Probably the best option right now is to use slf4j as the API, and logback as the implementation.
That'll allow you to change the physical repository of your log data, without changing your code.
I also suggest you to read this article:
http://glauche.de/2009/09/09/java-util-logging-vs-slf4j/
The JDK commes with the java.util.logging package. This gives you all you need for logging.
I am working on my bachelor thesis (this is my first research project) and trying to evaluate different possibilites to monitor a developers work during a day, aggregate it and illustrate it later. For this purpose, I defined some metrics, I want to measure (they might change over time and there might come more).
My questions are related to the monitors. To start, I have decided to monitor the developers work (only) within Eclipse (and add a TFS implementation later):
Work Items: I want to know how many work items the developer solved, edited and created. If possible, I want to access the data from MyLin, because multiple different task/bug-trackers can be used (Bugzilla, Mantis, etc.). Unfortunately, I am not sure, if this is possible somehow?
IDE-stuff: (To start), I want to count the number of selects and edits in the code and probably also the number of clicks a developer made within the IDE. I read, that I can create listeners on Eclipse and get the data. Is this right and does anyone know a nice tutorial on that topic?
Source Code Management: To track a developers work, I need to get the developers source code changes (commits). For SVN, GIT and CVS, there are multiple APIs, which I may access with Java or also via webrequests. But before I start that, I wanted to ask you, if someone knows a plug-in or something where I can access the commits from different version control systems? What is the easiest way to do something like that?
I also have a couple of other things, I am going to measure... My aim is to aggregate the data within an Eclipse plugin and then send it the server for the visualisation.
I am not necessarily looking for fully implemented solutions, but more for hints, tutorials, tips, your opinions and probably also questions and propositions!
thank you!
You can try out with rabbit plugins for eclipse to track the time spent on it. It has different trackers to record the statistics about different commands executed, duration of the perspective or views used, time spent on editors etc.. And this plugins has a view to see all the statistics collected. have a look at the following link:
http://code.google.com/p/rabbit-eclipse/
Some of the metrics you want are produced by Hudson/Jenkins plugins as a starting point you could look at how those plugins produce those metrics. Same applies for the Eclipse metrics find existing plugins which manipulate or consume some of the metrics you want and use that as a starting point - since most of them are OpenSource.
I've often found the debug mode of Eclipse useful over relying on print statements or logging. However, I've found that the debug mode's performance seems to be particularly sensitive to file I/O. Loading a file can be way slower (taking ~25 times as long), and since my workflow requires loading a rather large file before I get to anything interesting, this is particularly inconvenient for me.
Is there any sensible workaround for this issue? I don't actually need the debugging during the file loading part, so is there perhaps a way to only jump into debug mode at a certain point in the process?
Note that, unlike this question, I do not believe this is a problem with the state of my workspace.
You can hook into a running application with eclipse's remote debuging feature. You have to start your app with some parameters:
java -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y suspend=y -jar yourapp.jar
Then in your debug configuration choose Remote Java Application using port 8001.
More detailed with pics here
It's possible you're doing IO in a slow way (reading in small pieces), and debugging is just amplifying that, since there's overhead for every function call.
NetBeans has a way to specify which functions to profile, so I'd look for a similar option in Eclipse, and then tell it not to profile anything in the java.* namespace, or any of your IO specific code if that doesn't help.
I'd also make sure you're reading your file in a fast way (using buffered input streams, not using Scanner, etc.). There may also be things in NIO that could help, but I'm not that familiar with it.
Does anyone have links and resources to connect to an AS400 from Java?
I remember years ago, somebody told me about a connector that simulates KeyStrokes from the keyboard and other "purest" approach that connected directly.
On the web I have found a lot of links, but I cannot find a complete product to do this (I am probably not using the right keywords).
EDIT
Thanks for the answers:
What we are looking for is a way to access the data inside the AS400 and/or the screens it uses and expose them for other new applications re-use. Either as a webservice of some sort, or directly through Java ( and java will expose the operations using webservices )
Thanks in advance.
EDIT
As per MicSim post, I've also found this link:
http://www.ibm.com/developerworks/library/ws-as400/index.html
What you are looking for is probably the Toolbox for Java™ & JTOpen from IBM. There is also an AS400 class in the toolbox for performing specific AS400 tasks. You can look here and here for more details. Just googled it and hope it's helpful.
IBM's 5250 screen-scraping technology was "WebFacing" - I would post a link but you're probably better off Googling it, since IBM's documentation is so scattered. There are other technologies available too but: Screen-scraping was never anyone's favourite since typically you end up with something which, although it looks more up-to-date, actually is harder to use than a green screen and no more functional. The 5250 is probably the single best data entry platform I've ever used - web forms in a browser are one of the worst.
As mentioned, jt400 is the way to go for most other things. In particular:
JDBC - for all things SQL. If you do it right and address your files as though they really are tables, it's a way to get away from the 400 entirely.
Record-level access - write Java programs using a similar database API to RPGLE (all those chains, setlls that 400 programmers love)
Call programs, system commands, manage resources (data queues, data areas, prints / spools, jobs etc etc)
Good luck
If you just want to run Java on the AS/400 (or iSeries, or System i, or whatever IBM's marketing department has decided to call it this month), that's a supported language. You can access the pseudo-DB2 database directly. Or are you after some other form of integration?
This obviously depends on what you want to do, however if you want to simulate keystrokes across a network connection to an AS400 process then Expect4j may be the library you are looking for.
This is generally a really nasty hack though and there are frequently better ways to achieve your goals. What are you trying to do?
The expect4J library can be found here. Expect was originally a unix command that allowed you to specify a string that you are expecting to see and then a string of characters to return. It was frequently used for automating logins etc and for screen-scraping applications.
Even better is the TN5250j Console, which can be used to extract data from the AS/400.
jacada makes tools to do what your looking for
http://www.jacada.com/