I've often found the debug mode of Eclipse useful over relying on print statements or logging. However, I've found that the debug mode's performance seems to be particularly sensitive to file I/O. Loading a file can be way slower (taking ~25 times as long), and since my workflow requires loading a rather large file before I get to anything interesting, this is particularly inconvenient for me.
Is there any sensible workaround for this issue? I don't actually need the debugging during the file loading part, so is there perhaps a way to only jump into debug mode at a certain point in the process?
Note that, unlike this question, I do not believe this is a problem with the state of my workspace.
You can hook into a running application with eclipse's remote debuging feature. You have to start your app with some parameters:
java -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y suspend=y -jar yourapp.jar
Then in your debug configuration choose Remote Java Application using port 8001.
More detailed with pics here
It's possible you're doing IO in a slow way (reading in small pieces), and debugging is just amplifying that, since there's overhead for every function call.
NetBeans has a way to specify which functions to profile, so I'd look for a similar option in Eclipse, and then tell it not to profile anything in the java.* namespace, or any of your IO specific code if that doesn't help.
I'd also make sure you're reading your file in a fast way (using buffered input streams, not using Scanner, etc.). There may also be things in NIO that could help, but I'm not that familiar with it.
Related
I am a C++ dev, working in java technology for about a year. Forgive my ignorance, but I have tried to search for following two questions.
What is the best way to debug in console for a java application (ala gdb in good ol' C++).
It seems jdb is below par.
Eclipse is the way to go for its IDE. But Eclipse is a GUI and I do develop in console on unix servers.
What is the best way to tame huge jave open-source frameworks. eg, camel, hibernate, spring, logback, blah, blah
I do most of my debugging in the console.
I have found that for the majority of bugs, Java stack traces are sufficient to identify the bug and develop a fix. This of course means that it's critical to allow stack traces to be seen on the console. For most applications, I've found that it's best to allow unchecked exceptions to propagate until they terminate the thread, with a stack trace showing up in the console; unchecked exceptions should normally indicate programming errors or uncontrollable conditions where the thread should be terminated anyway.
In cases where the stack trace is not sufficient, I've found that using System.out.println() during the debugging process is actually more effective than using an interactive debugger. Having a full log of a run is for me more useful than running a debugger where I may easily step past an issue that later turns out to be important.
Wat is the problem in setting up the project in eclipse IDE, you can always get the source code and set up your workspace??
Using sysout is great for small java programs but as the code grows or for big application (according to my understanding you plan to debug something big :P) it would really be a headache placing all those sysout and then removing them once you are done with the debugging.
I would suggest you to set up your own workspace by choosing 'import existing project in workspace' option of IDE, though it would be tiring first but will help you in long run.
I would say that there is no need for such a tool, because Java supports remote debugging.
You need to start your (server) JVM with remote debugging enabled - this will work for newer JDKs:
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005
After that you connect to it from the IDE, and set breakpoints or conditional breakpoints wherever you like.
See this question for more details
Also, this is a personal preference, but I would say that IntelliJ IDEA is the way to go as far as Java IDEs go.
I'm working on a server app that may be extended by user-supplied Groovy scripts. It's evident that I want to make sure these scripts run in a very tight sandbox where they cannot disrupt the core application code or consume too much resources to overload the server.
I have studied various possibilities and the final solution may be a combination of these:
Run the script within a very restricted security manager. The script is run within a no permission SecurityManager. Additional permissions have to be declared (like Android).
Launch a new JVM. Create a ScriptProcess wrapper around Runtime.exec and spawning a new JVM with a security manager, limited heap, etc. Because we launch a full-blown process, we might get more control on monitor bad behaving ones? The cost in resource would be dire though... An alternative would be to use Ant here, but would it be scalable?
Java Monitor API In Java 6 there is a package with monitoring capacity. We could monitor threads and maybe detect infinite loops and memory consumption. Anyone used this?
These are what I have in mind today. What would be the best way to make sure these scripts behave correctly and still keep a certain scalability and performance?
An additional possibility is using Groovy 1.8 compilation customizers on the GroovyShell that runs the embedded scripts. You can pre-import classes and methods, restrict use of the Groovy AST, and pre-apply an AST transformation, such as #ThreadInterrupt, #TimedInterrupt, or #ConditionalInterrupt. Details at:
http://www.jroller.com/melix/entry/customizing_groovy_compilation_process
You should have a look at the project groovy-sandbox from kohsuke.
Have also a look to his blog post here on this topic and what is solution is addressing: sandboxing, but performance drawback.
Also have a look at the java-sandbox project and the accompanying blog post http://blog.datenwerke.net/2013/06/sandboxing-groovy-with-java-sandbox.html.
All right, I've hit a bug that has simply confused the bejeebus out of me. I'm looking for ideas about what it could be that I can investigate, because right now, I got nothing. It goes something like this:
I have a standalone Java application that occasionally needs to twiddle the Line-In volume of the computer it's running on (a WinXP machine). It does this by calling a pair of external executables (written in VB6*) that can get and set various component volumes. (They can handle Line-In, Mic, Wave, CD, and the master volume control.)
There are several hundred units in the field, running on hardware (Dell machines) that my company provided and controls. At least several dozen clients are using this feature, and it works perfectly -- except for one instance.
For this one troublemaking machine, it simply doesn't work. I watch the volume sliders when the app is running, and when the volume is supposed to drop, they stay put. When I check the app's log file, it throws no errors, and appears to be executing the code that drops the volume. When I run the executables from the command line, they work perfectly.
I can't vouch for this machine being 100% identical to all the ones that are behaving properly, but we've been buying the same line of Dells for quite some time now; at a bare minimum, it's very, very similar.
So, turning my confusion into a bullet list:
If I'm doing something stupid in the Java code (i.e., not clearing my STDOUT/STDERR buffers), why is it only an issue on this machine?
If there's something broken in the VB6 executables, why do they work on every other machine and on this machine from the command line?
If there's some sort of hardware oddity on this machine, what sort of oddity could cause the volume control executables to fail only when called from within a Java application?
I am very confused. I do not like being confused. Anybody have any suggestions that may lead to my enlightenment?**
-* -- I know, I know, VB6, 1998 called and they want their obsolescent proprietary bug generator back, etc. Wasn't my decision. But the code works. Usually.
-** -- Insert Buddhism joke here.
Update Edit: Customer service may have stumbled onto something; it may be something to do with client configuration settings in the database. New evidence suggests that either something's misconfigured for that client or my software is doing something stupid in response to a specific configuration. And the problem may be more widespread than we thought, due to this particular feature not being as commonly used as I thought.
Responding to the comments:
Debugger: Theoretically possible, but looks like a massive headache given our setup.
High Verbosity Logging, Java: Good idea this, particularly given than the problem may be more widespread than I originally believed. Time to start revisiting some assumptions. And possibly clubbing them. Like baby seals.
High Verbosity Logging, VB6: A possibility; will need to be rolled-into the high-verbosity Java logging to trap the output, since my VB6-fu is so pitiably weak I don't know how to output text to a file. But, yeah, knowing whether or not the script is even getting called would be valuable.
Window Event Viewer: Not familiar with this tool. May have to correct that.
PATH problem: Doesn't feel likely; the Java code constructs a relative path to the executable that doesn't look like it's relying on any environment variables.
My thanks for the suggestions people have provided; at the very least, you've gotten my brain moving in directions that feel promising.
Solution Edit: And the winner is ... That's Not A Bug, That's A Feature! A feature gone horribly, horribly wrong. A feature that will now be neutered so as to stop bothering us.
A batch of invalid assumptions kept me from seeing it sooner, not the least of which was "I don't need to tool the code with more debug statements -- the statements already in there are telling me all I need to know!" DaDaDom, if you'd like to turn your comment into an answer, there's a shiny checkmark in it for you.
Thanks to everybody who chimed in with a suggestion. Now if you'll excuse me, my head is late for a meeting with my desk.
Here goes an answer:
Can you create a version of the software with verbose logging or could you even debug the code? At least then you can tell if it's in the java or the VB part.
Hmmmm. I've been told that executing programs from Java is either easy or hard. The easy part is starting them up. The hard part is dealing with the I/O streams (see my earlier question on using Runtime.exec()). Maybe the VB program is doing or expecting something weird on these particular machines that the Java code isn't working with properly.
edit: I also found a link to Jakarta Commons Exec:
Rationale
Executing external processes from Java is a well-known problem area. It is inheriently platform dependent and requires the developer to know and test for platform specific behaviors, for example using cmd.exe on Windows or limited buffer sizes causing deadlocks. The JRE support for this is very limited, albeit better with the new Java SE 1.5 ProcessBuilder class.
Reliably executing external processes can also require knowledge of the environment variables before or after the command is executed. In J2SE 1.1-1.4 there is not support for this, since the method, System.getenv(), for retriving environment variables is deprecated.
There are currently several different libraries that for their own purposes have implemented frameworks around Runtime.exec() to handle the various issues outlined above. The proposed project should aim at coordinating and learning from these initatives to create and maintain a simple, reusable and well-tested package. Since some of the more problematic platforms are not readily available, it is my hope that the broad Apache community can be a great help.
Have you considered the possibility that the authenticated user may not have permission to edit volume settings on the workstation? Does the program run correctly if you run as an 'Administrator'?
A few of my various Eclipse 3.5 (actually SpringSource ToolSuite 2.2) workspaces are painfully slow when building, and I have a suspicion this is because they are continuously grabbing network resources from over the internet (e.g. schemas), whereas other workspaces, with similar content, are fine. Tracking these down is problematic.
Does anyone know how to track these down, if this is in fact what's happening? Something like a log file which tracks each URL being accessed, or something of similar utility.
Edit: I should have mentioned that I've considered using a packet sniffer like Wireshark, but Eclipse generates a lot of traffic, and it's hard to see the wood from the trees. I'm looking for something higher level.
Edit: In the end, it turned out that SpringSource ToolSuite was doing some highly inefficient classpath scanning, slowing things to a crawl. No network access at all, it just felt like that kind of performance hit.
You could simply look directly at the wire, e.g. using Wireshark or using a HTTP proxy like loxy (never tried this one though).
Most likely it's a single resource where the connection times out. This should be pretty easy to spot (i.e. search) and remove from your workspace.
At the Java level, you might use a profiler or instrumentation to track down methods that take a long time to complete.
You can analyze what's going on in your network using a protocol analyzer like Wireshark. Wireshark shows you all network traffic and you can even analyze it's content. You can get it free of charge (open source license).
There is also the possibility to analyze wich application is connected to wich endpoint using a tools like TcpView.
If you are using Spring XMLs, Eclipse will validate their schema, which might mean downloading it every time from the web.
You can add these entries to your XML validation catalog as local files, which will prevent them from being downloaded (this is also important when working offline).
Go to the preferences, to XML -> XML Catalog. Some of the entries might already be there (added by Spring IDE, latest versions add that). You can add the rest yourself.
You can tell Eclipse to use a SOCKS proxy, and then configure that proxy server to dump what it is doing.
I've used JSOCKS for this - http://jsocks.sourceforge.net/ - and put in my personal changes for an inhouse project at http://github.com/ravn/jsocks.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Unfortunately, sometimes the only way to debug a program is by going through its long log files.
I searched for a decent log viewer for a while now, and haven't found a real solution. The only program that seemed to be most appropriate was Chainsaw with its Socket connector but after a few short uses the program proved to be buggy and unresponsive at best.
For my purposes, a log viewer should at least be able to mark log levels (for example with different colors) and perform easy filtering based on packages and free-text.
Is there any other (free) log viewer? I'm looking for anything that could work well with log4j.
Just wanted to say that I've finally found a tool that I can get along with just fine...
It's called LogExpert (see http://www.log-expert.de/) and is free. Besides the usual tail function, it also has a filter and a search function - two crucial things that are missing from BareTail. And if you happen to want to customize the way it parses columns further, it's dead simple. Just implement an interface in .NET and you're done (and I'm a Java/Flex programmer...)
I'm using OtrosLogViewer. You can mark log events manually or using string/regular expression. You can filter events based on level, time thread, string or regular expression. Logs can be imported by listening on socket or connecting to Log4j SocketHubAppender
You can take a look at Youtube video or screenshots:
Disclaimer: I am the author of OtrosLogViewer
You didn't mention an OS, so I'll mention this though it is only on Windows.
Bare Metal Software makes a product called BareTail that has a nice interface and works well. They have a free version with a startup nag screen, a licensed version with no nag, and a pro version with additional features. It has configurable highlighting based on matching lines against keywords.
They also have a BareGrep product too, which provides similar grep capabilities. Both are excellent and very stable and better than anything I've seen on Windows. I liked them so much I bought the bundle with both pro versions for $50.
I am using Notepad++ with my custom log file highlighting UDL. Looks like this:
You can try logFaces, it has fantastic real-time log viewer based on eclipse-like design.
Disclosure: I am the author of this product.
Consider to use Log4j viewer eclipse plugin - that was fork of Ganemede plugin in the begging and now have more features and stability was improved significantly, and still in active development and free :)
I've always used 'tail -f | grep re' or occasionaly 'awk'.
LogSaw based on Eclipse and free. Log4j log file analyzer, simple to use with easy filtering. Supports several flavors of log4j log files: JBoss, Log4j pattern layout, Log4j XML layout, WebSphere. Works like a charm. After couple of hours googling and trying several recommended free log4j viewers, this one was pleasant surprise. Have tried Chainsaw, BareTail, Insight, LogExpert, logview4j. It is released weeks ago, and I guess still builds its way up on google.
I've rolled out Splunk (http://www.splunk.com/) for log viewing and searching with great success. The free version can be used locally and the paid version can collect all your logs into one location. We use it mostly for Log4J logs but with lots of other formats as well.
Beyond tail and grep support (without needing to know grep...) it automatically indexes logs and allows easy analysis (e.g. # of events in last xx timeframe) as well as basic charting, alerting, and event aggregation.
I won't say that the app is perfect or that the company has matured yet. But I don't hesitate at all to recommend that you try it.
I'll add that for Windows, WireShark makes for a handy syslog viewer, ironically enough. I've tried several other syslog tools, and really, Kiwi is the best for syslog out there, but the "free" version is a bit nerfed. Others I ran into were either poorly programmed (crashing on minor issues -- logview4net), had a poor interface (Star SysLog Daemon Lite), or didn't even run (nxlog)
You can use WireShark's filter language to drill down on log data. It's overkill, but until someone writes a free syslog viewer/collector for Windows and makes it decent, this is one field that will be a hard one for most people.
Example:
# Display level 6 alerts from 192.168.5.90 in WireShark
syslog.level == 6 && ip.addr == 192.168.5.90
LogMX is a crossplatform tool that parses any log format from any source, then displays log entries with many features. By default, it handles formats like Log4j, LogFactor, syslog,... and can read from local file or SFTP, FTP, HTTP... but you can write your own pluggins if your format is another one or if your logs cannot be accessed through classical protocols.
You can monitor logs in realtime like 'tail' or load a whole log file and stop monitoring it.
www.logmx.com
Depending on what platform you are running on and what other log viewing tools you have available, you can just use the appropriate log4j appender (syslog, Windows Event Logger) and just use your platform log viewing tools.
Other than that I have usually seen custom solutions developed.
Something that will drive your solution is what your overall system is like. Are you trying to aggregate logs from several computers? Or just view the logs from a single remote process?
You may want to use a custom log viewer that just works on files. I like Kiwi Log Viewer or Ganymede (an Eclipse plugin), but it's not hard to put a simple Swing app together that reads from the socket.
Take a look to http://jlogviewer.sourceforge.net/ or http://sourceforge.net/projects/jlogviewer/
Java log viewer is lightweight GUI to easily view the java application
logs generated by the "java.util.logging" package.
It's open source!!
You can use MindTree Insight, it is open source, efficient, and specific for that use case : analyze log4j files.
Another good log viewer is Lilith (http://sourceforge.net/projects/lilith/ and http://lilithapp.com/). It is open source and works well with Logback, log4j & java.util.logging.
Just published a node module for color highlighting log output log-color-highlight.
echo "this string" | lch -red.bold this -blue string
Works well on unix/linux/windows and supports config file for complex logging scenarios.
For windows I use it in combination with file-tail
I have written a custom tool for that: https://plus.google.com/u/0/102275357970232913798/posts/Fsu6qftH2ja
Alfa is a GUI tool for analyzing log files. Usually you are forced to search for data in them using editors. You open a log, press Ctrl-F and the "Next" button again and again, then reload the file as it was modified, and repeat the search. Alfa maps a log file to a database allowing you to use standard SQL queries to get data without any superfluous actions.
UPD: Google killed Google+ so please use other link: https://drive.google.com/drive/folders/0B-hYEtveqA0aN1E3Ul9NVlFlYWM