Develop Reference

High Performance Cluster Computing (and MPI) in Java [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I am trying to implement a cluster computing system using Java and the Windows OS. I'm looking for a solution that
is not too out of date
is reasonably easy to install and set up
has enough documentation to get started with the classes and methods without a previous knowledge of MPI
is at least somewhat user friendly
This might not be possible, but it would also be nice if it was somewhat close in usage to the Java Concurrent framework.
I initially learned a bit about the Java Concurrent package and was easily able to learn to write parallel programs on my local 8-core machine using the Runnable Interface and the ExecutorService, making all my classes threadsafe in the process. However, I have yet to find a standard mechanism to extend this programming framework to clusters.
I then learned of a GitHub project called Java-Interop-Library (https://github.com/MicrosoftHPC/Java-Interop-Library ) that could be used with Microsoft HPC Pack. I networked a few cloud computer via Amazon EC2 and installed the HPC Pack. The Java-Interop-Library was a nightmare to compile and set up. I had to manually edit several batch files and even some Java code to get it compiled. By the time I got most of it working (but not everything), I just started thinking that there had to be an easier way, and I started searching again.
My new search led me to MPJ-Express (http://mpj-express.org). I read through the documentation on the site, and it seems easy to set up. They even have documentation on how to integrate it with Eclipse and debug. But, I could never find any documentation on how the classes and methods are actually used (there's a simple hello world example, but it's not close to enough).
More searching led me to MPIJava, Hadoop, and GridGain. Having no experience with MPI or MPJ, and knowing that MPJ grew out of MPIJava, I started trying to find documentation for that instead. I found some docs, but some of it is quite old and I'm not really sure I'm on the right track. I saw the mention of GridGain on another StackOverflow post, and went to their website. They seem to have a cluster computing framework, and a simple posted example even uses what appear to be classes that use Runnable objects, which seemed attractive to me having had some experience with the Java Concurrent framework. I don't really know anything about Hadoop, other than that it might be a possibility.
I really just need some better direction on the best way to accomplish cluster computing i Java. I feel like I'm just spinning my wheels.

You can use MPJ Express and I assume that you have already given it a go and environment set up is done. MPJ Express can execute parallel Java applications on both multiocore mode and cluster mode. It must be mentioned there that the same application works for both modes without any modification so you have the option of developing and testing your application in multicore mode and when done it would seamly run cluster mode as well. Although It is not quite clear what you want to develop in the end but as far as its Java docs are concerned you can find them here: http://mpj-express.org/docs/javadocs/index.html. These are all the classes and methods that you need to develop any parallel java application.
Learning cluster Programming with MPJ Express is quite easy as you can access a comprehensive set of working examples. For this purpose unzip MPJ Express and explore test directory. It contains multiple test cases and working examples of parallel Java applications. You can have examples of point-to-point and collective communication in mpj-v0_xx/test/mpi/pt2pt and mpj-v0_xx/test/mpi/ccl respectively. You can develop any basic parallel application by just using point-to-point and collective communication methods. It would give you head start in learning cluster Programming using Java.
Let me know if you encounter any problem in setting up MPJ Express on cluster or executing examples provided in test directory. You can also post your queries on MPJ Express mailing list on following page: http://sourceforge.net/p/mpjexpress/mailman/?source=navbar

Media player libraries in Java [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I am evaluating libraries for playing audio/video in Java. It does not need to be 100% Java; Java bindings to native libraries are perfectly OK. An external application that can be controlled from Java is also fine, as long as it can render video on a Java component.
The target platform is Linux. Windows support is a plus, but not required.
I have played with VLC using the VLCj bindings, and it works pretty well in general, but keeps crashing occasionally when put under stress. I have also used mplayer in slave mode, which looks good. But I'd like to evaluate other options as well.
These are the requirements:
Live playback of H.264, MPEG4 through RTP using RTSP. Extra points for MJPEG over HTTP.
Able to render video on a Java (Swing or AWT) Component
Open source, and actively maintained
Stable, rock-solid
Suggestions? Advice?

Ok, I've spent the last month playing with several options and these are the results:
VLC. This was my first thought, as everybody keeps saying that it "plays everything". I have used both the VLCj Java bindings, and the built-in remote interface (-I rc). I found this to work pretty well in general, but had some issues. I kept seeing ocassional crashes under heavy load (not VLCj's fault as it also happens when using the binary directly). Also I've found memory consumption to be relatively high when compared to other options.
MPlayer. This actually performs better than VLC in my experience, no crashes, and memory consumption is lower. No Java bindings, although the slave mode works very well.
GStreamer. Very powerful, very flexible, while still easy to get started with. Tried both the Java bindings and running the gst-launch binary from Java. Both approaches work remarkably well.
Xuggler looked good, however it seems to have issues with RTP (as stated in the FAQ). Since both GStreamer and MPlayer worked so well, I did not get past the initial research.
I found GStreamer to be the best solution given the requirements, with MPlayer being the second option.

have you looked at gstreamer?

I've also spent a while researching my options, and I've actually come to the conclusion that VLCJ is the best option - however here's the clincher, you need to run it out of process (especially with multiple players) for it to give you 100% reliable operation. That's the approach I'm taking and I've yet to see it crash. With it rock solid in this way it also has other advantages:
It can play pretty much anything. Yes it's a bit of a cliche, but this includes DVDs, Youtube videos, pretty much any video file...
Should support Linux / Mac / Windows, though I've yet to verify Mac working.
Actively maintained, and if you ask a good (as in well thought out) question on the discussion group the owner often replies in minute in my experience!
Open source, and the Google group activity at the time of writing is high.
Uses all the graphics acceleration VLC does
Able to play to any heavyweight Canvas component (and if you use the direct player, you get the BufferedImage to display anywhere you like, even in JOGL 3D land if you're that crazy!
I haven't found it that heavy on the resources front either, though I haven't done any thorough testing (but 3 players work fine in tandem as well as my relatively power-hungry application on my 4 year old basic laptop!)
Disadvantages? There's no official framework for out of process players, so you have to build one yourself (see here for how I did it). And you do sometimes have to do some fiddling to get it working. But as far as I can see, it's working well for me so far!

Has Java ever been used in outer space? [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
In response to a question about examples of Java usages, I bumped across some articles where NASA used Java for ground control in a mission to Mars but I couldn't find out if it has ever been used outside of Earth. Do you know of any such instances?

Here you will find a paper discussing the current state of using Java in space applications:
REAL-TIME JAVA IN SPACE: POTENTIAL BENEFITS AND OPEN CHALLENGES
They say:
In short time, it is unlikely that
with current Java implementations the
whole spacecraft on-board software
could be written in Java. However,
Java could be used as an isolation
platform for software that has not
been assigned the highest criticality,
while the critical code would still be
written in Ada or C.
Here one of the same authors reports on a successful test flight of an unmanned aerial vehicle (UAV) that uses Ovm (open source RTSJ (Real Time Specification for Java) implementation):
A Real-time Java Virtual Machine for Avionics

One fact often overlooked is that most space probes use older processors. They simply cannot run current commercial software. They are not fast enough or powerful enough. IC chips in space are exposed to high levels of cosmic rays. They energetic particles can destroy the small nanometer components on modern CPUs and other IC chips. To work reliably in space you either need specially designed ICs with redundant components, spend a lot on heavy shielding, or use older IC chips which have larger components that can take a hit from a cosmic ray without being destroyed.
Most of these older CPUs will run with a RTOS such as VxWorks and are either programed in assembler or using a language like C and still get decent performance.

Probably on the documentation laptops they take up there on the Shuttle, at the very least.
Java is also mentioned here, which implies that there is at least Java code that can talk to some spacecraft, if not actually run on them: http://www.dtnrg.org/wiki/Code

According to this article, the Ground Operations Center uses it for 3D Mapping and Planning. A comment on this forum claims the Mars Lander runs VxWorks.
Edit: Confirmed by Wind River, the Spirit and Opportunity run VxWorks RTOS. No other references where Java has been explicitly used in Space.

I think I read a few years ago that some probe or satellite used Java as part of their analyzation equipment but I doubt that mission critical software will be developed in Java today.
However: With all the applications used by astronauts on their Laptops I wouldn't be surprised if there were applications written in Java. Maybe some locally installed webapp that is used as a bugtracker.

I cannot speak with any authority with respect to the software onboard spacecraft, but I can say definitively that Java is used by NASA (or its affiliates) to handle the data from the various missions. I recently worked at a laboratory for space physics, which handled data and telemetry for in-orbit spacecraft, and the development team (of which I was a part) worked almost exclusively in Java.

What software would you recommend for image enhancement prior to OCR (Optical Character Recognition)? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
We are currently researching ways of enhancing image quality prior to submission to OCR. The OCR engine we are currently utilizing is the Scansoft API from Nuance (v15). We were researching the Lead Tools but have since decided to look elsewhere. The licensing costs associated with Lead Tools is just too great. To start with we are looking for simple image enhancement features such as: deskewing, despeckling, line removal, punch hole removal, sharpening, etc. We are running a mix of .NET and Java software, but java solution would be preferred.

Kofax is good for pre-processing, but for the types of cleanup you are talking about may be overkill unless the images are really bad. Unless your specialty is in image processing, I'd recommend working with a provider that does the image cleanup and the OCR so you can focus on the value you actually add.
We license the OCR development kit from ABBYY (ABBY SDK) and have found it to be superb for both image processing and OCR. The API is quite extensive, and the sample apps, help and support have been beyond impressive. I definitely recommend taking a look.

Disclaimer: I work for Atalasoft
We have those functions and run-time royalty-free licensing for .NET.
http://www.atalasoft.com/products/dotimage/
We also have OCR components including a .NET wrapper for Abbyy, Tesseract and others and Searchable PDF generation (image on top of text in a PDF)

Not sure if this would be quite up to the standards that you guys would need, but perhaps you should look at some of the Paint.Net APIs. I don't know how easy it would be to extract their image processing algorithms for use in your project, but I believe they do some of the things you are looking for. Plus it is an open source project with an MIT License, so it should be pretty friendly for business use.

Research about KOFAX VRS at KOFAX.com

Maybe JMagick, it is an open source Java interface of ImageMagick. It is implemented in the form of a thin Java Native Interface (JNI) layer into the ImageMagick API. It's licensed under the LGPL so it shouldn't be a problem license wise.
http://sourceforge.net/projects/jmagick/

I would suggest Intel for its zero-cost runtime licensing.

Depends on the number and quality of the original images. Managed code and imaging tool kits will work but it's not always the best solution if you haved several million images to process. For small batches and tight budgets, I agree with the previous posters that projects like Aforge, Paint.NET, and other open source computer vision libraries will do the trick. Of course, you are on your own if the results are not improving... At least this let's you put everything you need under one application for a low cost.
If you are processing several hundred thousand images a month, then I would suggest you divide up the process into smaller workflow step and tweak each one until your cost per image gets as close to zero as you can. You will find that the OCR results rise quickly at first and then level off sooner than you expected. (I'm not a big fan of OCR but it has its place)
I use commercial Windows product from Recogniform to process and clean up the images prior to OCR in a batch mode using scripts adjusted for various kinds of images. If an image fails QC or is rejected by the OCR engine, it is "repaired" by hand using a custom .NET application built with Atalasoft's toolkit. Batch process everything and only touch what fails.

Are there any decent free Java data plotting libraries out there? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
On a recent Java project, we needed a free Java based real-time data plotting utility. After much searching, we found this tool called the Scientific Graphics Toolkit or SGT from NOAA. It seemed pretty robust, but we found out that it wasn't terribly configurable. Or at least not configurable enough to meet our needs. We ended up digging very deeply into the Java code and reverse engineering the code and changing it all around to make the plot tool look and act the way we wanted it to look and act. Of course, this killed any chance for future upgrades from NOAA.
So what free or cheap Java based data plotting tools or libraries do you use?
Followup: Thanks for the JFreeChart suggestions. I checked out their website and it looks like a very nice data charting and plotting utility. I should have made it clear in my original question that I was looking specifically to plot real-time data. I corrected my question above to make that point clear. It appears that JFreeChart support for live data is marginal at best, though. Any other suggestions out there?

I've had success using JFreeChart on multiple projects. It is very configurable. JFreeChart is open source, but they charge for the developer guide. If you're doing something simple, the sample code is probably good enough. Otherwise, $50 for the developer guide is a pretty good bargain.
With respect to "real-time" data, I've also used JFreeChart for these sorts of applications. Unfortunately, I had to create some custom data models with appropriate synchronization mechanisms to avoid race conditions. However, it wasn't terribly difficult and JFreeChart would still be my first choice. However, as the FAQ suggests, JFreeChart might not give you the best performance if that is a big concern.

I just ran into a similar issue (displaying fast-updating data for engineering purposes), and I'm using JChart2D. It's pretty minimalist and has a few quirks but it seems fairly fast: I'm running a benchmark speed test where it's adding 2331 points per second (333x7 traces) to a strip chart and uses 1% of the CPU on my 3GHz Pentium 4.

Live Graph supports real-time rendering.

I'm using GRAL for real-time plotting. It's an LGPL Java library. Although it's not as powerful as JFreeChart it has a nicer API. I got a plot up and running in very short time. They also ship a real-time plotting example.

I found this question when I was googling for open source plotting libraries for java. I wasn't quite happy with the answers posted here so I did some further research on the issue.
Although this question has been posted back in 2008 this might still be interesting to someone.
Here is a list of Open Source Charting & Reporting Tools in Java

http://autoplot.org/ allows for real-time updates and can be used to create many types of scientific plots.
To update the plot, specify the URL to a data file and then append &filePollUpdates=1&tail=100. See the example at http://autoplot.org/cookbook#Loading_Data

Waterloo Scientific Graphics is a new LGPL project. Data objects are observable and could be updated in a real time plotting scenario.
For details see http://waterloo.sourceforge.net/
A few screenshots:

The library I wrote, Plot4j, also supports real-time plotting.

I used JFreeChart (http://www.jfree.org/jfreechart/) on a previous project. It has some very good built-in capabilities, and the design was WAY extensible so you could always roll your own extension later if you needed some custom chart annotation or wanted an axis to render differently, or whatever. It's definitely worth checking out.

Check ILOG's JViews - they have a lot of stuff and something might fit your needs. All of them are extremely configurable and quite fast. Not free though.

I've used JFreeChart in a rather complex application that needed to visualize data streams and calculations based on the data. We implemented the ability to visually edit the data plots by mouse and had a very large set of data points. JFreeChart handled it very well.
Unfortunately I was stuck with v0.7, but the newest release are sooo much better when it comes to API clarity. The community is very helpful and the developers are responding to mails too.
If you're doing a web application and don't want to bother with libraries, you can check the Google Chart API. Didn't use it myself, but I started some tests which were very promising.

For real-time plotting you can use QN Plot, JOpenChart or its fork Openchart2.

JHandles is an alternative graphics package for Octave (a math package). It is probably worth looking into, but being Octave specific may not have what you need.
-Adam

PtPlot
may be a good choice. Formerly called Ptolemy.

jcckit can handle real-time plotting. It's a bear to use though.
I forked it, and made a very simple wrapper around it for non-realtime plotting. The underlying complicated interface can be used directly too.
https://bitbucket.org/hughperkins/easyjcckit

You might want to check out JMathPlot