I'm in the process of writing a Java and FFmpeg based video editor, and I'm trying to find a library that would allow me to create a video from frames rendered via Java2D.
By 'video' I mean in a standard format (preferably vp8/webm, but anything common should be alright). It would be a plus if there was some facility for modifying preexisting videos as well, but that may need to be left to ffmpeg. Audio isn't needed as I'll mainly be working with ffmpeg for that.
The obvious solution would be to save each frame as an image and have ffmpeg combine them - but I worry about performance and quality using this method. Additionally, some tests showed that even short videos (less than 5 minutes) at 1280x720 resolution would be pretty massive in size - which I'd like to avoid if possible. Working a little more directly with compressed formats rather than huge batches of image files would certainly be preferred, or at least some method that isn't too hungry for disk space.
I'm not against homebrew solutions either (I'm already writing the ffmpeg bindings from scratch), but I don't know how practical it would be to write my own vp8 encoder for something that seems like it should be fairly simple.
Any suggestions on where to go with this? Or is the best solution to generate a individual image files and combine them later?
Thanks!
Have you had a look at Xuggler? It should be able to encode videos in the way you describe, though I haven't tried producing videos from a series of BufferedImages it should be possible.
It sits on top of ffmpeg and is pretty powerful with what it can achieve - it's not the easiest API to start with but there are a number of good tutorials around.
Related
I know we can extract text from image using ocr. But I need to extract the text present in video, like those in video lectures. Or in other words is it possible to transcribe a video to text. Is that possible? If so please suggest me how to do it in java or any other language.
My naive linux driven approach would be:
check: does the OCR work in my operating system?
extract some samples from the video using the normal runner. Each runner (for example VLC) has such a functionality.
check: how good is the OCR in extracting text from image files?
check: how good is the OCR in extracting text from image files with the background the video is providing?
get software to extract frames from videos in batch -> there is various software which allows to create contact-sheets, this should also be able to extract images in full resolution at abitrary points in time out of the video. Full resolution might be necessary to allow the OCR to work. Perhaps you can clip the images first, if you know, that the text is positioned in fixed rectangles.
Worst case you let OCR analyse each frame of the movie.
That mostly depends on how good and how fast your OCR is working. Everything else to me is very proven software. The language might be bash-shell-script, since the components will probably be separate linux programs. As I mentioned, it depends on the quality, performance and runtime environment of your OCR.
Yes, You can do that and there are 3 ways you can achieve it.
Split, Classify and train on your own.
Get a performance server,
A. Extract images from the video
B. Develop and Train your machine learning model. You can use tensor flow to do the same.
Note: If you prefer to train models on your own, make sure you have enough time as sometime the developing and training requires few months and you should have data to train them.
Use an OCR framework
USE API(Freemium model). There are many available in the market. Just google them and your will have many in hand.
I was trying to open jpeg files in a Java program and noticed that neither ImageIO nor the Apache commons imaging library tools could open the images. The commons library showed me this error:
"Only sequential, baseline JPEGs are supported at the moment"
So, my image files are compressed in a way both libraries aren't able to read. I could make an ImageJ macro and transform all the images first but I would like to just use my program and not something extra.
Is there a way to find the compression mode in jpeg or even a java library that can read jpegs in several modes?
Thanks in advance
Suddenly it works with the ImageIO standard Class. Don't ask me why. Thanks for your help guys.
You need to get some tool that will allow you to dump a JPEG stream. There are a number of them out there.
What you are looking for is the start of frame marker.
FFC0 indicates baseline sequential.
FFC1 indicates extended sequential. It doesn't take much more code to do extended sequential than baseline. It is puzzling why a decoder would limit itself to baseline these day.
FFC2 is progressive.
There are others but those are the only ones you are likely to encounter and are widely supported.
You just need to find a tool that will save in baseline format. Finding one to read the other two formats is easy.
I have to associate a couple of text parameters (a UUID and a couple of strings representing integers) to a PNG image in a way they can follow the image when the PNG file is passed from an Android device to another through the Net. This is the typical situation in which I would use a couple of custom auxiliary chuncks (similar to EXIF fields ) to store my data inside the PNG image itself.
Maybe it is just me but the only info I was able to find about reading and writing PNG custom metadata from Java code on Android was this SO post:
Writing image metadata in Java, preferably PNG
that even offers some code (quite verbose, as usual with Java).
Those same SO post refers also to PNGJ: http://code.google.com/p/pngj/
To be honest, I would be happy to not use yet another library in this project.
Does anybody know of others ways to write and read text metadata in a PNG file in Android? Maybe a less verbose way... Maybe a way that does not require a separated library....
Any other source of information? Any tutorial? Any example?
I'm open to use a different (but equivalent) image file format, if needed (JPEG, whatever).
A solution working also on iOS and Windows 8 Phone would be a plus but it is not actually required.
I had to do something similar lately, so I had to study the subject.
Android offers you no option of manipulating png metadata. You will have to use an external library (PNGJ seems like a good option).
In my case, since I am making changes to the Android frameworks, I didn't use an external lib, but made changes to skia (the Android graphics library instead).
If you need more info on that I can give you, but I think it's irrelevant in your case.
I wanted to find out how can one capture screencast using java. I know that using Robot class one can get a screenshot but how do I go about capturing it as a video and then uploading it to the server? How exactly would that work?
ideas?
With a pure Java solution, I doubt that it will work, but it depends of course on what your interpretation of "video".
On my desktop with a 1920x1200 resolution, I am able to get about 20 frames per second when using the Java Robot to capture the entire screen. Since each image contains >6 MByte of uncompressed data, I would need more than 1 Gbps bandwidth to transmit the raw data of these images to a server. Most probably, requiring so much bandwidth is not acceptable, so you either have to decrease the number of frames per second or apply some kind of compression to the images.
One possibility is to compress each image using one of the image formats supported by ImageIO. The size of the compressed images will of course depend heavily on what is actually shown on the screen, but the performance of the compressors is not particularly good. Compressing to PNG ought to give the best lossless compression ratio for most desktop content, but at least my computer is only able to process just about 2 frames per second. Using the JPEG compressor with default quality settings reaches about 5 frames per second.
Using common video codecs through an abstraction layer like jffmpeg will probably achieve both better performance and better compression ratio, but I doubt that mainstream video codecs like WMV or H.264 are suitable for common desktop content.
If you really require a pure Java solution (and are not able to use any of the available standalone software, which do what you're asking for), I would make an attempt to implement my own, simple compression algorithm. With common desktop activity, there ought to be very little difference between most consecutive screen shots, so what might work quite well is to transmit the first frame completely and after that implement an algorithm to roughly detect rectangles, in which changes have been made and then transmit only these combined with JPG or preferrably (quality) PNG compression.
Or use Xuggler, a better wrapper for FFmpeg in Java. In fact, the code for capturing the screen and encoding the video is one of the standard tutorials.
I'm also curious about this. https://www.screencast.com/ is currently doing just this with a pure java (or at least straight out of the browser) experience.
You can just use something like Java to a native FFMPEG build, and execute the command line at runtime. Here is an applet that I made that does just that: http://have2chat.net/screencast/
I have downloaded the main capture *.JAR file for the Screencast-O-Matic.com. To download the file:
Go to http://screencast-o-matic.com/jars/ScreencastOMaticRun-1.0.5.jar
Save the file
Extract the contents (I DO NOT intend to use this commercially!)
Canon/Nikon/other cameras save raw output of their sensor in some of their proprietary formats (.CR2, whatever). Is there any Java library designed to read them and convert into manageable BufferedImages?
I don't reqlly care here about fully customizable conversion similar to ufraw or imagemagick, rather something simple that "just works" for rendering simple previews of such images.
I've been where you are, and I feel for you. Your best bet is to use an Adobe or dcraw-based program to create thumbnails automatically. Temporary DNG files using Adobe's converter may be easier to use.
IF you insist on doing it in Java, you're about to run into a mountain of pain. RAW formats change often, have all sorts of crazy nuances and are intentionally hard to work with. Camera makers want you to use THEIR RAW conversion software, to show the camera's abilities at its best and screw Adobe. The guy behind dcraw found that some are camera manufacturers even use encryption now!
The existing Java libraries are poor -- JRawIO has improved since I last looked at it, but it supports only a fraction of the formats that dcraw does. In addition to the listed libraries, the imagero library may provide the ability to display a thumbnail for your image.
From personal experience, don't even think about writing your own RAW file reader.
I tried to do this with a very simple RAW format once (just a solid array of sensor data, 12 bits per pixel). The dcraw source translates badly to Java. You haven't seen such a nightmare of bit-fiddling ever. Took me days to debug problems with byte alignment and endian-ness.
jrawio is a plugin for Java Image I/O. With it you can read the raster data, the thumbnails and the metadata from the raw image file.
nef and cr2 already contains preview images in jpeg. just find the right offset and the right length to extract it...
Laurent Clevy # lclevy.free.fr/raw
Laurent
Unless you want to write you own file parser/loader (sounds fun imho ;) ), perhaps JMagick will help you. I haven't tried it and it might not work given your target platform since JMagick uses JNI.
UPDATE: dcraw looks like a good resource/reference