How do I extract all the frames from a video in Java? - java

I'm trying to grab the frames from a video and put them into a collection of BufferedImage so I can use the images later. I tried to use the code in the second answer for this question, but it wasn't really helpful because the code they provided only grabs the first frame. How would I extract all the frames and put them into a collection?

So, a little bit of Googling (and reading the source code), was I able to hobble together this basic concept.
The "obvious" solution was to loop over the frames in the video and extract them, the "un-obvious" solution was "how"?!
The two methods you need are FFmpegFrameGrabber#getLengthInFrames and FFmpegFrameGrabber#setFrameNumber, from there it's just a simple method of converting the current frame and writing it to a file (which has already been demonstrated from the previous question)
File videoFile = new File("Storm - 106630.mp4");
FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(videoFile.getAbsoluteFile());
frameGrabber.start();
Java2DFrameConverter c = new Java2DFrameConverter();
int frameCount = frameGrabber.getLengthInFrames();
for (int frameNumber = 0; frameNumber < frameCount; frameNumber++) {
System.out.println("Extracting " + String.format("%04d", frameNumber) + " of " + String.format("%04d", frameCount) + " frames");
frameGrabber.setFrameNumber(frameNumber);
Frame f = frameGrabber.grab();
BufferedImage bi = c.convert(f);
ImageIO.write(bi, "png", new File("Frame " + String.format("%04d", frameNumber) + "-" + String.format("%04d", frameCount) + ".png"));
}
frameGrabber.stop();
Now, obviously, you could store the images into some kind of List in memory, just beware, depending on the size and length of the original video, this might run you into memory issues.

You can use OpenCV to extract frames in a video, try the below code. input is the video file path of your computer
VideoCapture cap = new VideoCapture();
String output = "resources/output";
cap.open(input);
int video_length = (int) cap.get(Videoio.CAP_PROP_FRAME_COUNT);
int frames_per_second = (int) cap.get(Videoio.CAP_PROP_FPS);
Mat frame = new Mat();
if (cap.isOpened()) {
System.out.println("Video is opened");
System.out.println("Number of Frames: " + video_length);
System.out.println(frames_per_second + " Frames per Second");
System.out.println("Converting Video to images...");
cap.read(frame);
int frame_number = 0;
while (cap.read(frame)) {
Imgcodecs.imwrite(output + "/" + frame_number + ".jpg", frame);
frame_number++;
}
cap.release();
processingStatus = video_length + " Frames extracted";
System.out.println(processingStatus);
} else {
processingStatus = "Failed";
System.out.println(processingStatus);
}

Related

OpenCV(4.3.0): error: (-215:Assertion failed) !_src.empty() in function 'GaussianBlur'

I am new to OpenCV. I am trying to apply a Gaussian filter to the images. The below java code is working fine for one image but for all other images in the same or other location in my machine, it is throwing an error.
static int width = 49;
static int height = 49;
public static void gausianBlur(){
try {
long startTime = System.currentTimeMillis();
nu.pattern.OpenCV.loadShared();
File f = new File("/Users/shankysharma/Desktop/login.png");
System.out.println("is file exists?: " + f.exists() + " path: " + f.getAbsolutePath());
//Reading the Image from the file and storing it in to a Matrix object
Mat src = Imgcodecs.imread(f.getAbsolutePath());
//Creating an empty matrix to store the result
//Mat dst = new Mat();
Mat dst = new Mat(src.rows(), src.cols(), src.type());
System.out.println(src + " rows: " + src.rows());
Imgproc.GaussianBlur(src, dst,new Size(width, height), 0);
Imgcodecs.imwrite(gausian_output + "_" + width + "x" + height + ".png", dst);
long endTme = System.currentTimeMillis();
System.out.println("Image processed for gausian blur. Total time : " + (endTme-startTime) + " milliseconds");
} catch (Exception e) {
System.out.println("Error:" + e.getMessage());
}
}
Output:
is file exists?: true path: /Users/shankysharma/Desktop/login.png
Mat [ -1*-1*CV_8UC1, isCont=false, isSubmat=false, nativeObj=0x7fa6f9c16030, dataAddr=0x0 ] rows: 0
Error:cv::Exception: OpenCV(4.3.0) /Users/jason/Projects/openpnp/opencv/opencv/opencv-4.3.0/modules/imgproc/src/smooth.dispatch.cpp:606: error: (-215:Assertion failed) !_src.empty() in function 'GaussianBlur'
I am using opencv maven library:
<dependency>
<groupId>org.openpnp</groupId>
<artifactId>opencv</artifactId>
<version>4.3.0-3</version>
</dependency>
There seems to be an issue while loading the image (Mat src = Imgcodecs.imread(f.getAbsolutePath());) as src matrix is not getting created. but not sure what is the root cause. I have checked the name and location of the image, those are fine.

how to take color screenshot using selenium java

screenshots is getting captured like below images but i want to capture it in actual colors also i don't know the reason behind it can anybody tell me why this is happening?
((JavascriptExecutor)driver).executeScript("window.scrollBy(0,131)");
int yPosition = Decision_Maker.getLocation().getY();
for(int j = 1; j<=list.size(); j++)
{
// Get entire page screenshot
File screenshots = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
BufferedImage fullImg = ImageIO.read(screenshots);
ImageIO.read(screenshots).getHeight();
// Get width and height of the element
int eleWidth = Decision_Maker.getSize().getWidth();
int eleHeight = Decision_Maker.getSize().getHeight();
//Crop the entire page screenshot to get only element screenshot
BufferedImage eleScreenshot= fullImg.getSubimage(465, 190,eleWidth,eleHeight);
ImageIO.write(eleScreenshot, "jpg", screenshots);
//Scroll vertically to the element
JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript("window.scroll (0, " + yPosition + ") ");
yPosition = yPosition + eleHeight;
//wait for sometime
Thread.sleep(3000);
//File Location
String location = "E:\\Automation\\Screenshots\\";
Thread.sleep(3000);
//capture screenshot
FileUtils.copyFile(screenshots, new File (location + "img" + i + ".jpg"));
Thread.sleep(3000L);
}
You can create utils class to take screenshot and use it
public static void TakeScreenshot(String filename) throws IOException
{
File file= ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(file, new File("path to store screenshot" + filename + ".jpg"));
}

Java Xuggler Metadata List of Chapters MP4/M4V Video

I'm trying to use Xuggler like FFMPEG Metadata Wrapper (I just need the list of Chapters of MP4/M4V Video).
So far I have not been able to find a solution.
Can anyone help me?
I was only able to get the following information:
final String filename = "...path...";
IContainer container = IContainer.make();
int result = container.open(filename, IContainer.Type.READ, null);
if (result < 0)
throw new RuntimeException("Failed to open media file");
int numStreams = container.getNumStreams();
long duration = container.getDuration();
long fileSize = container.getFileSize();
long bitRate = container.getBitRate();
System.out.println("Number of streams: " + numStreams);
System.out.println("Duration (ms): " + duration);
System.out.println("File Size (bytes): " + fileSize);
System.out.println("Bit Rate: " + bitRate);
for (int i = 0; i < numStreams; i++) {
IStream stream = container.getStream(i);
IStreamCoder coder = stream.getStreamCoder();
System.out.println("*** Start of Stream Info ***");
System.out.printf("stream %d: ", i);
System.out.printf("type: %s; ", coder.getCodecType());
System.out.printf("codec: %s; ", coder.getCodecID());
System.out.printf("duration: %s; ", stream.getDuration());
System.out.printf("start time: %s; ", container.getStartTime());
System.out.printf("timebase: %d/%d; ", stream.getTimeBase().getNumerator(),
stream.getTimeBase().getDenominator());
System.out.printf("coder tb: %d/%d; ", coder.getTimeBase().getNumerator(),
coder.getTimeBase().getDenominator());
System.out.println();
if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_AUDIO) {
System.out.printf("sample rate: %d; ", coder.getSampleRate());
System.out.printf("channels: %d; ", coder.getChannels());
System.out.printf("format: %s", coder.getSampleFormat());
} else if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO) {
System.out.printf("width: %d; ", coder.getWidth());
System.out.printf("height: %d; ", coder.getHeight());
System.out.printf("format: %s; ", coder.getPixelType());
System.out.printf("frame-rate: %5.2f; ", coder.getFrameRate().getDouble());
}
System.out.println();
System.out.println("*** End of Stream Info ***");
UPDATE 07.06.2017
I just tried it with VLCJ, but still I can not get the list of chapters.
File file = new File("ia_ISL_13_r720P.m4v");
NativeLibrary.addSearchPath(RuntimeUtil.getLibVlcLibraryName(), "vlc64/");
Native.loadLibrary(RuntimeUtil.getLibVlcLibraryName(), LibVlc.class);
MediaPlayerFactory mpf = new MediaPlayerFactory();
EmbeddedMediaPlayer emp = mpf.newEmbeddedMediaPlayer();
MediaMeta mediaMeta = mpf.getMediaMeta(file.getAbsolutePath(), true);
MediaMetaData asMediaMetaData = mediaMeta.asMediaMetaData();
System.out.println(asMediaMetaData.getAlbum());
System.out.println(asMediaMetaData.getArtist());
System.out.println(asMediaMetaData.getTitle());
emp.prepareMedia(file.getAbsolutePath());
emp.play();
emp.nextChapter(); // -> GO NEXT CHAPTER - SUCCESS
List<List<String>> allChapterDescriptions = emp.getAllChapterDescriptions();
for (List<String> list : allChapterDescriptions) {
for (String string : list) {
System.out.println(string);
}
}
have you tested Mp4Chapters? Is for .net but is open source and maybe helps.
Am using it in a C# project I guess similar to what you are doing, very similar...
Here am feeding a listbox with mp4 and m4v chapters, with old mp4 and m4v files seams to be working fine, but am having issues with new ones (2014 ->)
using Mp4Chapters;
// more code here...
private void readChapters(string inputFull)
{
using (var str = File.OpenRead(this.inputFull))
{
var extractor = new ChapterExtractor(new StreamWrapper(str));
extractor.Run();
// build the listbox
foreach (var c in extractor.Chapters ?? new ChapterInfo[0])
{
this.lb_chapters.Items.Add(new { chapterTime = c.Time, chapterName = c.Name });
}
}
}
Other options is to parse the video file using ffmpeg and read the result:
ffmpeg -i ia_ISL_13_r720P.m4v -f ffmetadata metadata.txt
I hope this helps, let me know.
(sorry for the english, is not my main language)

change text of slide in java using aspose api ppt

public void save()
{ Presentation pres = new Presentation(filename);
ISlide slide = pres.getSlides().get_Item(0);
IShape shape= null;
for (int i = 0 ; i < slide.getShapes().size() ; i++)
{ shape = slide.getShapes().get_Item(i);
if (shape.getPlaceholder() != null)
{
((IAutoShape)shape).getTextFrame().setText(txtArea.getText());
}
}
pres.save(filename,SaveFormat.Ppt);
}
This code is for changing the text but it's not working. I have used two APIs at a time, display code is below:
public void Display(int currentPage, String source)
{
try {
// Create a slideshow object; this creates an underlying POIFSFileSystem object for us
SlideShow ppt = new SlideShow(new HSLFSlideShow(source));
current=currentPage;
// Get all of the slides from the PPT file
Slide[] slides = ppt.getSlides();
Dimension pgsize = ppt.getPageSize();
all = slides.length;
String temp="";
lblPage.setText(currentPage+" / "+all);
BufferedImage img = new BufferedImage(pgsize.width, pgsize.height, BufferedImage.TYPE_INT_RGB);
Graphics2D graphics = img.createGraphics();
graphics.setPaint(Color.white);
graphics.fill(new Rectangle2D.Float(0, 0, pgsize.width, pgsize.height));
//render
slides[currentPage-1].draw(graphics);
//save the output
/*FileOutputStream out = new FileOutputStream("slide-" + (i + 1) + ".png");
javax.imageio.ImageIO.write(img, "png", out);
out.close();
//ImageIcon icon = new ImageIcon("slide-" + (i + 1) + ".png");*/
ImageIcon icon = new ImageIcon(img);
lblPresentasi.setIcon(icon);
// Obtain metrics about the slide: its number and name
int number = slides[currentPage-1].getSlideNumber();
String title = slides[currentPage-1].getTitle();
// Obtain the embedded text in the slide
TextRun[] textRuns = slides[currentPage-1].getTextRuns();
System.out.println("Slide " + number + ": " + title);
System.out.println("\tText Runs");
txtArea.setText("Slide : " + number + " Title : " + title + "\n");
for (int j = 0; j < textRuns.length; j++) {
// Display each of the text runs present on the slide
System.out.println("\t\t" + j + ": " + textRuns[j].getText());
temp=txtArea.getText();
txtArea.setText(temp+"\t\t" + textRuns[j].getText() + "\n");
}
// Obtain the notes for this slide
System.out.println("\tNotes: ");
Notes notes = slides[currentPage-1].getNotesSheet();
if (notes != null) {
// Notes are comprised of an array of text runs
TextRun[] notesTextRuns = notes.getTextRuns();
for (int j = 0; j < notesTextRuns.length; j++) {
System.out.println("\t\t" + notesTextRuns[j].getText());
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
Can anyone please help, I am trying to make a simple powerpoint editor in Java.
I want to change text in textarea press save button so text has to change and call display function.
I have observed your sample code and like to share that as far as code related to Aspose.Slides is concerned, there is no issue and it is perfectly right. You can verify this by opening the saved presentation with changed text in PowerPoint. If you encounter any issue in this, we will be glad to help you further. You may also contact us in Aspose.Slides support forums as well.
I work with Aspose as Developer evangelist.

Why does Java app crash after processing x number of files?

I'm only asking this as a last resort. I'm stumped.
I have written a small app which performs very simple image processing. It is made using JDK 1.8, written in Netbeans 8.0.1 and is running on Debian Linux.
The application captures a large number of individual frames at a certain framerate, set by the user, by calling the 'streamer' webcam program via a process builder. Once it has begun capturing, it begins to translate the frames into RGB values and checks whether or not any pixels are above a user defined threshold. If no pixels exceed this threshold, it simply deletes the frame. If any pixels do exceed it, it moves the frame to a different folder for user inspection. This all works fine. It keeps up with even relatively high framerates and selects appropriate frames as expected. However, when the number of frames processed reaches around 1500 (or fewer for lower framerates), the program is freezing. I've tried issuing the commands to streamer manually at the command line, and it seems perfectly capable of producing as many as required, so I have to assume the issue is with my coding. The images are only small (320x240). Am I somehow maxxing out the available memory (I am getting no errors, just freezing).
The purpose of this program is to detect cosmic ray impacts on a CMOS sensor, part of a friend's dissertation. If I can't get this working reliably, the poor kid's going to have to go through the footage manually!
The code is attached below. Apologies for the length of the code, but as all of it is fairly crucial, I didn't want to omit anything.
package cosmicraysiii;
import java.io.File;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.JOptionPane;
public class CosmicRaysIII {
public static int dark = 0;//Dark current determined by averaging
public static int tol = 0;//Tolerance set by user
public static int frames = 0;//Total number of frames set by user
public static int runs = 0;
public static int rate = 0;//Framerate set by user
public static int time = 0;//Total time calculated
public static String zeros = "";
public static int ready = 0;
public static void main(String[] args) throws IOException, InterruptedException {
//Get directory ID from user
String id = JOptionPane.showInputDialog("Enter a folder name for detections (no ., in name):");
//Get dark current
Dark d = new Dark();
dark = d.getCurrent();
//Get tolerance from user, will be added to dark current.
String t = JOptionPane.showInputDialog("Dark Current = " + dark + "/255\n"
+ "Enter a tolerance (integer values only).\n "
+ "This will be added to the dark current:");
tol = Integer.parseInt(t) + dark;
//Get number of frames from user
String fs = JOptionPane.showInputDialog("Enter the total number of frames required (Mulitples of 500 only):");
frames = Integer.parseInt(fs);
runs = frames / 500;
//Get framerate from user
String r = JOptionPane.showInputDialog("Enter the framerate required:");
rate = Integer.parseInt(r);
//Determine duration
time = (int) Math.round(frames / rate);
//Provide summary for user and request permission to continue
int secs = time % 60;
int mins = (time - secs) / 60;
int hrs = (mins - (mins % 60)) / 60;
if (hrs >= 1) {
mins = mins % 60;
}
String theMessage = "The following parameters have been set:\n"
+ "Tolerance (including dark current): " + tol + "\n"
+ "Frames: " + frames + "\n"
+ "Frame rate: " + rate + " fps\n"
+ "Total capture time: " + time + " sec\n"
+ " " + hrs + " h " + mins + " m " + secs + " s\n"
+ "\n"
+ "Would you like to proceed?";
int result = JOptionPane.showConfirmDialog(null, theMessage, "Continue?", JOptionPane.OK_CANCEL_OPTION);
if (result == 2) {
System.exit(0);
}
//Create directory for data acquisition
ProcessBuilder pb1 = new ProcessBuilder("mkdir", "data");
pb1.start();
//Establish array of filenames
String[] filenames = new String[frames];
//Fill filenames array with filenames
//Taking into consideration that the filename
//will have a varying number of zeros appended
//before the frame number, dependent on the
//order of the frame number
for (int i = 0; i < frames; i++) {
if (i < 10) {
zeros = "00000000";
} else if (i >= 10 && i < 100) {
zeros = "0000000";
} else if (i >= 100 && i < 1000) {
zeros = "000000";
} else if (i >= 1000 && i < 10000) {
zeros = "00000";
} else if (i >= 10000 && i < 100000) {
zeros = "0000";
} else if (i >= 100000 && i < 1000000) {
zeros = "000";
} else if (i >= 1000000 && i < 10000000) {
zeros = "00";
} else if (i >= 10000000 && i < 100000000) {
zeros = "0";
} else {
zeros = "";
}
filenames[i] = "./data/frame" + zeros + i + ".ppm";
}
//Begin data acquisition
new Thread(new Runnable() {
public void run() {
try {
//Capture images
ProcessBuilder pb2 = new ProcessBuilder("streamer", "-t", Integer.toString(frames), "-r", Integer.toString(rate), "-p", "0", "-o", "./data/frame000000000.ppm");
Process p = pb2.start();
p.waitFor();
ready = 1;
} catch (IOException | InterruptedException ex) {
Logger.getLogger(CosmicRaysIII.class.getName()).log(Level.SEVERE, null, ex);
}
}
}).start();
//Sleep to allow some image capture to prevent thread disordering
Thread.sleep(3000);
//Check array size
System.out.println("Array size: " + filenames.length);
//Conduct image analysis
new Thread(new Runnable() {
public void run() {
int done = 0;
int donea = 0;
while (ready == 0) {
for (int i = 0; i < frames; i++) {
File f = new File(filenames[i]);
if (f.exists() && !filenames[i].equals("")) {//Check file still exists
try {
//Perform analysis steps
Analysis a = new Analysis();
//STEP 1: Convert file from P6 to P3
String newfile = a.convert(filenames[i], zeros, i);
//STEP 2: Read file
a.read(newfile, tol, i, id);
filenames[i] = "";
done++;
} catch (IOException ex) {
Logger.getLogger(CosmicRaysIII.class.getName()).log(Level.SEVERE, null, ex);
}
}
if (done > donea) {
System.out.println(done + " files processed");
donea = done;
}
}
}
}
}).start();
}
}
Then the Analyse.java class is as follows:
package cosmicraysiii;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
public class Analysis {
public String convert(String ofile, String zeros, int number) throws IOException {
//Create new file name
String nfile = "./proc/frame" + zeros + number + ".ppm";
//Ensure process directory exists
ProcessBuilder mkdir = new ProcessBuilder("mkdir", "proc");
mkdir.start();
//Convert file to P3 PPM (RGB format) and move to process folder
ProcessBuilder convert = new ProcessBuilder("convert", ofile, "-compress", "none", nfile);
convert.start();
//Delete original file
ProcessBuilder del = new ProcessBuilder("sudo", "rm", ofile);
del.start();
//Return new filename
return nfile;
}
public void read(String filename, int tol, int ix, String id) throws FileNotFoundException, IOException {
int move = 0;
//Make directory for hits
ProcessBuilder mkdir = new ProcessBuilder("mkdir", "hits" + id);
mkdir.start();
//Open reader to read file
File f = new File(filename);
if (f.exists()) {
BufferedReader br = new BufferedReader(new FileReader(filename));
String line;
//To eliminate header
int x = 0;
//Iterate through text to find abnormal pixels
while ((line = br.readLine()) != null) {
x++;
String[] pixrgb = line.split("\\ ");
//Iterate through pixels on each line
for (int i = 0; i < pixrgb.length; i++) {
if (x >= 4) {//Eliminate header
//Check each pixel value
try {
int pixval = Integer.parseInt(pixrgb[i]);
if (pixval > tol) {
move = 1;
break;
}
} catch (NumberFormatException ne) {
}
}
}
}
if (move == 1) {
//Move file to hits folder
ProcessBuilder pb3 = new ProcessBuilder("sudo", "cp", filename, "./hits" + id + "/detection" + ix + ".ppm");
pb3.start();
//Delete original file
ProcessBuilder del = new ProcessBuilder("sudo", "rm", filename);
del.start();
}
}
//Delete original file
ProcessBuilder del = new ProcessBuilder("sudo", "rm", filename);
del.start();
}
}
I appreciate this is quite a lengthly chunk of code to be posting. Really appreciate any help that can be given.
G
OK I have managed to solve this by completely overhauling the analysis process. Firstly, rather than converting the image into a P3 .ppm file, I now examine the pixels directly from the image using BufferedReader. Then, I stopped looping through the file list repeatedly. Instead, the loop which calls Analysis() just runs through the list of filenames once. If it encounters a file which does not yet exist, it does Thread.sleep(500) and then tries again. I have now successfully run a batch of 50,000 frames without incident, and there is now much less of a drain on memory thanks to the improved process. Just thought I should place this answer up here in case anyone comes across it. I may post code if anyone wants it.

Categories

Resources