Issues with Processing OpenCV and QuickTime video capture - java

I'm having difficulty with the Processing OpenCV library:
http://ubaa.net/shared/processing/opencv/index.html
I am running the absolute most basic possible sample code:
import hypermedia.video.*;
OpenCV opencv;
void setup ()
{
opencv = new OpenCV( this );
opencv.capture( width, height );
}
void draw ()
{
opencv.read();
background( opencv.image() );
}
But every time, I get the console message:
SGIdle failed in icvGrabFrame_QT_Cam with error -1
and get no video input. My webcam turns on, but the program hangs.
I have seen others online with this problem, however I have not found a single source. I have a feeling it has something to do with QuickTime's video capture, but I'm not sure.
System info:
Recent MacBook Pro with built-in iSight camera
OS X 10.7.3 (Lion)
QuickTime 10.1
OpenCV 1.1
Processing 1.5.1
Does anyone know what is going on? It's difficult to get to the actual project when the most basic functionality of the library doesn't work...

do a workaround using the video lib and copying the video frames it into openCV layer: http://habu.phpfogapp.com/?p=3

Here is an example of a workaround using standard library to capture image
// OpenCV cannot capture anymore since a iTunes or QuickTime update.
// It returns this error:
// SGIdle failed in icvGrabFrame_QT_Cam with error -1
// This example shows how to use standard video library to capture then
// pass the image to OpenCV.
//
// Pierre Rossel
// 25.2.2013
import hypermedia.video.*;
import processing.video.*;
OpenCV opencv;
Capture video;
int captureW = 640;
int captureH = 480;
void setup ()
{
size(captureW, captureH);
opencv = new OpenCV( this );
opencv.allocate(captureW, captureH);
opencv.threshold(255); // Clears allocated image
video = new Capture(this, captureW, captureH);
video.start();
}
void draw ()
{
if (video.available()) {
video.read();
opencv.copy(video);
}
background( opencv.image() );
}

Related

Android Java - programmatically capture background before app window opens [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to programatically take a screenshot on Android?
How to capture the android device screen content and make an image file using the snapshot data? Which API should I use or where could I find related resources?
BTW:
not camera snapshot, but device screen
Use the following code:
Bitmap bitmap;
View v1 = MyView.getRootView();
v1.setDrawingCacheEnabled(true);
bitmap = Bitmap.createBitmap(v1.getDrawingCache());
v1.setDrawingCacheEnabled(false);
Here MyView is the View through which we need include in the screen. You can also get DrawingCache from of any View this way (without getRootView()).
There is also another way.. If we having ScrollView as root view then its better to use following code,
LayoutInflater inflater = (LayoutInflater) this.getSystemService(LAYOUT_INFLATER_SERVICE);
FrameLayout root = (FrameLayout) inflater.inflate(R.layout.activity_main, null); // activity_main is UI(xml) file we used in our Activity class. FrameLayout is root view of my UI(xml) file.
root.setDrawingCacheEnabled(true);
Bitmap bitmap = getBitmapFromView(this.getWindow().findViewById(R.id.frameLayout)); // here give id of our root layout (here its my FrameLayout's id)
root.setDrawingCacheEnabled(false);
Here is the getBitmapFromView() method
public static Bitmap getBitmapFromView(View view) {
//Define a bitmap with the same size as the view
Bitmap returnedBitmap = Bitmap.createBitmap(view.getWidth(), view.getHeight(),Bitmap.Config.ARGB_8888);
//Bind a canvas to it
Canvas canvas = new Canvas(returnedBitmap);
//Get the view's background
Drawable bgDrawable =view.getBackground();
if (bgDrawable!=null)
//has background drawable, then draw it on the canvas
bgDrawable.draw(canvas);
else
//does not have background drawable, then draw white background on the canvas
canvas.drawColor(Color.WHITE);
// draw the view on the canvas
view.draw(canvas);
//return the bitmap
return returnedBitmap;
}
It will display entire screen including content hidden in your ScrollView
UPDATED AS ON 20-04-2016
There is another better way to take screenshot.Here I have taken screenshot of WebView.
WebView w = new WebView(this);
w.setWebViewClient(new WebViewClient()
{
public void onPageFinished(final WebView webView, String url) {
new Handler().postDelayed(new Runnable(){
#Override
public void run() {
webView.measure(View.MeasureSpec.makeMeasureSpec(
View.MeasureSpec.UNSPECIFIED, View.MeasureSpec.UNSPECIFIED),
View.MeasureSpec.makeMeasureSpec(0, View.MeasureSpec.UNSPECIFIED));
webView.layout(0, 0, webView.getMeasuredWidth(),
webView.getMeasuredHeight());
webView.setDrawingCacheEnabled(true);
webView.buildDrawingCache();
Bitmap bitmap = Bitmap.createBitmap(webView.getMeasuredWidth(),
webView.getMeasuredHeight(), Bitmap.Config.ARGB_8888);
Canvas canvas = new Canvas(bitmap);
Paint paint = new Paint();
int height = bitmap.getHeight();
canvas.drawBitmap(bitmap, 0, height, paint);
webView.draw(canvas);
if (bitmap != null) {
try {
String filePath = Environment.getExternalStorageDirectory()
.toString();
OutputStream out = null;
File file = new File(filePath, "/webviewScreenShot.png");
out = new FileOutputStream(file);
bitmap.compress(Bitmap.CompressFormat.PNG, 50, out);
out.flush();
out.close();
bitmap.recycle();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}, 1000);
}
});
Hope this helps..!
AFAIK, All of the methods currently to capture a screenshot of android use the /dev/graphics/fb0 framebuffer. This includes ddms. It does require root to read from this stream. ddms uses adbd to request the information, so root is not required as adb has the permissions needed to request the data from /dev/graphics/fb0.
The framebuffer contains 2+ "frames" of RGB565 images. If you are able to read the data, you would have to know the screen resolution to know how many bytes are needed to get the image. each pixel is 2 bytes, so if the screen res was 480x800, you would have to read 768,000 bytes for the image, since a 480x800 RGB565 image has 384,000 pixels.
For newer Android platforms, one can execute a system utility screencap in /system/bin to get the screenshot without root permission.
You can try /system/bin/screencap -h to see how to use it under adb or any shell.
By the way, I think this method is only good for single snapshot.
If we want to capture multiple frames for screen play, it will be too slow.
I don't know if there exists any other approach for a faster screen capture.
[Based on Android source code:]
At the C++ side, the SurfaceFlinger implements the captureScreen API. This is exposed over the binder IPC interface, returning each time a new ashmem area that contains the raw pixels from the screen. The actual screenshot is taken through OpenGL.
For the system C++ clients, the interface is exposed through the ScreenshotClient class, defined in <surfaceflinger_client/SurfaceComposerClient.h> for Android < 4.1; for Android > 4.1 use <gui/SurfaceComposerClient.h>
Before JB, to take a screenshot in a C++ program, this was enough:
ScreenshotClient ssc;
ssc.update();
With JB and multiple displays, it becomes slightly more complicated:
ssc.update(
android::SurfaceComposerClient::getBuiltInDisplay(
android::ISurfaceComposer::eDisplayIdMain));
Then you can access it:
do_something_with_raw_bits(ssc.getPixels(), ssc.getSize(), ...);
Using the Android source code, you can compile your own shared library to access that API, and then expose it through JNI to Java. To create a screen shot form your app, the app has to have the READ_FRAME_BUFFER permission.
But even then, apparently you can create screen shots only from system applications, i.e. ones that are signed with the same key as the system. (This part I still don't quite understand, since I'm not familiar enough with the Android Permissions system.)
Here is a piece of code, for JB 4.1 / 4.2:
#include <utils/RefBase.h>
#include <binder/IBinder.h>
#include <binder/MemoryHeapBase.h>
#include <gui/ISurfaceComposer.h>
#include <gui/SurfaceComposerClient.h>
static void do_save(const char *filename, const void *buf, size_t size) {
int out = open(filename, O_RDWR|O_CREAT, 0666);
int len = write(out, buf, size);
printf("Wrote %d bytes to out.\n", len);
close(out);
}
int main(int ac, char **av) {
android::ScreenshotClient ssc;
const void *pixels;
size_t size;
int buffer_index;
if(ssc.update(
android::SurfaceComposerClient::getBuiltInDisplay(
android::ISurfaceComposer::eDisplayIdMain)) != NO_ERROR ){
printf("Captured: w=%d, h=%d, format=%d\n");
ssc.getWidth(), ssc.getHeight(), ssc.getFormat());
size = ssc.getSize();
do_save(av[1], pixels, size);
}
else
printf(" screen shot client Captured Failed");
return 0;
}
You can try the following library: Android Screenshot Library (ASL) enables to programmatically capture screenshots from Android devices without requirement of having root access privileges. Instead, ASL utilizes a native service running in the background, started via the Android Debug Bridge (ADB) once per device boot.
According to this link, it is possible to use ddms in the tools directory of the android sdk to take screen captures.
To do this within an application (and not during development), there are also applications to do so. But as #zed_0xff points out it certainly requires root.
Framebuffer seems the way to go, it will not always contain 2+ frames like mentioned by Ryan Conrad. In my case it contained only one. I guess it depends on the frame/display size.
I tried to read the framebuffer continuously but it seems to return for a fixed amount of bytes read. In my case that is (3 410 432) bytes, which is enough to store a display frame of 854*480 RGBA (3 279 360 bytes). Yes, the frame in binary outputed from fb0 is RGBA in my device. This will most likely depend from device to device. This will be important for you to decode it =)
In my device /dev/graphics/fb0 permissions are so that only root and users from group graphics can read the fb0. graphics is a restricted group so you will probably only access fb0 with a rooted phone using su command.
Android apps have the user id (uid) app_## and group id (guid) app_## .
adb shell has uid shell and guid shell, which has much more permissions than an app.
You can actually check those permissions at /system/permissions/platform.xml
This means you will be able to read fb0 in the adb shell without root but you will not read it within the app without root.
Also, giving READ_FRAME_BUFFER and/or ACCESS_SURFACE_FLINGER permissions on AndroidManifest.xml will do nothing for a regular app because these will only work for 'signature' apps.
if you want to do screen capture from Java code in Android app AFAIK you must have Root provileges.

Trying to set a texture from Java plugin in Unity 4.6

What i'm trying to do:
Pull an image from sd-card on phone using Java Plugin.
Unity passes a texture ID to plugin.
Plugin uses opengl to assign the image to the texture in Unity through the ID.
Will (eventually) be used to play a video clip from the phone in Unity, for now, it's just trying to change a texture outside of unity.
My issue:
When i call the method in the plugin, passing texture.GetNativeTextureID() into it, the texture does not change. I'm currently only using a simple black 50x50 texture for testing, and the original texture is a flat white.
I'm worried that i've missed something significant, as this is my first time working with Gl calls in java. Much of the answers to similar problems involve using native C++ instead of Java, but I can't find a concrete answer saying that C++ must be used. I'd like to do my best to avoid writing another set of plugins and plugin handlers for C++, but if it's the most efficient/only way to get this working, i'll do it as i'm not unfamiliar with OpenGL and C++
Code:
The plugin method is called from OnPreRender() in a script attached to the main camera:
if (grabTex) {
int texPtr = m_VideoTex.GetNativeTextureID();
Debug.Log( "texPtr = " + texPtr );
m_JVInterface.SetTex( texPtr );
}
m_VideoTex is a basic Texture2D( 50, 50 ) with all pixels set to white, attached to the diffuse shader on the quad in the scene.
The Java plugin code is as follows:
public void SetTexture(Context cont, int _texPointer) {
if (_texPointer != 0) {
final BitmapFactory.Options options = new BitmapFactory.Options();
options.inScaled = false;
options.inJustDecodeBounds = false;
final Bitmap bitmap = BitmapFactory.decodeFile("/storage/emulated/0/Pictures/black.jpg", options);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, _texPointer);
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bitmap, 0);
Log.i("VideoHandler", "Recieved ID: " + _texPointer);
bitmap.recycle();
}
}
This is most likely a problem with the OpenGL Context. The easiest way would be to send the texture as raw bytes to Unity and then upload as texture inside Unity.

IP Webcam on OpenCV for Java

I am using IP Webcam APP for android and it is streaming MJPEG video through the local url :
http://192.168.0.2:8080/video
I was able to show the video using VLC player and this piece of code in C++.
On the OpenCV 2.2 I opened the url using:
VideoCapture cap;
cap.open("http://192.168.0.2:8080/video?dummy=param.mjpg");
It worked in C++, but I want it to work in Java. I was able to run OpenCV2.4.9 using Java when taking pictures from my built in webcam. This is my code for taking the images from a url in Java.
System.loadLibrary("opencv_java249");
VideoCapture capture = new VideoCapture();
capture.open("http://192.168.0.2:8080/video?dummy=param.mjpg");
But the capture.open does not open the streaming and I could not debug it properly. I know that it might be a issue with the ffmpeg, since it works on OpenCV2.2. I also know that my OpenCV2.2 is specific for MS 2010 and might be more complete.
Would it help if I compile the OpenCV2.4.9 from sources? Is there a file that I could add to solve that problem? Is there another way of receiving the video from the IP camera and using on OpenCV?
I took a while to figure it out. I could not receive the stream directly from OpenCVJava.I downloaded
http://www.mediafire.com/download/ayxwnwnqv3mpg39/javacv-0.7-bin.zip http://www.mediafire.com/download/2rkk0rjwxov7ale/javacv-0.7-cppjars.zip
Which I believe to be a java wrapper into OpenCV in C. I took this link from this video.
htttp://www.youtube.com/watch?v=mIYaHCyZICI
After unziping the zip I added the jars into my project and Used this code:
package javaapplication7;
import java.io.IOException;
import com.googlecode.javacv.OpenCVFrameGrabber;
import com.googlecode.javacv.CanvasFrame;
import com.googlecode.javacv.cpp.opencv_core.IplImage;
public class JavaApplication7 {
public static void main(String[] args) throws Exception {
OpenCVFrameGrabber grabber = new OpenCVFrameGrabber("http://192.168.0.2:8080/video?dummy=param.mjpg");
grabber.setFormat("mjpeg");
grabber.start();
for (int k=0; k<20000; k++){
System.out.print(k);
}
IplImage frame = grabber.grab();
CanvasFrame canvasFrame = new CanvasFrame("Camera");
canvasFrame.setCanvasSize(frame.width(), frame.height());
while (canvasFrame.isVisible() && (frame = grabber.grab()) != null) {
canvasFrame.showImage(frame);
}
grabber.stop();
canvasFrame.dispose();
System.exit(0);
}
}
Which I got from:
htttp://stackoverflow.com/questions/14251290/cvcreatefilecapture-error-could-not-create-camera-capture-with-javacv
It takes 15-20 seconds to start catching the streaming. But I was impressed with the delay which is much smaller than VLC. It is 1-2 seconds comparing to 3-4 seconds on VLC. I would like to upvote the guy who I took the answer from but I dont have enough reputation/
I also bumped into the same problem as you but the easiest method i figured out was to use droid cam instead of the Ip webcam app.Check it out here

How can I remove background from an (video)image using OpenCV and Java?

I just started learning OpneCV and started my project in Java. As Java wrapper of OpenCV is released recently, there isn't much documentation available.
I am trying to separate the background and foreground from video captured through webcam. I tried using the BackgroundSubtractorMog class in java but failed to get the desired output.
Here is my code:
VideoCapture capture = new VideoCapture(0);
Mat camImage = new Mat();
if (capture.isOpened()) {
while (true) {
capture.read(camImage);
BackgroundSubtractorMOG backgroundSubtractorMOG=new BackgroundSubtractorMOG();
Mat fgMask=new Mat();
backgroundSubtractorMOG.apply(camImage, fgMask,0.1);
Mat output=new Mat();
camImage.copyTo(output,fgMask);
displayImageOnScreen(output);
}
}
This code just gives a blackscreen output.
Move this line:
BackgroundSubtractorMOG backgroundSubtractorMOG=new BackgroundSubtractorMOG();
Out of the loop.
unfortunately, as of now, you can't use any of the BackgroundSubtractorXXX from java
(problem with the wrappers, the code is there, but the creator function is missing)
there's a pull request for this already, hope it will get accepted soon.

converting webpage into jpeg image using java

I am building a web application, in Java, where i want the whole screenshot of the webpage, if i give the URL of the webpage as input.
The basic idea i have is to capture the display buffer of the rendering component..I have no idea of how to do it..
plz help..
There's a little trick I used for this app:
count down demo app http://img580.imageshack.us/img580/742/capturadepantalla201004wd.png
Java application featuring blog.stackoverflow.com page ( click on image to see the demo video )
The problem is you need to have a machine devoted to this.
So, the trick is quite easy.
Create an application that takes as
argument the URL you want to fetch.
Then open it with Desktop.open( url
) that will trigger the current
webbrowser.
And finally take the screenshot with
java.awt.Robot and save it to diks.
Something like:
class WebScreenShot {
public static void main( String [] args ) {
Desktop.getDesktop().open( args[0] );
Robot robot = new Robot();
Image image = robot.createScreenCapture( getScreenResolutionSize() );
saveToDisk( image );
}
}
This solution is far from perfect, because it needs the whole OS, but if you can have a VM devoted to this app, you can craw the web and take screenshots of it quite easy.
The problem of having this app as a non-intrusive app is that up to this date, there is not a good html engine renderer for Java.
For a pure-java solution that can scale to support concurrent rendering, you could use a java HTML4/CSS2 browser, such as Cobra, that provides a Swing component for the GUI. When you instantiate this component, you can call it's paint(Graphics g) method to draw itself into an off-screen image
E.g.
Component c = ...; // the browser component
BufferedImage bi = new BufferedImage(c.getWidth(), c.getHeight(), TYPE_INT_RGB)
Graphics2d g = bi.createGraphics();
c.paint(g);
You can then use the java image API to save this as a JPG.
JPEGImageEncoder encoder = JPEGCodec.createEncoder(new FileOutputStream("screen.jpg"));
enncoder.encode(bi); // encode the buffered image
Java-based browsers typically pale in comparison with the established native browsers. However, as your goal is static images, and not an interactive browser, a java-based browser may be more than adequate in this regard.

Categories

Resources