I've been working on an app that detects changes in pupil size. However, at the moment I'm stuck on a section of code using the Canny function in the openCV library.
private void runOpenCVCode() {
try {
File imageFile = new File(locations.get(0));
Uri uri = Uri.fromFile(imageFile);
Bitmap bitmap = MediaStore.Images.Media.getBitmap(getContentResolver(), uri);
Mat mat = Mat.zeros(100,400, CvType.CV_8UC3);
Utils.bitmapToMat(bitmap, mat);
//check mat
Bitmap bm = Bitmap.createBitmap(mat.cols(), mat.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mat, bm);
pic.setImageBitmap(bm);
Mat gray = new Mat(mat.size(), CvType.CV_8UC1);
Imgproc.cvtColor(mat, gray, Imgproc.COLOR_BGR2GRAY, 4);
Bitmap bmGray = Bitmap.createBitmap(gray.cols(), gray.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(gray, bmGray);
pic.setImageBitmap(bmGray);
Mat edges = gray;
double thresh = Imgproc.threshold(gray, edges, 0, 255, Imgproc.THRESH_OTSU);
Imgproc.Canny(gray, edges, 80, 100);
Bitmap bmEdges = Bitmap.createBitmap(edges.cols(), edges.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(edges, bmEdges);
pic.setImageBitmap(bmEdges);
} catch (IOException e) {
Log.e(TAG, e.toString());
e.printStackTrace();
} catch (Exception e) {
Log.e(TAG, e.toString());
e.printStackTrace();
}
}
The code runs fine until it hits the canny function call. If it is commented out the rest runs without a problem. I'm assuming it is a native crash since the app crashes without any errors in the log and goes straight back to the initial activity. I've tried instantiating the matrices gray and edges various ways, and using different thresholds but nothing seems to work.
Any help would be appreciated.
I have answered a similar question on: https://stackoverflow.com/a/50637228/1693327 but I am unable to comment to the post due to lack of reputation points so I will copy the answer below.
I believe the app is crashing when it hits the Canny detector because of the wrong type of OpenCV Manager installed on your device, be it version number or central processor instruction set. Checking the correct version should be straightforward. Just go to the OpenCV-android-sdk\apk directory and check for the 3 (x.y.z) numbers after OpenCV_
Checking instruction set of Android devices for Windows
To check the instruction set of your device, navigate to the adb (android debug bridge) directory commonly located at:
C:\Users\<'your username'>\AppData\Local\Android\Sdk\platform-tools
Run the command:
./adb.exe shell cat /proc/cpuinfo
After getting the correct instruction set, navigate back to the OpenCV-android-sdk\apk and locate the correct apk version and instruction set to be installed on your android device.
You can then transfer the apk to your device and install it. Another way I find useful is to navigate to the adb.exe directory and run the command:
./adb.exe install <path to OpenCV-android-sdk>/apk/OpenCV_x.y.z_Manager_x.yz_<platform instruction set>.apk
Apart from the steps above, make sure that you do not have any other environment variables that use other types of OpenCV Manager such as stating a different one in the Application.mk or build.gradle files.
After the steps above, your Canny detector should be able to run on your device without crashing.
Happy Developing :).
Related
I'm running Ubuntu 16.04. And on Android Studio when I try to run my application in the emulator I get the following error:
FATAL EXCEPTION: main
Process: project name here, PID: 2528
java.lang.RuntimeException: Canvas: trying to draw too large(216090000bytes) bitmap.
at android.view.DisplayListCanvas.throwIfCannotDraw(DisplayListCanvas.java:260)
at android.graphics.Canvas.drawBitmap(Canvas.java:1415)
at android.graphics.drawable.BitmapDrawable.draw(BitmapDrawable.java:528)
at android.widget.ImageView.onDraw(ImageView.java:1316)
at android.view.View.draw(View.java:17185)
at android.view.View.updateDisplayListIfDirty(View.java:16167)
at android.view.View.draw(View.java:16951)
at android.view.ViewGroup.drawChild(ViewGroup.java:3727)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3513)
at android.view.View.updateDisplayListIfDirty(View.java:16162)
at android.view.View.draw(View.java:16951)
at android.view.ViewGroup.drawChild(ViewGroup.java:3727)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3513) at
etc...
I did have to run through some hoops to get my emulator working however, needed to create a sym-link so I can run the emulator on AMD. Not sure if this is part of the problem. And for the life of me I cannot figure why it continues to do this. In my group there are others who emulate the project just fine on the same emulated phone and SDK.
Move your image in the (hi-res) drawable to drawable-xxhdpi. But in app development, you do not need to use large image. It will increase your APK file size.
The solution is to move the image from drawable/ folder to drawable-xxhdpi/ folder, as also others have mentioned.
But it is important to also understand why this somewhat weird suggestion actually helps:
The reason is that the drawable/ folder exists from early versions of android and is equivalent to drawable-mdpi. When an image that is only in drawable/ folder is used on xxhdpi device, the potentially already big image is upscaled by a factor of 3, which can then in some cases cause the image's memory footprint to explode.
This solution worked for me.
Add these lines in your Manifest application tag
android:largeHeap="true"
android:hardwareAccelerated="false"
I had the same problem.
If you try to upload an image that is too large on some low resolution devices, the app will collapse.
You can make several images of different sizes (hdpi, xxdpi and more) or simply use an external library to load images that solve the problem quickly and efficiently.
I used Glide library (you can use another library like Picasso).
panel_IMG_back = (ImageView) findViewById(R.id.panel_IMG_back);
Glide
.with(this)
.load(MyViewUtils.getImage(R.drawable.wallpaper)
.into(panel_IMG_back);
This issue can be resolved by 3 methods as follows:
Method 1:
By adding image into a res/drawable-nodpi folder (By doing this it will not pre-scale your image).
Method 2:
Generate all dpi(hdpi,ldpi,mdpi,xhdpi,xxhdpi,xxxhdpi) of image and add to drawable folder. (This process will increase APK size).
Method 3:
Add image to drawable/drawable-xxhdpi folder.
All these methods are verified.
Turns out the problem was the main image that we used on our app at the time. The actual size of the image was too large, so we compressed it. Then it worked like a charm, no loss in quality and the app ran fine on the emulator.
For this error was like others said a big image(1800px X 900px) which was in drawable directory, I edited the image and reduced the size proportionally using photoshop and it worked...!!
If you don't want your image to be pre-scaled you can move it to the res/drawable-nodpi/ folder.
More info: https://developer.android.com/training/multiscreen/screendensities#DensityConsiderations
if you use Picasso change to Glide like this.
Remove picasso
Picasso.get().load(Uri.parse("url")).into(imageView)
Change Glide
Glide.with(context).load("url").into(imageView)
More efficient Glide than Picasso draw to large bitmap
I also had this issue when i was trying to add a splash screen to the android app through the launch_backgrgound.xml . the issue was the resolution. it was too high so the images memory footprint exploded and caused the app to crash hence the reason for this error. so just resize your image using a site called nativescript image builder so i got the ldpi,mdpi and all the rest and it worked fine for me.
I just created directory drawable-xhdpi(You can change it according to your need) and copy pasted all the images to that directory.
This can be an issue with Glide. Use this while you are trying to load to many images and some of them are very large:
Glide.load("your image path")
.transform(
new MultiTransformation<>(
new CenterCrop(),
new RoundedCorners(
holder.imgCompanyLogo.getResources()
.getDimensionPixelSize(R.dimen._2sdp)
)
)
)
.error(R.drawable.ic_nfs_default)
.into(holder.imgCompanyLogo);
}
Try using an xml or a vector asset instead of a jpg or png.
The reason is quite obvious in the exception name itself i.e. the resolution of the resource is too large to render.
You can png to xml using online tools like https://svg2vector.com/ OR add your image to drawable-xxhdpi folder.
Solution for Picasso is add Transformation for resize image.
class ResizeTransformation(private val maxSize: Int) : Transformation {
override fun transform(source: Bitmap?): Bitmap? {
var result:Bitmap? = null
if (source != null) {
var width = source.width
var height = source.height
val bitmapRatio = width.toFloat() / height.toFloat()
if (bitmapRatio > 1) {
width = maxSize;
height = (width / bitmapRatio).toInt()
} else {
height = maxSize;
width = (height * bitmapRatio).toInt()
}
result = Bitmap.createScaledBitmap(source, width, height, true)
source.recycle()
}
return result
}
override fun key() = "resize()"
}
Use:
Picasso.get()
.load(url)
.transform(ResizeTransformation(2400)) //FHD+ resolution
.into(view)
Convert your all png formats into webs format. You can do it by Android Studio.
Why isOperational() in mobile vision text recognizer returns false?
At first, mobile vision only show preview camera and after many tries to get the result, I saw that the texts recognized but in one device it works and in other device does not.
What should I do?
For example, in one device, isOperational() returns false, and it goes to readstate() and after that goes to looper() and stays on it!
in other device it only return false and doesn't go to looper.
I want ask other questions about it:
My first question is: how does isOperational() work? I can't understand it.
Maybe it goes to looper to download the native library in a queue and after many try, at last download completes and work. Can it be correct? Or is it just a bug that it goes to looper? Anywhere, what should I do?
Can I work on this when it works in one device I tried and in other does not? Or it must work in every device to I can work on it? And I get .apk from project but it can't install in devices, why?
Should it check for network?
Should it check for access to the memory?
note: it works with camera API and its deprecated. maybe the problem is with this!
TextRecognizer textRecognizer = new TextRecognizer.Builder(context).build();
textRecognizer.setProcessor(new OcrDetectorProcessor(graphicOverlay));
if (!textRecognizer.**isOperational**()) {
// Note: The first time that an app using a Vision API is installed on a
// device, GMS will download a native libraries to the device in order to do detection.
// Usually this completes before the app is run for the first time. But if that
// download has not yet completed, then the above call will not detect any text,
// barcodes, or faces.
//
// isOperational() can be used to check if the required native libraries are currently
// available. The detectors will automatically become operational once the library
// downloads complete on device.
Log.w(TAG, "Detector dependencies are not yet available.");
// Check for low storage. If there is low storage, the native library will not be
// downloaded, so detection will not become operational.*
IntentFilter lowstorageFilter = new IntentFilter(Intent.ACTION_DEVICE_STORAGE_LOW);
boolean hasLowStorage = registerReceiver(null, lowstorageFilter) != null;
if (hasLowStorage) {
Toast.makeText(this, R.string.low_storage_error, Toast.LENGTH_LONG).show();
Log.w(TAG, getString(R.string.low_storage_error));
}
}
*// Creates and starts the camera. Note that this uses a higher resolution in comparison
// to other detection examples to enable the text recognizer to detect small pieces of text.*
cameraSource =
new CameraSource.Builder(getApplicationContext(), textRecognizer)
.setFacing(CameraSource.CAMERA_FACING_BACK)
.setRequestedPreviewSize(1280, 1024)
.setRequestedFps(2.0f)
.setFlashMode(useFlash ? Camera.Parameters.FLASH_MODE_TORCH : null)
.setFocusMode(autoFocus ? Camera.Parameters.FOCUS_MODE_CONTINUOUS_VIDEO : null)
.build();
}
It doesn't produce any error and show preview camera but doesn't recognize texts in some devices.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to programatically take a screenshot on Android?
How to capture the android device screen content and make an image file using the snapshot data? Which API should I use or where could I find related resources?
BTW:
not camera snapshot, but device screen
Use the following code:
Bitmap bitmap;
View v1 = MyView.getRootView();
v1.setDrawingCacheEnabled(true);
bitmap = Bitmap.createBitmap(v1.getDrawingCache());
v1.setDrawingCacheEnabled(false);
Here MyView is the View through which we need include in the screen. You can also get DrawingCache from of any View this way (without getRootView()).
There is also another way.. If we having ScrollView as root view then its better to use following code,
LayoutInflater inflater = (LayoutInflater) this.getSystemService(LAYOUT_INFLATER_SERVICE);
FrameLayout root = (FrameLayout) inflater.inflate(R.layout.activity_main, null); // activity_main is UI(xml) file we used in our Activity class. FrameLayout is root view of my UI(xml) file.
root.setDrawingCacheEnabled(true);
Bitmap bitmap = getBitmapFromView(this.getWindow().findViewById(R.id.frameLayout)); // here give id of our root layout (here its my FrameLayout's id)
root.setDrawingCacheEnabled(false);
Here is the getBitmapFromView() method
public static Bitmap getBitmapFromView(View view) {
//Define a bitmap with the same size as the view
Bitmap returnedBitmap = Bitmap.createBitmap(view.getWidth(), view.getHeight(),Bitmap.Config.ARGB_8888);
//Bind a canvas to it
Canvas canvas = new Canvas(returnedBitmap);
//Get the view's background
Drawable bgDrawable =view.getBackground();
if (bgDrawable!=null)
//has background drawable, then draw it on the canvas
bgDrawable.draw(canvas);
else
//does not have background drawable, then draw white background on the canvas
canvas.drawColor(Color.WHITE);
// draw the view on the canvas
view.draw(canvas);
//return the bitmap
return returnedBitmap;
}
It will display entire screen including content hidden in your ScrollView
UPDATED AS ON 20-04-2016
There is another better way to take screenshot.Here I have taken screenshot of WebView.
WebView w = new WebView(this);
w.setWebViewClient(new WebViewClient()
{
public void onPageFinished(final WebView webView, String url) {
new Handler().postDelayed(new Runnable(){
#Override
public void run() {
webView.measure(View.MeasureSpec.makeMeasureSpec(
View.MeasureSpec.UNSPECIFIED, View.MeasureSpec.UNSPECIFIED),
View.MeasureSpec.makeMeasureSpec(0, View.MeasureSpec.UNSPECIFIED));
webView.layout(0, 0, webView.getMeasuredWidth(),
webView.getMeasuredHeight());
webView.setDrawingCacheEnabled(true);
webView.buildDrawingCache();
Bitmap bitmap = Bitmap.createBitmap(webView.getMeasuredWidth(),
webView.getMeasuredHeight(), Bitmap.Config.ARGB_8888);
Canvas canvas = new Canvas(bitmap);
Paint paint = new Paint();
int height = bitmap.getHeight();
canvas.drawBitmap(bitmap, 0, height, paint);
webView.draw(canvas);
if (bitmap != null) {
try {
String filePath = Environment.getExternalStorageDirectory()
.toString();
OutputStream out = null;
File file = new File(filePath, "/webviewScreenShot.png");
out = new FileOutputStream(file);
bitmap.compress(Bitmap.CompressFormat.PNG, 50, out);
out.flush();
out.close();
bitmap.recycle();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}, 1000);
}
});
Hope this helps..!
AFAIK, All of the methods currently to capture a screenshot of android use the /dev/graphics/fb0 framebuffer. This includes ddms. It does require root to read from this stream. ddms uses adbd to request the information, so root is not required as adb has the permissions needed to request the data from /dev/graphics/fb0.
The framebuffer contains 2+ "frames" of RGB565 images. If you are able to read the data, you would have to know the screen resolution to know how many bytes are needed to get the image. each pixel is 2 bytes, so if the screen res was 480x800, you would have to read 768,000 bytes for the image, since a 480x800 RGB565 image has 384,000 pixels.
For newer Android platforms, one can execute a system utility screencap in /system/bin to get the screenshot without root permission.
You can try /system/bin/screencap -h to see how to use it under adb or any shell.
By the way, I think this method is only good for single snapshot.
If we want to capture multiple frames for screen play, it will be too slow.
I don't know if there exists any other approach for a faster screen capture.
[Based on Android source code:]
At the C++ side, the SurfaceFlinger implements the captureScreen API. This is exposed over the binder IPC interface, returning each time a new ashmem area that contains the raw pixels from the screen. The actual screenshot is taken through OpenGL.
For the system C++ clients, the interface is exposed through the ScreenshotClient class, defined in <surfaceflinger_client/SurfaceComposerClient.h> for Android < 4.1; for Android > 4.1 use <gui/SurfaceComposerClient.h>
Before JB, to take a screenshot in a C++ program, this was enough:
ScreenshotClient ssc;
ssc.update();
With JB and multiple displays, it becomes slightly more complicated:
ssc.update(
android::SurfaceComposerClient::getBuiltInDisplay(
android::ISurfaceComposer::eDisplayIdMain));
Then you can access it:
do_something_with_raw_bits(ssc.getPixels(), ssc.getSize(), ...);
Using the Android source code, you can compile your own shared library to access that API, and then expose it through JNI to Java. To create a screen shot form your app, the app has to have the READ_FRAME_BUFFER permission.
But even then, apparently you can create screen shots only from system applications, i.e. ones that are signed with the same key as the system. (This part I still don't quite understand, since I'm not familiar enough with the Android Permissions system.)
Here is a piece of code, for JB 4.1 / 4.2:
#include <utils/RefBase.h>
#include <binder/IBinder.h>
#include <binder/MemoryHeapBase.h>
#include <gui/ISurfaceComposer.h>
#include <gui/SurfaceComposerClient.h>
static void do_save(const char *filename, const void *buf, size_t size) {
int out = open(filename, O_RDWR|O_CREAT, 0666);
int len = write(out, buf, size);
printf("Wrote %d bytes to out.\n", len);
close(out);
}
int main(int ac, char **av) {
android::ScreenshotClient ssc;
const void *pixels;
size_t size;
int buffer_index;
if(ssc.update(
android::SurfaceComposerClient::getBuiltInDisplay(
android::ISurfaceComposer::eDisplayIdMain)) != NO_ERROR ){
printf("Captured: w=%d, h=%d, format=%d\n");
ssc.getWidth(), ssc.getHeight(), ssc.getFormat());
size = ssc.getSize();
do_save(av[1], pixels, size);
}
else
printf(" screen shot client Captured Failed");
return 0;
}
You can try the following library: Android Screenshot Library (ASL) enables to programmatically capture screenshots from Android devices without requirement of having root access privileges. Instead, ASL utilizes a native service running in the background, started via the Android Debug Bridge (ADB) once per device boot.
According to this link, it is possible to use ddms in the tools directory of the android sdk to take screen captures.
To do this within an application (and not during development), there are also applications to do so. But as #zed_0xff points out it certainly requires root.
Framebuffer seems the way to go, it will not always contain 2+ frames like mentioned by Ryan Conrad. In my case it contained only one. I guess it depends on the frame/display size.
I tried to read the framebuffer continuously but it seems to return for a fixed amount of bytes read. In my case that is (3 410 432) bytes, which is enough to store a display frame of 854*480 RGBA (3 279 360 bytes). Yes, the frame in binary outputed from fb0 is RGBA in my device. This will most likely depend from device to device. This will be important for you to decode it =)
In my device /dev/graphics/fb0 permissions are so that only root and users from group graphics can read the fb0. graphics is a restricted group so you will probably only access fb0 with a rooted phone using su command.
Android apps have the user id (uid) app_## and group id (guid) app_## .
adb shell has uid shell and guid shell, which has much more permissions than an app.
You can actually check those permissions at /system/permissions/platform.xml
This means you will be able to read fb0 in the adb shell without root but you will not read it within the app without root.
Also, giving READ_FRAME_BUFFER and/or ACCESS_SURFACE_FLINGER permissions on AndroidManifest.xml will do nothing for a regular app because these will only work for 'signature' apps.
if you want to do screen capture from Java code in Android app AFAIK you must have Root provileges.
What i'm trying to do:
Pull an image from sd-card on phone using Java Plugin.
Unity passes a texture ID to plugin.
Plugin uses opengl to assign the image to the texture in Unity through the ID.
Will (eventually) be used to play a video clip from the phone in Unity, for now, it's just trying to change a texture outside of unity.
My issue:
When i call the method in the plugin, passing texture.GetNativeTextureID() into it, the texture does not change. I'm currently only using a simple black 50x50 texture for testing, and the original texture is a flat white.
I'm worried that i've missed something significant, as this is my first time working with Gl calls in java. Much of the answers to similar problems involve using native C++ instead of Java, but I can't find a concrete answer saying that C++ must be used. I'd like to do my best to avoid writing another set of plugins and plugin handlers for C++, but if it's the most efficient/only way to get this working, i'll do it as i'm not unfamiliar with OpenGL and C++
Code:
The plugin method is called from OnPreRender() in a script attached to the main camera:
if (grabTex) {
int texPtr = m_VideoTex.GetNativeTextureID();
Debug.Log( "texPtr = " + texPtr );
m_JVInterface.SetTex( texPtr );
}
m_VideoTex is a basic Texture2D( 50, 50 ) with all pixels set to white, attached to the diffuse shader on the quad in the scene.
The Java plugin code is as follows:
public void SetTexture(Context cont, int _texPointer) {
if (_texPointer != 0) {
final BitmapFactory.Options options = new BitmapFactory.Options();
options.inScaled = false;
options.inJustDecodeBounds = false;
final Bitmap bitmap = BitmapFactory.decodeFile("/storage/emulated/0/Pictures/black.jpg", options);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, _texPointer);
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bitmap, 0);
Log.i("VideoHandler", "Recieved ID: " + _texPointer);
bitmap.recycle();
}
}
This is most likely a problem with the OpenGL Context. The easiest way would be to send the texture as raw bytes to Unity and then upload as texture inside Unity.
Below you will see a picture of beatiful pastry called "simit" from Turkey. It is taken from iPad 2, therefore it is a JPEG with dimensions 720*960.
The problem is, when I use javax.imageio.ImageIO.read method, the image it strangely imports is to a BufferedImage rotated to left and becomes 960*720.
I reproduced this in my Sun JVM 1.6.0_29 on OS X and Sun JVM 1.6.0_26 on Debian. Here's the code:
public class Main {
public static void main(String[] args) throws Exception {
FileInputStream stream = new FileInputStream(new File("IMG_0159.JPG"));
BufferedImage img = ImageIO.read(stream);
System.out.println("width:" + img.getWidth() + " height:"
+ img.getHeight());
}
}
It outputs width:960 height:720, and when I save this output image, it is rotated to left as I told before. If you would like to reproduce this, download code and picture from here and run the following commands to build and run:
javac Main.java && java Main
NOTE: You may see the JPG in the archive as already rotated, however it appears 720*960 on OS X, iPad, iPhone and as you see above, it is uploaded correctly to imgur.com. And it is also opened correctly in Adobe Photoshop, uploaded to Facebook correctly etc.
What could be the problem here?
The photo was probably taken holding the iPad in portrait mode, and therefore contains EXIF orientation information, which ImageIO ignores, but you can use other libraries, like Apache Sanselan to correctly handle it.
So the image itself is 960x720, but MacOS, ImgUR, Facebook etc correctly take the EXIF info into account.
And simit looks delicious :)