My List<FirebaseVisionFace> faces is always empty - java

I am implementing the MLKit face detection library with a simple application. The application is a facial monitoring system so i am setting up a preview feed from the front camera and attempting to detect a face. I am using camera2Api. At my ImageReader.onImageAvailableListener, I want to implement the firebase face detection on each read in the image. After creating my FirebaseVisionImage and running the FirebaseVisionFaceDetector I am getting an empty faces list, this should contain detected faces but I always get a face of size 0 even though a face is in the image.
I have tried other forms of creating my FirebaseVisionImage. Currently, I am creating it through the use of a byteArray which I created following the MlKit docs. I have also tried to create a FirebaseVisionImage using the media Image object.
private final ImageReader.OnImageAvailableListener onPreviewImageAvailableListener = new ImageReader.OnImageAvailableListener() {
/**Get Image convert to Byte Array **/
#Override
public void onImageAvailable(ImageReader reader) {
//Get latest image
Image mImage = reader.acquireNextImage();
if(mImage == null){
return;
}
else {
byte[] newImg = convertYUV420888ToNV21(mImage);
FirebaseApp.initializeApp(MonitoringFeedActivity.this);
FirebaseVisionFaceDetectorOptions highAccuracyOpts =
new FirebaseVisionFaceDetectorOptions.Builder()
.setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE)
.setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS)
.setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS)
.build();
int rotation = getRotationCompensation(frontCameraId,MonitoringFeedActivity.this, getApplicationContext() );
FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
.setWidth(480) // 480x360 is typically sufficient for
.setHeight(360) // image recognition
.setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
.setRotation(rotation)
.build();
FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(newImg, metadata);
FirebaseVisionFaceDetector detector = FirebaseVision.getInstance()
.getVisionFaceDetector(highAccuracyOpts);
Task<List<FirebaseVisionFace>> result =
detector.detectInImage(image)
.addOnSuccessListener(
new OnSuccessListener<List<FirebaseVisionFace>>() {
#Override
public void onSuccess(List<FirebaseVisionFace> faces) {
// Task completed successfully
if (faces.size() != 0) {
Log.i(TAG, String.valueOf(faces.get(0).getSmilingProbability()));
}
}
})
.addOnFailureListener(
new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
// Task failed with an exception
// ...
}
});
mImage.close();
The aim is to have the resulting faces list contain the detected faces in each processed image.

byte[] newImg = convertYUV420888ToNV21(mImage);
FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(newImg, metadata);
These two lines are important. make sure Its creating proper VisionImage.
Checkout my project for all functionality
MLKIT demo

Related

Keep receiving this error "Failed to initialize detector". Am I not loading the tflite model correctly?

I am trying to setup an ImageAnalyzer with my Android app so I can run object classification using Google's ML Kit API. The issue I am currently facing, as the title suggests, is constantly seeing the error "Failed to initialize detector".
I've reread this tutorial about three times now and followed this post about someone facing the same error (although for a different reason) to no avail. I've also made sure everything with the CameraX API (except the ImageAnalyzer code that I will show in a second) works as expected.
As mentioned in the ML Kit documentation, here is the code I have regarding setting up a LocalModel, a CustomObjectDetectorOptions, and an ObjectDetector:
LocalModel localModel = new LocalModel.Builder()
.setAssetFilePath("mobilenet_v1_1.0_224_quantized_1_metadata_1.tflite")
.build();
CustomObjectDetectorOptions customObjectDetectorOptions =
new CustomObjectDetectorOptions.Builder(localModel)
.setDetectorMode(CustomObjectDetectorOptions.STREAM_MODE)
.enableClassification()
.setClassificationConfidenceThreshold(0.5f)
.setMaxPerObjectLabelCount(3)
.build();
ObjectDetector objectDetector = ObjectDetection.getClient(customObjectDetectorOptions);
Here is the ImageAnalyzer code I have, which basically makes a call to the ML Kit API by way of the processImage helper method:
// Creates an ImageAnalysis for analyzing the camera preview feed
ImageAnalysis imageAnalysis = new ImageAnalysis.Builder()
.setTargetResolution(new Size(224, 224))
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.build();
imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this),
new ImageAnalysis.Analyzer() {
#Override
public void analyze(#NonNull ImageProxy imageProxy) {
#SuppressLint("UnsafeExperimentalUsageError") Image mediaImage =
imageProxy.getImage();
if (mediaImage != null) {
Log.i(TAG, "Obtained ImageProxy object");
processImage(mediaImage, imageProxy)
.addOnCompleteListener(new OnCompleteListener<List<DetectedObject>>() {
#Override
public void onComplete(#NonNull Task<List<DetectedObject>> task) {
imageProxy.close();
}
});
}
}
});
Here is the processImage helper method, where I actually call objectDetector.process(...), the line of code that actually runs the tflite model.
private Task<List<DetectedObject>> processImage(Image mediaImage, ImageProxy imageProxy) {
InputImage image =
InputImage.fromMediaImage(mediaImage,
imageProxy.getImageInfo().getRotationDegrees());
return objectDetector.process(image)
.addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
String error = "Failed to process. Error: " + e.getMessage();
Log.i(TAG, error);
}
})
.addOnSuccessListener(new OnSuccessListener<List<DetectedObject>>() {
#Override
public void onSuccess(List<DetectedObject> results) {
String success = "Object(s) detected successfully!";
Log.i(TAG, success);
for (DetectedObject detectedObject : results) {
Rect boundingBox = detectedObject.getBoundingBox();
Integer trackingId = detectedObject.getTrackingId();
for (DetectedObject.Label label : detectedObject.getLabels()) {
String text = label.getText();
int index = label.getIndex();
float confidence = label.getConfidence();
Log.i(TAG, "Object detected: " + text + "; "
+ "Confidence: " + confidence);
}
}
}
});
}
Essentially, once I run the app, logcat just keeps logging these two lines on repeat. I know it means the ImageAnalyzer is continuously trying to analyze the image input, but for some reason the LocalModel just cannot process the input
2021-01-21 22:02:24.020 9328-9328/com.example.camerax I/MainActivity: Obtained ImageProxy object
2021-01-21 22:02:24.036 9328-9328/com.example.camerax I/MainActivity: Failed to process. Error: Failed to initialize detector.
I have only just started to work with Android, especially ML in Android, so any sort of help would be appreciated!
I managed to fix my issue before anyone answered, but in case anyone who just started to learn Android like me I'll leave my solution here.
Basically, remember to create an asset folder in the /src/main directory rather than the /src/androidTest directory :P
Once I did that, the model loaded correctly and now I just have to figure out how to display the results in my application.
// Do NOT compress tflite model files (need to call out to developers!)
aaptOptions {
noCompress "tflite"
}
add this line in build gradle for app under android tag

(React Native/Android) Android getFilesDir() gives me '/data/user/' instead of '/data/data/'

I am building an app which has a feature to crop images using react-native-image-crop-picker. I am trying to implement the logic to store the cropped images locally in my react native app. I could successfully implement the logic for iOS, however, I am having trouble with the Android side.
My problem is that when I store the image using reactContext.getFilesDir(), the image is stored into the /data/user/0/com.myapp/files/ directory. And the images can be accessed via 'Google Photos' app or 'Files' app. I don't want to let the users access these images.
Here is the picture describing my problem.
The things I have tried so far:
1. Use getCurrentActivity() instead of reactContext
2. Use getReactApplicationContext() instead of context
Findings:
- After saving the image, it is stored into /data/user/0/com.myapp/files/, /data/data/0/com.myapp/files/ and storage/emulated/0/Pictures/.
FileStreamHandler.java
public class FileStreamHandler extends ReactContextBaseJavaModule {
private Context context;
// private Activity mActivity;
#Nonnull
#Override
public String getName() {
return "FileStreamHandler";
}
public FileStreamHandler(ReactApplicationContext reactContext) {
super(reactContext);
// mActivity = reactContext.getCurrentActivity();
this.context = reactContext;
}
#ReactMethod
private void saveImageData(String base64String, Callback callback) {
// Generate random image name
String fileName = UUID.randomUUID().toString() + ".png";
// File fileDirectory = mActivity.getFilesDir();
File fileDirectory = context.getFilesDir();
File imageFile = new File(fileDirectory, fileName);
String imageFilePath = imageFile.getAbsolutePath();
try {
OutputStream stream = new FileOutputStream(imageFile);
//decode base64 string to image
byte[] decodedBytes = Base64.decode(base64String, Base64.DEFAULT);
Bitmap decodedImage = BitmapFactory.decodeByteArray(decodedBytes, 0, decodedBytes.length);
decodedImage.compress(Bitmap.CompressFormat.PNG,100, stream);
stream.flush();
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
callback.invoke(imageFilePath);
}
}
The image is stored successfully without any errors. However, it is stored into /data/user/ and can be accessed via other applications such as 'Photos' or 'Files'.
Although I am using exactly the same logic in my pure Android app, I have never had this problem. Therefore, I am suspecting that the react application context is causing the problem.
Any help would be highly appreciated.
Thank you.
It turns out that the cause of the problem is the react native library that I am using. I don't know why they implemented in this way, however, it seems like the react-native-image-crop-picker library saves images into the /storage/0/Pictures/ directory after cropping.

display different 3D objects based on detected image's index in image database arcore

I have created an imgdb file consisting of images that I want to detect in arcore sceneform. Once I detect an image, I wish to display an augmented 3D object where the image was detected. Now, I wish to display a different 3D object depending on which image within the imgdb file was detected.
Something like this? But my app keeps crashing if I do this in the AugmentedImageNode.java file.
public class AugmentedImageNode extends AnchorNode {
private static final String TAG = "AugmentedImageNode";
public AugmentedImage image;
private static CompletableFuture<ModelRenderable> ulCorner;
public AugmentedImageNode(Context context) {
int imagenumber;
imagenumber = image.getIndex();
if (imagenumber == 0) {
if (ulCorner == null) {
ulCorner =
ModelRenderable.builder()
.setSource(context, Uri.parse("models/tinker.sfb"))
.build();
}
}
if (imagenumber == 1) {
if (ulCorner == null) {
ulCorner =
ModelRenderable.builder()
.setSource(context, Uri.parse("models/borderfence-small.sfb"))
.build();
}
}
}
Here are the logcat details:
enter image description here

showing image in listview with json [duplicate]

I am using a ListView to display some images and captions associated with those images. I am getting the images from the Internet. Is there a way to lazy load images so while the text displays, the UI is not blocked and images are displayed as they are downloaded?
The total number of images is not fixed.
Here's what I created to hold the images that my app is currently displaying. Please note that the "Log" object in use here is my custom wrapper around the final Log class inside Android.
package com.wilson.android.library;
/*
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
*/
import java.io.IOException;
public class DrawableManager {
private final Map<String, Drawable> drawableMap;
public DrawableManager() {
drawableMap = new HashMap<String, Drawable>();
}
public Drawable fetchDrawable(String urlString) {
if (drawableMap.containsKey(urlString)) {
return drawableMap.get(urlString);
}
Log.d(this.getClass().getSimpleName(), "image url:" + urlString);
try {
InputStream is = fetch(urlString);
Drawable drawable = Drawable.createFromStream(is, "src");
if (drawable != null) {
drawableMap.put(urlString, drawable);
Log.d(this.getClass().getSimpleName(), "got a thumbnail drawable: " + drawable.getBounds() + ", "
+ drawable.getIntrinsicHeight() + "," + drawable.getIntrinsicWidth() + ", "
+ drawable.getMinimumHeight() + "," + drawable.getMinimumWidth());
} else {
Log.w(this.getClass().getSimpleName(), "could not get thumbnail");
}
return drawable;
} catch (MalformedURLException e) {
Log.e(this.getClass().getSimpleName(), "fetchDrawable failed", e);
return null;
} catch (IOException e) {
Log.e(this.getClass().getSimpleName(), "fetchDrawable failed", e);
return null;
}
}
public void fetchDrawableOnThread(final String urlString, final ImageView imageView) {
if (drawableMap.containsKey(urlString)) {
imageView.setImageDrawable(drawableMap.get(urlString));
}
final Handler handler = new Handler(Looper.getMainLooper()) {
#Override
public void handleMessage(Message message) {
imageView.setImageDrawable((Drawable) message.obj);
}
};
Thread thread = new Thread() {
#Override
public void run() {
//TODO : set imageView to a "pending" image
Drawable drawable = fetchDrawable(urlString);
Message message = handler.obtainMessage(1, drawable);
handler.sendMessage(message);
}
};
thread.start();
}
private InputStream fetch(String urlString) throws MalformedURLException, IOException {
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet request = new HttpGet(urlString);
HttpResponse response = httpClient.execute(request);
return response.getEntity().getContent();
}
}
I made a simple demo of a lazy list (located at GitHub) with images.
Basic Usage
ImageLoader imageLoader=new ImageLoader(context); ...
imageLoader.DisplayImage(url, imageView);
Don't forget to add the
following permissions to your AndroidManifest.xml:
<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/> Please
create only one instance of ImageLoader and reuse it all around your
application. This way image caching will be much more efficient.
It may be helpful to somebody. It downloads images in the background thread. Images are being cached on an SD card and in memory. The cache implementation is very simple and is just enough for the demo. I decode images with inSampleSize to reduce memory consumption. I also try to handle recycled views correctly.
I recommend open source instrument Universal Image Loader. It is originally based on Fedor Vlasov's project LazyList and has been vastly improved since then.
Multithread image loading
Possibility of wide tuning ImageLoader's configuration (thread executors, downloader, decoder, memory and disc cache, display image options, and others)
Possibility of image caching in memory and/or on the device's file system (or SD card)
Possibility to "listen" loading process
Possibility to customize every display image call with separated options
Widget support
Android 2.0+ support
Multithreading For Performance, a tutorial by Gilles Debunne.
This is from the Android Developers Blog. The suggested code uses:
AsyncTasks.
A hard, limited size, FIFO cache.
A soft, easily garbage collect-ed cache.
A placeholder Drawable while you download.
Update: Note that this answer is pretty ineffective now. The Garbage Collector acts aggressively on SoftReference and WeakReference, so this code is NOT suitable for new apps. (Instead, try libraries like Universal Image Loader suggested in other answers.)
Thanks to James for the code, and Bao-Long for the suggestion of using SoftReference. I implemented the SoftReference changes on James' code. Unfortunately, SoftReferences caused my images to be garbage collected too quickly. In my case, it was fine without the SoftReference stuff, because my list size is limited and my images are small.
There's a discussion from a year ago regarding the SoftReferences on google groups: link to thread. As a solution to the too-early garbage collection, they suggest the possibility of manually setting the VM heap size using dalvik.system.VMRuntime.setMinimumHeapSize(), which is not very attractive to me.
public DrawableManager() {
drawableMap = new HashMap<String, SoftReference<Drawable>>();
}
public Drawable fetchDrawable(String urlString) {
SoftReference<Drawable> drawableRef = drawableMap.get(urlString);
if (drawableRef != null) {
Drawable drawable = drawableRef.get();
if (drawable != null)
return drawable;
// Reference has expired so remove the key from drawableMap
drawableMap.remove(urlString);
}
if (Constants.LOGGING) Log.d(this.getClass().getSimpleName(), "image url:" + urlString);
try {
InputStream is = fetch(urlString);
Drawable drawable = Drawable.createFromStream(is, "src");
drawableRef = new SoftReference<Drawable>(drawable);
drawableMap.put(urlString, drawableRef);
if (Constants.LOGGING) Log.d(this.getClass().getSimpleName(), "got a thumbnail drawable: " + drawable.getBounds() + ", "
+ drawable.getIntrinsicHeight() + "," + drawable.getIntrinsicWidth() + ", "
+ drawable.getMinimumHeight() + "," + drawable.getMinimumWidth());
return drawableRef.get();
} catch (MalformedURLException e) {
if (Constants.LOGGING) Log.e(this.getClass().getSimpleName(), "fetchDrawable failed", e);
return null;
} catch (IOException e) {
if (Constants.LOGGING) Log.e(this.getClass().getSimpleName(), "fetchDrawable failed", e);
return null;
}
}
public void fetchDrawableOnThread(final String urlString, final ImageView imageView) {
SoftReference<Drawable> drawableRef = drawableMap.get(urlString);
if (drawableRef != null) {
Drawable drawable = drawableRef.get();
if (drawable != null) {
imageView.setImageDrawable(drawableRef.get());
return;
}
// Reference has expired so remove the key from drawableMap
drawableMap.remove(urlString);
}
final Handler handler = new Handler() {
#Override
public void handleMessage(Message message) {
imageView.setImageDrawable((Drawable) message.obj);
}
};
Thread thread = new Thread() {
#Override
public void run() {
//TODO : set imageView to a "pending" image
Drawable drawable = fetchDrawable(urlString);
Message message = handler.obtainMessage(1, drawable);
handler.sendMessage(message);
}
};
thread.start();
}
Picasso
Use Jake Wharton's Picasso Library.
(A Perfect ImageLoading Library from the developer of ActionBarSherlock)
A powerful image downloading and caching library for Android.
Images add much-needed context and visual flair to Android applications. Picasso allows for hassle-free image loading in your application—often in one line of code!
Picasso.with(context).load("http://i.imgur.com/DvpvklR.png").into(imageView);
Many common pitfalls of image loading on Android are handled automatically by Picasso:
Handling ImageView recycling and download cancellation in an adapter.
Complex image transformations with minimal memory use.
Automatic memory and disk caching.
Picasso Jake Wharton's Library
Glide
Glide is a fast and efficient open-source media management framework for Android that wraps media decoding, memory and disk caching, and resource pooling into a simple and easy-to-use interface.
Glide supports fetching, decoding, and displaying video stills, images, and animated GIFs. Glide includes a flexible API that allows developers to plug into almost any network stack. By default, Glide uses a custom HttpUrlConnection based stack but also includes utility libraries plug-in to Google's Volley project or Square's OkHttp library instead.
Glide.with(this).load("your-url-here").into(imageView);
Glide's primary focus is on making scrolling any kind of a list of images as smooth and fast as possible, but Glide is also effective for almost any case where you need to fetch, resize, and display a remote image.
Glide Image Loading Library
Fresco by Facebook
Fresco is a powerful system for displaying images in Android applications.
Fresco takes care of image loading and display, so you don't have to. It will load images from the network, local storage, or local resources, and display a placeholder until the image has arrived. It has two levels of cache; one in memory and another in internal storage.
Fresco Github
In Android 4.x and lower, Fresco puts images in a special region of Android memory. This lets your application run faster - and suffer the dreaded OutOfMemoryError much less often.
Fresco Documentation
High-performance loader - after examining the methods suggested here,
I used Ben's solution with some changes -
I realized that working with drawable is faster than with bitmaps so I uses drawable instead
Using SoftReference is great, but it makes the cached image to be deleted too often, so I added a Linked list that holds images references, preventing the image to be deleted, until it reached a predefined size
To open the InputStream I used java.net.URLConnection which allows me to use web cache (you need to set a response cache first, but that's another story)
My code:
import java.util.Map;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.Collections;
import java.util.WeakHashMap;
import java.lang.ref.SoftReference;
import java.util.concurrent.Executors;
import java.util.concurrent.ExecutorService;
import android.graphics.drawable.Drawable;
import android.widget.ImageView;
import android.os.Handler;
import android.os.Message;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.io.IOException;
import java.net.URL;
import java.net.URLConnection;
public class DrawableBackgroundDownloader {
private final Map<String, SoftReference<Drawable>> mCache = new HashMap<String, SoftReference<Drawable>>();
private final LinkedList <Drawable> mChacheController = new LinkedList <Drawable> ();
private ExecutorService mThreadPool;
private final Map<ImageView, String> mImageViews = Collections.synchronizedMap(new WeakHashMap<ImageView, String>());
public static int MAX_CACHE_SIZE = 80;
public int THREAD_POOL_SIZE = 3;
/**
* Constructor
*/
public DrawableBackgroundDownloader() {
mThreadPool = Executors.newFixedThreadPool(THREAD_POOL_SIZE);
}
/**
* Clears all instance data and stops running threads
*/
public void Reset() {
ExecutorService oldThreadPool = mThreadPool;
mThreadPool = Executors.newFixedThreadPool(THREAD_POOL_SIZE);
oldThreadPool.shutdownNow();
mChacheController.clear();
mCache.clear();
mImageViews.clear();
}
public void loadDrawable(final String url, final ImageView imageView,Drawable placeholder) {
mImageViews.put(imageView, url);
Drawable drawable = getDrawableFromCache(url);
// check in UI thread, so no concurrency issues
if (drawable != null) {
//Log.d(null, "Item loaded from mCache: " + url);
imageView.setImageDrawable(drawable);
} else {
imageView.setImageDrawable(placeholder);
queueJob(url, imageView, placeholder);
}
}
private Drawable getDrawableFromCache(String url) {
if (mCache.containsKey(url)) {
return mCache.get(url).get();
}
return null;
}
private synchronized void putDrawableInCache(String url,Drawable drawable) {
int chacheControllerSize = mChacheController.size();
if (chacheControllerSize > MAX_CACHE_SIZE)
mChacheController.subList(0, MAX_CACHE_SIZE/2).clear();
mChacheController.addLast(drawable);
mCache.put(url, new SoftReference<Drawable>(drawable));
}
private void queueJob(final String url, final ImageView imageView,final Drawable placeholder) {
/* Create handler in UI thread. */
final Handler handler = new Handler() {
#Override
public void handleMessage(Message msg) {
String tag = mImageViews.get(imageView);
if (tag != null && tag.equals(url)) {
if (imageView.isShown())
if (msg.obj != null) {
imageView.setImageDrawable((Drawable) msg.obj);
} else {
imageView.setImageDrawable(placeholder);
//Log.d(null, "fail " + url);
}
}
}
};
mThreadPool.submit(new Runnable() {
#Override
public void run() {
final Drawable bmp = downloadDrawable(url);
// if the view is not visible anymore, the image will be ready for next time in cache
if (imageView.isShown())
{
Message message = Message.obtain();
message.obj = bmp;
//Log.d(null, "Item downloaded: " + url);
handler.sendMessage(message);
}
}
});
}
private Drawable downloadDrawable(String url) {
try {
InputStream is = getInputStream(url);
Drawable drawable = Drawable.createFromStream(is, url);
putDrawableInCache(url,drawable);
return drawable;
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
private InputStream getInputStream(String urlString) throws MalformedURLException, IOException {
URL url = new URL(urlString);
URLConnection connection;
connection = url.openConnection();
connection.setUseCaches(true);
connection.connect();
InputStream response = connection.getInputStream();
return response;
}
}
I have followed this Android Training and I think it does an excellent job at downloading images without blocking the main UI. It also handles caching and dealing with scrolling through many images: Loading Large Bitmaps Efficiently
1. Picasso allows for hassle-free image loading in your application—often in one line of code!
Use Gradle:
implementation 'com.squareup.picasso:picasso:(insert latest version)'
Just one line of code!
Picasso.get().load("http://i.imgur.com/DvpvklR.png").into(imageView);
2. Glide An image loading and caching library for Android focused on smooth scrolling
Use Gradle:
repositories {
mavenCentral()
google()
}
dependencies {
implementation 'com.github.bumptech.glide:glide:4.11.0'
annotationProcessor 'com.github.bumptech.glide:compiler:4.11.0'
}
// For a simple view:
Glide.with(this).load("http://i.imgur.com/DvpvklR.png").into(imageView);
3. fresco is a powerful system for displaying images in Android
applications.Fresco takes care of image loading and display, so you don't have
to.
Getting Started with Fresco
I've written a tutorial that explains how to do lazy-loading of images in a listview. I go into some detail about the issues of recycling and concurrency. I also use a fixed thread pool to prevent spawning a lot of threads.
Lazy loading of images in Listview Tutorial
The way I do it is by launching a thread to download the images in the background and hand it a callback for each list item. When an image is finished downloading it calls the callback which updates the view for the list item.
This method doesn't work very well when you're recycling views however.
I just want to add one more good example, XML Adapters. As it's is used by Google and I am also using the same logic to avoid an OutOfMemory error.
Basically this ImageDownloader is your answer (as it covers most of your requirements). Some you can also implement in that.
This is a common problem on Android that has been solved in many ways by many people. In my opinion the best solution I've seen is the relatively new library called Picasso. Here are the highlights:
Open source, but headed up by Jake Wharton of ActionBarSherlock fame.
Asynchronously load images from network or app resources with one line of code
Automatic ListView detection
Automatic disk and memory caching
Can do custom transformations
Lots of configurable options
Super simple API
Frequently updated
I have been using NetworkImageView from the new Android Volley Library com.android.volley.toolbox.NetworkImageView, and it seems to be working pretty well. Apparently, this is the same view that is used in Google Play and other new Google applications. Definitely worth checking out.
Google I/O 2013 volley image cache tutorial
Developers Google events
Well, image loading time from the Internet has many solutions. You may also use the library Android-Query. It will give you all the required activity. Make sure what you want to do and read the library wiki page. And solve the image loading restriction.
This is my code:
#Override
public View getView(int position, View convertView, ViewGroup parent) {
View v = convertView;
if (v == null) {
LayoutInflater vi = (LayoutInflater)getSystemService(Context.LAYOUT_INFLATER_SERVICE);
v = vi.inflate(R.layout.row, null);
}
ImageView imageview = (ImageView) v.findViewById(R.id.icon);
AQuery aq = new AQuery(convertView);
String imageUrl = "http://www.vikispot.com/z/images/vikispot/android-w.png";
aq.id(imageview).progress(this).image(imageUrl, true, true, 0, 0, new BitmapAjaxCallback() {
#Override
public void callback(String url, ImageView iv, Bitmap bm, AjaxStatus status) {
iv.setImageBitmap(bm);
}
));
return v;
}
It should be solve your lazy loading problem.
I think this issue is very popular among Android developers, and there are plenty of such libraries that claims to resolve this issue, but only a few of them seems to be on the mark. AQuery is one such library, but it is better than most of them in all aspects and is worth trying for.
You must try this Universal Loader is best.
I am using this after done many RnD on lazy loading .
Universal Image Loader
Features
Multithread image loading (async or sync)
Wide customization of ImageLoader's configuration (thread executors, downloader, decoder, memory and disk cache, display image options, etc.)
Many customization options for every display image call (stub images, caching switch, decoding options, Bitmap processing and displaying, etc.)
Image caching in memory and/or on disk (device's file system or SD card)
Listening loading process (including downloading progress)
Android 2.0+ support
Have a look at Shutterbug, Applidium's lightweight SDWebImage (a nice library on iOS) port to Android.
It supports asynchronous caching, stores failed URLs, handles concurrency well, and helpful subclasses are included.
Pull requests (and bug reports) are welcome, too!
DroidParts has ImageFetcher that requires zero configuration to get started.
Uses a disk & in-memory Least Recently Used (LRU) cache.
Efficiently decodes images.
Supports modifying bitmaps in background thread.
Has simple cross-fade.
Has image loading progress callback.
Clone DroidPartsGram for an example:
Novoda also has a great lazy image loading library and many apps like Songkick, Podio, SecretDJ and ImageSearch use their library.
Their library is hosted here on Github and they have a pretty active issues tracker as well. Their project seems to be pretty active too, with over 300+ commits at the time of writing this reply.
Just a quick tip for someone who is in indecision regarding what library to use for lazy-loading images:
There are four basic ways.
DIY => Not the best solution but for a few images and if you want to go without the hassle of using others libraries
Volley's Lazy Loading library => From guys at android. It is nice and everything but is poorly documented and hence is a problem to use.
Picasso: A simple solution that just works, you can even specify the exact image size you want to bring in. It is very simple to use but might not be very "performant" for apps that has to deal with humongous amounts of images.
UIL: The best way to lazy load images. You can cache images(you need permission of course), initialize the loader once, then have your work done. The most mature asynchronous image loading library I have ever seen so far.
If you want to display Shimmer layout like Facebook there is a official facebook library for that. FaceBook Shimmer Android
It takes care of everything, You just need to put your desired design code in nested manner in shimmer frame.
Here is a sample code.
<com.facebook.shimmer.ShimmerFrameLayout
android:id=“#+id/shimmer_view_container”
android:layout_width=“wrap_content”
android:layout_height="wrap_content"
shimmer:duration="1000">
<here will be your content to display />
</com.facebook.shimmer.ShimmerFrameLayout>
And here is the java code for it.
ShimmerFrameLayout shimmerContainer = (ShimmerFrameLayout) findViewById(R.id.shimmer_view_container);
shimmerContainer.startShimmerAnimation();
Add this dependency in your gradle file.
implementation 'com.facebook.shimmer:shimmer:0.1.0#aar'
Here is how it looks like.
Check my fork of LazyList. Basically, I improve the LazyList by delaying the call of the ImageView and create two methods:
When you need to put something like "Loading image..."
When you need to show the downloaded image.
I also improved the ImageLoader by implementing a singleton in this object.
All above code have their own worth but with my personal experience just give a try with Picasso.
Picasso is a library specifically for this purpose, in-fact it will manage cache and all other network operations automatically.You will have to add library in your project and just write a single line of code to load image from remote URL.
Please visit here : http://code.tutsplus.com/tutorials/android-sdk-working-with-picasso--cms-22149
Use the glide library. It worked for me and will work for your code too.It works for both images as well as gifs too.
ImageView imageView = (ImageView) findViewById(R.id.test_image);
GlideDrawableImageViewTarget imagePreview = new GlideDrawableImageViewTarget(imageView);
Glide
.with(this)
.load(url)
.listener(new RequestListener<String, GlideDrawable>() {
#Override
public boolean onException(Exception e, String model, Target<GlideDrawable> target, boolean isFirstResource) {
return false;
}
#Override
public boolean onResourceReady(GlideDrawable resource, String model, Target<GlideDrawable> target, boolean isFromMemoryCache, boolean isFirstResource) {
return false;
}
})
.into(imagePreview);
}
I can recommend a different way that works like a charm: Android Query.
You can download that JAR file from here
AQuery androidAQuery = new AQuery(this);
As an example:
androidAQuery.id(YOUR IMAGEVIEW).image(YOUR IMAGE TO LOAD, true, true, getDeviceWidth(), ANY DEFAULT IMAGE YOU WANT TO SHOW);
It's very fast and accurate, and using this you can find many more features like animation when loading, getting a bitmap (if needed), etc.
Give Aquery a try. It has amazingly simple methods to load and cache images asynchronously.
URLImageViewHelper is an amazing library that helps you to do that.
public class ImageDownloader {
Map<String, Bitmap> imageCache;
public ImageDownloader() {
imageCache = new HashMap<String, Bitmap>();
}
// download function
public void download(String url, ImageView imageView) {
if (cancelPotentialDownload(url, imageView)) {
// Caching code right here
String filename = String.valueOf(url.hashCode());
File f = new File(getCacheDirectory(imageView.getContext()),
filename);
// Is the bitmap in our memory cache?
Bitmap bitmap = null;
bitmap = (Bitmap) imageCache.get(f.getPath());
if (bitmap == null) {
bitmap = BitmapFactory.decodeFile(f.getPath());
if (bitmap != null) {
imageCache.put(f.getPath(), bitmap);
}
}
// No? download it
if (bitmap == null) {
try {
BitmapDownloaderTask task = new BitmapDownloaderTask(
imageView);
DownloadedDrawable downloadedDrawable = new DownloadedDrawable(
task);
imageView.setImageDrawable(downloadedDrawable);
task.execute(url);
} catch (Exception e) {
Log.e("Error==>", e.toString());
}
} else {
// Yes? set the image
imageView.setImageBitmap(bitmap);
}
}
}
// cancel a download (internal only)
private static boolean cancelPotentialDownload(String url,
ImageView imageView) {
BitmapDownloaderTask bitmapDownloaderTask = getBitmapDownloaderTask(imageView);
if (bitmapDownloaderTask != null) {
String bitmapUrl = bitmapDownloaderTask.url;
if ((bitmapUrl == null) || (!bitmapUrl.equals(url))) {
bitmapDownloaderTask.cancel(true);
} else {
// The same URL is already being downloaded.
return false;
}
}
return true;
}
// gets an existing download if one exists for the imageview
private static BitmapDownloaderTask getBitmapDownloaderTask(
ImageView imageView) {
if (imageView != null) {
Drawable drawable = imageView.getDrawable();
if (drawable instanceof DownloadedDrawable) {
DownloadedDrawable downloadedDrawable = (DownloadedDrawable) drawable;
return downloadedDrawable.getBitmapDownloaderTask();
}
}
return null;
}
// our caching functions
// Find the dir to save cached images
private static File getCacheDirectory(Context context) {
String sdState = android.os.Environment.getExternalStorageState();
File cacheDir;
if (sdState.equals(android.os.Environment.MEDIA_MOUNTED)) {
File sdDir = android.os.Environment.getExternalStorageDirectory();
// TODO : Change your diretcory here
cacheDir = new File(sdDir, "data/ToDo/images");
} else
cacheDir = context.getCacheDir();
if (!cacheDir.exists())
cacheDir.mkdirs();
return cacheDir;
}
private void writeFile(Bitmap bmp, File f) {
FileOutputStream out = null;
try {
out = new FileOutputStream(f);
bmp.compress(Bitmap.CompressFormat.PNG, 80, out);
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if (out != null)
out.close();
} catch (Exception ex) {
}
}
}
// download asynctask
public class BitmapDownloaderTask extends AsyncTask<String, Void, Bitmap> {
private String url;
private final WeakReference<ImageView> imageViewReference;
public BitmapDownloaderTask(ImageView imageView) {
imageViewReference = new WeakReference<ImageView>(imageView);
}
#Override
// Actual download method, run in the task thread
protected Bitmap doInBackground(String... params) {
// params comes from the execute() call: params[0] is the url.
url = (String) params[0];
return downloadBitmap(params[0]);
}
#Override
// Once the image is downloaded, associates it to the imageView
protected void onPostExecute(Bitmap bitmap) {
if (isCancelled()) {
bitmap = null;
}
if (imageViewReference != null) {
ImageView imageView = imageViewReference.get();
BitmapDownloaderTask bitmapDownloaderTask = getBitmapDownloaderTask(imageView);
// Change bitmap only if this process is still associated with
// it
if (this == bitmapDownloaderTask) {
imageView.setImageBitmap(bitmap);
// cache the image
String filename = String.valueOf(url.hashCode());
File f = new File(
getCacheDirectory(imageView.getContext()), filename);
imageCache.put(f.getPath(), bitmap);
writeFile(bitmap, f);
}
}
}
}
static class DownloadedDrawable extends ColorDrawable {
private final WeakReference<BitmapDownloaderTask> bitmapDownloaderTaskReference;
public DownloadedDrawable(BitmapDownloaderTask bitmapDownloaderTask) {
super(Color.WHITE);
bitmapDownloaderTaskReference = new WeakReference<BitmapDownloaderTask>(
bitmapDownloaderTask);
}
public BitmapDownloaderTask getBitmapDownloaderTask() {
return bitmapDownloaderTaskReference.get();
}
}
// the actual download code
static Bitmap downloadBitmap(String url) {
HttpParams params = new BasicHttpParams();
params.setParameter(CoreProtocolPNames.PROTOCOL_VERSION,
HttpVersion.HTTP_1_1);
HttpClient client = new DefaultHttpClient(params);
final HttpGet getRequest = new HttpGet(url);
try {
HttpResponse response = client.execute(getRequest);
final int statusCode = response.getStatusLine().getStatusCode();
if (statusCode != HttpStatus.SC_OK) {
Log.w("ImageDownloader", "Error " + statusCode
+ " while retrieving bitmap from " + url);
return null;
}
final HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream inputStream = null;
try {
inputStream = entity.getContent();
final Bitmap bitmap = BitmapFactory
.decodeStream(inputStream);
return bitmap;
} finally {
if (inputStream != null) {
inputStream.close();
}
entity.consumeContent();
}
}
} catch (Exception e) {
// Could provide a more explicit error message for IOException or
// IllegalStateException
getRequest.abort();
Log.w("ImageDownloader", "Error while retrieving bitmap from "
+ url + e.toString());
} finally {
if (client != null) {
// client.close();
}
}
return null;
}
}
I had this issue and implemented lruCache. I believe you need API 12 and above or use the compatiblity v4 library. lurCache is fast memory, but it also has a budget, so if you're worried about that you can use a diskcache... It's all described in Caching Bitmaps.
I'll now provide my implementation which is a singleton I call from anywhere like this:
//Where the first is a string and the other is a imageview to load.
DownloadImageTask.getInstance().loadBitmap(avatarURL, iv_avatar);
Here's the ideal code to cache and then call the above in getView of an adapter when retrieving the web image:
public class DownloadImageTask {
private LruCache<String, Bitmap> mMemoryCache;
/* Create a singleton class to call this from multiple classes */
private static DownloadImageTask instance = null;
public static DownloadImageTask getInstance() {
if (instance == null) {
instance = new DownloadImageTask();
}
return instance;
}
//Lock the constructor from public instances
private DownloadImageTask() {
// Get max available VM memory, exceeding this amount will throw an
// OutOfMemory exception. Stored in kilobytes as LruCache takes an
// int in its constructor.
final int maxMemory = (int) (Runtime.getRuntime().maxMemory() / 1024);
// Use 1/8th of the available memory for this memory cache.
final int cacheSize = maxMemory / 8;
mMemoryCache = new LruCache<String, Bitmap>(cacheSize) {
#Override
protected int sizeOf(String key, Bitmap bitmap) {
// The cache size will be measured in kilobytes rather than
// number of items.
return bitmap.getByteCount() / 1024;
}
};
}
public void loadBitmap(String avatarURL, ImageView imageView) {
final String imageKey = String.valueOf(avatarURL);
final Bitmap bitmap = getBitmapFromMemCache(imageKey);
if (bitmap != null) {
imageView.setImageBitmap(bitmap);
} else {
imageView.setImageResource(R.drawable.ic_launcher);
new DownloadImageTaskViaWeb(imageView).execute(avatarURL);
}
}
private void addBitmapToMemoryCache(String key, Bitmap bitmap) {
if (getBitmapFromMemCache(key) == null) {
mMemoryCache.put(key, bitmap);
}
}
private Bitmap getBitmapFromMemCache(String key) {
return mMemoryCache.get(key);
}
/* A background process that opens a http stream and decodes a web image. */
class DownloadImageTaskViaWeb extends AsyncTask<String, Void, Bitmap> {
ImageView bmImage;
public DownloadImageTaskViaWeb(ImageView bmImage) {
this.bmImage = bmImage;
}
protected Bitmap doInBackground(String... urls) {
String urldisplay = urls[0];
Bitmap mIcon = null;
try {
InputStream in = new java.net.URL(urldisplay).openStream();
mIcon = BitmapFactory.decodeStream(in);
}
catch (Exception e) {
Log.e("Error", e.getMessage());
e.printStackTrace();
}
addBitmapToMemoryCache(String.valueOf(urldisplay), mIcon);
return mIcon;
}
/* After decoding we update the view on the main UI. */
protected void onPostExecute(Bitmap result) {
bmImage.setImageBitmap(result);
}
}
}

Memory leak in javacv

I'm trying to make a program that takes a picture from a webcam, and afterwards resizes it, converts it to HSV, and makes some thresholding on it, to find a specific color. After this is done, I use the thresholded image to find contours, and print the x,y coords of the different contours. This is repeated over and over, to make the processing from the webcam realtime.
It all works quite well, except for the fact that I am using up about 100 mb of RAM every 2 second it runs.
So far I have discovered that if I use a static picture, instead of the live images from the webcam, I can minimize the memory leak significantly, although still there is memory being consumed.
Below my code is:
public class Application {
private CaptureImage ci;
private ImageUtils iu;
private CanvasFrame canvasContours;
IplImage grabbedFrame;
IplImage resizedFrame;
IplImage thresholdedFrame;
IplImage clonedImage;
public Application(){
ci = new CaptureImage();
iu = new ImageUtils();
canvasContours = new CanvasFrame("contours");
}
public void frameProcessing(){
grabbedFrame = ci.grabImage();
//below call used for testing purposes
//grabbedFrame = (IplImage) opencv_highgui.cvLoadImage("testingImage.jpg");
//cloning image due to highgui guidelines.
clonedImage = opencv_core.cvCloneImage(grabbedFrame);
resizedFrame = iu.resizeImage(clonedImage);
opencv_core.cvReleaseImage(clonedImage);
thresholdedFrame = iu.thresholdImage(resizedFrame);
IplImage contoursFrame = iu.findContours(thresholdedFrame, resizedFrame);
canvasContours.showImage(contoursFrame);
}
}
The grabImage is just the standard frameGrabber from javacv, which looks like this:
public class CaptureImage {
private final OpenCVFrameGrabber grabber;
private IplImage img = null;
public CaptureImage(){
// 0-default camera, 1 - next...so on
grabber = new OpenCVFrameGrabber(0);
try {
grabber.start();
} catch (Exception e) {
System.err.print("Failed to initialize camera");
e.printStackTrace();
}
}
public IplImage grabImage(){
try {
//A grabbed image from Logitech webcam is in following resolution: 1200x800px
img = grabber.grab();
} catch (Exception e) {
e.printStackTrace();
}
return img;
}
I appreciate any help you can give me, and if you need any more information, please just ask!
/Jesper
From your heap dump, the used memory is all byte and int arrays that are referenced from native code. Looking at your code I see that you only call cvReleaseImage for the cloned image and not for the original image.

Categories

Resources