Develop Reference

How to use a reprocessCaptureRequest with camera2 API

I'm trying to update a camera project to Android N and in consequence I'm moving my old CameraCaptureSession to a ReprocessableCaptureSession. I did it and it is working fine, but with this new feature I can use the CameraDevice.TEMPLATE_ZERO_SHUTTER_LAG template in my device and I can reprocess frames with the reprocessCaptureRequest.
Here is where my problem appear. Because I don't find any example, and I don't really understand the little documentation about how to use a reprocessCaptureRequest:
Each reprocess CaptureRequest processes one buffer from CameraCaptureSession's input Surface to all output Surfaces included in the reprocess capture request. The reprocess input images must be generated from one or multiple output images captured from the same camera device. The application can provide input images to camera device via queueInputImage(Image). The application must use the capture result of one of those output images to create a reprocess capture request so that the camera device can use the information to achieve optimal reprocess image quality. For camera devices that support only 1 output Surface, submitting a reprocess CaptureRequest with multiple output targets will result in a CaptureFailure.
I tried to have a look to the CTS tests about the camera in google.sources but they do the same than me. Using multiples imageReaders, saving the TotalCaptureResult of the pictures in a LinkedBlockingQueue<TotalCaptureResult>. And later just calling:
TotalCaptureResult totalCaptureResult = state.captureCallback.getTotalCaptureResult();
CaptureRequest.Builder reprocessCaptureRequest = cameraStore.state().cameraDevice.createReprocessCaptureRequest(totalCaptureResult);
reprocessCaptureRequest.addTarget(state.yuvImageReader.getSurface());
sessionStore.state().session.capture(reprocessCaptureRequest.build(), null, this.handlers.bg());
But it always throw me a RuntimeException:
java.lang.RuntimeException: Capture failed: Reason 0 in frame 170,
I just want to know which is the right way to work with the ReprocessableCaptureSession because I already tried everything and I don't know what I'm doing wrong.

Finally I found the solution to make my reprocessableCaptureSession work.
I use with Flux architecture so don't be confused when you see Dispatcher.dispatch(action), just see it as a callback. So, here is my code:
First How the session is created:
//Configure preview surface
Size previewSize = previewState.previewSize;
previewState.previewTexture.setDefaultBufferSize(previewSize.getWidth(), previewSize.getHeight());
ArrayList<Surface> targets = new ArrayList<>();
for (SessionOutputTarget outputTarget : state.outputTargets) {
Surface surface = outputTarget.getSurface();
if (surface != null) targets.add(surface);
}
targets.add(previewState.previewSurface);
CameraCharacteristics cameraCharacteristics = cameraStore.state().availableCameras.get(cameraStore.state().selectedCamera);
Size size = CameraCharacteristicsUtil.getYuvOutputSizes(cameraCharacteristics).get(0);
InputConfiguration inputConfiguration = new InputConfiguration(size.getWidth(),
size.getHeight(), ImageFormat.YUV_420_888);
CameraCaptureSession.StateCallback sessionStateCallback = new CameraCaptureSession.StateCallback() {
#Override
public void onConfigured(#NonNull CameraCaptureSession session) {
if (sessionId != currentSessionId) {
Timber.e("Session opened for an old open request, skipping. Current %d, Request %d", currentSessionId, sessionId);
//performClose(session);
return;
}
try {
session.getInputSurface();
//This call is irrelevant,
//however session might have closed and this will throw an IllegalStateException.
//This happens if another camera app (or this one in another PID) takes control
//of the camera while its opening
} catch (IllegalStateException e) {
Timber.e("Another process took control of the camera while creating the session, aborting!");
}
Dispatcher.dispatchOnUi(new SessionOpenedAction(session));
}
#Override
public void onConfigureFailed(#NonNull CameraCaptureSession session) {
if (sessionId != currentSessionId) {
Timber.e("Configure failed for an old open request, skipping. Current %d, request %d", currentSessionId, sessionId);
return;
}
Timber.e("Failed to configure the session");
Dispatcher.dispatchOnUi(new SessionFailedAction(session, new IllegalStateException("onConfigureFailed")));
}
};
if (state.outputMode == OutputMode.PHOTO) {
cameraState.cameraDevice.createReprocessableCaptureSession(inputConfiguration, targets, sessionStateCallback, handlers.bg());
} else if (state.outputMode == OutputMode.VIDEO) {
cameraState.cameraDevice.createCaptureSession(targets, sessionStateCallback, handlers.bg());
}
} catch (IllegalStateException | IllegalArgumentException e) {
Timber.e(e, "Something went wrong trying to start the session");
} catch (CameraAccessException e) {
//Camera will throw CameraAccessException if another we try to open / close the
//session very fast.
Timber.e("Failed to access camera, it was closed");
}
Photo session as been created with 4 surfaces(Preview, YUV(input), JPEG and RAW). After it, I configure my imageWriter:
Dispatcher.subscribe(Dispatcher.VERY_HIGH_PRIORITY, SessionOpenedAction.class)
.filter(a -> isInPhotoMode())
.subscribe(action -> {
PhotoState newState = new PhotoState(state());
newState.zslImageWriter = ImageWriter.newInstance(action.session.getInputSurface(), MAX_REPROCESS_IMAGES);
setState(newState);
});
Ok, now we have the ImageWriter and the session created. No we start the streaming with the repeating request:
CaptureRequest.Builder captureRequestBuilder =
cameraStore.state().cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_ZERO_SHUTTER_LAG);
captureRequestBuilder.addTarget(previewStore.state().previewSurface);
captureRequestBuilder.addTarget(photoStore.state().yuvImageReader.getSurface());
state.session.setRepeatingRequest(captureRequestBuilder.build(), state.zslCaptureCallback, handlers.bg());
To don't add a lot of code, just say that the zslCaptureCallback is a custom callback which save in a LinkedBlockingQueue<TotalCaptureRequest> the X last TotalCaptureRequests. Also, I do the same with the yuvImageReader(input one) saving the last X images in a queue.
Finally here is my "take photo" method:
try {
//Retrieve the last image stored by the zslImageReader
Image image = zslImageReaderListener.getImage();
//Retrieve the last totalCaptureResult from the zslCaptureCallback and create a reprocessableCaptureRequest with it
TotalCaptureResult captureResult = sessionStore.state().zslCaptureCallback.getCaptureResult(image.getTimestamp());
CaptureRequest.Builder captureRequest = cameraStore.state().cameraDevice.createReprocessCaptureRequest(captureResult);
//Add the desired target and values to the captureRequest
captureRequest.addTarget(state().jpegImageReader.getSurface());
//Queued back to ImageWriter for future consumption.
state.zslImageWriter.queueInputImage(image);
//Drain all the unused and queued CapturedResult from the CaptureCallback
sessionStore.state().zslCaptureCallback.drain();
//Capture the desired frame
CaptureRequest futureCaptureResult = captureRequest.build();
sessionStore.state().session.capture(futureCaptureResult, new CameraCaptureSession.CaptureCallback() {
#Override
public void onCaptureCompleted(#NonNull CameraCaptureSession session,
#NonNull CaptureRequest request,
#NonNull TotalCaptureResult result) {
Dispatcher.dispatchOnUi(new PhotoStatusChangedAction(PhotoState.Status.SUCCESS));
}
#Override
public void onCaptureFailed(#NonNull CameraCaptureSession session,
#NonNull CaptureRequest request,
#NonNull CaptureFailure failure) {
super.onCaptureFailed(session, request, failure);
Exception captureFailedException = new RuntimeException(
String.format("Capture failed: Reason %s in frame %d, was image captured? -> %s",
failure.getReason(),
failure.getFrameNumber(),
failure.wasImageCaptured()));
Timber.e(captureFailedException, "Cannot take mediaType, capture failed!");
Dispatcher.dispatchOnUi(new PhotoStatusChangedAction(PhotoState.Status.ERROR, captureFailedException));
}
}, this.handlers.bg());
//Capture did not blow up, we are taking the photo now.
newState.status = PhotoState.Status.TAKING;
} catch (CameraAccessException | InterruptedException| IllegalStateException | IllegalArgumentException | SecurityException e) {
Timber.e(e, "Cannot take picture, capture error!");
newState.status = PhotoState.Status.ERROR;
}

Facebook SDK blocking upload of String to Firebase

I'm developing an Android app which uses Facebook Login. Login's working fine and I'm able to get back info via Facebook Graph API calls.
I'm trying upload a string to my Firebase database and the string (titled parentFirstNameFromFacebook) is the first name of the user who signed into Facebook. I'm trying to eventually upload a parent object with parentFirstNameFromFacebook in its setName() method.
doneCreatingNameAndPasswordFAB.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
gettingTextFromNameAndPasswordEditTexts();
//region creating new Parent object and setting required variables
Parent coOpCreatingParent = new Parent();
coOpCreatingParent.setCoopCreator(true);
coOpCreatingParent.setNumOfHoursOwned(0);
Bundle params = new Bundle();
params.putString("fields", "id,first_name");
new GraphRequest(AccessToken.getCurrentAccessToken(), "me", params, HttpMethod.GET,
new GraphRequest.Callback() {
#Override
public void onCompleted(GraphResponse response) {
if (response != null) {
try {
JSONObject data = response.getJSONObject();
parentFirstNameFromFacebook = data.getString("first_name");
SharedPreferences fBookSharedPref;
SharedPreferences.Editor editor;
fBookSharedPref = getSharedPreferences(Constants.FBOOK_NAME_SHARED_PREF, Context.MODE_PRIVATE);
editor = fBookSharedPref.edit();
editor.putString(Constants.FBOOK_NAME_SHARED_PREF, parentFirstNameFromFacebook);
editor.apply();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}).executeAsync();
SharedPreferences fBookSharedPref = getSharedPreferences(Constants.FBOOK_NAME_SHARED_PREF, Context.MODE_PRIVATE);
fbookFirstNameForUpload = fBookSharedPref.getString(Constants.FBOOK_NAME_SHARED_PREF, null);
coOpCreatingParent.setName(fbookFirstNameForUpload);
getProfileImageUrlFromFBookGraph();
coOpCreatingParent.setImageUrl(parentImageIDFromFBookGraph);
ArrayList<Child> newChildArrayList = new ArrayList<>();
coOpCreatingParent.setChildren(newChildArrayList);
//endregion
//region creating new ArrayList<Parent> adding Parent object from above
ArrayList<Parent> coOpParentArrayList = new ArrayList<>();
coOpParentArrayList.add(coOpCreatingParent);
//endregion
getReferenceOfCoOpBeingCreated();
//region uploading entire new Co-Op object to Firebase
CoOp coOpObjectBeingUploaded = new CoOp(coOpKey, enteredNewCoOpPassword, enteredNewCoOpName, coOpParentArrayList);
referenceOfCoOp.setValue(coOpObjectBeingUploaded);
//endregion
//region going to AddChildren Activity with Co-Op key in intent
Intent intentForNewActivity = new Intent(getBaseContext(), AddChildrenActivity.class);
intentForNewActivity.putExtra(Constants.CO_OP_REFERENCE_TO_CHILD_ACTIVITY_KEY, coOpKey);
startActivity(intentForNewActivity);
//endregion
}
});
In the Firebase screenshot below, parentsInCoOp is an ArrayList and the 0 below it is the Parent object. I'm trying to set a name and Facebook URL string as other variables for that Parent object.
Firebase screenshot
Whatever I do, the name doesn't show up on Firebase! I'm not sure what I'm doing wrong!
I also tried using Shared Preferences in case there's an issue with setting parentFirstNameFromFacebook within the Facebook call's onCompleted() method. I put parentFirstNameFromFacebook in SharedPref and then got it out to pass it through parent.setName().
Once I successfully upload the first name, then I'll do the same process for the fbook image URl string.
Anyone have any advice? Thanks!

You are performing the Facebook API GraphRequest using the executeAsync() method. The request is processed on a separate thread and the onComplete() response is received asynchronously. I don't use the Facebook API, but am guessing the request requires communication with the Facebook servers and will require many milliseconds to deliver a response.
The result is that these statements
fbookFirstNameForUpload =
fBookSharedPref.getString(Constants.FBOOK_NAME_SHARED_PREF, null);
coOpCreatingParent.setName(fbookFirstNameForUpload);
execute before the statements in the callback have executed and stored the name in preferences.

Android app crashes when i try to reLogin

I have an android app that is connected to an API through retrofit, ive succesfully logged in, if i press back button to return back to the login activity again, if i try re-logging in again, the app crashes and give me a NullPointerException.
here's connection code
private void loginUser(String email, String password) {
UnifyAuthenticationApiInterface service = this.client.create(UnifyAuthenticationApiInterface.class);
Call<UnifyAuthenticationApiResponse> call = service.staffLogin(email, password);
call.enqueue(new Callback<UnifyAuthenticationApiResponse>() {
#Override
public void onResponse(Call<UnifyAuthenticationApiResponse> call,
Response<UnifyAuthenticationApiResponse> response) {
UnifyAuthenticationApiResponse result = response.body();
School school = new School();
com.peterstev.unify.login.Data data = result.getData();
mySchoolsList = new ArrayList<School>();
mySchoolsList = data.getSchools();
staff = data.getStaff();
gotoHomeActivity();
}
#Override
public void onFailure(Call<UnifyAuthenticationApiResponse> call, Throwable t) {
progressDialog.dismiss();
Toast.makeText(MainActivity.this, "Login Failed # onFailure", Toast.LENGTH_SHORT).show();
}
});
}
and the goToHomeActivity() is
private void gotoHomeActivity() {
progressDialog.dismiss();
if (mySchoolsList.size() > 1) {
schoolsListView = new ListView(MainActivity.this);
schoolsArrayAdapter = new SchoolListAdapter(MainActivity.this, android.R.layout.simple_list_item_1, mySchoolsList);
schoolsListView.setAdapter(schoolsArrayAdapter);
dialog = new Dialog(MainActivity.this);
dialog.setContentView(schoolsListView);
dialog.setTitle("Welcome " + staff.getFullName());
dialog.show();
} else {
Intent intent = new Intent(MainActivity.this, NavMainActivity.class);
startActivity(intent);
}
}
the NullPointerException gets thrown at
com.peterstev.unify.login.Data data = result.getData();
at first, it gets the data n succesfully logs in, but when i use the back button n try to log in again it crashes.

Debugger is your answer - check if you aren't loosing any data when going back - maybe you're storing login params somewhere in activity class but you're not saving instance state properly and second request is triggered without necessary data. Check state of variables just before calling your request first and second time.
In situation like that always best bet to place breakpoint and trigger your work step by step. You cannot be good developer without debugger skills.

I think for some reason, the data object wasn't receiving the result when i used the back button to navigate to the parent activity. so i used and if condition to make it get the required data.
private void loginUser(String email, String password) {
UnifyAuthenticationApiInterface service = this.client.create(UnifyAuthenticationApiInterface.class);
Call<UnifyAuthenticationApiResponse> call = service.staffLogin(email, password);
call.enqueue(new Callback<UnifyAuthenticationApiResponse>() {
#Override
public void onResponse(Call<UnifyAuthenticationApiResponse> call,
Response<UnifyAuthenticationApiResponse> response) {
if(response.isSuccessful()) {
UnifyAuthenticationApiResponse result = response.body();
School school = new School();
data = result.getData();
if(data == null) {
try{
this.onResponse(call, response);
}catch(NullPointerException NPE){
Log.d("NPE", NPE.getMessage());
}
}
mySchoolsList = new ArrayList<School>();
mySchoolsList = data.getSchools();
staff = data.getStaff();
gotoHomeActivity();
}
}
#Override
public void onFailure(Call<UnifyAuthenticationApiResponse> call, Throwable t) {
progressDialog.dismiss();
Toast.makeText(MainActivity.this, "Login Failed # onFailure", Toast.LENGTH_SHORT).show();
}
});
}

Volley request too slow

My app crashes because the images ArrayList is empty when I set the adapter, I figured that out by putting a toast message right after I parse my JSON request, and a Toast message after I initialize my adapter, "second" gets printed first on screen and the app crashes right after, does it have to do with my internet? Or am I missing something, here's my code, thanks!
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_page);
mViewPager = (ViewPager) findViewById(R.id.view_pager);
mVolleySingleton = VolleySingleton.getInstance();
mRequestQueue = mVolleySingleton.getRequestQueue();
//First Toast message inside this method
sendAPIRequest();
//after you get the images
mCustomSwipeAdapter = new CustomSwipeAdapter(this, images);
//SECOND TOAST
Toast.makeText(getApplicationContext(), "Second", Toast.LENGTH_LONG).show();
mViewPager.setAdapter(mCustomSwipeAdapter);
mCustomSwipeAdapter.notifyDataSetChanged();
}
public void sendAPIRequest(){
String requestURL = "";
JsonObjectRequest jsonObjectRequest = new JsonObjectRequest(Request.Method.GET, requestURL, (String) null, new Response.Listener<JSONObject>() {
#Override
public void onResponse(JSONObject response) {
parseJSONResponse(response);
//FIRST TOAST : SHOULD BE CALLED FIRST
Toast.makeText(getApplicationContext(), "First", Toast.LENGTH_LONG).show();
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
}
});
mRequestQueue.add(jsonObjectRequest);
}
public void parseJSONResponse(JSONObject response) {
if (response != null || response.length() != 0) {
try {
JSONObject GObject = response.getJSONObject("game");
String name = "N/A";
if (GObject.has("name") && !GObject.isNull("name")) { name = GObject.getString("name"); }
if (GObject.has("screenshots") && !GObject.isNull("screenshots")) {
JSONArray screenShotsArray = GObject.getJSONArray("screenshots");
for (int i = 0; i < screenShotsArray.length(); i++){
JSONObject screenshot = screenShotsArray.getJSONObject(i);
String screenshotURL = screenshot.getString("url");
images.add(screenshotURL);
}
}
} catch (JSONException e) {
e.printStackTrace();
}
}
}

Does it have to do with my internet? Or am I missing something ...
Both. It happens because you have a race condition.
From what I can make out, your images list is being populated asynchronously by the onResponse callback. Basically, that happens when your app gets the responses to the API requests that it is making. That is going to take at least milliseconds, and possibly seconds (or longer).
But your app is (so you say) crashing soon after the swipe adapter is registered, and the evidence is that the images list has not been populated.
There are three possibilities:
There is something wrong with the requests you are sending which is causing the API requests to not give you any response. (Hypothetically, you could have authentication wrong or something.)
The API requests are taking a long time because of internet connection speed, congestion, or the remote server being slow.
The API requests are taking a short time ... but the adapter registration is even quicker.
If (hypothetically) there is a problem with your requests you will need to fix that. But both of the other scenarios have to be fixed by:
modifying the code that uses the images to work properly if there are no images (yet), or
modifying the code to wait until the image loading has completed before registering the adapter.

Please use this code in your onResponse callback :
//after you get the images
mCustomSwipeAdapter = new CustomSwipeAdapter(this, images);
//SECOND TOAST
Toast.makeText(getApplicationContext(), "Second", Toast.LENGTH_LONG).show();
mViewPager.setAdapter(mCustomSwipeAdapter);
mCustomSwipeAdapter.notifyDataSetChanged();
Volley adds your requests in queue , so better do all the dependent tasks in Response or Error callback only.

Parse getCurrentUser() is returning null after login successful

In an android app, which uses Parse, the login flow is like this...
We have our own logic to see if the user has entered the correct credentials. Once we verify that, signUpOrLoginOnParse() is called. Parse is used just to store data and handle sessions locally. Users can not access the api without the token.
private void signUpOrLogin(final String username, final String token) {
ParseUser user = new ParseUser();
user.setUsername(username);
user.setPassword(username);
user.signUpInBackground(new SignUpCallback() {
#Override
public void done(ParseException e) {
if (e == null) {
// sign up success. ParseUser.getCurrentUser() populated
saveTokenToCloud(token);
} else if ("condition to check if the user already exists") {
// existing user, login.
ParseUser.logInInBackground(username, username, new LogInCallback() {
#Override
public void done(ParseUser parseUser, ParseException e) {
// login was successful, ParseUser.getCurrentUser() populated
saveTokenToCloud(token);
}
});
} else {
showProgress(false);
e.printStackTrace();
}
}
});
}
private void saveTokenToCloud(String token) {
// saving token to cloud
ParseUser user = ParseUser.getCurrentUser();
user.put("token", token); // THIS IS WHERE I GET NULL POINTER EXCEPTION
user.saveEventually();
// link installation to user.
ParseInstallation parseInstallation = ParseInstallation.getCurrentInstallation();
parseInstallation.put("user", user);
parseInstallation.saveEventually();
// Starting next activity
Intent i = new Intent(this, NextActivity.class);
i.setFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP);
startActivity(i);
finish();
}
All good when I first run the app. Once, logout button is pressed (contains - Parse.logoutInBackground()), it shows the LoginActivity (current one). When trying to log in, everything succeeds but I get a NullPointerException at line 3 of saveTokenToCloud().
It says - trying to invoke virtual method .put() over a null object reference.
But, isn't Parse.currentUser() already populated since this method is called from callback of methods that do that ?
It works after restarting the app. But then the same continues if logout is pressed.

After calling logoutInBackground , future calls to getCurrentUser() will return null.
You will need to initialize the user again.
signUpInBackground will create a new ParseUser on the server, and also persist the session on disk so that you can access the user using ParseUser.getCurrentUser().
However i am not sure you should be calling it every single time you log in.
Instead of calling the getCurrentuser inside the saveToken method you can pass the user to the saveTokenMethod from the done callback parameter.
Separate you logic in distinct methods for sign up and logIn. I suggest you check before calling signUp, and not abusing it every time you want to login