Develop Reference

Google MLKit pose detection not executing onSuccessListener or onFailureListener

I am using MediaMetadataRetriever as well as MLkit to do pose detection on a saved video file. I have attached some of my code here:
public ArrayList<ArrayList<Float>> extractPoses() throws FileNotFoundException {
//public Bitmap extractPoses() throws FileNotFoundException {
Log.e("DATA SOURCE: ", videoUri.getPath());
URIPathHelper uph = new URIPathHelper();
String videoInputPath = uph.getPath(applicationContext, videoUri).toString();
File vidInputFile = new File(videoInputPath);
File inputFile = new File(vidInputFile.getAbsolutePath());
if (!inputFile.canRead()) {
throw new FileNotFoundException("Unable to read $inputFile");
}
mdr.setDataSource(inputFile.getAbsolutePath());
ArrayList<ArrayList<Float>> totalVideoPose = new ArrayList<>();
int count = 0;
while (true) {
try {
Bitmap bmp = mdr.getFrameAtIndex(count);
InputImage inputImg = InputImage.fromBitmap(bmp, 0);
poseDetector.process(inputImg)
.addOnSuccessListener(new OnSuccessListener<Pose>() {
#Override
public void onSuccess(Pose pose) {
Log.e("POSE DETECT: ", "success");
if(!(pose.getAllPoseLandmarks().isEmpty())) {
ArrayList<Float> framePoseMarks = new ArrayList<Float>();
PoseLandmark left_shoulder = pose.getPoseLandmark(PoseLandmark.LEFT_SHOULDER);
framePoseMarks.add(left_shoulder.getPosition().x);
framePoseMarks.add(left_shoulder.getPosition().y);
totalVideoPose.add(framePoseMarks);
}
}
})
.addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
Log.e("POSE DETECT: ", "pose detect on failure listener");
}
});
Log.e("Pose processing: ", "incrementing count");
count++;
} catch (IllegalArgumentException e) {
Log.e("POSE DETEC:", "Pose detec ended");
Log.e("POSE DETEC length: ", String.valueOf(totalVideoPose.size()));
for (ArrayList<Float> frame: totalVideoPose) {
Log.e("POSE DETEC SUBS: ", String.valueOf(frame.size()));
}
break;
}
}
return totalVideoPose;
//return null;
}
I know the bitmaps I am using are correct, as I have displayed them in an ImageView to verify. I am not sure why neither is executing - I dont see the output for Log.e("POSE DETECT: ", "success"); or for Log.e("POSE DETECT: ", "pose detect on failure listener");. I appreciate any help and I am happy to answer any follow up questions!

Waiting inside loop for the method to finish executing

I have a For loop in which I call a method to upload images to the server , the problem that i am facing that at a certain limit the server will force stop the opened socket so i have to upload every image at once
for (int i = 0; i < paths.size(); i++) {
transferData(paths.get(i), i);
}
and the transferData Function I am using the transfer Utility aws s3 function
TransferUtility transferUtility =
TransferUtility.builder()
.context(this)
.awsConfiguration(AWSMobileClient.getInstance().getConfiguration())
.s3Client(amazonS3Client)
.defaultBucket("name")
.build();
TransferObserver uploadObserver = transferUtility.upload("name", name, new File(path), CannedAccessControlList.PublicRead);
uploadObserver.setTransferListener(new TransferListener() {
#Override
public void onStateChanged(int id, TransferState state) {
Log.d(TAG, "onStateChanged: " + id + ", " + state);
if (TransferState.COMPLETED == state) {
}
}
#Override
public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
float percentDonef = ((float) bytesCurrent / (float) bytesTotal) * 100;
int percentage = (int) percentDonef;
Log.d(TAG, "onProgressChanged: " + percentage);
}
#Override
public void onError(int id, Exception ex) {
Log.e(TAG, "Error during upload: " + id, ex);
try {
showToast(ex.getMessage());
} catch (Exception e) {
e.printStackTrace();
}
ex.printStackTrace();
}
});
How to wait for the method to finish execution then continue the loop

to wait loop use this code :
CountDownLatch latch=new CountDownLatch(1);
for (int i = 0; i < paths.size(); i++) {
transferData(paths.get(i), i);
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
and put this at the end of your method that finished upload image or failed
latch.countDown();
when your method rich to latch.countDown(); your loop will be continue
notice that you must put your loop in another thread except main thread

Issues using android studio to get time of travel between two points

For the last week, I have been using Android Studio to write code that achieves the following goals:
Wait for the user to be within a certain distance of the start waypoint
Once at start waypoint, begin a timer that logs gps data and current time
Stops timer when the end waypoint is crossed
At the moment, I have the start and end waypoints hard coded but I seem to run into an error that I have been trying to trace with the step through function on my IDE but can't seem to find it. Below is the code I have been using:
void StartTimer (View view){
//Location l = null;
boolean hasLoc = false; //are we at the start?
float speed = 0;
float topSpeed = 0;
while(hasLoc == false && cancel == false){
float d = l.distanceTo(t);
if(d < 2.0)
hasLoc = true;
//if(!l.equals(lm.getLastKnownLocation("")))
String msg = "Latitude: " + l.getLatitude() + "\nLongitude: "+ l.getLongitude();
Toast.makeText(getBaseContext(), msg, Toast.LENGTH_LONG).show();
}
hasLoc = false;
Handler handler = new Handler();
handler.postDelayed(new Runnable() {
public void run() {
// Actions to do after 10 seconds
buzzer();
}
}, 10000);
while(l.distanceTo(tf) > 2.0 && cancel == false){
float cSpeed = l.getSpeed();
if(cSpeed>topSpeed)
topSpeed = cSpeed;
String msg = "Current Speed: "+cSpeed+"Top Speed: "+topSpeed;
Toast.makeText(getBaseContext(), msg, Toast.LENGTH_LONG).show();
}
cancel = false;
}
When I run the code, the phone I test it one will run it but it won't respond, which leads me to believe there is an unsatisfied loop that I have not considered.
Any suggestions would be helpful, thank you in advance for advice!

Your while loops are clogging up the CPU's execution which is what is causing it to not respond. Instead you should place your code inside a thread and call Thread.sleep(1000); inside the thread, this way the the while loop is paused for 1 second after every execution of the code inside it.
Something like this:
new Thread(new Runnable() {
#Override
public void run() {
while (hasLoc == false && cancel == false) {
float d = l.distanceTo(t);
if (d < 2.0)
hasLoc = true;
String msg = "Latitude: " + l.getLatitude() + "\nLongitude: " + l.getLongitude();
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
runOnUiThread(new Runnable() {
#Override
public void run() {
Toast.makeText(getBaseContext(), msg, Toast.LENGTH_LONG).show();
}
});
}
hasLoc = false;
new Handler().postDelayed(new Runnable() {
public void run() {
// Actions to do after 10 seconds
buzzer();
}
}, 10000);
while (l.distanceTo(tf) > 2.0 && cancel == false) {
float cSpeed = l.getSpeed();
if (cSpeed > topSpeed)
topSpeed = cSpeed;
String msg = "Current Speed: " + cSpeed + "Top Speed: " + topSpeed;
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
runOnUiThread(new Runnable() {
#Override
public void run() {
Toast.makeText(getBaseContext(), msg, Toast.LENGTH_LONG).show();
}
});
}
cancel = false;
}
}).start();

mediacodec doesn't work smoothly

using this url I wrote below code to encode onpreviewframe data to mp4 video and I used a thread to do this job well, but it seems that it doesn't work properly.
private void initCodec() {
String root = Environment.getExternalStorageDirectory().toString();
File myDir = new File(root + "/Vocalist");
if(!myDir.exists()) {
myDir.mkdirs();
}
try {
File file = new File (myDir, "myVideo.mp4");
if(file.exists()){
file.delete();
}
fos = new FileOutputStream(file, false);
} catch (FileNotFoundException e) {
e.printStackTrace();
}try {
mMediaCodec = MediaCodec.createEncoderByType("video/avc");
}
catch (Exception e){
}
MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc",
320,
240);
mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 500000);
mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 15);
mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT,
MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar);
mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
mMediaCodec.configure(mediaFormat,
null,
null,
MediaCodec.CONFIGURE_FLAG_ENCODE);
mMediaCodec.start();
inputBuffers = mMediaCodec.getInputBuffers();
outputBuffers = mMediaCodec.getOutputBuffers();
}
private synchronized void encode(byte[] dataInput)
{
byte[] data = dataInput;
inputBuffers = mMediaCodec.getInputBuffers();// here changes
outputBuffers = mMediaCodec.getOutputBuffers();
int inputBufferIndex = mMediaCodec.dequeueInputBuffer(-1);
if (inputBufferIndex >= 0) {
ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
inputBuffer.clear();
inputBuffer.put(data);
mMediaCodec.queueInputBuffer(inputBufferIndex, 0, data.length, 0, 0);
} else {
return;
}
MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
int outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, 0);
Log.i("tag", "outputBufferIndex-->" + outputBufferIndex);
do {
if (outputBufferIndex >= 0) {
ByteBuffer outBuffer = outputBuffers[outputBufferIndex];
System.out.println("buffer info-->" + bufferInfo.offset + "--"
+ bufferInfo.size + "--" + bufferInfo.flags + "--"
+ bufferInfo.presentationTimeUs);
byte[] outData = new byte[bufferInfo.size];
outBuffer.get(outData);
try {
if (bufferInfo.offset != 0) {
fos.write(outData, bufferInfo.offset, outData.length
- bufferInfo.offset);
} else {
fos.write(outData, 0, outData.length);
}
fos.flush();
Log.i("camera", "out data -- > " + outData.length);
mMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo,
0);
} catch (IOException e) {
e.printStackTrace();
}
} else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
outputBuffers = mMediaCodec.getOutputBuffers();
} else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
MediaFormat format = mMediaCodec.getOutputFormat();
}
} while (outputBufferIndex >= 0);
}
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
if (mHolder.getSurface() == null) {
return;
}
try {
initCodec();
mCamera.setPreviewDisplay(mHolder);
mCamera.setPreviewCallback(new Camera.PreviewCallback() {
#Override
public void onPreviewFrame(final byte[] bytes, Camera camera) {
if (recording == true) {
if(mThread.isAlive())
encode(bytes);
}
}
});
} catch (Exception e) {
Log.d("TAG", "Error starting camera preview: " + e.getMessage());
}
}
}
public void newOpenCamera() {
if (mThread == null) {
mThread = new CameraHandlerThread();
}
synchronized (mThread) {
mThread.openCamera();
}
}
private static void oldOpenCamera() {
try {
c = Camera.open(1);
Camera.Parameters parameters = c.getParameters();
parameters.set("orientation", "portrait");
parameters.setJpegQuality(100);
parameters.setPreviewFormat(ImageFormat.NV21);
parameters.setPreviewSize(320, 240);
c.setParameters(parameters);
}
catch (RuntimeException e) {
Log.e("camera", "failed to open front camera");
}
}
public CameraHandlerThread mThread = null;
public static class CameraHandlerThread extends HandlerThread {
Handler mHandler = null;
CameraHandlerThread() {
super("CameraHandlerThread");
start();
mHandler = new Handler(getLooper());
}
synchronized void notifyCameraOpened() {
notify();
}
public void openCamera() {
mHandler.post(new Runnable() {
#Override
public void run() {
oldOpenCamera();
notifyCameraOpened();
}
});
}
}
I converted onpreviewframe data to a video but after first second video doesn't play smoothly. what should I do ?

First, you're not forwarding the timing information with the frames:
mMediaCodec.queueInputBuffer(inputBufferIndex, 0, data.length, 0, 0)
So your BufferInfo.presentationTimeUs will always be zero when you dequeue the buffer.
Second, you don't appear to be using MediaMuxer, which means you're just writing raw the raw H.264 stream to a file. This is not ".mp4"; it doesn't include the timing information at all. Many video players don't even know what to do with plain H.264.
Wrapping the file as .mp4, with the frame timing from the camera, should yield better results.
Your code structure appears to be assuming that it can feed one frame of input and get one frame of output, which isn't always the case. You want to keep the input full, and drain the output as it becomes available.
You can find more information and some sample code on bigflake and in Grafika.

How can I read and write data about the accelerometer in android?

I am trying to write a step counter for android. It currently includes four buttons in one activity:
a button to start recording accelerometer data which is stored in an arraylist. The arraylist takes a type Trace, which is a class I created to hold the data of one sensor change. I also have a stop button, and buttons to read or write the data from text file.
This program keeps giving me a NullPointerException on the arraylist and I can't figure out why. Any help would be greatly appreciated!
EDIT: Sorry if the indentation is off or the code is unclear, this is a school assignment and I'm on a strict deadline so I have to rush to make the code usable before I can worry about readability or efficiency.
EDIT 2: I no longer get any exceptions, however I still cannot read/write properly. I was able to write to file one successfully, and then somehow it stops functioning.
package com.myApp.playpool;
//imports
public class MainActivityBAK extends Activity implements SensorEventListener{
//global fields (traces is instantiated here as new arraylist)
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
acceleration=(TextView)findViewById(R.id.acceleration);
startButton = (Button) findViewById(R.id.startButton);
stopButton = (Button) findViewById(R.id.stopButton);
readButton = (Button) findViewById(R.id.readButton);
writeButton = (Button) findViewById(R.id.writeButton);
sm = (SensorManager) getSystemService(Context.SENSOR_SERVICE);
accelerometer = sm.getDefaultSensor(Sensor.TYPE_ACCELEROMETER);
sm.registerListener(this, accelerometer, SensorManager.SENSOR_DELAY_GAME);
dataFilePath = getString(R.string.data_file_path);
acceleration.setText("Current file: " + dataFilePath);
lastCheck = System.currentTimeMillis();
defineButtons();
}
#Override
public boolean onCreateOptionsMenu(Menu menu) {
getMenuInflater().inflate(R.menu.activity_main, menu);
return true;
}
#Override
public void onSensorChanged(SensorEvent event1) {
if (running && ((System.currentTimeMillis() - lastCheck) > 1000)) {
acceleration.setText("X: "+event1.values[0]+"\nY: "+event1.values[1]+"\nZ: "+event1.values[2]);
traces.add(new Trace(System.currentTimeMillis(), event1.values[0], event1.values[1], event1.values[2]));
lastCheck = System.currentTimeMillis();
}
}
#Override
public void onAccuracyChanged(Sensor sensor, int accuracy) {
}
public void defineButtons() { //defines onClick methods for the buttons
startButton.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
traces = new ArrayList<Trace>();
running = true;
}
});
stopButton.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
traces = new ArrayList<Trace>();
running = false;
}
});
readButton.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
try {
scan = new Scanner(new File(dataFilePath));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
while (scan.hasNext()) {
String str = scan.nextLine();
String [] strings = str.split(";");
double time = Double.parseDouble(strings[0]);
float x = Float.parseFloat(strings[0]), y = Float.parseFloat(strings[1]), z = Float.parseFloat(strings [2]);
traces.add(new Trace(time, x, y, z));
}
Toast.makeText(getBaseContext(),
("Done reading to SD file: '" + dataFilePath + "'"),
Toast.LENGTH_SHORT).show();
scan.close();
}
});
writeButton.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
try {
File file = new File(dataFilePath);
print = new PrintWriter(file);
file.createNewFile();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
for (int i = 0; i < traces.size(); i++) {
double time = traces.get(i).time;
float x = traces.get(i).x, y = traces.get(i).y, z = traces.get(i).z;
print.println(time + ";" + x + ";" + y + ";" + z + ";");
}
print.close();
Toast.makeText(getBaseContext(),
("Done writing to SD file: '" + dataFilePath + "'"),
Toast.LENGTH_SHORT).show();
}
});
}
}
It seems to be crashing in the for loop in the write method:
for (int i = 0; i < traces.size(); i++) {

You are only initializing the traces variable if your buttons start or stop are clicked.
If you click the "write" button before that, then the reference is still null.

It looks perhaps like traces has not been initialized in either onSensorChanged() or in your readButton.setOnClickListener, although it's hard to tell without the full source code or stack trace. If traces is a "// global field" then you should probably initialize it where it's declared, like:
protected ArrayList<Trace> traces = new ArrayList<Trace>();
EDIT for 2nd Question
My guess is that using the File.createNewFile() method won't work if the file already exists. Maybe try a different approach, like this sample:
Writer writer = null;
try {
OutputStream out = mContext.openFileOutput(dataFileName, Context.MODE_PRIVATE);
writer = new OutputStreamWriter(out);
for (int i = 0; i < traces.size(); i++) {
double time = traces.get(i).time;
float x = traces.get(i).x, y = traces.get(i).y, z = traces.get(i).z;
writer.write(time + ";" + x + ";" + y + ";" + z + ";");
}
}
finally {
if (writer != null) {
writer.close();
}
}