Develop Reference

How to concatenate clips getting from Kinesis Video Stream in Java

I'm using AWS Kinesis Video Stream service to get my video recordings. So due to the Kinesis Video Stream fragment limitation, it turns out I can only retrieve up to ~30 minutes video at one request. And I was intend to retrieve a 2 hour video.
So I loop the request and get all 4 response into a List of InputStream, then I turn them into SequenceInputStream because I try to chain them all together.
However when I success uploaded them to S3 bucket and try to download from there. It shows me file are corrupted. I researched on SequenceInputStream however it seems that my design was okay.
Furthermore, if I extend my video length, let say I have 24 InputStream, and I chained them all to a single SequenceInputStream, it will encounter the SSL Socket Exception: Connection Reset when I run the readAllBytes operation on the sequence input stream.
Is there any way I can achieve what I want or something wrong in my code to cause this?
Here are my source code:
private String downloadMedia(Request request, JSONObject response, JSONObject metaData, Date startDate, Date endDate) throws Exception {
long duration = endDate.getTime() - startDate.getTime();
long durationInMinutes = TimeUnit.MILLISECONDS.toMinutes(duration);
long intervalsCount = durationInMinutes / 30;
ArrayList<GetClipResult> getClipResults = new ArrayList<>();
for (int i = 0; i < intervalsCount; i++){
Media currentMedia = constructMediaAfterIntervalsBreakdown(metaData, request, startDate, endDate);
String deviceName = metaData.getString("name") + "_" + request.getId();
Stream stream = getStreamByName(name, request.getId());
String endPoint = getDataEndpoint(stream.getStreamName());
GetClipResult clipResult = downloadMedia(currentMediaDto, endPoint, stream.getStreamName());
if(clipResult != null){
getClipResults.add(clipResult);
}
startDate = currentMediaDto.getEndTime();
}
//Get presigned URL from S3 service response
String url = response.getJSONArray("data").getJSONObject(0).getJSONArray("parts").getJSONObject(0).getString("url");
if (getClipResults.size() > 0) {
Vector<InputStream> inputStreams = new Vector<>();
for (GetClipResult clipResult : getClipResults){
InputStream videoStream = clipResult.getPayload();
inputStreams.add(videoStream);
}
Enumeration<InputStream> inputStreamEnumeration = inputStreams.elements();;
SequenceInputStream sequenceInputStream = new SequenceInputStream(inputStreamEnumeration);
if (sequenceInputStream.available() > 0){
sequenceInputStream.readAllBytes();
byte[] bytes = sequenceInputStream.readAllBytes();
String message = uploadFileUsingSecureUrl(url, bytes, metaData);
return message;
}
}
return "failed";
}
Edited: I came across couple package that called Xuggler and FFMPEG, however most of them are getting the video file from disk (which has a path), but for my case there isn't any video file because I do not download them to local, they only existed in the runtime and will upload to S3 later on after concatenated.
Appreciates any help! Thank you!

So in the end I just downloaded the clips, saved it to the disk on runtime, merged them using mp4parser and upload to S3. Afterwards I just deleted those on my disk.
If anyone curious about the code, it is taken from https://github.com/sannies/mp4parser/blob/master/examples/src/main/java/com/googlecode/mp4parser/AppendExample.java
Thank you.

How to set HTTP header in Apache JClouds?

I'm using Apache JClouds to connect to my Openstack Swift installation. I managed to upload and download objects from Swift. However, I failed to see how to upload dynamic large object to Swift.
To upload dynamic large object, I need to upload all segments first, which I can do as usual. Then I need to upload a manifest object to combine them logically. The problem is to tell Swift this is a manifest object, I need to set a special header, which I don't know how to do that using JClouds api.
Here's a dynamic large object example from openstack official website.
The code I'm using:
public static void main(String[] args) throws IOException {
BlobStore blobStore = ContextBuilder.newBuilder("swift").endpoint("http://localhost:8080/auth/v1.0")
.credentials("test:test", "test").buildView(BlobStoreContext.class).getBlobStore();
blobStore.createContainerInLocation(null, "container");
ByteSource segment1 = ByteSource.wrap("foo".getBytes(Charsets.UTF_8));
Blob seg1Blob = blobStore.blobBuilder("/foo/bar/1").payload(segment1).contentLength(segment1.size()).build();
System.out.println(blobStore.putBlob("container", seg1Blob));
ByteSource segment2 = ByteSource.wrap("bar".getBytes(Charsets.UTF_8));
Blob seg2Blob = blobStore.blobBuilder("/foo/bar/2").payload(segment2).contentLength(segment2.size()).build();
System.out.println(blobStore.putBlob("container", seg2Blob));
ByteSource manifest = ByteSource.wrap("".getBytes(Charsets.UTF_8));
// TODO: set manifest header here
Blob manifestBlob = blobStore.blobBuilder("/foo/bar").payload(manifest).contentLength(manifest.size()).build();
System.out.println(blobStore.putBlob("container", manifestBlob));
Blob dloBlob = blobStore.getBlob("container", "/foo/bar");
InputStream input = dloBlob.getPayload().openStream();
while (true) {
int i = input.read();
if (i < 0) {
break;
}
System.out.print((char) i); // should print "foobar"
}
}
The "TODO" part is my problem.
Edited:
I've been pointed out that Jclouds handles large file upload automatically, which is not so useful in our case. In fact, we do not know how large the file will be or when the next segment will arrive at the time we start to upload the first segment. Our api is designed to make client able to upload their files in chunks of their own chosen size and at their own chosen time, and when done, call a 'commit' to make these chunks as a file. So this makes us want to upload the manifest on our own here.

According to #Everett Toews's answer, I've got my code correctly running:
public static void main(String[] args) throws IOException {
CommonSwiftClient swift = ContextBuilder.newBuilder("swift").endpoint("http://localhost:8080/auth/v1.0")
.credentials("test:test", "test").buildApi(CommonSwiftClient.class);
SwiftObject segment1 = swift.newSwiftObject();
segment1.getInfo().setName("foo/bar/1");
segment1.setPayload("foo");
swift.putObject("container", segment1);
SwiftObject segment2 = swift.newSwiftObject();
segment2.getInfo().setName("foo/bar/2");
segment2.setPayload("bar");
swift.putObject("container", segment2);
swift.putObjectManifest("container", "foo/bar2");
SwiftObject dlo = swift.getObject("container", "foo/bar", GetOptions.NONE);
InputStream input = dlo.getPayload().openStream();
while (true) {
int i = input.read();
if (i < 0) {
break;
}
System.out.print((char) i);
}
}

jclouds handles writing the manifest for you. Here are a couple of examples that might help you, UploadLargeObject and largeblob.MainApp.

Try using
Map<String, String> manifestMetadata = ImmutableMap.of(
"X-Object-Manifest", "<container>/<prefix>");
BlobBuilder.userMetadata(manifestMetadata)
If that doesn't work you might have to use the CommonSwiftClient like in CrossOriginResourceSharingContainer.java.

WebSocket disconnected when trying to send large size of images

I am testing a WebSocket at localhost using java and javascirpt, running Tomcat 7.0.42 and no proxy in between. It works fine on sending text and small size of image via websocket. However, it will be forced to close the connection on client side(chrome browser) when trying to send a large size of photo (Notice that the tomcat's 'onClose callback in MessageInbound' does not be notified after websocket on browser closed the connection).
how can I solve it? thx.
Here is the capture from chrome development tool
Below is my code on client side:
for (var i = 0, f; f = files[i]; i++) {
// step 1: tell server who the people you want to send
ws.send(JSON.stringify({
action: "binary",
receiver: <%=selectedfriend.getUserId()%>,
timestamp: new Date().getTime()
}));
// step 2: send file
ws.send(f);
var reader = new FileReader();
reader.onload = (function(theFile) {
return function(e) {
var span = document.createElement('span');
span.innerHTML = ['<img class="thumb" style="width: 50px;height: 30px;" src="', e.target.result,
'" title="', escape(theFile.name), '"/>'].join('');
appendImage(span.innerHTML, "pullleft");
};
})(f);
reader.readAsDataURL(f);
}

Finally, I found a solution for my question, what i did on Tomcat is to use:
protected StreamInbound createWebSocketInbound(String string, HttpServletRequest hsr) {
MyMessageInbound inbound = new MyMessageInbound();
inbound.setByteBufferMaxSize(9999999);
inbound.setOutboundByteBufferSize(9999999);
return inbound;
}
But there is another problem is:
what is the proper value I should use, here is 9999999, the value should be tested well when I upload under 8-9MB file using WebSocket, but why, how can I measure it?
Please help and discuss here again, thx!

Try to send as ArrayBuffer:
ws.send(f.readAsArrayBuffer());

Connecting to PC to view shared folders from Android device

I am working on a samba client for Android. Given an IP address it should connect to it and browse the shared folders.
For this I use JCIFS. I dropped the jar in my Android project and added following code to connect to PC and get the list of files:
private void connectToPC() throws IOException {
String ip = "x.x.x.x";
String user = Constants.username + ":" + Constants.password;
String url = "smb://" + ip;
NtlmPasswordAuthentication auth = new NtlmPasswordAuthentication(user);
SmbFile root= new SmbFile(url, auth);
String[] files = root.list();
for (String fileName : files) {
Log.d("GREC", "File: " + fileName);
}
}
And I get in return: jcifs.smb.SmbAuthException: Logon failure: unknown user name or bad password.
But the credentials are correct. I also tried with another samba client from the android market that uses JCIFS and it successfully connected to that ip, so obviously I am doing something wrong here but don't know what especially.
Any help is highly appreciated.

In the end I managed successfully to connect to PC. The issue turned out to be in the NtlmPasswordAuthentication(); constructor.
So, instead of this:
String user = Constants.username + ":" + Constants.password;
NtlmPasswordAuthentication auth = new NtlmPasswordAuthentication(user);
I changed to this:
NtlmPasswordAuthentication auth = new NtlmPasswordAuthentication("",
Constants.username, Constants.password);
I don't know why, perhaps it's because of ":" special character, perhaps because of Android, but passing an empty domain name, the user name, and password separately to the constructor, solved the issue.

Since some people will get to this topic if they got a similar problem with android and JCIFS,
these are other common problems when trying to make it work:
*Put the .jar specifically in /libs folder of your android project (not just via "build path")
*Be sure that your project has internet permission What permission do I need to access Internet from an android application?
*Also be sure that your JCIFS code is running in a separate thread from the UI (in other words, use AsyncTask class) how to use method in AsyncTask in android?
*Code:
protected String doInBackground(String... params) {
SmbFile[] domains;
String username = USERNAME;
String password = PASSWORD;
NtlmPasswordAuthentication auth = new NtlmPasswordAuthentication("",
username, password);
try {
SmbFile sm = new SmbFile(SMB_URL, auth);
domains = sm.listFiles();
for (int i = 0; i < domains.length; i++) {
SmbFile[] servers = domains[i].listFiles();
for (int j = 0; j < servers.length; j++) {
Log.w(" Files ", "\t"+servers[j]);
}
}
} catch (SmbException e) {
e.printStackTrace();
} catch (MalformedURLException e) {
e.printStackTrace();
}
return "";
}
these were the problems i encounter while trying to make work JCIFS on android, hope to help anyone, regards.

maybe i can help other people too.
I had the problem that i used thread.run() instead of thread.start() to execute the Smb-Code in a Runnable. I searched a lot of time for an answer but nothing fixed my problem.
But then a friend explained me the different between thread.run() and thread.start():
run(): Execute the Methode (for example the run() Methode of a Runnable) like a normal Method (synchronous)
start(): Start the Thread with the Runnable in an own task (asynchronous)
And for Smb you need a asynchronous Thread. Because of this you need to call thread.start()!
Maybe someone make the same mistake as i did.

Serving mp3 to mobile Safari browser in Java using Jersey

I am having trouble with this code,
#Path("/play")
public class Player {
#GET
#Produces("audio/mpeg")
public Response get(#DefaultValue("C:\\Users\\Ben\\Music\\sample.mp3") #QueryParam("file") String file) {
File song = new File(file);
return Response.ok().entity(song).build();
}
}
Chrome is able to play the content returned from this, but Safari mobile can't.
When I move the sample.mp3 into static web folder it is able to play in Safari mobile browser.
How can I get mobile Safari to play audio returned using JAX-RS?

I used the AudioAttributes and EncodingAttributes classes to convert the file to the right codec. It's pretty slow and wastes a lot of storage because a new file has to be create each time a song is played. I'll probably change this code later so that the files are cached after converted. Then before I convert I check if I already converted. Also it would be nice to test if the raw file is compatible with the device before converting it. Here's current the code.
#GET
#Path("/audio")
#Produces("audio/mpeg")
public Response getAudio(
#DefaultValue("C:\\Users\\Ben\\Music\\sample.mp3") #QueryParam("file") String file,
#DefaultValue("medium") #QueryParam("quality") String quality) throws EncoderException, IOException {
File song = new File(file);
File rootMusicDir = new File(AUDIO_PATH);
File rootVideoDir = new File(VIDEO_PATH);
if (!directoryService.isSubDirectory(rootMusicDir, song) && !directoryService.isSubDirectory(rootVideoDir, song)) {
return Response.status(500).build();
}
AudioAttributes audio = new AudioAttributes();
audio.setCodec("libmp3lame");
if (quality.equalsIgnoreCase("high")) {
audio.setBitRate(new Integer(256000));
audio.setChannels(new Integer(2));
audio.setSamplingRate(new Integer(44100));
} else if (quality.equalsIgnoreCase("medium")) {
audio.setBitRate(new Integer(128000));
audio.setChannels(new Integer(2));
audio.setSamplingRate(new Integer(44100));
} else {
audio.setBitRate(new Integer(64000));
audio.setChannels(new Integer(1));
audio.setSamplingRate(new Integer(22050));
}
EncodingAttributes attrs = new EncodingAttributes();
attrs.setFormat("mp3");
attrs.setAudioAttributes(audio);
Encoder encoder = new Encoder();
String random = new BigInteger(130, new SecureRandom()).toString(32);
File songMP4 = new File(TEMP_PATH + file.replaceAll("[^\\dA-Za-z ]", "").replaceAll("\\s+", "+") + random);
encoder.encode(song, songMP4, attrs);
return Response.ok().entity(songMP4).build();
}

Could you clarify what the root cause of your problem was? I don't quite follow why re-encoding the file should solve the problem? From what I understood, you have an .mp3 which works fine in Safari mobile when served statically, but not when served as per your code above. I presume, therefore, that your original .mp3 is encoded in a fashion that iOS can cope with (or else the statically served file would not play correctly).
Could there be an issue with the headers that you are sending along with the served .mp3? The reason I ask is that I have a similar problem serving an mp3 to mobile Safari which is served by a Perl script. Works fine when files statically served. When the same file is served using Perl it gives 404 in Safari mobile (Safari on OSX works fine).