Develop Reference

How to putObject without InputStream lenght knowdlege

I'm using SDK 2.0 and trying to putObject into the bucket.
From different API I'm receiving InputStream which holds file (CSV or plain text)
InputStream stream = otherApi.get();
S3Client s3 = S3Client.build();
s3.putObject(PutObjectRequest.builder(), RequestBody ? )
RequestBody has multiple useful methods as well RequestBody.fromInputStream but require to provide contentLenght which I don't know. Files could be (1MB or even 20MB).
Anyone faced with this problem during using the new API version?
Old 1.X did not requered knowledge of contentLength.

How do I upload a pdf to elasticsearch when using the elastic search java client?

This link explains how to use the REST API to upload an attachment.
But I want to upload an attachment with the java client...
I assume the following classes are relevant (though I may be wrong)...
org.elasticsearch.ingest.IngestService
org.elasticsearch.ingest.PipelineStore
I realize that I can just fall back to the REST interface but I'd rather try and use the native client first...

Just send a BASE64 encoded PDF in a field like:
String base64;
try (InputStream is = YourClass.class.getResourceAsStream(pathToYourFile)) {
byte bytes[] = IOUtils.toByteArray(is);
base64 = Base64.getEncoder().encodeToString(bytes);
}
IndexRequest indexRequest = new IndexRequest("index", "type", "id")
.setPipeline("foo")
.source(
jsonBuilder().startObject()
.field("field", base64)
.endObject()
);
In case you are not aware of it, I'm also linking to FSCrawler project in case it solves something you want to do already.

Here is four options that you can use to index PDFs to ElasticSearch
Ingest Attachment Plugin
Apache Tika
FsCrawler
Ambar
Pros/cons described in this post

How to update the content of a file in Google Drive?

I am trying to update the content of a Google Doc file with the content of another Google Doc file. The reason I don't use the copy method of the API is because that creates another file with another ID. My goal is to keep the current ID of the file. This is a code snippet which unfortunately does nothing:
com.google.api.services.drive.Drive.Files.Get getDraft = service.files().get(draftID);
File draft = driveManager.getFileBackoffExponential(getDraft);
com.google.api.services.drive.Drive.Files.Update updatePublished = service.files().update(publishedID, draft);
driveManager.updateFileBackoffExponential(updatePublished);
The two backoffExponential functions just launch the execute method on the object.
Googling around I found out that the update method offers another constructor:
public Update update(java.lang.String fileId, com.google.api.services.drive.model.File content, com.google.api.client.http.AbstractInputStreamContent mediaContent)
Thing is, I have no idea how to retrieve the mediaContent of a Google file such as a Google Doc.
The last resort could be a Google Apps Script but I'd rather avoid that since it's awfully slow and unreliable.
Thank you.
EDIT: I am using Drive API v3.

Try the Google Drive REST update.
Updates a file's metadata and/or content with patch semantics.
This method supports an /upload URI and accepts uploaded media with
the following characteristics:
Maximum file size: 5120GB Accepted Media MIME types: /*
To download a Google File in the format that's usable, you need to specify the mime-type. Since you're using Spreadsheets, you can try application/vnd.openxmlformats-officedocument.spreadsheetml.sheet. Link to Download files for more info.

Create Empty CloudBlockBlob in Azure

I'm hoping the answer to this question is quite simple, but I can't get it working after looking at the Azure Java API documentation.
I am trying to create an empty CloudBlockBlob, which will have blocks uploaded to it at a later point. I have successfully uploaded blocks before, when the blob is created upon the first block being uploaded, but I can't seem to get anything other than ("the specified blob does not exist") when I try to create a new blob without any data and then access it. I require this because in my service, a call is first made to create the new blob in Azure, and then later calls are used to upload blocks (at which point a check is made to see if the blob exists). Is it possible to create an empty blob in Azure, and upload data to it later? What have I missed?

I've not worked with Java SDK so I may be wrong but I tried creating an empty blob using C# code (storage client library 2.0) and if I upload an empty input stream an empty blob with zero byte size is created. I did something like the following:
CloudBlockBlob emptyBlob = blobContainer.GetBlockBlobReference("emptyblob.txt");
using (MemoryStream ms = new MemoryStream())
{
emptyBlob.UploadFromStream(ms);//Empty memory stream. Will create an empty blob.
}
I did look at Azure SDK for Java source code on Github here: https://github.com/WindowsAzure/azure-sdk-for-java/blob/master/microsoft-azure-api/src/main/java/com/microsoft/windowsazure/services/blob/client/CloudBlockBlob.java and found this "upload" function where you can specify an input stream. Try it out and see if it works for you.

Amazon rest call via android

I would like to perform rest calls to Amazon API from Android.
Amazon demands that all ws calls will be authenticated using HMAC signatures (Hash-based Message Authentication Code).
I'm missing a similar object to Apache Base64 object to sign my request.
Is there a simple way to do that in Android, or even better is there an Android client for Amazon web service (Product Advertising API).

You should be able to just include the Apache Base64 package in your project.
See this: http://www.delaytolerant.com/android-http-managing-base64-with-apache-commons-codec/
Or if there are any Java based Amazon clients, have you tried including those jars in your Android project?
Apparently the link above is now dead. Here's the contents of the page from Google's cache:
This post continues on programming
HTTP within Android. In the following,
I’ll show how to manage Base64 coded
content in Android and how to render
an image on WebView from a String that
we encoded.
First, the tool to use is commons
codec package from Apache. The
documentation can be found here. The
source is available here. You can just
include the source of the package to
your project, it is all Android
compatible.
The commons codec package has also
convenient method for Base64 decoding,
String imageString = "";
try {
FileInputStream fin = openFileInput("camera.jpg");
int jpeg_size = fin.available();
byte[] imagedata = new byte[jpeg_size];
fin.read(imagedata);
byte[] encodedData = Base64.encodeBase64(imagedata);
imageString = new String(encodedData);
final String mimetype = "text/html";
final String encoding = "UTF-8";
// replace below [ with html "<" and ] similarly ] with ">"
String html = "[html][body][center][img height=\"200\" width=\"200\"
src=\"data:image/jpeg;base64,"+imageString+"\"/][/center][/body][/html]";
mWebView.loadData(html, mimetype, encoding);
} catch (Exception e) {
e.printStackTrace();
}
There is also convenient Base64
decoding functionality in the package,
which can be used for example, to
decode Base64 encoded content in MIME
messages, which were covered in
previous post.

Make sure to encode the result as a url (signature = URLEncoder.encode(signature);) or you any end up in some misfortunes