im using Java and the jclouds SDK to upload files to a Swift container as multipart, the uploading is going fine yet i need to add metadata to the file, i noticed that there is a function called getContentMetadata() which can get a metadata such as content length and type, yet i was unable to add custom metadata, tried to cast put options to metadata, didn't generate error but that generated an exception when i ran the code, the code is here
try {
PutOptions putOptions = PutOptions.Builder.metadata(ImmutableMap.of("test", String.valueOf(strValue)));
ByteSource fileBytes = Files.asByteSource(file);
Payload payload = Payloads.newByteSourcePayload(fileBytes);
///setting the header for the request
payload.getContentMetadata().setContentLength(file.length());
payload.getContentMetadata().setContentType(contentType);
payload.setContentMetadata((MutableContentMetadata) putOptions);
Blob blob = blobStore.blobBuilder(file.getName()).payload(payload).build();
///sednig the request
blobStore.putBlob("testContainer", blob, multipart());
return contentLength;
}
the line payload.setContentMetadata((MutableContentMetadata) putOptions); generated an exception
any idea how to solve this?
Thanks
You should set metadata via BlobBuilder, e.g.,
Blob blob = blobStore.blobBuilder(file.getName())
.payload(fileBytes)
.contentLength(file.length()
.contentType(contentType)
.userMetadata(ImmutableMap.of("test", String.valueOf(strValue)))
.build();
Related
I am creating Azure function using Java, My requirement I need to copy blob from one container to another container with encryption
so, for encrypting blob I am adding 4bites before and after the blob while uploading to sink container
now, I need to fetch blob content, for this I found one class in azure i.e,
#BlobInput(
name = "InputFileName",
dataType = "binary",
path = sourceContainerName+"/{InputFileName}")
byte[] content,
Here byte[] content, fetching content of blob
but I am facing some errors like, if I pass any file name as InputFileName parameter it is giving 200ok means returning successful. also it is difficult to mefor exception handling
so I am looking for other ways for fetching blob content.... please answer me if any methods or classes we have
If you are looking for more control, instead of using the bindings, you can use the Azure Storage SDK directly. Check out the quickstart doc for getting
setup.
This sample code has full end-to-end code that you could build upon. Here is the code that you are looking for in it for reference
String data = "Hello world!";
InputStream dataStream = new ByteArrayInputStream(data.getBytes(StandardCharsets.UTF_8));
/*
* Create the blob with string (plain text) content.
*/
blobClient.upload(dataStream, data.length());
dataStream.close();
/*
* Download the blob's content to output stream.
*/
int dataSize = (int) blobClient.getProperties().getBlobSize();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(dataSize);
blobClient.downloadStream(outputStream);
outputStream.close();
So I'm working with retrofit 2 which uses okhttp underneath. The below code snippet works but I get OOM errors for large files.
I believe this is because I am reading the file to a byte array.
What is the recommended way to work with this?
private void appendFileContentsToBody(Attachment attachment, MultipartBody.Builder requestBodyBuilder) throws IOException {
File file = new File(attachment.getAbsolutePath());
if(file.exists()){
RequestBody attachmentPart = RequestBody.create(null, Base64.getEncoder().encode(FileUtils.readFileToByteArray(file)));
requestBodyBuilder.addPart(Headers.of("X-Filename", attachment.getFilename()), attachmentPart);
}
}
You should not encode a file into Base64 before sending it, this should be done using stream, which will be done by Retrofit for you. So your code should look like
private void appendFileContentsToBody(Attachment attachment, MultipartBody.Builder requestBodyBuilder) throws IOException {
File file = new File(attachment.getAbsolutePath());
if(file.exists()){
RequestBody attachmentPart = RequestBody.create(MediaType.parse("application/pdf"), file);
requestBodyBuilder.addPart(Headers.of("X-Filename", attachment.getFilename()), attachmentPart);
}
}
where "application/pdf" - is MIME type of the particular file.
That way you will not suffer of OOM ever. However this approach might be implemented on a backend first cuz seems like now your backend implementation made applicable for web-apps only cuz it expects encoded file in a request body.
This link explains how to use the REST API to upload an attachment.
But I want to upload an attachment with the java client...
I assume the following classes are relevant (though I may be wrong)...
org.elasticsearch.ingest.IngestService
org.elasticsearch.ingest.PipelineStore
I realize that I can just fall back to the REST interface but I'd rather try and use the native client first...
Just send a BASE64 encoded PDF in a field like:
String base64;
try (InputStream is = YourClass.class.getResourceAsStream(pathToYourFile)) {
byte bytes[] = IOUtils.toByteArray(is);
base64 = Base64.getEncoder().encodeToString(bytes);
}
IndexRequest indexRequest = new IndexRequest("index", "type", "id")
.setPipeline("foo")
.source(
jsonBuilder().startObject()
.field("field", base64)
.endObject()
);
In case you are not aware of it, I'm also linking to FSCrawler project in case it solves something you want to do already.
Here is four options that you can use to index PDFs to ElasticSearch
Ingest Attachment Plugin
Apache Tika
FsCrawler
Ambar
Pros/cons described in this post
I have an URL to file which I can download. It looks like this:
http://<server>/recruitment-mantis/plugin.php?page=BugSynchronizer/getfile&fileID=139&filehash=3e7a52a242f90c23539a17f6db094d86
How to get content type of this file? I have to admin that in this case simple:
URL url = new URL(stringUrl);
URLConnection urlConnection = url.openConnection();
urlConnection.connect();
String urlContent = urlConnection.getContentType();
returning me application/force-download content type in every file (no matter is jpg or pdf file).
I want to do this cause I want to set extension of downloaded file (which can be various). How to 'get around' of this application/force-download content type? Thanks in advance for your help.
Check urlConnection.getHeaderField("Content-Disposition") for a filename. Usually that header is used for attachments in multipart content, but it doesn't hurt to check.
If that header is not present, you can save the URL to a temporary file, and use probeContentType to get a meaningful MIME type:
Path tempFile = Files.createTempFile(null, null);
try (InputStream urlStream = urlConnection.getInputStream()) {
Files.copy(urlStream, tempFile, StandardCopyOption.REPLACE_EXISTING);
}
String mimeType = Files.probeContentType(tempFile);
Be aware that probeContentType may return null if it can't determine the type of the file.
How to 'get around' of this application/force-download content type?
I had the same problem with my uploaded content-type. Although you can trust the content-type from the URL, I chose to go looking for a content-type utilities to determine the content from the byte content.
After trying 5 or so implementations I decided to reinvent the wheel and released my SimpleMagic package which makes use of the magic(5) Unix content-type files to implement the same functionality as the Unix file(1) command. It uses either internal config files or can read /etc/magic, /usr/share/file/magic, or other magic(5) files and determine file content from File, InputStream, or byte[].
Location of the github sources, javadocs, and some documentation are available from the home page.
With SimpleMagic, you do something like the following:
ContentInfoUtil util = new ContentInfoUtil();
ContentInfo info = util.findMatch(byteArray);
It works from the contents of the data (File, InputStream, or byte[]), not the file name.
I guess this content type is set from the server your are downloading from. Some server use these kind of content type to force browsers to download the file instead of trying to open it. For example when my server return content type "application/pdf" chrome will try to open it as pdf, but when the server returns "application/force-download" the browser will save it to disk, because he has no clue what to do with this.
So you need to change the server to return the correct content type or better try some other heuristic to get the correct file type, because the server can always lie to you by setting it to jpg but giving you an exe.
I see with Java 7 you can try this method:
http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#probeContentType%28java.nio.file.Path%29
I have a table with blob content in it, in which I store different file types(text, audio, image, etc). After retrieving the byte [] successfully, I don't have any idea of how to convert the [] in such a way as to convert it into a download dialog box.
Here is my code
trns = session.beginTransaction();
Query query = session.createQuery("from FileDownload as fu where fu.Id =:Id");
query.setInteger("Id", id);
FileDownload fileDownload = (FileDownload) query.iterate().next();
byte[] byteArray = fileDownload.getFile();
The above code works fine and I receive a byte []. But I don't know how to proceed further in order to convert it into a file with the dialog appearing.
Can anyone please help me?
Assuming you know the mimeType and filename of your file, you can set the content type and the HTTP header Content-Disposition.
Just write the byte array to the OutputStream:
// The response from your servlet.
HttpServletResponse resp;
resp.setContentType(mimeType);
resp.setHeader("Content-Disposition", "attachment;filename=" + filename);
resp.getOutputStream().write(byteArray);
The byte array can be sent to the client from a servlet. You can find many discussions on the topic here and elsewhere.
Here are folks discussing efficiency of streaming (with code).
Here is a discussion on how to map the servlet to a url (with examples).
The last thing for you to do is just link the user to the servlet's URL when they click the button.
You'll also want to look into what additional info you can provide in the header before streaming the byte array. For example, if you provide a mime type the browser then has a clue what to do with the file; open PDFs in the browser, display images in the browser; open xls files in Excel.