Develop Reference

In Spring Boot Java Extracting the zip file from response without physically saving it

In Spring Boot the zip file that comes as a response has a corrupted structure before saving, but there is no problem when I save it physically. I need to take the file in the zip file and process the information in the database, but I cannot physically download this file because I am using GCP. How can I extract the file in this zip file that comes as a response?. How can I solve this please help.
Here is what it looks like in response.body() before saving (part of it):
"PK C`iUq �=n 緰) bu_customerfile_22110703_B001_10292121141�]i��������RI%U�v��CJ� ���×��My��y/ίϹ�������=>����}����8���׿}~}~yz�������ͲL��
�o�0�fV�29f�����6΋$K�c$�F��/�8˳�L��_�QaZ-q�F�d4γE�[���(f�8�D�0��2_��P"�I�A��D��4�߂�����D��(�T�$.��<�,���i]Fe�iM�q<ʨ�Olmi�(&���?�y�y4��<��Q�X�ޘp�#�6f-.F����8����"I㢨ҤU]�E��WI� %#������(W�8*0c�p:L��:� �}�G����e<����a�"
Here is the request call:
OkHttpClient client1 = new OkHttpClient().newBuilder()
.build();
MediaType mediaType1 = MediaType.parse("text/plain");
RequestBody body1 = RequestBody.create(mediaType1, "");
Request request1 = new Request.Builder()
.url(vers)
.method("POST", body1)
.addHeader("Cookie", "ASP.NET_SessionId=44dxexdxass5mtf00udjfwns")
.build();
Response response1 = client1.newCall(request1).execute();
String data = response1.body().string();

Would it be a matter of encoding type? String is of type UTF-16 (I think). Try a different datatype that is more like an array/vector of bytes.
Try something like what is mentioned here:
Having trouble reading http response into input stream
Update: Get the response as a stream of bytes and feed it into a ZipInputStream object as shown here https://zetcode.com/java/zipinputstream/#:~:text=ZipInputStream%20is%20a%20Java%20class,both%20compressed%20and%20uncompressed%20entries.
Then iterate over the contained files to find the one you need. Then retrieve the stream associated with the zipped file. Then you can read from there. (I realize that is a bit of handwaving, but it's been a while since I used Zip files and Java.) That should get you down the correct path.

How to upload a file with a presigned URL to an object storage minio with a Java client Api?

I want to upload a file to an object storage minio which created a presigned URL using a Java client API.
In the documentation only refers to creating presigned URL or creating some. Is there a to upload using the presigned url.

I think you probably got the answer by now, gonna put it here for others who might have stumbled upon similar task.
There are a few different HTTP clients that you could use in JAVA, so implementations may vary. The idea is, once you get the URL, it is just a matter of sending an HTTP PUT request using the URL with the file's binary content, just like you would do in any file upload procedure. As far as I know, you cannot send multipart file data directly using PUT, you have to send binary stream.
Here's an example of uploading a jpeg file with OkHttpClient:
OkHttpClient client = new OkHttpClient().newBuilder()
.build();
MediaType mediaType = MediaType.parse("image/jpeg");
RequestBody body = RequestBody.create(mediaType, "<file contents here>");
Request request = new Request.Builder()
.url("<minio presigned url here>")
.method("PUT", body)
.addHeader("Content-Type", "image/jpeg")
.build();
Response response = client.newCall(request).execute();
Another example with Spring's RestTemplate where the incoming request to the controller is a MultipartFile. If it's a File object instead, you can use your favorite utility method such as byte[] org.apache.commons.io.FileUtils.readFileToByteArray(File file) to get a byte array from that file.
HttpHeaders headers = new HttpHeaders();
HttpEntity<byte[]> entity = new HttpEntity<>(multipartFile.getBytes(), headers);
restTemplate.exchange(new URI("<presignedUrl>"),
org.springframework.http.HttpMethod.PUT, entity, Void.class);
You can search for your specific HTTP Client, just need to look for "RESTful file upload with PUT request" Or something similar.

Spring RestTemplate streaming response into another request

I am trying to stream the result of a file download directly into another post using spring's RestTemplate
My current approach is the following:
ResponseEntity<InputStreamResource> downloadResponse = restTemplate.getForEntity(fileToDownloadUri, InputStreamResource.class);
InputStreamResource imageInputStreamResource = downloadResponse.getBody();
ResponseEntity<String> response = restTemplate.exchange(storageUri, POST, new HttpEntity<>(imageInputStreamResource), String.class);
However, I get the following exception running the code above:
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://host:port/path/some.jpg": stream is closed; nested exception is java.io.IOException: stream is closed
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:6
...
Caused by: java.io.IOException: stream is closed
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.ensureOpen(HttpURLConnection.java:3348)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3373)
It seems that the response is always closed as the final step of processing. With the response, the HttpURLConnection is closed, and the stream is no longer processable.
I would like to be able to implement this scenario without having to hold the file completely in memory or writing it to a file (as described here).
Any hints are highly appreciated.

If you want to forward the response directly without ever holding it in memory, you have to directly write to the response:
#RequestMapping(value = "/yourEndPoint")
public void processRequest(HttpServletResponse response) {
RestTemplate restTemplate = new RestTemplate();
response.setStatus(HttpStatus.OK.value());
restTemplate.execute(
fileToDownloadUri,
HttpMethod.GET,
(ClientHttpRequest requestCallback) -> {},
responseExtractor -> {
IOUtils.copy(responseExtractor.getBody(), response.getOutputStream());
return null;
});
}

Since you tell RestTemplate to expect InputStreamResource it will try and use an appropriate converter to convert your message to a InputStreamResource. ( I'm guessing there is none that handles this as you want )
You should be able to let it expect a Resource from where you can get an input stream and read that.
import org.springframework.core.io.Resource;
ResponseEntity<Resource> exchange = RestTemplate.exchange(url, HttpMethod.GET, new HttpEntity(httpHeaders), Resource.class);
InputStream inputStream = exchange.getBody().getInputStream();
using this you can write the response to somewhere else. Files.write(inputStream, new File("./test.json")); wrote the file for me, so I assume the inputstream can also be used somewhere else. ( I used Spring 4.3.5 )
edit:
As the OP states, this will still load the file in memory. Behind the scene the InputStream is a ByteArrayInputStream.
The default RestTemplate and MessageConverters are not made for streaming content at all.
You could write your own implementation of a org.springframework.web.client.ResponseExtractor and maybe a MessageConverter. In ResponseExtractor you have access to the org.springframework.http.client.ClientHttpResponse
imho for your use case, you might be better of using Apache Httpcomponents HttpClient where you find HttpEntity#writeTo(OutputStream).

Download file from REST service using JAX-RS client

I am trying to download a file from a REST service using JAX-RS.
This is my code which invokes the download by sending a GET request:
private Response invokeDownload(String authToken, String url) {
// Creates the HTTP client object and makes the HTTP request to the specified URL
Client client = ClientBuilder.newClient();
WebTarget target = client.target(url);
// Sets the header and makes a GET request
return target.request().header("X-Tableau-Auth", authToken).get();
}
However I am facing problems converting the Response into an actual File object. So what I did is the following:
public File downloadWorkbook(String authToken, String siteId, String workbookId, String savePath)
throws IOException {
String url = Operation.DOWNLOAD_WORKBOOK.getUrl(siteId, workbookId);
Response response = invokeDownload(authToken, url);
String output = response.readEntity(String.class);
String filename;
// some code to retrieve the filename from the headers
Path path = Files.write(Paths.get(savePath + "/" + filename), output.getBytes());
File file = path.toFile();
return file;
}
The file which is created is not valid, I debugged the code and noticed that output contains a String like that (much larger):
PK ͢�F���� �[ Superstore.twb�ysI�7����ߡ���d�m3��f���
Looks like binary. Obviously there is something wrong with the code.
How do I get the HTTP response body as a string from the Response object?
Edit:
Quote from the REST API reference about the HTTP response:
Response Body
One of the following, depending on the format of the workbook:
The workbook's content in .twb format (Content-Type: application/xml)
The workbook's content in .twbx format (Content-Type: application/octet-stream)

As you noticed yourself, you're dealing with binary data here. So you shouldn't create a String from your response. Better get the input stream and pipe it to your file.
Response response = invokeDownload(authToken, url);
InputStream in = response.readEntity(InputStream.class);
Path path = Paths.get(savePath, filename);
Files.copy(in, path);

1) I assume by this point you're clear on the difference between "binary file" and "text file". And that you can only capture the latter into a "string".
2) Sebastian gave you excellent advice for capturing a binary file (+1, Sebastian!). VERY IMPORTANT: you should always set the MIME type (Content-Type: xxx/yyy)in cases like this. Here is another link that might be useful.
3) Finally, there are cases where you might WANT to treat "binary" data as text. This is how e-mail attachments work with SMTP (a text protocol). In these cases, you want to use Base64 Encoding. For example: JAX-RS | Download PDF from Base64 encoded data

Download a PDF file via REST service with “Content-Disposition” Header in java

I am trying to download a PDF file from a response of Java REST call after custom authentication check.
I can see downloaded file but it is empty file.
Below is my code snippet.
//Custom HTTPClient
HTTPAuthClient client = new HTTPAuthClient(url,username,password)
Request request = new Request(downloadURL); //I'm downloading file content of an URL.
Response response = client.executeGet(request);
String response1 = response.getResponseBody();
InputStream is = new ByteArrayInputStream(response.getBytes());
response.setContentType("Content-type",application/pdf); //here response is //javax.servlet.HttpServletResponse
response.setHeader("Content-Disposition","attachment;filename="myfile.pdf");
IOUtils.copy(is,response.getOutPutStream());
response.flushBuffer();
With this code I could download the file but when I open the file and verified there is no data.
As part of response body also I can see some data.
Could you please help me out where I'm doing mistake I tried many options but did not find solution.

How can you use setContentType like this
response.setContentType("Content-type",application/pdf);
If only one avalible param in this method is String void setContentType(String type) so your method should be:
response.setContentType("application/pdf");
Java Doc to be sure.