Develop Reference

(spring boot or java) I have a problem opening URL PDF

spring boot or java read/open pdf url and ResponseEntity attachment file .pdf
Call the URL https://xxxxx.xxx/file.pdf
Read the file from step 1 and display it. By setting the response value as follows:
Content-Type : application/pdf
Content-Transfer-Encoding : binary
Content-disposition : attachment; filename=filename.pdf
Content-Length : xxxx
URL url = new URL(apiReportDomain
+ "/rest_v2/reports/reports/cms/loan_emergency/v1_0/RTP0003_02.pdf?i_ref_code=" + documentId);
System.out.println(url);
String encoding = Base64.getEncoder().encodeToString(
(apiReportUsername + ":" + apiReportPassword).getBytes(StandardCharsets.UTF_8));
HttpURLConnection connectionApi = (HttpURLConnection) url.openConnection();
connectionApi.setRequestMethod("GET");
connectionApi.setDoOutput(true);
connectionApi.setRequestProperty("Authorization", "Basic " + encoding);
connectionApi.setRequestProperty("Content-Type", "application/pdf");
InputStream content = connectionApi.getInputStream();
BufferedReader in = new BufferedReader(
new InputStreamReader(content));
StringBuilder sb = new StringBuilder();
int cp;
while ((cp = in.read()) != -1) {
sb.append((char) cp);
}
byte[] output = sb.toString().getBytes();
HttpHeaders responseHeaders = new HttpHeaders();
responseHeaders.set("charset", "utf-8");
responseHeaders.setContentType(MediaType.valueOf("application/pdf"));
responseHeaders.setContentLength(output.length);
responseHeaders.set("Content-disposition", "attachment; filename=filename.pdf");
return new ResponseEntity<byte[]>(output, responseHeaders, HttpStatus.OK);
enter image description here
which the result i got is a blank page But in fact, this PDF contains a full sheet of text.

Update this if it does or does not operate, I think the problem would be the https and certificate verification at client download by your original connection.
You need the certificate to decrypt the pdf and formally accept the certificate. See JCA cryptography API.
Also the following is best MIME type for sending binary download.
Content-Type : application/octet-stream
https://docs.oracle.com/javase/7/docs/api/javax/net/ssl/HttpsURLConnection.html

The issue is that the server needs to fetch the file from the internet, and then pass it on. Except of a redirect (which would look like cross-site traffic).
First write local code to fetch the PDF in a local test application.
It could be that you need to use java SE HttpClient.
It just might be you need to fake a browser as agent, and accept cookies, follow a redirect. That all can be tested by a browser's development page looking at the network traffic in detail.
Then test that you can store a file with the PDF response.
And finally wire the code in the spring application, which is very similar on yielding the response. You could start with a dummy response, just writing some hard-coded bytes.
After info in the question
You go wrong in two points:
PDFs are binary data, String is Unicode, with per char 2 bytes, requiring a conversion back and forth: the data will be corrupted and the memory usage twice, and it will be slow.
String.getBytes(Charset) and new String(byte[], Charset) prevent that the default Charset of the executing PC is used.
Keeping the PDF first entirely in memory is not needed. But then you are missing the Content-Length header.
InputStream content = connectionApi.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
content.transferTo(baos);
byte[] output = baos.toByteArray();
HttpHeaders responseHeaders = new HttpHeaders();
responseHeaders.set("charset", "utf-8");
responseHeaders.setContentType(MediaType.valueOf("application/pdf"));
responseHeaders.setContentLength(output.length);
responseHeaders.set("Content-disposition",
"attachment; filename=filename.pdf");

Needs to download big file in chunks

I have 20 MB of the file. I just want to download the file in the browser in chunks/streams so that If there is another request it is not blocked for a long period of time.
I also see the pause behavior in a long file. It is not needed right now. If someone can explain this because of my curiosity to know how it works.
I have the file saved in DB. When I get the request to download the file. I will get the file from DB. I push the byte code in the response body and send it to the browser. In the browser, I have created one link and download the file in one go.
Back-end code:
#GetMapping("/job-detail/{jobExecutionId}/file-detail")
public ResponseEntity<Resource> getFileDetail(#PathVariable final Long jobExecutionId) {
final FileDetailDto fileDetailDto =
fileTaskletService.getFileDetailByExecutionId(jobExecutionId);
return ResponseEntity.ok().contentType(new MediaType("text", "csv"))
.header(HttpHeaders.CONTENT_DISPOSITION,
"attachment; filename=\"" + fileDetailDto.getFileName() + "\"")
.body(new ByteArrayResource(fileDetailDto.getData()));
}
I wants to download the file in chunks so that other requests can also start the downloading on the same time.

From the wording of your question it sounds like you want the server framework to set the following header for you so that the data flows back in chunked blocks:
Transfer-Encoding: chunked
To do that you need to supply a Spring resource where the full size is not known in advance. You have used ByteArrayResource where the full size is known and results in a Content-Length header being set in the response and chunked is not used.
Change your code to use InputStreamResource and the service will stream the response back to the client with a chunked transfer encoding and no content length. Here's a sample (syntax unchecked):
try(ByteArrayInputStream bis = new ByteArrayInputStream(fileDetailDto.getData())) {
return ResponseEntity.ok().contentType(new MediaType("text", "csv"))
.header(HttpHeaders.CONTENT_DISPOSITION,
"attachment; filename=\"" + fileDetailDto.getFileName() + "\"")
.body(new InputStreamResource(bis));
}
While this will get you a chunked response I'm not convinced it's the root of your problems with the browser because they are all very capable of asynchronously streaming back data regardless of how the server provides it.

HttpsUrlConnection response returned from servlet contains extra 'b''0''\r\n' characters when read through python library

I am using HttpsURLConnection to call a server and return the response returned from the HttpsURLConnection from my servlet. I am copying the response from HttpssURLConnection to HttpServletresponse using streams, copying bytes from the httpconnection response input stream to the response's output stream, checking the end by seeing if read returns < 0.
Following is the code for copying the response. The variable response is of type HttpServletResponse and the variable httpCon is of type HttpsURLConnection.
InputStream responseStream = httpCon.getInputStream();
if (responseStream != null)
{
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[1024];
int len;
while ((len = responseStream.read(buffer)) >= 0)
{
os.write(buffer, 0, len);
}
os.flush();
os.close();
}
On the client side, I am using python requests library to read the response.
What I am seeing that if I use the curl to test my servlet, I am getting the proper response json, response = u'{"key":"value"}'.
If i read it from the requests python, it is putting some extra characters in the response , the response looks like the following
response = u'b0\r\n{"key":"value"}\r\n0\r\n\r\n'
Both the strings are unicode. But the second one has extra characters.
Same resonse if I try from curl/Postman restclient, I am able to get it properly. But from python requests, it is not working. I tried another livetest library in python, with that also, it is not working and the response has same characters. I also tried to change the accept-encoding header but it did not have any effect.
Because of this, I am not able to parse the json.
I don't want to change the client to parse this kind of string.
Can I change something on the server so that it will work correctly?

Did the response contain the below header "Transfer-Encoding: chunked"?
The response should be in Chunked transfer encoding
https://en.wikipedia.org/wiki/Chunked_transfer_encoding.
In this case, you get \r\n0\r\n\r\n at the end of the response is as expected since it is terminating symbol of this encoding. I guest curl/Postman just help us to handle Chunked transfer encoding, so you can't find these chunked symbols.

Download file from REST service using JAX-RS client

I am trying to download a file from a REST service using JAX-RS.
This is my code which invokes the download by sending a GET request:
private Response invokeDownload(String authToken, String url) {
// Creates the HTTP client object and makes the HTTP request to the specified URL
Client client = ClientBuilder.newClient();
WebTarget target = client.target(url);
// Sets the header and makes a GET request
return target.request().header("X-Tableau-Auth", authToken).get();
}
However I am facing problems converting the Response into an actual File object. So what I did is the following:
public File downloadWorkbook(String authToken, String siteId, String workbookId, String savePath)
throws IOException {
String url = Operation.DOWNLOAD_WORKBOOK.getUrl(siteId, workbookId);
Response response = invokeDownload(authToken, url);
String output = response.readEntity(String.class);
String filename;
// some code to retrieve the filename from the headers
Path path = Files.write(Paths.get(savePath + "/" + filename), output.getBytes());
File file = path.toFile();
return file;
}
The file which is created is not valid, I debugged the code and noticed that output contains a String like that (much larger):
PK ͢�F���� �[ Superstore.twb�ysI�7����ߡ���d�m3��f���
Looks like binary. Obviously there is something wrong with the code.
How do I get the HTTP response body as a string from the Response object?
Edit:
Quote from the REST API reference about the HTTP response:
Response Body
One of the following, depending on the format of the workbook:
The workbook's content in .twb format (Content-Type: application/xml)
The workbook's content in .twbx format (Content-Type: application/octet-stream)

As you noticed yourself, you're dealing with binary data here. So you shouldn't create a String from your response. Better get the input stream and pipe it to your file.
Response response = invokeDownload(authToken, url);
InputStream in = response.readEntity(InputStream.class);
Path path = Paths.get(savePath, filename);
Files.copy(in, path);

1) I assume by this point you're clear on the difference between "binary file" and "text file". And that you can only capture the latter into a "string".
2) Sebastian gave you excellent advice for capturing a binary file (+1, Sebastian!). VERY IMPORTANT: you should always set the MIME type (Content-Type: xxx/yyy)in cases like this. Here is another link that might be useful.
3) Finally, there are cases where you might WANT to treat "binary" data as text. This is how e-mail attachments work with SMTP (a text protocol). In these cases, you want to use Base64 Encoding. For example: JAX-RS | Download PDF from Base64 encoded data

How can I set the encoding of a httpExchange response?

I'm trying to modify some server code which uses an httpExchangeobject to handle the server's response to the client.
My issue is that for responses containing characters not supported by iso-8859-1, such as Chinese characters, I get something like '????' in place of the characters. I'd like to set the encoding of the response to utf-8, but have thus far been unsuccessful in doing so.
I tried adding this line:
httpExchange.getResponseHeaders().put("charset", Arrays.asList("UTF-8"));
This successfully puts a "charset" header in the response, but I still can't send the characters I want in the response.
How do I set the encoding of the response to allow for these characters?

Use Content-Type header to specify encoding.
String encoding = "UTF-8";
httpExchange.getResponseHeaders().set("Content-Type", "text/html; charset=" + encoding);
Writer out = new OutputStreamWriter(httpExchange.getResponseBody(), encoding));
out.write(something);