Develop Reference

(spring boot or java) I have a problem opening URL PDF

spring boot or java read/open pdf url and ResponseEntity attachment file .pdf
Call the URL https://xxxxx.xxx/file.pdf
Read the file from step 1 and display it. By setting the response value as follows:
Content-Type : application/pdf
Content-Transfer-Encoding : binary
Content-disposition : attachment; filename=filename.pdf
Content-Length : xxxx
URL url = new URL(apiReportDomain
+ "/rest_v2/reports/reports/cms/loan_emergency/v1_0/RTP0003_02.pdf?i_ref_code=" + documentId);
System.out.println(url);
String encoding = Base64.getEncoder().encodeToString(
(apiReportUsername + ":" + apiReportPassword).getBytes(StandardCharsets.UTF_8));
HttpURLConnection connectionApi = (HttpURLConnection) url.openConnection();
connectionApi.setRequestMethod("GET");
connectionApi.setDoOutput(true);
connectionApi.setRequestProperty("Authorization", "Basic " + encoding);
connectionApi.setRequestProperty("Content-Type", "application/pdf");
InputStream content = connectionApi.getInputStream();
BufferedReader in = new BufferedReader(
new InputStreamReader(content));
StringBuilder sb = new StringBuilder();
int cp;
while ((cp = in.read()) != -1) {
sb.append((char) cp);
}
byte[] output = sb.toString().getBytes();
HttpHeaders responseHeaders = new HttpHeaders();
responseHeaders.set("charset", "utf-8");
responseHeaders.setContentType(MediaType.valueOf("application/pdf"));
responseHeaders.setContentLength(output.length);
responseHeaders.set("Content-disposition", "attachment; filename=filename.pdf");
return new ResponseEntity<byte[]>(output, responseHeaders, HttpStatus.OK);
enter image description here
which the result i got is a blank page But in fact, this PDF contains a full sheet of text.

Update this if it does or does not operate, I think the problem would be the https and certificate verification at client download by your original connection.
You need the certificate to decrypt the pdf and formally accept the certificate. See JCA cryptography API.
Also the following is best MIME type for sending binary download.
Content-Type : application/octet-stream
https://docs.oracle.com/javase/7/docs/api/javax/net/ssl/HttpsURLConnection.html

The issue is that the server needs to fetch the file from the internet, and then pass it on. Except of a redirect (which would look like cross-site traffic).
First write local code to fetch the PDF in a local test application.
It could be that you need to use java SE HttpClient.
It just might be you need to fake a browser as agent, and accept cookies, follow a redirect. That all can be tested by a browser's development page looking at the network traffic in detail.
Then test that you can store a file with the PDF response.
And finally wire the code in the spring application, which is very similar on yielding the response. You could start with a dummy response, just writing some hard-coded bytes.
After info in the question
You go wrong in two points:
PDFs are binary data, String is Unicode, with per char 2 bytes, requiring a conversion back and forth: the data will be corrupted and the memory usage twice, and it will be slow.
String.getBytes(Charset) and new String(byte[], Charset) prevent that the default Charset of the executing PC is used.
Keeping the PDF first entirely in memory is not needed. But then you are missing the Content-Length header.
InputStream content = connectionApi.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
content.transferTo(baos);
byte[] output = baos.toByteArray();
HttpHeaders responseHeaders = new HttpHeaders();
responseHeaders.set("charset", "utf-8");
responseHeaders.setContentType(MediaType.valueOf("application/pdf"));
responseHeaders.setContentLength(output.length);
responseHeaders.set("Content-disposition",
"attachment; filename=filename.pdf");

Getting 404 Not Found/ File Not Found When a specific URL is hit but in browser it does give the result

I know this may be a duplicate question but I applied all the patches as suggested in other related questions, but still have no luck resolving the issue.
The issue is When I hit a URL (It's an API provided by SMS provider through which we send SMS to our clients. It worked for more than a year and suddenly we start getting an exception.) from Browser I get the required result but When I do it through code then it gives me 404 Not Found. It freaks me out when 1 out (10/20) hit is successful and I get a response to that specific hit.
URL we hit:
https://rest.nexmo.com/sms/xml
In response, we git XML based text file. e.g if I hit the above URL from a browser I get a file with a name api.txt which has an XML based response.
This is the code I have written.
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new
InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
This is the exception:
java.io.FileNotFoundException: https://rest.nexmo.com/sms/xml?api_key=apiKey&api_secret=password&from=ourprojectname&to=mobilenumber&account-ref=account_registeration_number&text=encoded_message&type=text
As the provider accepts requests as post also.We converted this code for sending post request still getting the same exception.
As this URL is when hitting through a browser we are getting a response so I added a header user-agent in request property still got the same exception.
we have some certificate problem for other URLs before which we fixed. Added the same configuration for this URL still getting the same 404 response.
Need some suggestion on this matter any small help will be appreciated.

submit a POST request with parameters and a body using Java & REST

I'm trying to use an existing API that calls for a json object to be "submitted" using POST, but the page also requires a few parameters in request, for this example we will use name, and email. I'm super new to REST so I'm probably making an ignorant mistake somewhere here.
Here is the code I have so far in my servlet:
String path = "http://www.test.com/submit";
URL url = new URL(path);
conn = (HttpURLConnection)url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
Gson gson = new Gson();
//julySchedule is the object I want to submit with this request alongside the parameters.
String input = gson.toJson(julySchedule);
OutputStream os = conn.getOutputStream();
os.write(input.getBytes());
os.flush();
if (conn.getResponseCode() != 200) {
throw new RuntimeException("HTTP POST Request Failed with Error code : "
+ conn.getResponseCode());
}
I've tried to put the parameters in the URL, like:
/submit?name=Name&email=test#gmail.com
But that didn't work, because POST requests wont accept parameters like that.
Then I tried to add it to the Output stream like:
String params = "name=Name&email=test#gmail.com"
OutputStream os = conn.getOutputStream();
os.write(path.toBytes());
os.write(input.getBytes());
os.flush();
But that didn't work either. Am I making a really dumb mistake somewhere?

It is perfectly fine for a POST request to contain query string(Though not best practice).
Since you use application/json as Content-Type, I assume your payload has to be a JSON object. You cannot mix your params in POST body. You have 2 options:
Kep email&name in QueryString, and json in POST payload: What you did should work.
Use POST payload only: Set Content-Type: application/x-www-form-urlencoded as follows: name=NAME&email=EMAIL&jsonPayLoad={Your Json Object}. Server should extract jsonPayLoad from POST payload.
Make sure to escape using encodeURIComponent before writing to POST output stream.

OAuth2 requesting token returns 401

I'm trying to authenticate to a site that uses OAuth2 and store the token in my session object. My web app initially checks to see if there's a token already there, and if there isn't it redirects the user to the login page on the external site, where the user logs in and gets redirected back to my app. So far, so good, this works. My app directs me to the external site (Mendeley), I log in there, and then it redirects me back to the url in my app that I expect it to.
When it redirects back to my app, I expect a code and a state parameter on the request, and I do see these, so I assume I'm on the right track (stop me if I'm wrong). So then, if I understand correctly, I'm supposed to post the code back to the Mendeley service to get my authorization token, and that's where it all blows up.
URL url = new URL("https://api-oauth2.mendeley.com/oauth/token");
HttpsURLConnection connection = (HttpsURLConnection) url
.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
String authString = getClientId() + ":" + "[MY CLIENT SECRET]";
System.out.println("auth string: " + authString);
byte[] authEncBytes = Base64.getUrlEncoder().encode(
authString.getBytes());
String authStringEnc = new String(authEncBytes);
System.out.println("Base64 encoded auth string: " + authStringEnc);
connection.addRequestProperty("Authorization", "Basic "
+ authStringEnc);
connection.setDoOutput(true);
OutputStream os = connection.getOutputStream();
OutputStreamWriter writer = new OutputStreamWriter(os);
writer.write("scope=all&grant_type=authorization_code");
writer.write("&client_id=");
writer.write(getClientId());
writer.write("&code=");
writer.write(code);
writer.write("&redirect_uri=");
writer.write(getMendeleyRedirectUrl(request));
writer.write("&client_secret=");
writer.write("[MY CLIENT SECRET]");
writer.flush();
writer.close();
int responseCode = connection.getResponseCode();
BufferedReader reader = new BufferedReader(new InputStreamReader(
connection.getInputStream()));
The response code I get is 401. On that last line where it tries to get the inputStream from the connection it throws an exception, and that makes sense to me sense it returned a 401 and doesn't have one.
Yes, the redirect_uri is encoded. (I don't think the initial redirect to the login would work otherwise.)
My Spidey Sense tells me I'm overlooking something that should be obvious to me, but I've tried everything I could think of. Any help would be greatly appreciated.
Edit: changed how auth header is added, now getting response code 400.

You should check if you are creating the correct basic auth header. It should be something like this:
String user = "your app id";
String password = "your app secret";
String authValue = user + ":" + password;
Base64.Encoder encoder = Base64.getEncoder();
Bytes[] btyes = authValue.getBytes(StandardCharsets.UTF_8);
String authValueEncoded = encoder.encodeToString(bytes);
connection.addRequestProperty("Authorization",
"Basic "+authValueEncoded);
This values for user and password are specific for Mendeley. See step 4 of http://dev.mendeley.com/reference/topics/authorization_auth_code.html
Regarding the error 400, you might want to check the grant_type, code or redirect_uri. Remember that the code can only be used once.
from the docs:
Errors due to incorrect or missing values for grant_type, code and
redirect_uri result in a HTTP bad request response with a status of
400 Bad Request and a JSON format error code and message:
HTTP/1.1 400 Bad Request Content-Type: application/json
Content-Length: 82
{"error":"invalid_grant","error_description":"Invalid access code"}
Missing values generate a response with an invalid_request error code.
Invalid values (including previously used codes) generate a response
with an invalid_grant error code. Specifying a value other than
authorization_code (or refresh_token) generate a response with an
unsupported_grant_type error code.
So you might wan to look inside the response body to see what's wrong.

Different behavior when sending POST Request through Java application and Chrome's POSTMAN Extension. How do I debug and find the mistake?

I am making a POST Request to a server with Content-Type set to application/x-www-form-urlencoded. One of the query parameter has a value which contains & in it. I replace the & with & before sending the request.
When I send this request using POSTMAN (Chrome Extension), the request goes fine and I receive the expected response. But when I send this request using a Java application, the server throws an error (unable to parse document). Here is the code that I'm using to send the request:
URL url = new URL(url); // url => "http://myserver.com/api/update"
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setConnectTimeout(60000);
urlConnection.setReadTimeout(60000);
urlConnection.setDoOutput(true);
urlConnection.setRequestMethod("POST");
urlConnection.setDoInput(true);
urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
PrintWriter printWriter = new PrintWriter(urlConnection.getOutputStream());
printWriter.print(params); // params => api_key=abcd123&update_data_id=123&update_data_value=Test&Value
printWriter.flush();
InputStream inputStream = urlConnection.getInputStream();
String contentType = urlConnection.getContentType();
FileUtil.closeWriter(printWriter);
// Parse response ...
Here, the problem occurs in the parameter update_data_value. If I remove the &, the request goes fine from both, POSTMAN as well as my application. But when the & is present, only the request through POSTMAN works fine.
What could be wrong ?
Thanks a lot for your help!

After a long conversation in chat, the problem was this:
It's about a XML string, where an ampersand is used. The ampersand needs to replaced with "&", according to the XML-Standard.
This XML string needs to be sent in a POST request, so the ampersand and likely the semicolon need to be escaped.
The final replacement string looks like this: "%26amp%3B".