Develop Reference

Can't simulate Postman request in Java

I'm trying to login to a portal. It works using Postman. When I try the same request using plain Java or OkHttp the login fails and I will be redirected to the login page.
HttpUrl.Builder httpBuilder = HttpUrl.parse("https://test58.cashctrl.com/auth/login.html").newBuilder();
httpBuilder.addQueryParameter("JMCF_AUTH_EMAIL", "email");
httpBuilder.addQueryParameter("JMCF_AUTH_PASSWORD", "password");
Request request = new Request.Builder()
.url(httpBuilder.build())
.get()
.build();
I know the Url looks weird but it works this way using Postman or even simply use a browser.
Alternative with plain Java, which I tried:
Map<String, String> parameters = new HashMap<>();
parameters.put(PARAM_EMAIL, EMAIL);
parameters.put(PARAM_PASSWORD, PASSWORD);
URL url = new URL(LOGIN_URL + "?" + ParameterStringBuilder.getParamsString(parameters));
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("POST");
con.setInstanceFollowRedirects(true);
con.setDoOutput(true);
DataOutputStream out = new DataOutputStream(con.getOutputStream());
out.writeBytes(ParameterStringBuilder.getParamsString(parameters));
out.flush();
out.close();
int status = con.getResponseCode();
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer content = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
content.append(inputLine + "\n");
}
in.close();
con.disconnect();
System.out.println(status);
System.out.println(content.toString());
Postman must be doing something special or also a browser which I don't see.

I had the same issue, I got to know that Postman has "code" feature. Below the send button you can see the code option it will generate the code for you. There is a list of language to choose from and java is one of them. Do check that out. Also you must be missing the cookie, see the temporary headers in Postman add all in your code and do include the cookie one.
Thanks I hope it helps.

Java Http request PUT : 401 unauthorized

I'm trying to send a PUT request from a Java app to a server. I successfully send GET, POST and DELETE requests but the PUT one won't succeed (I'm getting a 401 Error with the code below, 405 Error with an other code using the HttpPut of the apache package).
I'm using java.net.HttpURLConnection, here is a small region of my code :
URL obj = new URL(urlPost);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
//add request header
con.setRequestMethod(typeRequest); //typeRequest = PUT
String credentials = adminOC + ":" + pwdOC;
String encoding = Base64.encode(credentials.getBytes("UTF-8"));
con.setRequestProperty("Authorization", String.format("Basic %s", encoding));
if (!typeRequest.equals("GET")){
con.setDoOutput(true);
try (DataOutputStream wr = new DataOutputStream(con.getOutputStream())) {
wr.writeBytes(postParam);
wr.flush();
}
}
if (con.getResponseCode() == 200){
try (BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()))) {
String inputLine;
while ((inputLine = in.readLine()) != null) {
response += inputLine;
}
}
}
I tried sending my PUT parameters the "POST" way and also directly in the URL.
It seems to be an error from my Java code and not from the server because I tried to do the PUT request with cURL and it worked.
Thanks for reading, I hope you will be able to give me some hints to debug the problem.

What is missing in your code is con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded")

Receiving encoded response to HttpsURLConnection GET request

I am working on an Android app which will connect to a webpage using the java class HttpsURLConnection and parse the HTML response using JSoup. The issue is that the HTML response from the website appears to be encoded. Any ideas on what I can do to get the actual HTML?
Here is my code for contacting the website:
private String GetPageContent(String url) throws Exception {
URL obj = new URL(url);
conn = (HttpsURLConnection) obj.openConnection();
// default is GET
conn.setRequestMethod("GET");
conn.setUseCaches(false);
// act like a browser
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.8,en-GB;q=0.6");
conn.setRequestProperty("Accept-Encoding" , "gzip, deflate, sdch");
conn.setRequestProperty("Connection" , "keep-alive");
if (cookies != null) {
for (String cookie : this.cookies) {
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
}
int responseCode = conn.getResponseCode();
Log.v(TAG,"\nSending 'GET' request to URL : " + url);
Log.v(TAG,"Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(
conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// Get the response cookies
setCookies(conn.getHeaderFields().get("Set-Cookie"));
return response.toString();
}
And a snippet of the response:
��������������]�r�6��۞�w#ՙ�NDQ�ﱥ|�siv�Kkw�m&�HH�M, Z��ff_c_o�d�#���9�l�6����� �_=w|����/A{��!W� LZ��������f]�=wc߽�2,˨�|�8x��~�}�x1�$Ib�Uq�7�j�X|;��K
EDIT: The HTML was encoded with GZIP, as shown in the request headers here.
The solution to this issue was to use the GZIPInputStream class as shown below:
BufferedReader in = new BufferedReader(new InputStreamReader(
new GZIPInputStream(conn.getInputStream())));

Based on the headers returned with the request, we can conclude that the content is encoded using gzip. Luckily, there is an easy method to decode a gzip encoding stream, using the GZIPInputStream class.

Don't know which URL you are trying to access, but have you tried setting the charset ?
BufferedReader in = new BufferedReader(new InputStreamReader(
conn.getInputStream(), "UTF8"));

post data to server using Java, like jQuery's $.post()

I have a web backend, which works with the following jQuery post:
$.post(path + "login",
{"params": {"mode":"self", "username": "aaa", "password": "bbb"}},
function(data){
console.log(data);
}, "json");
How can I implement the same POST from Java, with HttpURLConnection? I'm trying with
URL url = new URL(serverUrl + loginUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("Content-Length",
Integer.toString(postData.getBytes().length));
connection.setRequestProperty("Content-Language", "en-US");
connection.setUseCaches (false);
connection.setDoInput(true);
connection.setDoOutput(true);
DataOutputStream wr =
new DataOutputStream(connection.getOutputStream ());
wr.writeBytes(postData);
wr.flush ();
wr.close ();
BufferedReader br =
new BufferedReader(
new InputStreamReader(connection.getInputStream()));
, where postData = "{\"mode\": \"...\", ..... }"
but it doesn't work the same way.
The code on the server is written id Django, and tries to get the data in this way:
mode=request.POST.get("params[mode]")

You seem to be thinking all the time that jQuery sends JSON in its raw form to the server and that the HTTP server flawlessly understands it. This is not true. The default format for HTTP request parameters is application/x-www-form-urlencoded, exactly like as HTML forms in HTTP websites are using and exactly like as how GET query strings in URLs look like: name1=value1&name2=value2.
In other words, jQuery doesn't send JSON unmodified to the server. jQuery just transparently converts them to true request parameters. Pressing F12 in a sane browser and inspecting the HTTP traffic monitor should also have shown you that. The "json" argument which you specified there in end of $.post just tells jQuery which data format the server returns (and thus not which data format it consumes).
So, just do exactly the same as jQuery is doing under the covers:
String charset = "UTF-8";
String mode = "self";
String username = "username";
String password = "bbb";
String query = String.format("%s=%s&%s=%s&%s=%s",
URLEncoder.encode("param[mode]", charset), URLEncoder.encode(mode, charset),
URLEncoder.encode("param[username]", charset), URLEncoder.encode(username, charset),
URLEncoder.encode("param[password]", charset), URLEncoder.encode(password, charset));
// ... Now create URLConnection.
connection.setDoOutput(true); // Already implicitly sets method to POST.
connection.setRequestProperty("Accept-Charset", charset);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + charset);
try (OutputStream output = connection.getOutputStream()) {
output.write(query.getBytes(charset));
}
// ... Now read InputStream.
Note: do NOT use Data(Input|Output)Stream! Those are for creating/reading .dat files.
See also:
Using java.net.URLConnection to fire and handle HTTP requests

You should use efficient libraries to build (valid) json objects. Here is an example from the PrimeFaces library:
private JSONObject createObject() throws JSONException {
JSONObject object = new JSONObject();
object.append("mode", "...");
return object;
}
If you wish to have a nice and clean code to send and retrieve objects, take a look at the answer from Emil Adz ( Sending Complex JSON Object ).

Why am i getting 403 error reading a website url on java?

I'm trying to read http://www.meuhumor.com.br/ on java using this:
URL url;
HttpURLConnection connection = null;
try{
url = new URL(targetURL);
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("Content-Language", "en-US");
connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11");
connection.setUseCaches(false);
connection.setDoInput(true);
connection.setDoOutput(true);
DataOutputStream dataout = new DataOutputStream(connection.getOutputStream());
dataout.flush();
dataout.close();
InputStream is = connection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
String line;
StringBuffer response = new StringBuffer();
while((line = br.readLine()) != null){
response.append(line);
response.append('\n');
}
br.close();
String html = response.toString();
I can access the website using any browser, but when i try to get the html with Java im getting java.io.IOException: Server returned HTTP response code: 403 for URL:
Someone know a way to get the html?

You are most likely getting an HTTP 403 response because your POST request has no body. Your code looks like it's trying to submit a form. If your intention was to simply pull down the page content without submitting a form, try a GET request, remove the Content-Type header, remove connection.setDoOutput(true), and remove the 3 DataOutputStream lines.