Develop Reference

Getting java.io.IOException: Server returned HTTP response code: 400 for URL: when using a url which return 400 status code

I am trying to perform a get request using Groovy using the below code:
String url = "url of endpoint"
def responseXml = new XmlSlurper().parse(url)
If the endpoint returns status as 200 then everything works good but there is one case where we have to validate the error response like below and status returned is 400:
<errors>
<error>One of the following parameters is required: xyz, abc.</error>
<error>One of the following parameters is required: xyz, mno.</error>
</errors>
In this case parse method throws :
java.io.IOException: Server returned HTTP response code: 400 for URL: "actual endpoint throwing error"
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1900)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:646)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:150)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:831)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:796)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:142)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1216)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:644)
at groovy.util.XmlSlurper.parse(XmlSlurper.java:205)
at groovy.util.XmlSlurper.parse(XmlSlurper.java:271)
Can anyone pls suggest how to handle if server give error message by throwing 400 status code?

In the question since we are getting 400 status code for GET request. So in built XmlSlurper().parse(URI) method does not work as it throw io.Exception.
Groovy also support HTTP methods for api request and response and the below worked for me:
def getReponseBody(endpoint) {
URL url = new URL(endpoint)
HttpURLConnection get = (HttpURLConnection)url.openConnection()
get.setRequestMethod("GET")
def getRC = get.getResponseCode()
BufferedReader br = new BufferedReader(new InputStreamReader(get.getErrorStream()))
StringBuffer xmlObject = new StringBuffer()
def eachLine
while((eachLine = br.readLine()) !=null){
xmlObject.append(eachLine)
}
get.disconnect()
return new XmlSlurper().parseText(xmlObject.toString())
}

Getting the response text from the HttpURLConnection class rather than implicitly through XmlSlurper allows you much more flexibility in handling unsuccessful responses. Try something like this:
def connection = new URL('https://your.url/goes.here').openConnection()
def content = { ->
try {
connection.content as String
} catch (e) {
connection.responseMessage
}
}()
if (content) {
def responseXml = new XmlSlurper().parseText(content)
doStuffWithResponseXml(responseXml)
}
Even better would be to use an actual full-featured HTTP client, like the Spring Framework's HttpClient or RestTemplate classes.

You should check the return code and than obtain the error stream from http request instance in case of an error. The problem itself has nothing to do with JsonSlurper, as no instance of "input stream" is returned from http request instance if service returns not successfull return codes (400, 401, 500 etc.) POST example can be seen below:
http= new URL("yourUrl").openConnection() as HttpURLConnection
http.setRequestMethod('POST')
http.setDoOutput(true)
http.setRequestProperty("Content-Type", 'application/json')
http.setRequestProperty("Accept", 'application/json')
http.setRequestProperty("Authorization", "Bearer $yourTokenVariable")
http.outputStream.write(data.getBytes("UTF-8"))
http.connect()
if(http.getResponseCode() != 200 && http.getResponseCode() != 201){
throw new InvalidInputException("There was an error: " + http.getErrorStream().getText("UTF-8"))
} else {
//You can take input stream here
}

Authentication Details Not Provided REST API POST Request

I'm posting to an API server and I'm getting the following reply:
Server Response: {"detail":"Authentication credentials were not provided."}
I'm writing the requests in Java and I'm not sure exactly why it's saying credentials were not provided.
Here is a snippet of my code:
hconn.setRequestProperty("Content-Type", "application/json");
hconn.setRequestProperty( "Accept", "application/json" );
if( urlconn instanceof HttpsURLConnection )
{
String encoded = new String(Base64.getEncoder().encode( ( username + ":" + password).getBytes()));
String auth = "Basic " + encoded;
urlconn.setRequestProperty("Encoded Authorization", auth );
}
Where hconn is of type HttpURLConnection
This is the only relvant snippet that you guys need. Are there properties i'm missing to set here?
I know the server response is from a Django framework but the documentation is not clear on spotting what IS required to prevent this.
Any help appreciated.

Authorization is the correct request header name and not 'Encoded Authorization'. So, you should set
urlConn.setRequestProperty("Authorization", auth);

How do you set a token as an HTTPHeader for a get request in java?

I'm trying to write a test to receive a JSON response from an API and I need to set a security token in the header for the API call. I've already verified that I am receiving a valid token from the get/token API. When I try to execute the HttpGet I am receiving a 401 status code.
Update: Does anyone have a complete list of authorization token types?
public void listAllDoctors() throws IOException {
String listAllDoctors = "/api/doctors/search";
HttpGet getDEV = new HttpGet(DEVBASE_ENDPOINT + listAllDoctors);
getDEV.setHeader(HttpHeaders.AUTHORIZATION, "token " + TOKEN);
getDEV.setHeader(HttpHeaders.CONTENT_TYPE, "application/json");
response = client.execute(getDEV);
int actualStatus = response.getStatusLine().getStatusCode();
assertEquals(actualStatus, 200);
}

I figured out that the API uses a custom header token authentication. So the line of code goes like this:
getDev.setHeader("token", "Token value goes here");

Android HttpURLConnection receives HTTP 301 response code

I'm trying to do a HTTP GET using the HttpURLConnection object in Android.
UPDATE
I tried connection to a different server. This is also hosted within Cloud 9 (c9.io) and also returns a json response. This time I'm not getting a 301 redirect, but I am getting the actual response the server is supposed to send.
Since this means the problem is localised within the server, I've reorganized the following sections in order to focus reading onto the server-related information. Android related information has been moved to the end of the question.
Where I am connecting:
Development server on Cloud9
Using the Laravel Framework 5.2 (we cannot upgrade to 5.3 at this time, due to unsupported project dependencies)
The server should return a JSON answer
If I connect to the same URL through the browser I get the correct response (JSON string. Required HTTP Headers and a '200' HTTP Response Code)
Where I am connecting FROM
Android phone (Oneplus 3, on Android 6.0)
Compile SDK version: 23
Using Build Tools: "23.0.3"
Using Min SDK verion: 19
Using Target SDK version: 22
I'm connectiong using a HttpURLConnection object, using HTTP Method 'GET'
HTTP Response on Android
When I run my code I get the folling result from the server:
The HTTP response code is 301 but the message is null.
The new URL is exactly the same, but using HTTPS. It seems server is somehow forcing SSL/TSL encryption. Which does not happen when accessing HTTP from the browser.
HTTP Header (on Android):
date => Tue, 04 Oct 2016 05:56:26 GMT
location => https://domain.com/route/ (I modified this bit)
content-length => 382
content-type => text/html; charset=iso-8859-1
X-BACKEND => apps-proxy
X-Android-Selected-Protocol => http/1.1
X-Android-Sent-Millis => 1475560583894
X-Android-Received-Millis => 1475560585637
X-Android-Response-Source => NETWORK 301
null => HTTP/1.1 301
Other data
Since it seems the server wants Android to use HTTPS, I tried modifying the code to use HTTPS (HttpsURLConnection). This may or may not solve this problem, but I am unable to check it since I get an annoying SSL handshake failed error. Plus I have no need for encryption on this application, and therefore I'm reluctant to solve the problems coming with it.
This is all running within an AsyncTask object (since Android get moody when you try to use a network connection on the main thread).
Setting up a new server (outside of Cloud 9 and without any SSL/TSL) could be an option, but I'm reluctant to do this since it would be quite time consuming.
I tried connecting to another Cloud 9 server (which also returns a json response), using the exact same code, and everything works correctly. This suggests that the problem arises from the HTPP 301 error.
I will try to share with you any other information you may require to answer my question!
Native Android stuff (moved on UPDATE, see above)
The response content seems to be an incomplete JSON:
{ 'status':'ERROR'
Note I did NOT forget the closing } character, that's what the response actually containts. This is injected somewhere unknown (to me) during the workflow. When I capture the HTTP response (using Charles on my PC, which is set as a Proxy for my phone's Wi-Fi connection) it's content is (as expected) a simple HTML telling you to redirect (HTPP code 301) to a new route.
The invalid JSON code (above) isn't there, but a valid HTML is.
This would suggest that the invalid JSON appears somewhere internally to my code (not on the server, or transport). But there is no code on my app that generates a JSON string, let alone inject it into the response I'm processing.
Code for the HttpURLConnection
this.setURL(ruta); //gets correct url
HttpURLConnection cxn = (HttpURLConnection) this.getURL().openConnection(); //init
cxn.setRequestMethod("GET"); //use HTTP GET verb
cxn.setUseCaches(false); //no cache
cxn.setRequestProperty("Cache-Control", "no-cache"); //even less cache
cxn.setDoOutput(false); //only true in POST/PUT requests
cxn.setRequestProperty("Connection","keep-alive");
cxn.setRequestProperty("DNT", "1"); //TEMP
cxn.setInstanceFollowRedirects(true); //should follow redirects
cxn.setRequestProperty( "charset", "utf-8");
Code for the reading the result
int status_code = cxn.getResponseCode();
InputStream responseStream = new BufferedInputStream(cxn.getInputStream());
BufferedReader responseStreamReader = new BufferedReader(new InputStreamReader(responseStream));
String line = "";
StringBuilder stringBuilder = new StringBuilder();
while ((line = responseStreamReader.readLine()) != null) {
stringBuilder.append(line).append("\n");
}
responseStreamReader.close();
String response = stringBuilder.toString();
cxn.disconnect();

Remove the code you've used to create the HttpURLConnection and try with this one:
URL url;
HttpURLConnection urlConnection = null;
try {
url = new URL("http://www.domain.com/index.aspx?parameter1=X&parameter2=X"); //Use your url and add the GET parameters
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setInstanceFollowRedirects(false); /* added line */
InputStream in = urlConnection.getInputStream();
InputStreamReader isw = new InputStreamReader(in);
int data = isw.read();
while (data != -1) {
char current = (char) data;
data = isw.read();
System.out.print(current);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
}
This should be all you need to set for your GET request.
EDIT:
I've tested the webservice using Volley, here's the code I've used in order to retrieve the webservice response:
public class MainActivity extends AppCompatActivity {
public String response;
TextView textView;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
textView = (TextView) findViewById(R.id.rTextView);
RequestQueue queue = Volley.newRequestQueue(this);
String url = "yourWebserviceUrl";
// Request a string response from the provided URL.
StringRequest stringRequest = new StringRequest(Request.Method.GET, url,
new Response.Listener < String > () {
#Override
public void onResponse(String response) {
textView.setText("Response is: " + response);
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
textView.setText("That didn't work!");
}
});
// Add the request to the RequestQueue.
queue.add(stringRequest);
}
}
And this is the response I got:
{"status":"ok","found":false,"extra":"App\\Scanners"}

Changing the protocol to https worked for me.

I faced the same problem, and I fixed it after reading this source.
All we need to do is handling 3** errors like shown below
if(responseCode > 300 && responseCode < 400) {
String redirectHeader = conn.getHeaderField("Location");
if(TextUtils.isEmpty(redirectHeader)) {
return new JsonResponse(responseCode, "Failed to redirect");
}
JsonRequest newRequest = request;
newRequest.url = redirectHeader;
return getJsonFromUrl(newRequest);
}
Each 3** response should have a header with name Location which contains a redirect link which we should use.

Change the line :
HttpURLConnection cxn = (HttpURLConnection) this.getURL().openConnection();
with :
HttpsURLConnection cxn = (HttpsURLConnection) this.getURL().openConnection();
So you will able to handle https

Parse REST API, Validate User session

I am using Parse REST API for a game, altough there are java libraries for Parse I would like to handle the transfer protocolls my self using java.net for learning purposes. Please look away from things like, why dont I use Apache HttpClient.
Following the Parse REST API Guide
Here is what I am trying to achive:
Signup
User Login
validating session tokens / Retriving current user
The first two steps works just fine, the former using POST request method and the latter using GET with some paramaters.
Keeping the Request and Response format in mind I also provide the Application-ID and the REST-API-Key which are the appropriate request headers needed.
Now, for the third step using GET request with no paramaters, but with an additional header, the API expects there to be a Session-Token provided.
Code
private static void validateSessionToken(String token) throws IOException {
System.out.println("Token: " + token);
URL url = new URL("https://api.parse.com/1/users/me");
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("X-Parse-Application-Id", "xxxxxxx");
con.setRequestProperty("X-Parse-REST-API-Key", "xxxxxxxx");
con.setRequestProperty("X-Parse-Session-Token", token);
con.setRequestProperty("content-type", "application/json");
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
if(responseCode == 400) {
System.out.println("Bad request!");
return;
}
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
System.out.println(response.toString());
}
Outputs
Sending 'GET' request to URL : https://api.parse.com/1/users/me
Response Code : 400
Bad request!
Debugging
I have been using the PARSE API CONSOLE and Chrome network debugging tool to try and see what the difference is, but cannot see any.
From wiki:
400 Bad Request
The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing)
Some photos that may help

So when I asked my JsonObject for the session token:
jsonObject.get("sessionToken").toString();
It returned: "r:WzB7qdmhkcW5qd2moM8gbBLDp" in quotation, but using:
jsonObject.get("sessionToken").getAsString();
it returned: r:WzB7qdmhkcW5qd2moM8gbBLDp
I was so focused on the request I did not even notice...