Develop Reference

Getting java.io.IOException: Server returned HTTP response code: 400 for URL: when using a url which return 400 status code

I am trying to perform a get request using Groovy using the below code:
String url = "url of endpoint"
def responseXml = new XmlSlurper().parse(url)
If the endpoint returns status as 200 then everything works good but there is one case where we have to validate the error response like below and status returned is 400:
<errors>
<error>One of the following parameters is required: xyz, abc.</error>
<error>One of the following parameters is required: xyz, mno.</error>
</errors>
In this case parse method throws :
java.io.IOException: Server returned HTTP response code: 400 for URL: "actual endpoint throwing error"
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1900)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:646)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:150)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:831)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:796)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:142)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1216)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:644)
at groovy.util.XmlSlurper.parse(XmlSlurper.java:205)
at groovy.util.XmlSlurper.parse(XmlSlurper.java:271)
Can anyone pls suggest how to handle if server give error message by throwing 400 status code?

In the question since we are getting 400 status code for GET request. So in built XmlSlurper().parse(URI) method does not work as it throw io.Exception.
Groovy also support HTTP methods for api request and response and the below worked for me:
def getReponseBody(endpoint) {
URL url = new URL(endpoint)
HttpURLConnection get = (HttpURLConnection)url.openConnection()
get.setRequestMethod("GET")
def getRC = get.getResponseCode()
BufferedReader br = new BufferedReader(new InputStreamReader(get.getErrorStream()))
StringBuffer xmlObject = new StringBuffer()
def eachLine
while((eachLine = br.readLine()) !=null){
xmlObject.append(eachLine)
}
get.disconnect()
return new XmlSlurper().parseText(xmlObject.toString())
}

Getting the response text from the HttpURLConnection class rather than implicitly through XmlSlurper allows you much more flexibility in handling unsuccessful responses. Try something like this:
def connection = new URL('https://your.url/goes.here').openConnection()
def content = { ->
try {
connection.content as String
} catch (e) {
connection.responseMessage
}
}()
if (content) {
def responseXml = new XmlSlurper().parseText(content)
doStuffWithResponseXml(responseXml)
}
Even better would be to use an actual full-featured HTTP client, like the Spring Framework's HttpClient or RestTemplate classes.

You should check the return code and than obtain the error stream from http request instance in case of an error. The problem itself has nothing to do with JsonSlurper, as no instance of "input stream" is returned from http request instance if service returns not successfull return codes (400, 401, 500 etc.) POST example can be seen below:
http= new URL("yourUrl").openConnection() as HttpURLConnection
http.setRequestMethod('POST')
http.setDoOutput(true)
http.setRequestProperty("Content-Type", 'application/json')
http.setRequestProperty("Accept", 'application/json')
http.setRequestProperty("Authorization", "Bearer $yourTokenVariable")
http.outputStream.write(data.getBytes("UTF-8"))
http.connect()
if(http.getResponseCode() != 200 && http.getResponseCode() != 201){
throw new InvalidInputException("There was an error: " + http.getErrorStream().getText("UTF-8"))
} else {
//You can take input stream here
}

Calling a WS with Postman is done, but not from Java code

I did a Post request by using Postman and I got a response, but when did the same resquest using OkHttpClient (same problem with HttpsURLConnection) in java I got a connection refused exception.
Below is my code (with fake data) :
OkHttpClient client = new OkHttpClient();
String req = "<?xml version=\"1.0\"?>\r\n" +
"<ApplicantTestRequest\r\n" +
"PositionID=\"48939014-b24f-4d74-8a44-9913cd9f8936\"\r\n" +
"ThirdPartyCandidateID=\"4152ab4r\"\r\n" +
"FirstName=\"Danny\"\r\n" +
"LastName=\"Givaty\"\r\n" +
"UserName=\"dannyg\"\r\n" +
"Password=\"2sEr#d!w#\"\r\n" +
"email=\"dannyg#careerharmony.com\"\r\n" +
"Telephone=\"5558586858\"\r\n" +
"Source=\"LinkeIn\"\r\n" +
"SkipToFirstRecruiterComponent = \"1\"\r\n" +
"Gender = \"1\"\r\n" +
"/>";
RequestBody reqbody = RequestBody.create(null, req);
Request request = new Request.Builder()
.url("https://staging.direct-assessment.net/RomaTestUI/forms/xmlregistrationandtestentry.aspx?XMLReadType=1")
.method("POST",reqbody)
.addHeader("Content-Type", "application/x-www-form-urlencoded")
.addHeader("cache-control", "no-cache")
.build();
Response response = client.newCall(request).execute();
int responseCode = response.code();
System.out.println("Response Code : " + responseCode);
The result I got is :
Exception in thread "main" java.net.ConnectException: Failed to connect to staging.direct-assessment.net/185.52.110.193:443
...........
Caused by: java.net.ConnectException: Connection refused: connect
Any suggestion ?

I can see below reason for this, I am also including a possible solution -
The issue could be related to HTTP_TRANSPORT_VERSION, can you check what version is sent via Postman and what is sent from the Java program. If the versions are different then set the HTTP_TRANSPORT_VERSION (similar to that of Postman request) in your java call.

Use resteasy with user and password

I am trying to convert this line to resteasy stuff.
curl -X GET --digest -u myuser:mypassword --header 'Accept:application/json; charset=utf-8' http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/live/instances
this is how I am trying to connect to that endpoint
clientREST = new ResteasyClientBuilder().build();
clientREST.register(new BasicAuthentication(userWowza, passWowza));
String targetStr = "http://" + host + ":" + portEndpoint + endPoint + red5App + "/instances";
System.out.println(targetStr);
ResteasyWebTarget target = clientREST
.target(targetStr);
Response response = target.request().post(Entity.entity("", "application/json"));
if (response.getStatus() != 200) {
throw new RuntimeException("Failed : HTTP error code : " + response.getStatus());
}
But I get HTTP error code : 401. So it is like user and password is no working. Am I sending the headers correctly?
Update
So I couldnt find any example of resteasy with digest auth, so I found that wowza seems to suppport basic and digest at the same time

Actually your curl uses --digest not BASIC authentication. So it looks like Server does not accept BASIC Authorization schema.

Google Safe Browsing HTTP POST - 403 response

I'm working on a program that queries Google Safe Browsing for certain urls, but I'm getting an error that I don't think I should be getting.
I'm sending the following request:
2
http://google.com
http://facebook.com
via POST to: https://sb-ssl.google.com/safebrowsing/api/lookup?client=api&apikey=[KEY]&appver=1.5.2&pver=3.1
However, I'm getting a 403 response.
This is what the documentation says for HTTP POST lookup errors:
The server generates the following HTTP error codes for the POST request:
•200: AT LEAST ONE of the queried URLs are matched in either the phishing, malware, or unwanted software lists. The actual results are returned through the response body.
•204: NONE of the queried URLs matched the phishing, malware, or unwanted software lists, and no response body is returned.
•400: Bad Request—The HTTP request was not correctly formed.
•401: Not Authorized—The API key is not authorized.
•503: Service Unavailable—The server cannot handle the request. Besides the normal server failures, this could also indicate that the client has been “throttled” for sending too many requests.
The response code 403 isn't listed, yet I'm getting it.
I have triple-checked my API-key and made sure the API is enabled for my project. I'm using a Server-key, but I also tried a Browser-key.
I tried doing a GET request also, and that did work, but I cannot get POST to work. What's going on?
Here is my code:
try {
String baseURL="https://sb-ssl.google.com/safebrowsing/api/lookup";
String arguments = "";
arguments +=URLEncoder.encode("client", "UTF-8") + "=" + URLEncoder.encode("api", "UTF-8") + "&";
arguments +=URLEncoder.encode("apikey", "UTF-8") + "=" + URLEncoder.encode("[KEY]", "UTF-8") + "&";
arguments +=URLEncoder.encode("appver", "UTF-8") + "=" + URLEncoder.encode("1.5.2", "UTF-8") + "&";
arguments +=URLEncoder.encode("pver", "UTF-8") + "=" + URLEncoder.encode("3.1", "UTF-8");
// Construct the url object representing cgi script
URL url = new URL(baseURL + "?" + arguments);
// Get a URLConnection object, to write to POST method
HttpURLConnection connect = (HttpURLConnection) url.openConnection();
connect.setRequestMethod("POST");
// Specify connection settings
connect.setDoInput(true);
connect.setDoOutput(true);
// Get an output stream for writing
OutputStream output = connect.getOutputStream();
PrintStream pout = new PrintStream (output);
pout.print("2");
pout.println();
pout.print("http://www.google.com");
pout.println();
pout.print("http://www.facebook.com");
pout.close();
BufferedReader in = new BufferedReader(new InputStreamReader(connect.getInputStream()));
String decodedString;
while ((decodedString = in.readLine()) != null) {
System.out.println("w: " + decodedString);
}
in.close();
} catch(Exception e) {
e.printStackTrace();
}

I found the error. The CGI parameter was incorrect. It should have been key and not apikey. Still weird that you get an undocumented response-code though.

server cookie = null

I am doing authentication and receiving a null cookie. I want to store this cookie but sever is not returning me a cookie. But the response code is 200 ok.
httpConn.setRequestProperty(
"Authorization",
"Basic " + Base64OutputStream.encodeAsString(
login.getBytes(), 0, login.getBytes().length, false, false));
String tmpCookie = httpConn.getHeaderField("set-cookie");
This is my code.
String login = username + ":" + password;
String base = "http://mysever/login";
HttpConnection httpConn = null;
httpConn = (HttpConnection)Connector.open(base);
// Setup HTTP Request to POST
httpConn.setRequestMethod(HttpsConnection.POST);
httpConn.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
httpConn.setRequestProperty("Accept",
"text/html,application/xhtml+xml,application/xml,application/x-javascript,*/*;q=0.5 ");
//Add the authorized header.
httpConn.setRequestProperty("Authorization",
"Basic " + Base64OutputStream.encodeAsString(
login.getBytes(), 0, login.getBytes().length, false, false));
message = httpConn.getResponseMessage();
status = httpConn.getResponseCode();
tmpCookie = httpConn.getHeaderField("Set-Cookie");
EventLogger.logEvent(guid, status);
if (status == HttpConnection.HTTP_OK)
{
String tmpCookie = httpConn.getHeaderField("set-cookie");
authForm.append("\nConnected");
authForm.append("\n\nLogin Response:" + message +
"\nHTTP response code:" + status + "\nCookie: "
+ tmpCookie);
//getNewZipFile();
}
else if(status !=HttpConnection.HTTP_OK){
throw new IOException("HTTP response code: " + status);
}
httpConn.close();

Have you actually made a connection? Your code shows you setting a request property and then immediately trying to find a header value, with no indication that the request has actually been sent.
If you are doing so (in which case fuller code would be welcome) you should use Wireshark or something similar to find out what the network traffic actually looks like.

How request Authorization header connected with response Cookie?
I suppose that it is in no way.
May be server doesn't return any cookies? Or may be "set-cookie" is case sensitive, and you must use "Set-Cookie" instead.