Develop Reference

Remove headers from openfeign request set by default http client

I am calling an API, that blacklists certain HttpHeaders including Content-Length which seems to be preset by the HttpClient underneath spring-openfeign.
To properly receive an API response, I'd need to remove the Content-Length header.
The following workarounds had been tried:
I tried to set the header to null or an empty String using the available Feign annotations #Headers, #RequestHeaders
I implemented a RequestInterceptor that creates a copy of the available (immutable) header map, deletes the blacklisted header and sets the Map as requestTemplate.headers(newHeaders). But only new headers can be added and the available ones not modified (seems to be really immutable ;))
I researched on overriding the used HttpClient but wasn't successful until now.
Experienced errors/ issues:
The API I am calling returns a 400 based on their header schema validation.
Code:
In case any code-snippets are needed, I am happy to provide them but to me the issue does not seem to be related to any code issue as I am not running into any exceptions.
Thanks in advance!!

The Apache Http Client included in feign-httpclient will always set the content length header if there is a request body present. One way to address this to configure the Apache Client directly and provide it to Feign via the builder:
This custom client can have an Apache Http Client interceptor applied that allows you to modify the request after it leaves Feign and before Apache sends it. Review their javadoc for more information.
public class Example {
public static void main(String[] args) {
HttpClient httpClient = HttpClients.custom.build();
GitHub github = Feign.builder()
.client(new ApacheHttpClient(httpClient))
.target(GitHub.class, "https://api.github.com");
}
}

FeignClient will preset Content_Length in the request header. In a keep-alive connection mode, either Content-Length or Transfer-Encoding header field must be set to signal the presence of a message body, so you can set Transfer-Encoding=chunked and Content-Length will be ignored by the serverside.
You can refer to rfc7230#section-3.3.1
"The presence of a message body in a request is signaled by a
Content-Length or Transfer-Encoding header field. Request message
framing is independent of method semantics, even if the method does
not define any use for a message body."
"In order to remain persistent, all messages on a connection need
to have a self-defined message length (i.e., one not defined by
closure of the connection), as described in Section 3.3. A server
MUST read the entire request message body or close the connection
after sending its response, since otherwise the remaining data on a
persistent connection would be misinterpreted as the next request.
Likewise, a client MUST read the entire response message body if it
intends to reuse the same connection for a subsequent request."
and from here , you can read:
"All HTTP/1.1 applications that receive entities MUST accept the
"chunked" transfer-coding (section 3.6), thus allowing this mechanism
to be used for messages when the message length cannot be determined
in advance.
Messages MUST NOT include both a Content-Length header field and a
non-identity transfer-coding. If the message does include a non-
identity transfer-coding, the Content-Length MUST be ignored.
When a Content-Length is given in a message where a message-body is
allowed, its field value MUST exactly match the number of OCTETs in
the message-body. HTTP/1.1 user agents MUST notify the user when an
invalid length is received and detected."

How does cookies work, when calling cookie-setting REST services from java VM?

I'm in the process of learning how to use HP Quality Center's REST api to query and manipulate data. Unlike REST standard, this API is not completely stateless. It uses cookies to store authentication sessions.
I've tried to implement a very simple test, using the Jersey Client library. I can successfully authenticate my user, by sending my credentials. The API reference claims that this will set a cookie, and I am good to go with further calling the REST api. However, a simple "is-authenticated" call returns a 401, Authentication failed.
I have a feeling that the cookie writing or reading is not working properly, as everything else seems to work as it should. But I haven't been able to find out if or how cookies are set and read, when no browser is involved. So How does cookies work, when calling cookie-setting REST services from java VM? Does it work at all? Where are they stored?
I am using Eclipse Kepler as my IDE, if that matters at all, and a 32-bit java 1.6 JDK and JRE.
Code, and response strings below:
1. Logging in:
Client client = ClientBuilder.newClient();
Response response = client
.target("http://[host]:[port]").path("qcbin/authentication-
point/alm-authenticate")
.request().post(Entity.entity("<alm-authentication>
<user>username</user>
<password>secret</password></alm-authentication>",
MediaType.TEXT_XML_TYPE));
System.out.println(response.toString());
Output:
InboundJaxrsResponse{ClientResponse{method=POST,
uri=http://[host]:[port]/qcbin/authentication-point/alm-authenticate,
status=200, reason=OK}}
API Return description:
One of:
HTTP code 200 and sets the LWSSO cookie (LWSSO_COOKIE_KEY).
HTTP code 401 for non-authenticated request. Sends header
WWW-Authenticate: ALMAUTH
2. Verifying Logged in:
response = client.target("http://[host]:[port]")
.path("qcbin/rest/is-authenticated")
.request().get();
System.out.println(response.toString());
Output:
InboundJaxrsResponse{ClientResponse{method=GET,
uri=http://[host]:[port]/rest/is-authenticated, status=401,
reason=Authentication failed. Browser based integrations - to login append
'?login-form-required=y to the url you tried to access.}}
PS: adding the ?login-form-required=y to the URL, will bring up a log-in window when called in a browser, but not here. Appending the line to the URL actually still gives the same error message, and suggestion to append it again. Also, when called in a browser, the is-authenticated returns a 200, success, even without the login-form.

When you log in, you're getting a cookie which is a name plus a value.
The REST server expects you to pass this in the request header with every request you make.
Look into the object which you get for client.request(); there should be a way to specify additional headers to send to the server. The header name must be Cookie and the header value must be name=value.
So if the server responds with a cookie called sessionID with the value 1234, then you need something like:
client.request().header("Cookie", "sessionID=1234")
Related:
http://en.wikipedia.org/wiki/HTTP_cookie

Java HttpURLConnection | POST method | Sequence of HTTP header fields

I am trying to "spoof" a Firefox HTTP POST request in Java using java.net.HttpURLConnection.
I use Wireshark to check the HTTP headers being sent, so I have (hopefully) reliable source of information, why the Java result doesn't match the ideal situation (using Firefox).
I have set all header fields exactly to the values that Firefox sends via HTTP and noticed, that the sequence of the header fields is not the same.
The output for Firefox is like:
POST ...
**Host**
User-Agent
Accept
Accept-Language
Accept-Encoding
Referer
Connection
Content-Type
Content-Length
When I let wireshark tap off my implementation in Java, it gives me a slightly different sequence of fields:
POST...
**User-Agent**
Accept
Accept-Language
Accept-Encoding
Referer
Content-Type
Host
Connection
Content-Length
So basically, I have all the fields, just in a different order.
I have also noticed that the Host field is sent with a different value:
www.thewebsite.com (Firefox) <---> thewebsite.com (Java HttpURLConnection), although I pass on the String to httpUrlConnection.setRequestProperty with the "www."
I have not yet analyzed the byte output of Wireshark, but I know that the server is not returning the same Location in the header fields of my response.
My questions are:
(1) Is is possible to control the sequence the header fields in the request, and if yes is it possible to do using HttpURLConnection? If not, is it possible to directly control the bytes in the HTTP header using Java? [I don't own the server, so my only hope to get the POST method working is through my application pretending to be Firefox, the server is not really verbose, my only info are: Apache with PHP]
(2) Is there a way to fix the setRequestProperty() problem ("www") as described above?
(3) What else could matter? (Do I need to concern the underlying layers, TCP....?)
Thanks for any comments.
PS. I am trying to model a situation without cookies being sent, so that I can ignore the effect.

First, the order of the headers is irrelevant.
Second, in order to manually override the host header you need to set sun.net.http.allowRestrictedHeaders=true either in code
System.setProperty("sun.net.http.allowRestrictedHeaders", "true")
or at JVM start
-Dsun.net.http.allowRestrictedHeaders=true
This is a security precaution introduced by Oracle a while ago. That's because according to RFC
The Host request-header field specifies the Internet host and port
number of the resource being requested, as obtained from the original
URI given by the user or referring resource (generally an HTTP URL).

the headers order is not important. the headers got by server are also out-of-order. And you can not control httpUrlConnection header order. But if you write your own TCP client, you can control your header order. like:
clientSocket = new Socket(serverHost, serverPort);
OutputStream os = clientSocket.getOutputStream();
String send = "GET /?id=y2y HTTP/1.1\r\nConnection: keep-alive\r\nKeep-Alive: timeout=15, max=200\r\nHost: chillyc.info\r\n\r\nGET /?id=y2y HTTP/1.1\r\nConnection: keep-alive\r\nKeep-Alive: timeout=15, max=200\r\nHost: chillyc.info\r\n\r\n";
os.write(send.getBytes());
The Second question is answered by Marcel Stör in the first answer.
a

I got lucky with Apache Http Components, my guess is that the "Host" header's missing "www." made the difference, which can be set exactly as intended using Apache's HttpPost:
httpPost.setHeader("Host", "www.thewebsite.com");
The Wireshark output confirmed my suspicion. Also this time the TCP communication prior to my HTTP post looks different (client ---> server, server ---> client, client ---> server) instead of (client ---> server, server ---> client, client ---> server, client---> server).
Now I get the desired Location header value and the server is also setting the cookies. :)
For the most part, this question is resolved.
Actually I wanted to use the lightweihgt HttpUrlConnection because that's what the Android Developers blog suggesting. The System.setProperty("sun.net.http.allowRestrictedHeaders", "true") might work as well, if it allows to "www." in the Host value.

Java - send HTTP POST request without downloading all the content

Is it possible to send HTTP POST request to a webserver and retrieve just headers of response or read just few bytes of the body, so the rest won't be downloaded at all (so it won't consume traffic)? If yes, how?
I know that there is a HEAD method for this, but I need to achieve it by POST method .. well, I am not sure if I need the POST method, I just need to post the data. Maybe if the webserver isn't secured well enough (it doesn't check what method it's used - it's just directly access the post data), is it possible to send "post data" by HEAD request?

There is no built-in HTTP mechanism for this, and HTTP HEAD requests do not allow content in the body. If however you are the one writing the server code then anything is possible.
If this is the case, I would suggest a URL parameter that triggers this behavior. For example:
POST /myURL - This would return the whole response
POST /myURL?body=minimal - Returns the reduced size response that you are looking for.
And you would have to code your server method to construct and return the appropriate response based on the URL parameter.

Does the server send response only when its HTTP 200?

im writing a java application that sends a post request to a server and expect a json from the server. Now when i need to get the response from the server do i only need to get it from the inputStream when the http code is 200 (HTTP OK) or is there any other cases ? , example :
//...
if (urlConn.getResponseCode() == HttpURLConnection.HTTP_OK) {
// only here try to get the response
}
//...

It depends on how the server is implemented. Check the API, if the server has one. If it's internal, ask your server guy.
Generally speaking, if your response code is either 2xx or 3xx, I would check the response anyway...

If the server your communicating with is following the spec then either 200 or 201 responses are valid to contain an entity. A 204 response is successful but has no data in the response.
See section 9.5 here: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5 for details of acceptable responses to a POST. Extract below:
The action performed by the POST method might not result in a resource
that can be identified by a URI. In this case, either 200 (OK) or 204
(No Content) is the appropriate response status, depending on whether
or not the response includes an entity that describes the result.
If a resource has been created on the origin server, the response
SHOULD be 201 (Created) and contain an entity which describes the
status of the request and refers to the new resource, and a Location
header (see section 14.30).

There are three things to consider:
All 2xx codes denote success of some sort. But depending on the exact code, your reading code might be different. (204 for example means success but no content.)
There are redirecting codes (3xx). These are usually automatically followed by the http client library but you can also set them not to, in which case you need to have custom code that handles these cases.
There can be valuable information returned in the stream even if you get a code that denotes an error. Whether you want to process it depends on your exact needs.