I'm trying to handle a simple POST Request in Java using a Socket.
I can receive the request header and answer the request without any problem, but I certainly can not get the body of the request.
I read somewhere that I'd need to open a second InputStream to achive this, but this doesn't really makes sense to me. Do you have any tips on how to get the request body?
This is what I basically use to get the header:
BufferedReader in = new BufferedReader(new InputStreamReader(
clientSocket.getInputStream()));
char[] inputBuffer = new char[INPUT_BUFFER_LENGTH];
int inputMessageLength = in.read(inputBuffer, 0,
INPUT_BUFFER_LENGTH);
String inputMessage = new String(inputBuffer, 0, inputMessageLength);
So, the message I get is something like:
POST / HTTP/1.1
User-Agent: Java/1.8.0_45
Host: localhost:5555
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
But I can't get the parameters of the POST request.
Edit:
So it turned out I just had INPUT_BUFFER_LENGTH up high enough (I know, shame on me).
So as it worked I changed my ServerSocket to SSLServerSocket and tried again to send a request with a HttpsUrlConnection from Java, now I have the same problem again (already checked the buffer), getting something like this:
POST / HTTP/1.1
User-Agent: Java/1.8.0_45
Host: localhost:5555
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-type: application/x-www-form-urlencoded
Content-Length: 128
*Missing Body*
It turned out I only get this when sending requests with my Java-Client - Sending requests from Chrome, etc are working fine - so I assume I got something wrong in my code.
This is what I use to send the request:
System.setProperty("javax.net.ssl.trustStore", ...);
System.setProperty("javax.net.ssl.trustStorePassword", ...);
SSLSocketFactory socketFactory = (SSLSocketFactory) SSLSocketFactory
.getDefault();
String url = "https://...";
URL obj = new URL(url);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
HttpsURLConnection.setDefaultSSLSocketFactory(socketFactory);
con.setRequestMethod("POST");
con.setDoOutput(true);
OutputStreamWriter writer = new OutputStreamWriter(con.getOutputStream());
writer.write(*Some String*);
writer.flush();
writer.close();
Any tips on what might be wrong with my code?
The code you have shown is not the correct way to read HTTP requests.
First off, Java has its own HttpServer and HttpsServer classes. You should consider using them.
Otherwise, you have to implement the HTTP protocol manually. You need to read the input line-by-line until you reach an empty line indicating the end of the request headers, then look at the headers you have read, in particular the Transfer-Encoding and Content-Length headers, to know how to read the remaining bytes of the request, per RFC 2616 Section 4.4:
4.4 Message Length
The transfer-length of a message is the length of the message-body as
it appears in the message; that is, after any transfer-codings have
been applied. When a message-body is included with a message, the
transfer-length of that body is determined by one of the following
(in order of precedence):
Any response message which "MUST NOT" include a message-body (such
as the 1xx, 204, and 304 responses and any response to a HEAD
request) is always terminated by the first empty line after the
header fields, regardless of the entity-header fields present in
the message.
If a Transfer-Encoding header field (section 14.41) is present and
has any value other than "identity", then the transfer-length is
defined by use of the "chunked" transfer-coding (section 3.6),
unless the message is terminated by closing the connection.
If a Content-Length header field (section 14.13) is present, its
decimal value in OCTETs represents both the entity-length and the
transfer-length. The Content-Length header field MUST NOT be sent
if these two lengths are different (i.e., if a Transfer-Encoding
header field is present). If a message is received with both a
Transfer-Encoding header field and a Content-Length header field,
the latter MUST be ignored.
If the message uses the media type "multipart/byteranges", and the
ransfer-length is not otherwise specified, then this self-
elimiting media type defines the transfer-length. This media type
UST NOT be used unless the sender knows that the recipient can arse
it; the presence in a request of a Range header with ultiple byte-
range specifiers from a 1.1 client implies that the lient can parse
multipart/byteranges responses.
A range header might be forwarded by a 1.0 proxy that does not
understand multipart/byteranges; in this case the server MUST
delimit the message using methods defined in items 1,3 or 5 of
this section.
By the server closing the connection. (Closing the connection
cannot be used to indicate the end of a request body, since that
would leave no possibility for the server to send back a response.)
For compatibility with HTTP/1.0 applications, HTTP/1.1 requests
containing a message-body MUST include a valid Content-Length header
field unless the server is known to be HTTP/1.1 compliant. If a
request contains a message-body and a Content-Length is not given,
the server SHOULD respond with 400 (bad request) if it cannot
determine the length of the message, or with 411 (length required) if
it wishes to insist on receiving a valid Content-Length.
All HTTP/1.1 applications that receive entities MUST accept the
"chunked" transfer-coding (section 3.6), thus allowing this mechanism
to be used for messages when the message length cannot be determined
in advance.
Messages MUST NOT include both a Content-Length header field and a
non-identity transfer-coding. If the message does include a non-
identity transfer-coding, the Content-Length MUST be ignored.
When a Content-Length is given in a message where a message-body is
allowed, its field value MUST exactly match the number of OCTETs in
the message-body. HTTP/1.1 user agents MUST notify the user when an
invalid length is received and detected.
Try something more like this (semi-pseudo code):
String readLine(BufferedInputStream in)
{
// HTTP carries both textual and binary elements.
// Not using BufferedReader.readLine() so it does
// not "steal" bytes from BufferedInputStream...
// HTTP itself only allows 7bit ASCII characters
// in headers, but some header values may be
// further encoded using RFC 2231 or 5987 to
// carry Unicode characters ...
InputStreamReader r = new InputStreamReader(in, StandardCharsets.US_ASCII);
StringBuilder sb = new StringBuilder();
char c;
while ((c = r.read()) >= 0) {
if (c == '\n') break;
if (c == '\r') {
c = r.read();
if ((c < 0) || (c == '\n')) break;
sb.append('\r');
}
sb.append(c);
}
return sb.toString();
}
...
BufferedInputStream in = new BufferedInputStream(clientSocket.getInputStream());
String request = readLine(in);
// extract method, resource, and version...
String line;
do
{
line = readLine(in);
if (line.isEmpty()) break;
// store line in headers list...
}
while (true);
// parse headers list...
if (request method has a message-body) // POST, etc
{
if ((request version >= 1.1) &&
(Transfer-Encoding header is present) &&
(Transfer-Encoding != "identity"))
{
// read chunks...
do
{
line = readLine(in); // read chunk header
int size = extract value from line;
if (size == 0) break;
// use in.read() to read the specified
// number of bytes into message-body...
readLine(in); // skip trailing line break
}
while (true);
// read trailing headers...
line = readLine(in);
while (!line.isEmpty())
{
// store line in headers list, updating
// any existing header as needed...
}
// parse headers list again ...
}
else if (Content-Length header is present)
{
// use in.read() to read the specified
// number of bytes into message-body...
}
else if (Content-Type is "multipart/...")
{
// use readLine(in) and in.read() as needed
// to read/parse/decode MIME encoded data into
// message-body until terminating MIME boundary
// is reached...
}
else
{
// fail the request...
}
}
// process request and message-body as needed..
Related
I have a SOAP request, which needs to be redesigned, because SoapUI can't handle binary responses properly.
I decided to make it Java based. I found this really useful, but not sure, how functions come on code snippets. I have
DigestValue
SignatureValue
X509Certificate
defined in SOAP request and not sure how to transform these information to send request to my tsendpint.
I tried TSAClientBouncyCastle too, but not sure why we need login credentials. I left empty those fields, but it finish all the time with
TSAClientBouncyCastle#1f0e140b
message.
I call TSAClientBouncyCastle class from Main with constructor.
It is the main part, it should decode data.
// Get TSA response as a byte array
InputStream inp = tsaConnection.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int bytesRead = 0;
while ((bytesRead = inp.read(buffer, 0, buffer.length)) >= 0) {
baos.write(buffer, 0, bytesRead);
}
byte[] respBytes = baos.toByteArray();
String encoding = tsaConnection.getContentEncoding();
if (encoding != null && encoding.equalsIgnoreCase("base64")) {
respBytes = Base64.decode(new String(respBytes));
}
A Time Stamp Authority (TSA) generates a proof that a datum existed before a particular time. It uses a protocol and format defined in RFC3161.
A time-stamping response is as follows (see RFC3161-section 2.4.2):
TimeStampResp ::= SEQUENCE {
status PKIStatusInfo,
timeStampToken TimeStampToken OPTIONAL }
You can parse the response of content-type application/timestamp-reply with BouncyCastle to obtain PKIStatusInfo
TimeStampResponse response = new TimeStampResponse(tsaInputStream);
int status = response.getStatus();
The possible values are
PKIStatus ::= INTEGER {
granted (0),
-- when the PKIStatus contains the value zero a TimeStampToken, as
requested, is present.
grantedWithMods (1),
-- when the PKIStatus contains the value one a TimeStampToken,
with modifications, is present.
rejection (2),
waiting (3),
revocationWarning (4),
-- this message contains a warning that a revocation is
-- imminent
revocationNotification (5)
-- notification that a revocation has occurred }
We have a restful API (which we cannot change) where if the Content-Type value for a part is null, we have a specific processing.
Until httpclient3.x, we could do a the following and set the content type as null.
StringPart sp = new StringPart("name, "value")
sp.setContentType(null);
Now, we have moved to http components (httpclient4.x jar) and I realised that we need to pass an instance of ContentBody (StringBody to be precise). Found the following 2 ways of doing the same in the new version:
multipartEntityBuilder.addPart(FormBodyPartBuilder.create().setName(propName).setBody(new StringBody(propValue, ContentType.create("application/octet-stream", "UTF-8"))).build());
multipartEntityBuilder.addPart(propName, new StringBody(propValue, ContentType.create("application/octet-stream", "UTF-8")));
However, I couldn't pass the body with no Content-Type. Even if I use the deprecated constructor for StringBody, it sets the Content-Type to plain-text.
I know that any part of a request should always have a Content-Type but this is a special case of an outdated destination server which we cannot upgrade.
I think your only option is to use deprecated FormBodyPart#FormBodyPart(String name, ContentBody) constructor AND to override deprecated FormBodyPart#generateContentType method.
FormBodyPart bodyPart = new FormBodyPart("stuff", new StringBody("stuff", ContentType.DEFAULT_TEXT)) {
#Override
protected void generateContentType(ContentBody body) {
}
};
HttpEntity entity = MultipartEntityBuilder.create().addPart(bodyPart).build();
entity.writeTo(System.out);
std out >
--yJHqjnFj0u-dWGPGkGtK_NnOgUeOspQ
Content-Disposition: form-data; name="stuff"
Content-Transfer-Encoding: 8bit
stuff
--yJHqjnFj0u-dWGPGkGtK_NnOgUeOspQ--
I feel like I am missing something here. I have a filter which prints out my server's returned information and I report that I am returning the correct response (403). I wrote a JUnit test to verify this logic and many times I am reporting 200 instead of 403. The weird part is that my server logs still show that I sent a 403. Is there some known bug in Jersey 1.17 that I am not aware of and I need to upgrade to resolve? I am not really in a position to upgrade at this point in time so I am hoping there is some bug in my logic. Here is my test case.
#Test
public void testIdsOwnedBySomeoneElse()
{
final Login user1Cred = Logins.getInstance().getLogin(Logins.LoginType.User1);
final Login user2Cred = Logins.getInstance().getLogin(Logins.LoginType.User2);
final ServiceEndpointAuthenticated authUser1 = LoginHelper.Login(user1Cred);
final ServiceEndpointAuthenticated authUser2 = LoginHelper.Login(user2Cred);
// Create generic entry owned by user 1
BigInteger user1Id = null;
{
final Object payload = endpoint.CreateEntity(authUser1.getUserId());
final ClientResponse response = endpoint.Post(authUser1, payload);
assertTrue(Status.OK == response.getClientResponseStatus());
final byte[] data = Utilities.getBytes(response.getEntityInputStream());
user1Id = endpoint.getEntityId(data);
}
// Using User2, try to use that id from user1!
{
// test 1
final MyEndpointWrapper endpoint = new MyEndpointWrapper(user1Id, validId);
final Object payload = endpoint.CreateEntity(authUser2.getUserId());
final ClientResponse response = endpoint.Post(authUser2, payload);
final Status status = response.getClientResponseStatus();
System.out.println("Returned status = " + status);
if (status != Status.FORBIDDEN)
{
byte[] data = Utilities.getBytes(response.getEntityInputStream());
String toString = null;
try
{
toString = new String(data, "UTF-8");
}
catch (UnsupportedEncodingException e)
{
}
System.out.println("data: " + toString);
}
assertEquals("Status " + status + " is not forbidden!", Status.FORBIDDEN, status);
}
{
// test 2
final MyEndpointWrapper endpoint = new MyEndpointWrapper(validId, user1Id);
final Object payload = endpoint.CreateEntity(authUser2.getUserId());
final ClientResponse response = endpoint.Post(authUser2, payload);
final Status status = response.getClientResponseStatus();
System.out.println("Returned status = " + status);
if (status != Status.FORBIDDEN)
{
int i = 9;
}
assertEquals("Status " + status + " is not forbidden!", Status.FORBIDDEN, status);
}
// Go ahead and delete this data for cleanup
assertTrue(Status.OK == endpoint.Delete(authUser1, user1Id).getClientResponseStatus());
}
My generic code first logs into our server for the creds. These creds are "attached" to the WebResource and it attaches the proper headers automatically when I build my request. I first create an entity, post it, and store the returned id to be used by another user. I create another endpointwrapper which references that violation id and I attempt to post with that id. The server logs:
INFO: RESPONSE: 403 http://myendpoint MediaType:(application/json) Payload: 232 MyErrorMessage
I can even print this message out (as shown above)! The part I dont understand is that getClientResponseStatus returned to me OK. Why?
My Post code looks like:
#Override
public ClientResponse Post(ServiceEndpointAuthenticated endpoint, Object entity)
{
MyUploadData uploadData = (MyUploadData)entity;
return endpoint.getResourceBuilder("/myendpoint")
.accept(MediaTypeExt.APPLICATION_JSON)
.type(MediaTypeExt.APPLICATION_JSON)
.post(ClientResponse.class, gson.toJson(uploadData));
}
[UPDATE]
I ran wire capture and actually do see 200 being sent back! This does appear to be something inside of Jersey Server. Here is what I see:
When working:
Request: 1099 17.021219000 127.0.0.1 127.0.0.1 HTTP 2214 POST /myEndpoint HTTP/1.1 (application/json)
Response: 1153 17.042535000 127.0.0.1 127.0.0.1 HTTP 628 HTTP/1.1 400 Bad Request (application/json)
When not working:
Request: 1161 17.044313000 127.0.0.1 127.0.0.1 HTTP 250 POST /myEndpoint HTTP/1.1 (application/json)
Response: 1217 17.066059000 127.0.0.1 127.0.0.1 HTTP 412 HTTP/1.1 200 OK
When it works I see my normal headers in the response (eg: Access-Control-*, Pragma no cache, etc). When it doesn't work I dont see any of my headers but I do see "Transfer-Encoding: chunked" and my response is my error message but my response code is 200. I added an explicit Trace statement in the server right before I sent my response to ensure I am sending the right Status and I am.
I am okay with allowing chunked transfer but I am not really okay with losing my desired http response.
Incase anyone else encounters something similar. After digging around I finally found the problem. We have a heartbeat on some of our endpoints. Some endpoints can take longer than expected time. To ensure the client doesn't disconnect prematurely we have a component which attaches to the ServletOutputStream. This sends a space to the client to keep the connection alive.
When an error is thrown (caught by our new exception remapper), this keep-alive component was not being shutdown properly. This caused Jersey to switch into chunked mode. Ensuring the keep-alive component was shutdown properly resolved the problem.
I am trying to transfer form data from an Android application to a NodeJs server.
My client code is the following (the strings that can contain UTF-8 characters are the values of params):
final HttpPost post = new HttpPost(url);
final MultipartEntityBuilder mpb = MultipartEntityBuilder.create()
.setCharset(Charset.forName("UTF-8")) // tried with or without this line
.setMode(HttpMultipartMode.BROWSER_COMPATIBLE); // tried with or without this line
for (final Entry<String, String> e : params.entrySet()) {
mpb.addTextBody(e.getKey(), e.getValue());
}
post.setEntity(mpb.build());
final HttpClient httpClient = new DefaultHttpClient();
final HttpResponse response = httpClient.execute(request);
And my server code is the following:
app.post('/accesspoint', function(req, res) {
var body = req.body;
var form = new formidable.IncomingForm();
form.encoding = 'utf-8';
form.parse(req, function(err, fields, files) {
console.log(fields);
...
When my input java params has a value containing an UTF-8 character, the log I get server side prints the corresponding value without this character, so it is kind of swallowed at some point. For instance if my input string is "ê", then my server log will print a "" value.
I use a multipart form as I read that it was the best way to send data that can contain non-ASCII characters. Formidable is also apparently the best node package to handle form that can contain UTF-8 characters.
My client side uses Apache HttpClient 4.3.3.
What am I doing wrong?
Ok so I tested with a simple query and the key value ("foo","[ê]") looked at the headers and I saw that my query was still using ISO-8859-1
Content-Disposition: form-data; name="foo"
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
[]
in contradiction to my builder parameter.
I found the solution via https://stackoverflow.com/a/21020435/592392 and changed my code to:
for (final Entry<String, String> e : params.entrySet()) {
mpb.addTextBody(e.getKey(), e.getValue(), ContentType.create("text/plain", Charset.forName("UTF-8")));
}
And now the server gets the UTF8 chars :)
Anyway, the form builder is quite misleading in my opinion.
I'm trying to uncompress a GZIPed HTTP Response by using GZIPInputStream. However I always have the same exception when I try to read the stream : java.util.zip.ZipException: invalid bit length repeat
My HTTP Request Header:
GET www.myurl.com HTTP/1.0\r\n
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7\r\n
Keep-Alive: 115\r\n
Connection: keep-alive\r\n
X-Requested-With: XMLHttpRequest\r\n
Cookie: Some Cookies\r\n\r\n
At the end of the HTTP Response header, I get path=/Content-Encoding: gzip, followed by the gziped response.
I tried 2 similars codes to uncompress :
UPDATE : In the following codes, tBytes = (the string after 'path=/Content-Encoding: gzip').getBytes ();
GZIPInputStream gzip = new GZIPInputStream (new ByteArrayInputStream (tBytes));
StringBuffer szBuffer = new StringBuffer ();
byte tByte [] = new byte [1024];
while (true)
{
int iLength = gzip.read (tByte, 0, 1024); // <-- Error comes here
if (iLength < 0)
break;
szBuffer.append (new String (tByte, 0, iLength));
}
And this one that I get on this forum :
InputStream gzipStream = new GZIPInputStream (new ByteArrayInputStream (tBytes));
Reader decoder = new InputStreamReader (gzipStream, "UTF-8");//<- I tried ISO-8859-1 and get the same exception
BufferedReader buffered = new BufferedReader (decoder);
I guess this is an encoding error.
Best regards,
bill0ute
You don't show how you get the tBytes that you use to set up the gzip stream here:
GZIPInputStream gzip = new GZIPInputStream (new ByteArrayInputStream (tBytes));
One explanation is that you are including the entire HTTP response in tBytes. Instead, it should be only the content after the HTTP headers.
Another explanation is that the response is chunked.
edit: You are taking the data after the content-encoding line as the message body. However, according to the HTTP 1.1 specification the header fields do not come in any particular order, so this is very dangerous.
As explained in this part of the HTTP specification, the message body of a request or response doesn't come after a particular header field but after the first empty line:
Request (section 5) and Response
(section 6) messages use the generic
message format of RFC 822 [9] for
transferring entities (the payload of
the message). Both types of message
consist of a start-line, zero or more
header fields (also known as
"headers"), an empty line (i.e., a
line with nothing preceding the CRLF)
indicating the end of the header
fields, and possibly a message-body.
You still haven't show how exactly you compose tBytes, but at this point I think you're erroneously including the empty line in the data that you try to decompress. The message body starts after the CRLF characters of the empty line.
May I suggest that you use the httpclient library instead to extract the message body?
Well there is the problem I can see here;
int iLength = gzip.read (tByte, 0, 1024);
Use following to fix that;
byte[] buff = new byte[1024];
byte[] emptyBuff = new byte[1024];
StringBuffer unGzipRes = new StringBuffer();
int byteCount = 0;
while ((byteCount = gzip.read(buff, 0, 1024)) > 0) {
// only append the buff elements that
// contains data
unGzipRes.append(new String(Arrays.copyOf(
buff, byteCount), "utf-8"));
// empty the buff for re-usability and
// prevent dirty data attached at the
// end of the buff
System.arraycopy(emptyBuff, 0, buff, 0,
1024);
}