Develop Reference

Java: What is the maximum size of a POST request body in an HTTPServer?

I'm using a simple Java HTTPServer instance to connect my program to a client application. In this architecture, the server is a very barebones layer used to serve HTML pages statically and deal with the body of POST requests.
The HTTPServer is then linked to an HTTPHandler, whose implementation looks as follows:
#Override
public void handle(HttpExchange exchange) throws IOException {
Headers h = exchange.getResponseHeaders();
h.add("Content-Type", getContentType(exchange));
if ("GET".equals(exchange.getRequestMethod())) {
File file = // Static file location
h.add("Access-Control-Allow-Origin", "*");
h.add("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
h.add("Access-Control-Allow-Methods", "OPTIONS, GET, POST, PUT, DELETE");
h.add("Access-Control-Allow-Credentials", "true");
OutputStream out = exchange.getResponseBody();
exchange.sendResponseHeaders(200, file.length());
exchange.getResponseBody().write(Files.readAllBytes(file.toPath()));
exchange.close();
out.close();
} else if ("POST".equals(exchange.getRequestMethod())) {
String req = new String(exchange.getRequestBody().readAllBytes());
String response = "{}";
if (path.startsWith("api/input")) {
// deal with request body
}
exchange.sendResponseHeaders(200, response.length());
exchange.getResponseBody().write(response.getBytes(StandardCharsets.UTF_8));
exchange.close();
}
}
When sending a POST request with a longer body (roughly 3 MB), the exchange fails with an Error 413: "Request entity too large". Reducing the request size to, let's say, 300 KB seems to do the trick, so I am led to think that the error is genuine.
While I am familiar with the concept of server implementations limiting the request body size of POST requests (e.g. Express limiting the size to 100 kb unless otherwise specified), I am not able to find the corresponding parameter in a vanilla com.sun.net.httpserver.HTTPServer/HTTPHandler implementation - nor does documentation help me further in this regard.
Does anyone know the limit? Is there a way for the limit to be increased?

How to calculate Content-Length of a request body of a Http Post Request

I am trying to calculate the Content-Length of the request body of a http post request but I keep getting error that indicated wrong Content-Length. Request body looks like following:
Map<String, String> body = {
'grant_type': 'authorization_code',
'client_id': 'clientid',
'code': authCode,
'redirect_uri': 'http://localhost:8080/login/callback',
'code_verifier':
'codeverifier',
};
I tried couple solutions like concatenating content of the body into one string such as following and then convert it into byte array and send the length of it but it didn't work.
String bodyStr = "grant_type:authorization_code" +
"client_id:clientid" +
"code:{$authCode}" +
"redirect_uri:http://localhost:8080/login/callback" +
"code_verifier:codeverifier";
List<int> bytes = utf8.encode(bodyStr);
The post request body is x-www-form-urlencoded format and content length has to be calculated correctly. Any help is appreciated, thanks.

I encapsulated it myself. Generally, I don't need to calculate it. Unless it's a special occasion.
Okhttp3 is recommended

You don't need to make a list of integer...
String bodyStr = "...";
byte[] bytes = bodyStr.getBytes(Charset.forName("UTF-8"));
int len = bytes.length;
response.setContentLength(len);
response.getOutputStream().write(bytes);
This example is using the HttpServletResponse object from a HttpServlet. Not sure if that's what you need.
I have used this a fair amount, it works well.

Wrong encoding on header response Java

I am try to do POST REQUEST to api, let say the API will return a response header in thailand character 10,000 บาท. after i do the request the response i am getting the header value is 10,000 à¸à¸²à¸.
Here is the code:
RequestSpecification httpRequest = RestAssured.given().header(HeaderKey.appId,header.getAppId());
Response response = httpRequest.post(uri);
String errMessage = response.header("key-to-value-i-want");
System.out.println(errMessage);
what's missing from my code? Thanks you

Content length for MultipartFormDataInput request always -1

I am using RESTEasy to PUT some data by a key and value. I'd like to only accept data that's less than 4k in size, which I figured I'd check by getting the content length of the request. Here's the method I'm working with:
#PUT
#Path("/{key}")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response addData(
final #PathParam("key") String dataKey,
MultipartFormDataInput data) throws IOException, ClassNotFoundException {
final HttpServletRequest request = ....getHttpRequest(); //returns the request
final Map<String, List<InputPart>> parts = data.getFormDataMap();
final List<InputPart> inputParts = parts.get("data");
final InputPart part= inputParts.get(0);
final InputStream dataStream = part.getBody(InputStream.class, null);
int length = request.getContentLength();
String length2 = request.getHeader("Content-Length");
System.out.println(length);
System.out.println(length2);
service.addData(key, dataStream);
return Response.status(204).build();
}
However, length and length2 return -1 and null respectively. If I monitor Fiddler and look at the request, I notice that the contentLength on the request is not -1 but it rather a correct value instead.
Any ideas on why I can't get the request content length? Thanks.
Edit: My PUT request look like this using the Advanced Rest Client Chrome Extension: http://i.imgur.com/aI6WNDy.png
Edit: Not sure I have much else to add, but any thoughts on this would be great.

see if #HeaderParam works for you
http://www.mkyong.com/webservices/jax-rs/get-http-header-in-jax-rs/

HTTP request compression

General Use-Case
Imagine a client that is uploading large amounts of JSON. The Content-Type should remain application/json because that describes the actual data. Accept-Encoding and Transfer-Encoding seem to be for telling the server how it should format the response. It appears that responses use the Content-Encoding header explicitly for this purpose, but it is not a valid request header.
Is there something I am missing? Has anyone found an elegant solution?
Specific Use-Case
My use-case is that I have a mobile app that is generating large amounts of JSON (and some binary data in some cases but to a lesser extent) and compressing the requests saves a large amount of bandwidth. I am using Tomcat as my Servlet container. I am using Spring for its MVC annotations primarily just to abstract away some of the JEE stuff into a much cleaner, annotation-based interface. I also use Jackson for auto (de)serialization.
I also use nginx, but I am not sure if thats where I want the decompression to take place. The nginx nodes simply balance the requests which are then distributed through the data center. It would be just as nice to keep it compressed until it actually got to the node that was going to process it.
Thanks in advance,
John
EDIT:
The discussion between myself and #DaSourcerer was really helpful for those that are curious about the state of things at the time of writing this.
I ended up implementing a solution of my own. Note that this specifies the branch "ohmage-3.0", but it will soon be merged into the master branch. You might want to check there to see if I have made any updates/fixes.
https://github.com/ohmage/server/blob/ohmage-3.0/src/org/ohmage/servlet/filter/DecompressionFilter.java

It appears [Content-Encoding] is not a valid request header.
That is actually not quite true. As per RFC 2616, sec 14.11, Content-Encoding is an entity header which means it can be applied on the entities of both, http responses and requests. Through the powers of multipart MIME messages, even selected parts of a request (or response) can be compressed.
However, webserver support for compressed request bodies is rather slim. Apache supports it to a degree via the mod_deflate module. It's not entirely clear to me if nginx can handle compressed requests.

Because the original code is not available any more. In case someone come here need it.
I use "Content-Encoding: gzip" to identify the filter need to decompression or not.
Here's the codes.
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException
{
HttpServletRequest httpServletRequest = (HttpServletRequest) request;
String contentEncoding = httpServletRequest.getHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.indexOf("gzip") > -1)
{
try
{
final InputStream decompressStream = StreamHelper.decompressStream(httpServletRequest.getInputStream());
httpServletRequest = new HttpServletRequestWrapper(httpServletRequest)
{
#Override
public ServletInputStream getInputStream() throws IOException
{
return new DecompressServletInputStream(decompressStream);
}
#Override
public BufferedReader getReader() throws IOException
{
return new BufferedReader(new InputStreamReader(decompressStream));
}
};
}
catch (IOException e)
{
mLogger.error("error while handling the request", e);
}
}
chain.doFilter(httpServletRequest, response);
}
Simple ServletInputStream wrapper class
public static class DecompressServletInputStream extends ServletInputStream
{
private InputStream inputStream;
public DecompressServletInputStream(InputStream input)
{
inputStream = input;
}
#Override
public int read() throws IOException
{
return inputStream.read();
}
}
Decompression stream code
public class StreamHelper
{
/**
* Gzip magic number, fixed values in the beginning to identify the gzip
* format <br>
* http://www.gzip.org/zlib/rfc-gzip.html#file-format
*/
private static final byte GZIP_ID1 = 0x1f;
/**
* Gzip magic number, fixed values in the beginning to identify the gzip
* format <br>
* http://www.gzip.org/zlib/rfc-gzip.html#file-format
*/
private static final byte GZIP_ID2 = (byte) 0x8b;
/**
* Return decompression input stream if needed.
*
* #param input
* original stream
* #return decompression stream
* #throws IOException
* exception while reading the input
*/
public static InputStream decompressStream(InputStream input) throws IOException
{
PushbackInputStream pushbackInput = new PushbackInputStream(input, 2);
byte[] signature = new byte[2];
pushbackInput.read(signature);
pushbackInput.unread(signature);
if (signature[0] == GZIP_ID1 && signature[1] == GZIP_ID2)
{
return new GZIPInputStream(pushbackInput);
}
return pushbackInput;
}
}

Add to your header when you are sending:
JSON : "Accept-Encoding" : "gzip, deflate"
Client code :
HttpUriRequest request = new HttpGet(url);
request.addHeader("Accept-Encoding", "gzip");
#JulianReschke pointed out that there can be a case of:
"Content-Encoding" : "gzip, gzip"
so extended server code will be:
InputStream in = response.getEntity().getContent();
Header encodingHeader = response.getFirstHeader("Content-Encoding");
String gzip = "gzip";
if (encodingHeader != null) {
String encoding = encodingHeader.getValue().toLowerCase();
int firstGzip = encoding.indexOf(gzip);
if (firstGzip > -1) {
in = new GZIPInputStream(in);
int secondGzip = encoding.indexOf(gzip, firstGzip + gzip.length());
if (secondGzip > -1) {
in = new GZIPInputStream(in);
}
}
}
I suppose that nginx is used as load balancer or proxy, so you need to set tomcat to do decompression.
Add following attributes to the Connector in server.xml on Tomcat,
<Connector
compression="on"
compressionMinSize="2048"
compressableMimeType="text/html,application/json"
... />
Accepting gziped requests in tomcat is a different story. You'll have to put a filter in front of your servlets to enable request decompression. You can find more about that here.