In Google's Appengine with Java, the servlet:
#Override protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
Writer w=resp.getWriter();
w.write("a");
resp.resetBuffer();
w=resp.getWriter();
w.write("b");
}
gives "ab" in the response, in the development server. I was expected just "b", from reading Servlet Spec 2.5.
What is wrong?
I'm reading the documentation at https://cloud.google.com/appengine/docs/java/requests.
I tried resp.setBufferSize(8192) and then getBufferSize() but just got 1.
appengine version: 1.9.34, java version: openjdk version "1.8.0_66-internal", ubuntu 14.10
Here is what the Java Servlet Specification (Version 3.0) says:
"The reset method clears data in the buffer when the response is not
committed. Headers and status codes set by the servlet prior to the
reset call must be cleared as well. The resetBuffer method clears
content in the buffer if the response is not committed without
clearing the headers and status code.
If the response is committed and the reset or resetBuffer method is
called, an IllegalStateException must be thrown. The response and its
associated buffer will be unchanged."
Facially, that says that resetBuffer should either clear out the "a" or throw an exception.
I guess, you could make the argument that the characters are buffered in the Writer not the response buffer, but the Servlet spec doesn't make any such distinction. Furthermore, such an interpretation of the spec would render resetBuffer effectively useless.
If Google AppEngine is really behaving the way you say, I'd call that a bug.
Related
I need to make a java REST service that will return an inputstream as a response. My problem is that I don't know how to close the stream after the client receives the entire stream. I'm using Java and CXF. Thanks
#GET
#Path("/{uuid}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getAttachmentByUuid(#PathParam("uuid")String uuid)
{
//getting inputstream from AWS S3
InpputSream is=getStreamFromS3(uuid);
return Response.status(Response.Status.OK).entity(is).build();
// will this "is" stream causes memory leak,, do I have to close it. Client side is not controlled by me
}
JAX-RS is implemented using Java servlets. In case of CXF is used CXFServlet. Your stream will be sent to client using the HttpServletResponse of the servlet interface
public void doGet(HttpServletRequest request, HttpServletResponse response)
You should not close an stream source (HttpServletResponse) if you have not created it. It is responsability of the container, and you can interfere with the life cycle of the request
See also Is is necessary to close the input stream returned from HttpServletRequest?
If you have a stream to close, consider try with resources:
#GET
#Path("/{uuid}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getAttachmentByUuid(#PathParam("uuid")String uuid)
{
//getting inputstream from AWS S3
// the try block opens the stream and guarantees to close it
try (InputStream is=getStreamFromS3(uuid)) {
return Response.status(Response.Status.OK).entity(from(is)).build();
}
}
This requires Java 7 and onwards. It's also awesome!
If you're in Java 6, then you would have to make your own finally block to remember to close the stream for you.
You might be looking to use a 'Conduit'
See CXF Apache Custom Transport for more info.
Be Careful though, the documentation states :
It is strongly recommended to don’t break streaming in Conduit and Destination implementations, if physical protocol supports it. CXF is completely streaming oriented – it causes high performance and scalability.
I've been having an issue with Jetty processing application/json formatted request body data. Essentially, when the request body is processed by Jetty, the request data is cut off.
I have a relatively large POST body of around 74,000 bytes. As per some advice I found online, I instantiated a new context handler with the setMaxFormContentSize property set to a sufficiently large size of 500,000 bytes.
ServletContextHandler handler = new ServletContextHandler(server, "/");
handler.setMaxFormContentSize(500000);
However, this did not seem to work correctly. I also read online that this property might only work for form encoded data, not application/json, which is a strict requirement of our application.
Is there any way to circumvent this issue? Is there some special constraint class that I can subclass to allow the processing size to increase to at least 500KB?
Edit #1: I should add that I also tried to drop the size of the limit to 5 bytes to see if it would cut off more of the data in the payload. That also didn't working, which seems to imply that's definitely ignoring the property entirely.
Edit #2: Here is where I read the information from the request stream.
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
try {
String json = CharStreams.toString(new InputStreamReader(req.getInputStream()));
....
} catch (Exception e) {
logger.error("Exception in internal api forwarder", e);
throw e;
}
}
This seems to a standard way of reading from a request stream. I also tried using a BufferedReader from req.getReader() with the same issue.
Vivek
What is this CharStreams object?
It doesn't seem to know, or care, or honor the request character encoding. (Bad idea)
Suggest that you use the servlet request.getReader() instead of request.getInputStream() (which is really only designed for binary request body content)
Using request.getReader() will at the very least support your request character encoding properly.
Another bit of information you might want to look into is request.getContentLength() and verify that the request headers does indeed contain the size you are expecting.
I am writing a REST application using Tomcat and Spring WebMVC.
I want to signal errors to my client using HTTP status codes along with some XML payload that contains more information about what went wrong.
To catch all errors regardless of where they occur, I have written a Filter which wraps the response and overrides the sendError() method:
private static final class GenericErrorResponseWrapper
extends HttpServletResponseWrapper
{
#Override
public void sendError(int sc, String msg) throws IOException {
final HttpServletResponse wrappedResponse = (HttpServletResponse) getResponse();
wrappedResponse.setStatus(sc, msg);
wrappedResponse.setContentType("application/xml");
PrintWriter writer = wrappedResponse.getWriter();
try {
SimpleXmlWriter xmlWriter = SimpleXmlWriterWrapper.newInstance(writer);
xmlWriter.writeStartElement("ns2", "genericError")
.writeAttribute("xmlns:ns2", "http://mynamespace")
.writeCharacters(msg)
.writeEndDocument().flush();
writer.flush();
wrappedResponse.flushBuffer();
} finally {
writer.close();
}
}
}
This implementation has two problems:
It generates a deprecation warning in Eclipse, since HttpServletResponse.setStatus(sc, msg) is deprecated.
The HTTP response header generated by Tomcat is not correct, it starts with the first line "HTTP/1.1 500 OK". 500 is correct, but instead of OK the reason phrase should be "Internal Server Error".
How can I implement my filter so that it does the right thing and is free of deprecation warnings? Both alternatives named in the Javadoc are not usable for me:
sendError(sc, msg) is not usable, since it commits the response body and I can't write XML payload any more
setStatus(sc) with just the error code is theoretically usable, but it also creates the hardcoded "OK" string in the first line of the response header.
There is unfortunately no way to avoid the deprecation warning. As you already mention yourself, the two alternatives which are referred to in the API documentation do not cover the same functionality. You may of course annotate your method with #SuppressWarnings("deprecation") to indicate that the usage of the deprecated method is intended.
The other thing, that Tomcat does not use your message string, even if one is provided, is a configuration issue. For some strange reason, Tomcat will by default ignore the provided message string and use a default error message based on the passed return code. You must set the system property org.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER to true to force Tomcat to use your provided error message instead. More details on this can be found in the Tomcat documentation.
As an alternative answer - you could first write the XML payload, without calling flush/flushBuffer, and only after that do sendError(int, String), which would flush the buffer.
I have this code where I read the input from a request input stream and use a JacksonMapper to convert into a POJO. Its running in a jetty 7 container with guice support.
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
try {
RequestType requestType = mapper.readValue(req.getInputStream(), RequestType.class);
} Catch(Exception ex) {
....
}
}
However, sometimes under load the following exception is thrown. I have checked my client and I am sure its sending a valid json string. What is going wrong? Is it expected behavior for Jetty 7 under load?
java.io.EOFException: No content to map to Object due to end of input
at org.codehaus.jackson.map.ObjectMapper._initForReading(ObjectMapper.java:2433)
at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2385)
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1637)
at com.ea.wsop.user.LoginServlet.processRequest(LoginServlet.java:69)
at com.ea.wsop.user.LoginServlet.doPost(LoginServlet.java:63)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd.CGLIB$doPost$0(<generated>)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd$$FastClassByGuice$$c6f479ee.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.ea.monitor.MethodExecutionTimer.invoke(MethodExecutionTimer.java:130)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd.doPost(<generated>)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd.CGLIB$service$8(<generated>)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd$$FastClassByGuice$$c6f479ee.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.ea.monitor.MethodExecutionTimer.invoke(MethodExecutionTimer.java:130)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd.service(<generated>)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd.CGLIB$service$9(<generated>)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd$$FastClassByGuice$$c6f479ee.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.ea.monitor.MethodExecutionTimer.invoke(MethodExecutionTimer.java:130)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at com.ea.wsop.user.LoginServlet$$EnhancerByGuice$$a91c2ebd.service(<generated>)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
I had a similar problem running a Spring Boot application. My Spring Boot app is a simple Dispatcher servlet that reads the request body and processes it.
In my case, the client (curl) sets a content-type header of application/x-www-form-urlencoded if the curl command line uses -d {some-data} and does not set an specific content-type header via -Hcontent-type=some-other-media-type.
Inside the Apache Catalina servlet engine that Spring Boot runs, the Request class makes the following test in parseParameters()
if (!("application/x-www-form-urlencoded".equals(contentType))) {
success = true;
return;
}
For other content-type values, Request returns here, done.
However, if the content type matches application/x-www-form-urlencoded, Request continues:
try {
if (readPostBody(formData, len) != len) {
parameters.setParseFailedReason(FailReason.REQUEST_BODY_INCOMPLETE);
return;
}
} catch (....)
which will consume the body. So in my case, even though my servlet does nothing other than call request.getInputStream() and try to read() from it, it is already too late - the runtime Request already reads the input and does not buffer or unread it. The only workaround is to set a different Content-Type.
The culprit is
OrderedHiddenHttpMethodFilter(HiddenHttpMethodFilter).doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain) line 70
which is looking for the "_method" query parameter.
I was able to disable the filter by adding
#Bean
public FilterRegistrationBean registration(HiddenHttpMethodFilter filter) {
FilterRegistrationBean registration = new FilterRegistrationBean(filter);
registration.setEnabled(false);
return registration;
}
(which was used to solve another problem)
It will be empty if it's already consumed beforehand. This will be implicitly done whenever you call getParameter(), getParameterValues(), getParameterMap(), getReader(), etc on the HttpServletRequest. Make sure that you don't call any of those kind of methods which by themselves need to gather information from the request body before calling getInputStream(). If your servlet isn't doing that, then start checking the servlet filters which are mapped on the same URL pattern.
Update: this seems to be GAE 1.5 specific. See also
http://code.google.com/p/googleappengine/issues/detail?id=5161
http://code.google.com/p/googleappengine/issues/detail?id=5898
I'm afraid that there's no solution/workaround until they get it fixed. You could try to check if it's available inside a Filter and if so, then copy and store it as request attribute. But this might affect further processing by some GAE servlet.
I had the problem that my request InputStream was always empty with Jetty 6.1.15, and found out that it was caused by a missing or wrong "Content-Type" header.
I generate the requests in another Java program with HttpUrlConnection. When I did not set the Content-Type header explicitly, the InputStream returned by request.getInputStream() in the receiving program was always empty. When I set the content type to "binary/octet-stream", the InputStream of the request contained the correct data.
The only method that is called on the request object before getInputStream() is getContentLength().
I was using mod_jk 1.2.39 which had a bug that caused this issue. After updating to 1.2.40 it started working.
I've had this problem with a post. I solved it by FIRST reading the inputstream and putting it in a cache, before reading the parameters. That seemed to do the trick
Systematic approach is:
Get source code for your container, or at least it's web part (can be hard to find), import in your IDE.
Make break point in your code where before HttpServletRequest->getInputStream() is called.
Step into HttpServletRequest->getInputStream() method, now you are in some ...Impl class.
Set a new break point in that getInputStream() implemmentation, or even in its read() method.
Repeat test call and see what is consuming your data.
I ended up with the problem when enabling debug logging for org.springframework in a Spring Boot 2.2.1 project, and thus using spring-webmvc 5.2.1.
This is caused by the request logging of the parameter-map, which reads the input stream if the Content-Type is application/x-www-form-urlencoded. I believe this spring issue is related to it.
See the following code which causes the problem.
private void logRequest(HttpServletRequest request) {
LogFormatUtils.traceDebug(logger, traceOn -> {
String params;
if (isEnableLoggingRequestDetails()) {
params = request.getParameterMap().entrySet().stream()
.map(entry -> entry.getKey() + ":" + Arrays.toString(entry.getValue()))
.collect(Collectors.joining(", "));
}
else {
params = (request.getParameterMap().isEmpty() ? "" : "masked");
}
...
source
I ended up reporting an issue and and changing the content-type in the request instead.
I want to check the header of the request whether it contains a certain header or not before continuing with the body. For example, I want to check whether a multipart/form-data contains "Authorization" in the header or not. If it is not then there is no need to continue with uploading the multipart body which are generally quite large for file uploading.
Does servlet allow you to do this? I have tried to search on google randomly but there is no luck. Here is the code i try in my servlet but it still continues with recieving the body before this doPost method is called. It seems that the stream is fully received before the servlet is invoked.
#Override
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
PrintWriter out = response.getWriter();
response.setContentType("text/plain");
if (request.getHeader("Authorization") == null) {
response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
out.println("Status: " + HttpServletResponse.SC_UNAUTHORIZED + " - UNAUTHORIZED");
return;
}
// ... the rests
}
That's the limitation of HTTP. You can't send a response when the request hasn't been read fully to end.
RFC 2616 says:
An HTTP/1.1 (or later) client sending a message-body SHOULD monitor the network connection for an error status while it is transmitting the request. If the client sees an error status, it SHOULD immediately cease transmitting the body.
So I disagree, this is not a HTTP limitation but a servlet one.