I have spring mvc application using RequestBody and ResponseBody annotations. They are configured with MappingJackson2HttpMessageConverter. I also have slf4j set up. I would like to log all json as it comes in and out from my controller.
I did extend
MappingJackson2HttpMessageConverter
#Override
public Object read(Type type, Class<?> contextClass, HttpInputMessage inputMessage)
throws IOException, HttpMessageNotReadableException {
logStream(inputMessage.getBody());
return super.read(type, contextClass, inputMessage);
}
I can get the input stream, but if I read the content it becomes empty and I loose the message. Moreover mark() and reset() is not supported. It is implemented by PushbackInputStream, so I tried to read it's content and push it back like this:
public void logStream(InputStream is) {
if (is instanceof PushbackInputStream)
try {
PushbackInputStream pushbackInputStream = (PushbackInputStream) is;
byte[] bytes = new byte[20000];
StringBuilder sb = new StringBuilder(is.available());
int red = is.read();
int pos =0;
while (red > -1) {
bytes[pos] = (byte) red;
pos=1 + pos;
red = is.read();
}
pushbackInputStream.unread(bytes,0, pos-1);
log.info("Json payload " + sb.toString());
} catch (Exception e) {
log.error("ignoring exception in logger ", e);
}
}
but I get exception
java.io.IOException: Push back buffer is full
I also tried to turn on logging on http level as described here:Spring RestTemplate - how to enable full debugging/logging of requests/responses? without luck.
After more than whole work day of experimenting I got working solution.
It consists of Logging filter, two wrappers for request and response and registration of Logging filter:
the filter class is:
/**
* Http logging filter, which wraps around request and response in
* each http call and logs
* whole request and response bodies. It is enabled by
* putting this instance into filter chain
* by overriding getServletFilters() in
* AbstractAnnotationConfigDispatcherServletInitializer.
*/
public class LoggingFilter extends AbstractRequestLoggingFilter {
private static final Logger log = LoggerFactory.getLogger(LoggingFilter.class);
#Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
long id = System.currentTimeMillis();
RequestLoggingWrapper requestLoggingWrapper = new RequestLoggingWrapper(id, request);
ResponseLoggingWrapper responseLoggingWrapper = new ResponseLoggingWrapper(id, response);
log.debug(id + ": http request " + request.getRequestURI());
super.doFilterInternal(requestLoggingWrapper, responseLoggingWrapper, filterChain);
log.debug(id + ": http response " + response.getStatus() + " finished in " + (System.currentTimeMillis() - id) + "ms");
}
#Override
protected void beforeRequest(HttpServletRequest request, String message) {
}
#Override
protected void afterRequest(HttpServletRequest request, String message) {
}
}
this class is using stream wrappers, which was suggested by
Master Slave and David Ehrmann.
Request wrapper looks like this:
/**
* Request logging wrapper using proxy split stream to extract request body
*/
public class RequestLoggingWrapper extends HttpServletRequestWrapper {
private static final Logger log = LoggerFactory.getLogger(RequestLoggingWrapper.class);
private final ByteArrayOutputStream bos = new ByteArrayOutputStream();
private long id;
/**
* #param requestId and id which gets logged to output file. It's used to bind request with
* response
* #param request request from which we want to extract post data
*/
public RequestLoggingWrapper(Long requestId, HttpServletRequest request) {
super(request);
this.id = requestId;
}
#Override
public ServletInputStream getInputStream() throws IOException {
final ServletInputStream servletInputStream = RequestLoggingWrapper.super.getInputStream();
return new ServletInputStream() {
private TeeInputStream tee = new TeeInputStream(servletInputStream, bos);
#Override
public int read() throws IOException {
return tee.read();
}
#Override
public int read(byte[] b, int off, int len) throws IOException {
return tee.read(b, off, len);
}
#Override
public int read(byte[] b) throws IOException {
return tee.read(b);
}
#Override
public boolean isFinished() {
return servletInputStream.isFinished();
}
#Override
public boolean isReady() {
return servletInputStream.isReady();
}
#Override
public void setReadListener(ReadListener readListener) {
servletInputStream.setReadListener(readListener);
}
#Override
public void close() throws IOException {
super.close();
// do the logging
logRequest();
}
};
}
public void logRequest() {
log.info(getId() + ": http request " + new String(toByteArray()));
}
public byte[] toByteArray() {
return bos.toByteArray();
}
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
}
and response wrapper is different only in close/flush method (close doesn't get called)
public class ResponseLoggingWrapper extends HttpServletResponseWrapper {
private static final Logger log = LoggerFactory.getLogger(ResponseLoggingWrapper.class);
private final ByteArrayOutputStream bos = new ByteArrayOutputStream();
private long id;
/**
* #param requestId and id which gets logged to output file. It's used to bind response with
* response (they will have same id, currenttimemilis is used)
* #param response response from which we want to extract stream data
*/
public ResponseLoggingWrapper(Long requestId, HttpServletResponse response) {
super(response);
this.id = requestId;
}
#Override
public ServletOutputStream getOutputStream() throws IOException {
final ServletOutputStream servletOutputStream = ResponseLoggingWrapper.super.getOutputStream();
return new ServletOutputStream() {
private TeeOutputStream tee = new TeeOutputStream(servletOutputStream, bos);
#Override
public void write(byte[] b) throws IOException {
tee.write(b);
}
#Override
public void write(byte[] b, int off, int len) throws IOException {
tee.write(b, off, len);
}
#Override
public void flush() throws IOException {
tee.flush();
logRequest();
}
#Override
public void write(int b) throws IOException {
tee.write(b);
}
#Override
public boolean isReady() {
return servletOutputStream.isReady();
}
#Override
public void setWriteListener(WriteListener writeListener) {
servletOutputStream.setWriteListener(writeListener);
}
#Override
public void close() throws IOException {
super.close();
// do the logging
logRequest();
}
};
}
public void logRequest() {
byte[] toLog = toByteArray();
if (toLog != null && toLog.length > 0)
log.info(getId() + ": http response " + new String(toLog));
}
/**
* this method will clear the buffer, so
*
* #return captured bytes from stream
*/
public byte[] toByteArray() {
byte[] ret = bos.toByteArray();
bos.reset();
return ret;
}
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
}
at last LoggingFilter needs to be registered in AbstractAnnotationConfigDispatcherServletInitializer like this:
#Override
protected Filter[] getServletFilters() {
LoggingFilter requestLoggingFilter = new LoggingFilter();
return new Filter[]{requestLoggingFilter};
}
I know, there is maven lib for this, but I don't want to include whole lib because of small logging utility. It was much harder than I originally thought. I expected to achieve this just by modifying log4j.properties. I still think this should be part of Spring.
It sounds like you want to decorate HttpInputMessage so it returns a decorated InputStream that logs all reads in an internal buffer, then on close() or finalize() logs what was read.
Here's an InputStream that will capture what was read:
public class LoggingInputStream extends FilterInputStream {
private ByteArrayOutputStream out = new ByteArrayOutputStream();
private boolean logged = false;
protected LoggingInputStream(InputStream in) {
super(in);
}
#Override
protected void finalize() throws Throwable {
try {
this.log();
} finally {
super.finalize();
}
}
#Override
public void close() throws IOException {
try {
this.log();
} finally {
super.close();
}
}
#Override
public int read() throws IOException {
int r = super.read();
if (r >= 0) {
out.write(r);
}
return r;
}
#Override
public int read(byte[] b) throws IOException {
int read = super.read(b);
if (read > 0) {
out.write(b, 0, read);
}
return read;
}
#Override
public int read(byte[] b, int off, int len) throws IOException {
int read = super.read(b, off, len);
if (read > 0) {
out.write(b, off, read);
}
return read;
}
#Override
public long skip(long n) throws IOException {
long skipped = 0;
byte[] b = new byte[4096];
int read;
while ((read = this.read(b, 0, (int)Math.min(n, b.length))) >= 0) {
skipped += read;
n -= read;
}
return skipped;
}
private void log() {
if (!logged) {
logged = true;
try {
log.info("Json payload " + new String(out.toByteArray(), "UTF-8");
} catch (UnsupportedEncodingException e) { }
}
}
}
And now
#Override
public Object read(Type type, Class<?> contextClass, final HttpInputMessage inputMessage)
throws IOException, HttpMessageNotReadableException {
return super.read(type, contextClass, new HttpInputMessage() {
#Override
public InputStream getBody() {
return new LoggingInputStream(inputMessage.getBody());
}
#Override
public HttpHeaders getHeaders() {
return inputMessage.getHeaders();
}
});
}
Decorating HttpInputMessage as David Ehrmann suggested is one likely solution.
The whole trouble with this functionality is that it requires InputStream to be read more than once. However, this is not possible, once you read a portion or a stream, its "consumed" and there no way to go back and read it again.
A typical solution is to apply a filter that will create a wrapper for a request that will allow re-reads of the inputStream. One approach is by using the TeeInputStream which copies all the bytes read from InputStream to a secondary OutputStream.
There's a github project that uses just that kind of a filter and in fact just for the same purpose spring-mvc-logger The RequestWrapper class used
public class RequestWrapper extends HttpServletRequestWrapper {
private final ByteArrayOutputStream bos = new ByteArrayOutputStream();
private long id;
public RequestWrapper(Long requestId, HttpServletRequest request) {
super(request);
this.id = requestId;
}
#Override
public ServletInputStream getInputStream() throws IOException {
return new ServletInputStream() {
private TeeInputStream tee = new TeeInputStream(RequestWrapper.super.getInputStream(), bos);
#Override
public int read() throws IOException {
return tee.read();
}
};
}
public byte[] toByteArray(){
return bos.toByteArray();
}
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
}
A similar implementation wraps the response as well
Related
I have this code:
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
throws IOException, ServletException {
logger.info("Filter start...");
HttpServletRequest httpRequest = (HttpServletRequest) request;
HttpServletResponse httpResponse = (HttpServletResponse) response;
String ba = getBaId(getBody(httpRequest));
if (ba == null) {
logger.error("Wrong XML");
httpResponse.setStatus(HttpServletResponse.SC_BAD_REQUEST);
} else {
if (!clients.containsKey(ba)) {
clients.put(ba, 1);
logger.info("Client map : init...");
} else {
clients.put(ba, clients.get(ba).intValue() + 1);
logger.info("Threads for " + ba + " = " + clients.get(ba).toString());
}
chain.doFilter(request, response);
}
}
and this web.xml (packages are shortened and names changed, but it looks the same)
<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app>
<filter>
<filter-name>TestFilter</filter-name>
<filter-class>pkg.TestFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>TestFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>WEB-INF/applicationContext.xml</param-value>
</context-param>
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>
<servlet>
<servlet-name>Name</servlet-name>
<display-name>Name</display-name>
<servlet-class>pkg.Name</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>Name</servlet-name>
<url-pattern>/services/*</url-pattern>
</servlet-mapping>
</web-app>
I want to invoke the Servlet after the Filter. I was hoping chain.doFilter(...) could do the trick, but i always get this error on the line with chain.doFilter(...):
java.lang.IllegalStateException: getInputStream() can't be called after getReader()
at com.caucho.server.connection.AbstractHttpRequest.getInputStream(AbstractHttpRequest.java:1933)
at org.apache.cxf.transport.http.AbstractHTTPDestination.setupMessage(AbstractHTTPDestination.java:249)
at org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:82)
at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:283)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:166)
at org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:174)
at org.apache.cxf.transport.servlet.AbstractCXFServlet.doPost(AbstractCXFServlet.java:152)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:153)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
at pkg.TestFilter.doFilter(TestFilter.java:102)
at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:187)
at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:265)
at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:273)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:682)
at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:743)
at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:662)
at java.lang.Thread.run(Thread.java:619)
Working code based on the accepted answer.
public class CustomHttpServletRequestWrapper extends HttpServletRequestWrapper {
private static final Logger logger = Logger.getLogger(CustomHttpServletRequestWrapper.class);
private final String body;
public CustomHttpServletRequestWrapper(HttpServletRequest request) {
super(request);
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
InputStream inputStream = request.getInputStream();
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
char[] charBuffer = new char[128];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
} else {
stringBuilder.append("");
}
} catch (IOException ex) {
logger.error("Error reading the request body...");
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException ex) {
logger.error("Error closing bufferedReader...");
}
}
}
body = stringBuilder.toString();
}
#Override
public ServletInputStream getInputStream () throws IOException {
final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(body.getBytes());
ServletInputStream inputStream = new ServletInputStream() {
public int read () throws IOException {
return byteArrayInputStream.read();
}
};
return inputStream;
}
}
You probably start consuming the HttpServletRequest using getReader() in :
String ba = getBaId(getBody(httpRequest));
Your servlet tries to call getInputStream() on the same request, which is not allowed. What you need to do is use a ServletRequestWrapper to make a copy of the body of the request, so you can read it with multiple methods. I dont have the time to find a complete example right know ... sorry ...
This worked for me. It implements getInputStream.
private class MyHttpServletRequestWrapper extends HttpServletRequestWrapper {
private byte[] body;
public MyHttpServletRequestWrapper(HttpServletRequest request) {
super(request);
try {
body = IOUtils.toByteArray(request.getInputStream());
} catch (IOException ex) {
body = new byte[0];
}
}
#Override
public ServletInputStream getInputStream() throws IOException {
return new ServletInputStream() {
ByteArrayInputStream bais = new ByteArrayInputStream(body);
#Override
public int read() throws IOException {
return bais.read();
}
};
}
}
Then you use in your method:
//copy body
servletRequest = new MyHttpServletRequestWrapper(servletRequest);
For Servlet 3.1
class MyHttpServletRequestWrapper extends HttpServletRequestWrapper {
private byte[] body;
public MyHttpServletRequestWrapper(HttpServletRequest request) {
super(request);
try {
body = IOUtils.toByteArray(request.getInputStream());
} catch (IOException ex) {
body = new byte[0];
}
}
#Override
public ServletInputStream getInputStream() throws IOException {
return new DelegatingServletInputStream(new ByteArrayInputStream(body));
}
}
public class DelegatingServletInputStream extends ServletInputStream {
private final InputStream sourceStream;
private boolean finished = false;
/**
* Create a DelegatingServletInputStream for the given source stream.
*
* #param sourceStream the source stream (never {#code null})
*/
public DelegatingServletInputStream(InputStream sourceStream) {
this.sourceStream = sourceStream;
}
/**
* Return the underlying source stream (never {#code null}).
*/
public final InputStream getSourceStream() {
return this.sourceStream;
}
#Override
public int read() throws IOException {
int data = this.sourceStream.read();
if (data == -1) {
this.finished = true;
}
return data;
}
#Override
public int available() throws IOException {
return this.sourceStream.available();
}
#Override
public void close() throws IOException {
super.close();
this.sourceStream.close();
}
#Override
public boolean isFinished() {
return this.finished;
}
#Override
public boolean isReady() {
return true;
}
#Override
public void setReadListener(ReadListener readListener) {
throw new UnsupportedOperationException();
}
}
inputStream in servlet request can only be used once because of it is stream,you can store it and then get it from a byte array,this can resolve.
public class HttpServletRequestWrapper extends javax.servlet.http.HttpServletRequestWrapper {
private final byte[] body;
public HttpServletRequestWrapper(HttpServletRequest request)
throws IOException {
super(request);
body = StreamUtil.readBytes(request.getReader(), "UTF-8");
}
#Override
public BufferedReader getReader() throws IOException {
return new BufferedReader(new InputStreamReader(getInputStream()));
}
#Override
public ServletInputStream getInputStream() throws IOException {
final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(body);
return new ServletInputStream() {
#Override
public int read() throws IOException {
return byteArrayInputStream.read();
}
#Override
public boolean isFinished() {
return false;
}
#Override
public boolean isReady() {
return false;
}
#Override
public void setReadListener(ReadListener arg0) {
}
};
}
}
in filter:
ServletRequest requestWrapper = new HttpServletRequestWrapper(request);
request.getInputStream() is allowed to read only one time. In order to use this method many times, we need to do extra the custom task to HttpServletReqeustWrapper class. see my sample wrapper class below.
public class MultiReadHttpServletRequest extends HttpServletRequestWrapper {
private ByteArrayOutputStream cachedBytes;
public MultiReadHttpServletRequest(HttpServletRequest request) {
super(request);
}
#Override
public ServletInputStream getInputStream() throws IOException {
if (cachedBytes == null)
cacheInputStream();
return new CachedServletInputStream();
}
#Override
public BufferedReader getReader() throws IOException {
return new BufferedReader(new InputStreamReader(getInputStream()));
}
private void cacheInputStream() throws IOException {
/*
* Cache the inputstream in order to read it multiple times. For convenience, I use apache.commons IOUtils
*/
cachedBytes = new ByteArrayOutputStream();
IOUtils.copy(super.getInputStream(), cachedBytes);
}
/* An inputstream which reads the cached request body */
public class CachedServletInputStream extends ServletInputStream {
private ByteArrayInputStream input;
public CachedServletInputStream() {
/* create a new input stream from the cached request body */
input = new ByteArrayInputStream(cachedBytes.toByteArray());
}
#Override
public int read() throws IOException {
return input.read();
}
}
}
In my case, I trace all incoming requests into the log. I created a Filter
public class TracerRequestFilter implements Filter {
private static final Logger LOG = LoggerFactory.getLogger(TracerRequestFilter.class);
#Override
public void destroy() {
}
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException,
ServletException {
final HttpServletRequest req = (HttpServletRequest) request;
try {
if (LOG.isDebugEnabled()) {
final MultiReadHttpServletRequest wrappedRequest = new MultiReadHttpServletRequest(req);
// debug payload info
logPayLoad(wrappedRequest);
chain.doFilter(wrappedRequest, response);
} else {
chain.doFilter(request, response);
}
} finally {
LOG.info("end-of-process");
}
}
private String getRemoteAddress(HttpServletRequest req) {
String ipAddress = req.getHeader("X-FORWARDED-FOR");
if (ipAddress == null) {
ipAddress = req.getRemoteAddr();
}
return ipAddress;
}
private void logPayLoad(HttpServletRequest request) {
final StringBuilder params = new StringBuilder();
final String method = request.getMethod().toUpperCase();
final String ipAddress = getRemoteAddress(request);
final String userAgent = request.getHeader("User-Agent");
LOG.debug(String.format("============debug request=========="));
LOG.debug(String.format("Access from ip:%s;ua:%s", ipAddress, userAgent));
LOG.debug(String.format("Method : %s requestUri %s", method, request.getRequestURI()));
params.append("Query Params:").append(System.lineSeparator());
Enumeration<String> parameterNames = request.getParameterNames();
for (; parameterNames.hasMoreElements();) {
String paramName = parameterNames.nextElement();
String paramValue = request.getParameter(paramName);
if ("password".equalsIgnoreCase(paramName) || "pwd".equalsIgnoreCase(paramName)) {
paramValue = "*****";
}
params.append("---->").append(paramName).append(": ").append(paramValue).append(System.lineSeparator());
}
LOG.debug(params.toString());
/** request body */
if ("POST".equals(method) || "PUT".equals(method)) {
try {
LOG.debug(IOUtils.toString(request.getInputStream()));
} catch (IOException e) {
LOG.error(e.getMessage(), e);
}
}
LOG.debug(String.format("============End-debug-request=========="));
}
#Override
public void init(FilterConfig arg0) throws ServletException {
}
}
It works for me both Servlet 2.5 and 3.0. I see all request params both form-encoded and request json body.
(Java ver. 8)
I need to process the request body in a filter. Using the below code, I read the body.
private static String convertInputStreamToString(InputStream is) throws IOException {
ByteArrayOutputStream result = new ByteArrayOutputStream();
byte[] buffer = new byte[1024 * 50];
int length;
while ((length = is.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
return result.toString("UTF-8");
}
The issue is if there are parameters posted by request body with the content type "application/x-www-form-urlencoded", then the parameters won't be available after reading the body. They are available to get using request.getParameter(), if I don't read the body.
Moreover, I tried using the below code to wrap the request and provide the body, so it would be available to the rest of the solution (e.g. servlets), but the issue with losing the parameters happens yet. code is copied/adopted from this post
public class RequestWrapper extends HttpServletRequestWrapper {
private final String body;
public RequestWrapper(HttpServletRequest request) throws IOException {
super(request);
body = convertInputStreamToString(request.getInputStream());
}
private static String convertInputStreamToString(InputStream is) throws IOException {
ByteArrayOutputStream result = new ByteArrayOutputStream();
byte[] buffer = new byte[1024 * 50];
int length;
while ((length = is.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
return result.toString("UTF-8");
}
#Override
public ServletInputStream getInputStream() throws IOException {
final byte[] myBytes = body.getBytes("UTF-8");
ServletInputStream servletInputStream = new ServletInputStream() {
private int lastIndexRetrieved = -1;
private ReadListener readListener = null;
#Override
public boolean isFinished() {
return (lastIndexRetrieved == myBytes.length - 1);
}
#Override
public boolean isReady() {
return isFinished();
}
#Override
public void setReadListener(ReadListener readListener) {
this.readListener = readListener;
if (!isFinished()) {
try {
readListener.onDataAvailable();
} catch (IOException e) {
readListener.onError(e);
}
} else {
try {
readListener.onAllDataRead();
} catch (IOException e) {
readListener.onError(e);
}
}
}
#Override
public int read() throws IOException {
int i;
if (!isFinished()) {
i = myBytes[lastIndexRetrieved + 1];
lastIndexRetrieved++;
if (isFinished() && (readListener != null)) {
try {
readListener.onAllDataRead();
} catch (IOException ex) {
readListener.onError(ex);
throw ex;
}
}
return i;
} else {
return -1;
}
}
};
return servletInputStream;
}
#Override
public BufferedReader getReader() throws IOException {
return new BufferedReader(new InputStreamReader(this.getInputStream()));
}
}
I tried to run the code you mentioned you're using and I think the accepted answer may not solve your issue as it's quite old. Seems you also need to overwrite the getParameter, getParameterMap and getParameterValues methods. I tried to do that based on this answer from the same post and seems it works. Here is the code:
public class MultiReadHttpServletRequest extends HttpServletRequestWrapper {
private ByteArrayOutputStream cachedBytes;
private String body;
private Map<String, String[]> parameterMap;
public MultiReadHttpServletRequest(HttpServletRequest request) throws IOException {
super(request);
parameterMap = super.getParameterMap();
cacheBodyAsString();
System.out.println("The Body read into a String is: " + body);
}
#Override
public ServletInputStream getInputStream() throws IOException {
if (cachedBytes == null)
cacheInputStream();
return new CachedServletInputStream(cachedBytes.toByteArray());
}
#Override
public BufferedReader getReader() throws IOException {
return new BufferedReader(new InputStreamReader(getInputStream()));
}
#Override
public String getParameter(String key) {
Map<String, String[]> parameterMap = getParameterMap();
String[] values = parameterMap.get(key);
return values != null && values.length > 0 ? values[0] : null;
}
#Override
public String[] getParameterValues(String key) {
Map<String, String[]> parameterMap = getParameterMap();
return parameterMap.get(key);
}
#Override
public Map<String, String[]> getParameterMap() {
return parameterMap;
}
private void cacheInputStream() throws IOException {
// Cache the inputstream in order to read it multiple times
cachedBytes = new ByteArrayOutputStream();
byte[] buffer = new byte[1024 * 50];
int length;
InputStream is = super.getInputStream();
while ((length = is.read(buffer)) != -1) {
cachedBytes.write(buffer, 0, length);
}
}
private void cacheBodyAsString() throws IOException {
ByteArrayOutputStream result = new ByteArrayOutputStream();
byte[] buffer = new byte[1024 * 50];
int length;
InputStream is = getInputStream();
while ((length = is.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
body = result.toString("UTF-8");
}
}
public class CachedServletInputStream extends ServletInputStream {
private final ByteArrayInputStream buffer;
public CachedServletInputStream(byte[] contents) {
this.buffer = new ByteArrayInputStream(contents);
}
#Override
public int read() {
return buffer.read();
}
#Override
public boolean isFinished() {
return buffer.available() == 0;
}
#Override
public boolean isReady() {
return true;
}
#Override
public void setReadListener(ReadListener listener) {
throw new RuntimeException("Not implemented");
}
}
This is just a sample implementation. I highly recommend to follow the steps specified in the answer mentioned above as it seems to be newer and it also ensures that the parameters are being read from both body and query string. My code is just a sample sketch to see if it works as expected.
Thank you #zaerymoghaddam for helping with this.
I was concerning if I am affecting the request object implicitly, so the rest of the solution is lacking something in it.
Moreover, I found that parameterMap = super.getParameterMap(); is not icluding the parameters from body (in case of post with content type of "application/x-www-form-urlencoded")
With a little bit of change of your code I came up with below solution:
public class MyRequestWrapper extends HttpServletRequestWrapper {
private ByteArrayOutputStream cachedBytes;
private String body;
private Map<String, String[]> parameterMap;
private static int bufferLength = 1024 * 50;
public MyRequestWrapper(final HttpServletRequest request) throws IOException {
super(request);
cacheBodyAsString();
parameterMap = new HashMap<>(super.getParameterMap());
addParametersFromBody();
}
#Override
public ServletInputStream getInputStream() throws IOException {
return new CachedServletInputStream(cachedBytes.toByteArray());
}
#Override
public BufferedReader getReader() throws IOException {
return new BufferedReader(new InputStreamReader(this.getInputStream()));
}
public String GetRequestBodyAsString() {
return this.body;
}
#Override
public String getParameter(String key) {
Map<String, String[]> parameterMap = getParameterMap();
String[] values = parameterMap.get(key);
return values != null && values.length > 0 ? values[0] : null;
}
#Override
public String[] getParameterValues(String key) {
Map<String, String[]> parameterMap = getParameterMap();
return parameterMap.get(key);
}
#Override
public Map<String, String[]> getParameterMap() {
return parameterMap;
}
private void cacheInputStream() throws IOException {
cachedBytes = new ByteArrayOutputStream();
byte[] buffer = new byte[bufferLength];
int length;
InputStream is = super.getInputStream();
while ((length = is.read(buffer)) != -1) {
cachedBytes.write(buffer, 0, length);
}
}
private void cacheBodyAsString() throws IOException {
if (cachedBytes == null)
cacheInputStream();
this.body = cachedBytes.toString("UTF-8");
}
private void addParametersFromBody() {
if(this.body == null || this.body.isEmpty())
return;
String[] params = this.body.split("&");
String[] value = new String[1];
for (String param : params) {
String key = param.split("=")[0];
value[0] = param.split("=")[1];
parameterMap.putIfAbsent(key, value);
}
}
class CachedServletInputStream extends ServletInputStream {
private final ByteArrayInputStream buffer;
public CachedServletInputStream(byte[] contents) {
this.buffer = new ByteArrayInputStream(contents);
}
#Override
public int read() {
return buffer.read();
}
#Override
public boolean isFinished() {
return buffer.available() == 0;
}
#Override
public boolean isReady() {
return true;
}
#Override
public void setReadListener(ReadListener listener) {
throw new RuntimeException("Not implemented");
}
}
}
Strangely HttpServletRequest content may only be read once. It comes as a stream so once you read the stream it is gone. So you need some wrapper that allows you multiple reads. Spring actually provides such wrapper. The name of the class is ContentCachingRequestWrapper. Here its Javadoc. Here is the answer that explains how to use it if you work with Spring boot: How to get request body params in spring filter?
I have a spring java application that has a EncryptDecryptFilter.java which is a "OncePerRequestFilter" within which I am trying to modify the response payload based on certain conditions. All is going well for happy path.
During the un-happy path, when my application code in the RestController throws an exception, that exception is captured by a #ControllerAdvice class and the controller advice class returns a ResponseEntity. Let's say this response is "string1" with a length 105 characters. After this, the call is intercepted by the "OncePerRequestFilter" and when I attempt to modify the response in this filter with a new response, lets say the modified response is "string2" with a length 200 characters, the modified response string2 is truncated to 105 characters and delievered to the client. So, while the response it self is being modified, the setContentLength() on the response has no effect.
Any suggestions on how to fix this?
See response.setContentLength(encryptedResponse.length()); in the below filter
#Component
public class EncryptDecryptFilter extends OncePerRequestFilter {
public static final Gson gson = new GsonBuilder()
.addSerializationExclusionStrategy(new GsonExclusionStrategy())
.setPrettyPrinting()
.create();
#Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
String requestURI = request.getRequestURI();
String requestMethod = request.getMethod();
HttpServletRequest requestToUse = new ResettableStreamHttpServletRequest(request);
BufferResponseWrapper responseToUse = new BufferResponseWrapper(response); // declared as a member of this filter class below
interceptRequest(requestToUse, request);
filterChain.doFilter(requestToUse, responseToUse);
interceptResponse(response, responseToUse);
}
private void interceptRequest(HttpServletRequest requestToUse, HttpServletRequest request) throws IOException {
byte[] payload = IOUtils.toByteArray(requestToUse.getReader(), requestToUse.getCharacterEncoding());
String urlEncodedEncryptedBody = new String(payload, requestToUse.getCharacterEncoding());
System.out.println(urlEncodedEncryptedBody);
AesDecryptionRequest decryptionRequest = gson.fromJson(urlEncodedEncryptedBody, AesDecryptionRequest.class);
decryptionRequest.initialize();
String plainTextBody = AesUtil.decrypt(decryptionRequest);
requestToUse.setAttribute("requestBody", plainTextBody);
}
private void interceptResponse(HttpServletResponse response, BufferResponseWrapper responseToUse) throws IOException {
String responseStr = new String(responseToUse.getWrapperBytes());
System.out.println(responseStr); // full response string with length 105 from #ControllerAdvice
AesEncryptionResponse encryptionResponse = AesUtil.encrypt(responseStr);
String encryptedResponse = gson.toJson(encryptionResponse); // encrypted form of #ControllerAdvice response length 200n chars
response.setContentLength(encryptedResponse.length()); // this does not seem to have any effect, the response is still truncated to 105 chars
response.getOutputStream().write(encryptedResponse.getBytes(StandardCharsets.UTF_8));
}
private final class BufferResponseWrapper extends HttpServletResponseWrapper
{
MyServletOutputStream stream = new MyServletOutputStream();
public BufferResponseWrapper(HttpServletResponse httpServletResponse)
{
super(httpServletResponse);
}
public ServletOutputStream getOutputStream() throws IOException
{
return stream;
}
public PrintWriter getWriter() throws IOException
{
return new PrintWriter(stream);
}
public byte[] getWrapperBytes()
{
return stream.getBytes();
}
}
private final class MyServletOutputStream extends ServletOutputStream
{
private ByteArrayOutputStream out = new ByteArrayOutputStream();
public void write(int b) throws IOException
{
out.write(b);
}
public byte[] getBytes()
{
return out.toByteArray();
}
#Override
public boolean isReady() {
return false;
}
#Override
public void setWriteListener(WriteListener writeListener) {
}
}
}
public class ResettableStreamHttpServletRequest extends HttpServletRequestWrapper {
// this class allows the input stream to be read more than once
private static final String UTF_8 = "UTF-8";
private byte[] rawData;
private HttpServletRequest request;
private ResettableServletInputStream servletStream;
public ResettableStreamHttpServletRequest(HttpServletRequest request) {
super(request);
this.request = request;
this.servletStream = new ResettableServletInputStream();
}
public void resetInputStream() {
servletStream.stream = new ByteArrayInputStream(rawData);
}
public void resetInputStream(byte[] newRawData) {
servletStream.stream = new ByteArrayInputStream(newRawData);
}
#Override
public ServletInputStream getInputStream() throws IOException {
if (rawData == null) {
rawData = IOUtils.toByteArray(this.request.getReader(), UTF_8);
servletStream.stream = new ByteArrayInputStream(rawData);
}
return servletStream;
}
#Override
public BufferedReader getReader() throws IOException {
if (rawData == null) {
rawData = IOUtils.toByteArray(this.request.getReader(), UTF_8);
servletStream.stream = new ByteArrayInputStream(rawData);
}
return new BufferedReader(new InputStreamReader(servletStream, UTF_8));
}
private class ResettableServletInputStream extends ServletInputStream {
private ByteArrayInputStream stream;
#Override
public int read() throws IOException {
return stream.read();
}
#Override
// Returns true when all the data from the stream has been read else it returns false.
public boolean isFinished() {
return stream.available() == 0;
}
#Override
// Returns true if data can be read without blocking else returns false.
public boolean isReady() {
return true;
}
#Override
public void setReadListener(ReadListener listener) {
throw new RuntimeException("Not implemented");
}
}
}
Controller Advice class
#ControllerAdvice
public class ExceptionAdvice extends ResponseEntityExceptionHandler {
#ExceptionHandler(value = {Exception.class})
protected ResponseEntity<Object> handleException(Exception exception, WebRequest request) {
return handleExceptionInternal(exception, exception.getMessage(), getStandardHeaders(), HttpStatus.INTERNAL_SERVER_ERROR, request);
}
}
Try using .reset() method on response object before you rewrite it.
private void interceptResponse(HttpServletResponse response, BufferResponseWrapper responseToUse) throws IOException {
String responseStr = new String(responseToUse.getWrapperBytes());
System.out.println(responseStr); // full response string with length 105 from #ControllerAdvice
AesEncryptionResponse encryptionResponse = AesUtil.encrypt(responseStr);
String encryptedResponse = gson.toJson(encryptionResponse); // encrypted form of #ControllerAdvice response length 200n chars
response.reset(); <-------------
response.setContentLength(encryptedResponse.length()); // this does not seem to have any effect, the response is still truncated to 105 chars
response.getOutputStream().write(encryptedResponse.getBytes(StandardCharsets.UTF_8));
}
According to the doc
void reset() Clears any data that exists in the buffer as well as the
status code and headers. If the response has been committed, this
method throws an IllegalStateException.
Throws: IllegalStateException
if the response has already been committed
I am trying to intercept and log all the request-responses. To make requests i am using RestTemplate.exchange().
When i make a GET request and get an 4** error i can call the ClientHttpResponse.getBody() and can access the response body but for PUT and POST requests ClientHttpResponse.getBody() method throws an exception.
What might be causing this and how can i get the response body for POST and PUT requests as well?
This is where i make the request:
apiResponse = restTemplate.exchange(url, vCloudRequest.getHttpMethod(), entity, responseType);
This is the part of the interceptor that gets the exception:
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
ClientHttpResponse response = execution.execute(request, body);
String requestString = new String(body);
String responseString = new
// Below line throws exception
String(ByteStreams.toByteArray(response.getBody()), Charset.forName("UTF-8"));
This is the stack.
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: https://176.235.57.11/api/admin/org/bd154aaf-2e7c-446d-91be-f0a45138476b/users
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
at org.springframework.http.client.SimpleClientHttpResponse.getBody(SimpleClientHttpResponse.java:85)
at org.springframework.http.client.BufferingClientHttpResponseWrapper.getBody(BufferingClientHttpResponseWrapper.java:69)
at roma.api_utils.model.Interceptors.RequestLoggingInterceptor.intercept(RequestLoggingInterceptor.java:39)
at org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:86)
at org.springframework.http.client.InterceptingClientHttpRequest.executeInternal(InterceptingClientHttpRequest.java:70)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:652)
Update :
When i call response.getStatusCode() before calling response.getBody() it doesn't throw IOException.
Basic knowledge:
HttpURLConnection has two similar fields, errorStream and inputStream. When we invoke its getInputSteam method, it checks whether the response has an error code. If so, it throws an IOException and records it- that's why you got the exception. Furthermore, it also copies the contents in inputStream to errorStream, thus we can get its response body by invoking its getErrorStream method. This is exactly what SimpleClientHttpResponse does with its getBody method:
#Override
public InputStream getBody() throws IOException {
InputStream errorStream = this.connection.getErrorStream();
this.responseStream =
(errorStream != null ? errorStream : this.connection.getInputStream());
return this.responseStream;
}
It first checks if errorStream is not null. If true, it returns it. If false, it calls connection.getInputStream() and returns that.
Now here are the answers
Why does calling response.getBody() not throw an IOException after you called response.getStatusCode()? It is because getStatusCode calls getInputStream internally. Thus, errorStream will be not null when getBody is called.
Why does it not throw an exception when the http method is GET?
See method org.springframework.http.client.SimpleBufferingClientHttpRequest#executeInternal.
.
#Override
protected ClientHttpResponse executeInternal(HttpHeaders headers, byte[] bufferedOutput)
throws IOException {
addHeaders(this.connection, headers);
// JDK <1.8 doesn't support getOutputStream with HTTP DELETE
if (HttpMethod.DELETE == getMethod() && bufferedOutput.length == 0) {
this.connection.setDoOutput(false);
}
if (this.connection.getDoOutput() && this.outputStreaming) {
this.connection.setFixedLengthStreamingMode(bufferedOutput.length);
}
this.connection.connect();
if (this.connection.getDoOutput()) {
FileCopyUtils.copy(bufferedOutput, this.connection.getOutputStream());
}
else {
// Immediately trigger the request in a no-output scenario as well
this.connection.getResponseCode();
}
return new SimpleClientHttpResponse(this.connection);
}
It eagerly executes this.connection.getResponseCode(); when the http method is GET.
I had a similar requirement of logging every request and response. I wrote a filter and hooked into the filter chain.
The code looks like something below:
public class CustomRequestFilter implements Filter {
#Override
public void init(FilterConfig filterConfig) throws ServletException {
//No custom initialisation required
}
#Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse,
FilterChain filterChain) throws IOException, ServletException {
try {
HttpServletRequest httpServletRequest = (HttpServletRequest) servletRequest;
HttpServletResponse httpServletResponse = (HttpServletResponse) servletResponse;
Map<String, String> requestMap = this
.getTypesafeRequestMap(httpServletRequest);
BufferedRequestWrapper bufferedRequest = new BufferedRequestWrapper(
httpServletRequest);
BufferedResponseWrapper bufferedResponse = new BufferedResponseWrapper(
httpServletResponse);
final StringBuilder logMessage = new StringBuilder(
"REST Request - ").append("[HTTP METHOD:")
.append(httpServletRequest.getMethod())
.append("] [PATH INFO:")
.append(httpServletRequest.getServletPath())
.append("] [REQUEST PARAMETERS:").append(requestMap)
.append("] [REQUEST BODY:")
.append(bufferedRequest.getRequestBody())
.append("] [REMOTE ADDRESS:")
.append(httpServletRequest.getRemoteAddr()).append("]");
log.info("=======================REQUEST PAYLOAD=================================");
log.info(bufferedRequest.getRequestBody());
log.info("========================================================");
filterChain.doFilter(bufferedRequest, bufferedResponse);
logMessage.append(" [RESPONSE:")
.append(bufferedResponse.getContent()).append("]");
log.info("=======================REST RESPONSE=================================");
log.info(bufferedResponse.getContent());
log.info("========================================================");
} catch (Exception a) {
log.error("Error while filtering ", a);
}
}
private Map<String, String> getTypesafeRequestMap(HttpServletRequest request) {
Map<String, String> typesafeRequestMap = new HashMap<>();
Enumeration<?> requestParamNames = request.getParameterNames();
while (requestParamNames.hasMoreElements()) {
String requestParamName = (String) requestParamNames.nextElement();
String requestParamValue;
if ("password".equalsIgnoreCase(requestParamName)) {
requestParamValue = "********";
} else {
requestParamValue = request.getParameter(requestParamName);
}
typesafeRequestMap.put(requestParamName, requestParamValue);
}
return typesafeRequestMap;
}
#Override
public void destroy() {
//not yet implemented
}
private static final class BufferedRequestWrapper extends
HttpServletRequestWrapper {
private ByteArrayInputStream bais = null;
private ByteArrayOutputStream baos = null;
private BufferedServletInputStream bsis = null;
private byte[] buffer = null;
public BufferedRequestWrapper(HttpServletRequest req)
throws IOException {
super(req);
// Read InputStream and store its content in a buffer.
InputStream is = req.getInputStream();
this.baos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int read;
while ((read = is.read(buf)) > 0) {
this.baos.write(buf, 0, read);
}
this.buffer = this.baos.toByteArray();
}
#Override
public ServletInputStream getInputStream() {
this.bais = new ByteArrayInputStream(this.buffer);
this.bsis = new BufferedServletInputStream(this.bais);
return this.bsis;
}
String getRequestBody() throws IOException {
BufferedReader reader = new BufferedReader(new InputStreamReader(
this.getInputStream()));
String line;
StringBuilder inputBuffer = new StringBuilder();
do {
line = reader.readLine();
if (null != line) {
inputBuffer.append(line.trim());
}
} while (line != null);
reader.close();
return inputBuffer.toString().trim();
}
}
private static final class BufferedServletInputStream extends
ServletInputStream {
private ByteArrayInputStream bais;
public BufferedServletInputStream(ByteArrayInputStream bais) {
this.bais = bais;
}
#Override
public int available() {
return this.bais.available();
}
#Override
public int read() {
return this.bais.read();
}
#Override
public int read(byte[] buf, int off, int len) {
return this.bais.read(buf, off, len);
}
#Override
public boolean isFinished() {
return false;
}
#Override
public boolean isReady() {
return true;
}
#Override
public void setReadListener(ReadListener readListener) {
//No specific readListener changes required
}
}
public class TeeServletOutputStream extends ServletOutputStream {
private final TeeOutputStream targetStream;
public TeeServletOutputStream(OutputStream one, OutputStream two) {
targetStream = new TeeOutputStream(one, two);
}
#Override
public void write(int arg0) throws IOException {
this.targetStream.write(arg0);
}
#Override
public void flush() throws IOException {
super.flush();
this.targetStream.flush();
}
#Override
public void close() throws IOException {
super.close();
this.targetStream.close();
}
#Override
public boolean isReady() {
return false;
}
#Override
public void setWriteListener(WriteListener writeListener) {
//not yet implemented
}
}
public class BufferedResponseWrapper implements HttpServletResponse {
HttpServletResponse original;
TeeServletOutputStream tee;
ByteArrayOutputStream bos;
public BufferedResponseWrapper(HttpServletResponse response) {
original = response;
}
public String getContent() {
return bos.toString();
}
#Override
public PrintWriter getWriter() throws IOException {
return original.getWriter();
}
#Override
public ServletOutputStream getOutputStream() throws IOException {
if (tee == null) {
bos = new ByteArrayOutputStream();
tee = new TeeServletOutputStream(original.getOutputStream(),
bos);
}
return tee;
}
#Override
public String getCharacterEncoding() {
return original.getCharacterEncoding();
}
#Override
public String getContentType() {
return original.getContentType();
}
#Override
public void setCharacterEncoding(String charset) {
original.setCharacterEncoding(charset);
}
#Override
public void setContentLength(int len) {
original.setContentLength(len);
}
#Override
public void setContentLengthLong(long l) {
original.setContentLengthLong(l);
}
#Override
public void setContentType(String type) {
original.setContentType(type);
}
#Override
public void setBufferSize(int size) {
original.setBufferSize(size);
}
#Override
public int getBufferSize() {
return original.getBufferSize();
}
#Override
public void flushBuffer() throws IOException {
tee.flush();
}
#Override
public void resetBuffer() {
original.resetBuffer();
}
#Override
public boolean isCommitted() {
return original.isCommitted();
}
#Override
public void reset() {
original.reset();
}
#Override
public void setLocale(Locale loc) {
original.setLocale(loc);
}
#Override
public Locale getLocale() {
return original.getLocale();
}
#Override
public void addCookie(Cookie cookie) {
original.addCookie(cookie);
}
#Override
public boolean containsHeader(String name) {
return original.containsHeader(name);
}
#Override
public String encodeURL(String url) {
return original.encodeURL(url);
}
#Override
public String encodeRedirectURL(String url) {
return original.encodeRedirectURL(url);
}
#SuppressWarnings("deprecation")
#Override
public String encodeUrl(String url) {
return original.encodeUrl(url);
}
#SuppressWarnings("deprecation")
#Override
public String encodeRedirectUrl(String url) {
return original.encodeRedirectUrl(url);
}
#Override
public void sendError(int sc, String msg) throws IOException {
original.sendError(sc, msg);
}
#Override
public void sendError(int sc) throws IOException {
original.sendError(sc);
}
#Override
public void sendRedirect(String location) throws IOException {
original.sendRedirect(location);
}
#Override
public void setDateHeader(String name, long date) {
original.setDateHeader(name, date);
}
#Override
public void addDateHeader(String name, long date) {
original.addDateHeader(name, date);
}
#Override
public void setHeader(String name, String value) {
original.setHeader(name, value);
}
#Override
public void addHeader(String name, String value) {
original.addHeader(name, value);
}
#Override
public void setIntHeader(String name, int value) {
original.setIntHeader(name, value);
}
#Override
public void addIntHeader(String name, int value) {
original.addIntHeader(name, value);
}
#Override
public void setStatus(int sc) {
original.setStatus(sc);
}
#SuppressWarnings("deprecation")
#Override
public void setStatus(int sc, String sm) {
original.setStatus(sc, sm);
}
#Override
public String getHeader(String arg0) {
return original.getHeader(arg0);
}
#Override
public Collection<String> getHeaderNames() {
return original.getHeaderNames();
}
#Override
public Collection<String> getHeaders(String arg0) {
return original.getHeaders(arg0);
}
#Override
public int getStatus() {
return original.getStatus();
}
}
}
For PUT and POST, it depends on your target resource. If your target resource do not add anything in the response's body after a PUT or POST request, it's normal to get an exception. In general, you know the resource that you send with PUT or POST so you can just check the response's status to know if your resource had been created or modified. You do not need to check the response body again.
You can use the following to accesss the response body in the interceptor. I did a quick unit test to confirm it works even on a POST with a 403 response.
However be careful, getBody returns an InputStream. Which means you can only read it once. You won't be able to read the same stream again outside the interceptor unless you provide a new response with a new body.
...
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
final ClientHttpResponse response = execution.execute(request, body);
final InputStream body = response.getBody();
return response;
}
...
I am trying to get the progress of the actual file upload using HttpPost. I have a stable solution so far (that i found here in SO) that has a progress but after uploading large files i realized it only counts the bytes written to the output buffer NOT the post transfer progress. I would like to somehow get the progress of the actual "post". Can someone explain how I can achieve this using what I worked so hard to get so far? Most of the solutions I have found online are only counting bytes written to output buffer (which is good enough for small files but not when transferring large files).
public static String postFile(final Context context, String fileName) throws Exception {
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("http://my.url/");
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
final File file = new File(fileName);
final long totalSize = file.length();
FileBody fb = new FileBody(file);
builder.addPart("uploaded_file", new FileBody(new File(fileName)));
final HttpEntity yourEntity = builder.build();
int progressPercent = 0;
class ProgressiveEntity implements HttpEntity {
#Override
public void consumeContent() throws IOException {
yourEntity.consumeContent();
}
#Override
public InputStream getContent() throws IOException,
IllegalStateException {
return yourEntity.getContent();
}
#Override
public Header getContentEncoding() {
return yourEntity.getContentEncoding();
}
#Override
public long getContentLength() {
return yourEntity.getContentLength();
}
#Override
public Header getContentType() {
return yourEntity.getContentType();
}
#Override
public boolean isChunked() {
return yourEntity.isChunked();
}
#Override
public boolean isRepeatable() {
return yourEntity.isRepeatable();
}
#Override
public boolean isStreaming() {
return yourEntity.isStreaming();
} // CONSIDER put a _real_ delegator into here!
#Override
public void writeTo(OutputStream outstream) throws IOException {
class ProxyOutputStream extends FilterOutputStream {
public ProxyOutputStream(OutputStream proxy) {
super(proxy);
}
public void write(int idx) throws IOException {
out.write(idx);
}
public void write(byte[] bts) throws IOException {
out.write(bts);
}
public void write(byte[] bts, int st, int end) throws IOException {
out.write(bts, st, end);
}
public void flush() throws IOException {
out.flush();
}
public void close() throws IOException {
out.close();
}
} // CONSIDER import this class (and risk more Jar File Hell)
class ProgressiveOutputStream extends ProxyOutputStream {
long totalSent;
public ProgressiveOutputStream(OutputStream proxy) {
super(proxy);
totalSent = 0;
}
public void write(byte[] bts, int st, int end) throws IOException {
// end is the amount being sent this time
// st is always zero and end=bts.length()
totalSent += end;
int progress = (int) ((totalSent / (float) totalSize) * 100);
out.write(bts, st, end);
}
}
yourEntity.writeTo(new ProgressiveOutputStream(outstream));
}
};
ProgressiveEntity myEntity = new ProgressiveEntity();
post.setEntity(myEntity);
//Output to buffer is complete at this point!
HttpResponse response = client.execute(post);
String jsonResponseStr = getContent(response);
Log.d("MYTAG",jsonResponseStr);
return jsonResponseStr;
}
In my receiving script on the remote server, I am only echo-ing a string so that I can send an immediate response (no file/database processing at all) and the response from server still takes a very long time. I strongly believe at this point the transfer happens after write to buffer completes.
class ProgressiveOutputStream extends ProxyOutputStream {
long totalSent;
public ProgressiveOutputStream(OutputStream proxy) {
super(proxy);
totalSent = 0;
}
public void write(byte[] bts, int st, int end) throws IOException {
// FIXME Put your progress bar stuff here!
// end is the amount being sent this time
// st is always zero and end=bts.length()
totalSent += end;
progress.publish((int) ((totalSent / (float) totalSize) * 100));
out.write(bts, st, end);
}
Since I don't see any solution for this, I suppose the answer is to use a spinning animation without a progress percent. Since nothing can be done until the transmission is complete anyway. Oh well... atleast it solved my problem.
I know this is quite old, but I've just found the solution.
If totalSize is your entity content length, then you can base your progress on that:
class ProgressiveOutputStream extends ProxyOutputStream {
long totalSent;
long totalSize;
public ProgressiveOutputStream(OutputStream proxy, long total) {
super(proxy);
totalSent = 0;
totalSize = total;
}
public void write(byte[] bts, int st, int end) throws IOException {
totalSent += end;
publishProgress((int) ((totalSent / (float) totalSize) * 100));
out.write(bts, st, end);
}
}
yourEntity.writeTo(new ProgressiveOutputStream(outstream, yourEntity.getContentLength()));
You update your progressbar in OnProgressUpdate of your asyncTask (pb is the progressBar):
#Override
protected void onProgressUpdate(Integer... values) {
super.onProgressUpdate(values);
pb.setProgress(values[0]);
}
maybe you could flush the data every time you call write:
totalSent += end;
int progress = (int) ((totalSent / (float) totalSize) * 100);
out.write(bts, st, end);
out.flush(); //flush
Edit:
you can also try this since i get the feeling end represents the end index in the outputstream named out:
totalSent += (end-st);
Please try next solution instead of using ProxyOutputStream in your writeTo method:
#Override
public void writeTo(OutputStream outstream) throws IOException {
ByteArrayInputStream reader = new ByteArrayInputStream(mImageData);
byte[] fileBuffer = new byte[2048];
int bytesRead;
while ((bytesRead = reader.read(fileBuffer)) != -1) {
outstream.write(fileBuffer, 0, bytesRead);
int progress = bytesRead;
}
reader.close();
yourEntity.writeTo(outstream);
}