Develop Reference

Wiremock withBodyFile OutOfMemoryError for large files

In the following stub request, I use withBodyFile which loads large files (>300MB) from specific locations on local disk using streams:
public void mockGetUrlContent(String url) {
stubFor(get(urlEqualTo(url))
.willReturn(ok()
.withBodyFile(FilenameUtils.getName(url))));
}
Once the stub is being called, wiremock tries to complete a served event and in order to log the response it uses the from method below:
public static LoggedResponse from(Response response) {
return new LoggedResponse(
response.getStatus(),
response.getHeaders() == null || response.getHeaders().all().isEmpty()
? null
: response.getHeaders(),
response.getBody(),
response.getFault());
}
When reached to wiremock's Response getBody method it converts the stream in to byte[] and in that points it blows on OutOfMemoryError (Java heap space).:
public byte[] getBody() {
try (InputStream stream = bodyStreamSource == null ? null : getBodyStream()) {
return stream == null ? null : ByteStreams.toByteArray(stream);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Any ideas about what am I doing wrong?
Your help is highly appreciated!

WireMock 2.34.0 has a solution to this.
To ensure that logged response bodies are limited to a particular size you need to start WireMock with a new configuration parameter specifying the max size in bytes
When running standalone add the CLI parameter:
--logged-response-body-size-limit=1000000
When running in Java add the following to the configuration builder:
wireMockConfig().maxLoggedResponseSize(1000000)

Apache Wicket: File download that would not lock the page

I want to create a link that would initiate a file download which would be asynchronous to the page itself, i.e. I want the page not to be locked during the file download. Should I make it be initiated outside wicket? Or is there something inside wicket that would let me set up a resource stream which would bypass the page locks?
Things I tried:
DownloadLink - locks the page, as stated in its doc. This was my starting point.
ResourceLink - did not state the locking explicitly in the doc, so I tried this, but it also locked the page.
At this point I've investigated the code of both links a bit and noticed they both schedule the download via ResourceStreamRequestHandler. Expecting that his kind of behavior could be just handler-specific I've attempted to schedule a custom handler I've written:
private void sendFile(final File file) throws IOException {
IRequestHandler fileDownloadHandler = new IRequestHandler() {
#Override
public void respond(IRequestCycle requestCycle) {
WebResponse response = (WebResponse) requestCycle.getResponse();
OutputStream outStream = response.getOutputStream();
response.setContentType("audio/x-wav");
response.setContentLength((int)file.length());
String fileName = "Somethingsomething.wav";
// sets HTTP header
response.setHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
byte[] byteBuffer = new byte[1024];
DataInputStream in = null;
try {
in = new DataInputStream(new FileInputStream(file));
int length = 0;
// reads the file's bytes and writes them to the response stream
while ((in != null) && ((length = in.read(byteBuffer)) != -1))
{
outStream.write(byteBuffer,0,length);
}
in.close();
outStream.close();
} catch (IOException e) {
throw new PortalError("IOException trying to write the response", e);
}
}
#Override
public void detach(IRequestCycle requestCycle) {
}
};
getRequestCycle().scheduleRequestHandlerAfterCurrent(fileDownloadHandler);
}
This did not quite work either, so I've investigated further. I've noticed that unlike I expected, the "scheduled" request handlers would not get executed on a separate request, as I expected, but on the same one. I figured that it must be that the page gets locked for the first handler and then remains locked while the second one is executing as well. So I've attempted to force the download handler into a separate request (via an ajax behaviour):
public void startDownload(AjaxRequestTarget target) throws DownloadTargetNotFoundException{
target.appendJavaScript("setTimeout(\"window.location.href='" + getCallbackUrl() + "'\", 100);");
}
#Override
public void onRequest() {
sendFile(getFile());
logger.debug("Download initiated");
}
I've found this here and hoped it could potentially be what I've been looking for. However, unsurprisingly so, the page gets locked still (I would imagine because the behaviour still has to be retrieved from the page, for which the page lock has to be acquired).
I'm at a loss where I should be looking next, especially after all this time trying to get a simple download link working. I was considering creating another web filter one layer above wicket, which could be signaled from within wicket to create the download after the wicket filter is finished with its work (and hence the page lock is already released), but that seems a bit excessive for a task like this.
Any suggestions are welcome.

You have to download from a resource, see
http://wicketinaction.com/2012/11/uploading-files-to-wicket-iresource/ and read http://wicket.apache.org/guide/guide/resources.html

Working with Servlets?

A normal java file executes only those methods called in the main. But what does a servlet do? I thought maybe it executes down the file but I just tried to split up a HTTP servlets request and response but it doesn't work. What is a neat way of working with a servlet to read an input stream, then execute some functions which can call the response writer method?
Here was my tryout code:
public void requestReader(HttpServletRequest req) {
try {
int len = req.getContentLength();
byte[] input = new byte[len];
ServletInputStream sin = req.getInputStream();
int c, count = 0 ;
while ((c = sin.read(input, count, input.length-count)) != -1) {
count +=c;
}
sin.close();
String inString = new String(input);
int index = inString.indexOf("=");
String value = inString.substring(index + 1);
inputStream = URLDecoder.decode(value, "UTF-8");
} catch (IOException e) {
}
}//end of requestReader
public void responseWriter(HttpServletResponse resp) {
try{
resp.setStatus(HttpServletResponse.SC_OK);
OutputStreamWriter writer = new OutputStreamWriter(resp.getOutputStream());
writer.write("Working");
writer.flush();
writer.close();
} catch (IOException e) {
}
}//end of responseWriter

A normal java file executes only those methods called in the main.
A "Java file" does not execute anything. The JVM, started by the java command line tool, executes the main method.
But what does a servlet do? I thought maybe it executes down the file
A servlet runs inside a Servlet Container, e.g. Apache Tomcat, or Jetty. The container itself is started via a main method, it learns about servlet classes in its web.xml file, and it calls the service() method of the servlet when a request arrives.
What is a neat way of working with a servlet to read an input stream, then execute some functions which can call the response writer method?
That's exactly what a servlet does (though the headers of a HTTP request are parsed by the servlet container and passed to the servlet as part of the request object). What are you really trying to do?
If you're not running a servlet container, it makes no sense to use servlets.

No Java class simply executes methods sequentially.
If you want to use multiple methods, you must break up your functionality and explicitly call the other methods you want to use, just like you do in a standalone program.
Servlet execution starts with a request handler, generally doPost or doGet.
Here's a tutorial to read through, but there are countless others--just search for "servlet tutorial".

Methods are never executed sequentially. They are executed WHEN called.
The function of a servlet is to service HTTP requests from a web browser. For this, your servlet must implement a service() method and then call your methods from inside it.
You could also implement doPost() or doGet() but they only respond to the POST and GET commands.
You can start by looking here

Servlets are typically used in the context of a web server. This is in contrast with a "normal Java file" which is typically executed from the command line and starts with the main method.
In order to get your servlet to work, you need to make it part of a Java web application. If you've never done this before, it would be best to follow a tutorial about Java web applications.
Briefly, the doPost, doGet, or service methods of HttpServlet are roughly equivalent to the main method (though with some important differences such as the main method is entered once while service can be entered many times).

When a GET HTTP request comes in, and the servlet is mapped to the path of the request, then the servlet doGet method is called. If it's a POST, then doPost is called. You need to override one (or both) of these methods to do what you want with the request and the response.
Read the javadoc of HttpServlet to learn more.

HttpServlet already define methods you can implement for different type of http request:
doGet()
doPost()
doDelete()
doPut()
etc...
All these methods have a request and response parameter.
You can read requested data, call any service you want, then write the response using the httpServletResponse from the parameter.
You don't have to define your own methods like you did in your question.

To anyone interested, here is an update code that does work by passing the ServletResponse to another class:
public void doPost(HttpServletRequest req, HttpServletResponse resp) {
try {
int len = req.getContentLength();
byte[] input = new byte[len];
ServletInputStream sin = req.getInputStream();
int c, count = 0 ;
while ((c = sin.read(input, count, input.length-count)) != -1) {
count +=c;
}
sin.close();
String inString = new String(input);
int index = inString.indexOf("=");
String value = inString.substring(index + 1);
inputStream = URLDecoder.decode(value, "UTF-8");
} catch (IOException e) {
}
responseWriter(resp);
}//end of requestReader
public void responseWriter(HttpServletResponse resp) {
try{
resp.setStatus(HttpServletResponse.SC_OK);
OutputStreamWriter writer = new OutputStreamWriter(resp.getOutputStream());
writer.write("Working");
writer.flush();
writer.close();
} catch (IOException e) {
}
}//end of responseWriter
This was all I was trying to achieve when I asked the question.

Relative paths in Flying Saucer XHTML?

I am using Flying Saucer to render some PDF documents from strings to XHTML. My code is something like:
iTextRenderer.setDocument(documentGenerator.generate(xhtmlDocumentAsString));
iTextRenderer.layout();
iTextRenderer.createPDF(outputStream);
What I'm trying to understand is, when using this method, where are relative paths in the XHTML resolved from? For example, for images or stylesheets. I am able to use this method to successfully generate a text-based document, but I need to understand how to reference my images and CSS.

The setDocument() method takes two parameters: document and url.
The url parameter indicates the base url used to prepend to relative paths that appear in the xhtml, such as in img tags.
Suppose you have:
<img src="images/img1.jpg">
Now suppose the folder "images" is located at:
C:/physical/route/to/app/images/
You may use setDocument() as:
renderer.setDocument(xhtmlDoc, "file:///C:/physical/route/to/app/");
Notice the trailing slash, it won't work without it.
This is the way it worked for me. I assume you could use other types of urls such as "http://...".

This week I worked on this, and I give you what worked fine for me.
In real life, your XHTML document points to multiple resources (images, css, etc.) with relative paths.
You also have to explain to Flying Saucer where to find them. They can be in your classpath, or in your file system. (If they are on the network, you can just set the base url, so this won't help)
So you have to extend the ITextUserAgent like this:
private static class ResourceLoaderUserAgent extends ITextUserAgent {
public ResourceLoaderUserAgent(ITextOutputDevice outputDevice) {
super(outputDevice);
}
protected InputStream resolveAndOpenStream(String uri) {
InputStream is = super.resolveAndOpenStream(uri);
String fileName = "";
try {
String[] split = uri.split("/");
fileName = split[split.length - 1];
} catch (Exception e) {
return null;
}
if (is == null) {
// Resource is on the classpath
try{
is = ResourceLoaderUserAgent.class.getResourceAsStream("/etc/images/" + fileName);
} catch (Exception e) {
}
if (is == null) {
// Resource is in the file system
try {
is = new FileInputStream(new File("C:\\images\\" + fileName));
} catch (Exception e) {
}
}
return is;
}
}
And you use it like this:
// Output stream containing the result
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ITextRenderer renderer = new ITextRenderer();
ResourceLoaderUserAgent callback = new ResourceLoaderUserAgent(renderer.getOutputDevice());
callback.setSharedContext(renderer.getSharedContext());
renderer.getSharedContext().setUserAgentCallback(callback);
renderer.setDocumentFromString(htmlSourceAsString);
renderer.layout();
renderer.createPDF(baos);
renderer.finishPDF();
Cheers.

The best solution for me was:
renderer.setDocumentFromString(htmlContent, new ClassPathResource("/META-INF/pdfTemplates/").getURL().toExternalForm());
Then all the provided styles and images in html (like
<img class="logo" src="images/logo.png" />
<link rel="stylesheet" type="text/css" media="all" href="css/style.css"></link>
) were rendered as expected.

AtilaUy's answer is spot-on for the default way things work in Flying Saucer.
The more general answer is that it asks the UserAgentContext. It will call setBaseURL() on the UserAgentContext when the document is set in. Then it will call resolveURL() to resolve relative URLs and ultimately resolveAndOpenStream() when it wants to read the actual resource data.
Well, this answer is probably way too late for you to make use of it anyway, but I needed an answer like this when I set out, and setting a custom user agent context is the solution I ended up using.

You can either have file paths, which should be absolute, or http:// urls. Relative paths can work but aren't portable because it depends on what directory you ran your program from

I think a easier approach would be:
DomNodeList<DomElement> images = result.getElementsByTagName("img");
for (DomElement e : images) {
e.setAttribute("src", result.getFullyQualifiedUrl(e.getAttribute("src")).toString());
}

Another way to resolve paths is to override UserAgentCallback#resolveURI, which offers a more dynamic behavior than a fixed URL (as in AtilaUy's answer, which looks quite valid for most cases).
This is how I make an XHTMLPane use dynamically-generated stylesheets:
public static UserAgentCallback interceptCssResourceLoading(
final UserAgentCallback defaultAgentCallback,
final Map< URI, CSSResource > cssResources
) {
return new UserAgentCallback() {
#Override
public CSSResource getCSSResource( final String uriAsString ) {
final URI uri = uriQuiet( uriAsString ) ; // Just rethrow unchecked exception.
final CSSResource cssResource = cssResources.get( uri ) ;
if( cssResource == null ) {
return defaultAgentCallback.getCSSResource( uriAsString ) ;
} else {
return cssResource ;
}
}
#Override
public String resolveURI( final String uriAsString ) {
final URI uri = uriQuiet( uriAsString ) ;
if( cssResources.containsKey( uri ) ) {
return uriAsString ;
} else {
return defaultAgentCallback.resolveURI( uriAsString ) ;
}
}
// Delegate all other methods to defaultUserAgentCallback.
} ;
}
Then I use it like that:
final UserAgentCallback defaultAgentCallback =
xhtmlPanel.getSharedContext().getUserAgentCallback() ;
xhtmlPanel.getSharedContext().setUserAgentCallback(
interceptCssResourceLoading( defaultAgentCallback, cssResources ) ) ;
xhtmlPanel.setDocumentFromString( xhtml, null, new XhtmlNamespaceHandler() ) ;

How to clear the screen output of a Java HttpServletResponse

I'm writing to the browser window using servletResponse.getWriter().write(String).
But how do I clear the text which was written previously by some other similar write call?

The short answer is, you cannot -- once the browser receives the response, there is no way to take it back. (Unless there is some way to abnormally stop a HTTP response to cause the client to reload the page, or something to that extent.)
Probably the last place a response can be "cleared" in a sense, is using the ServletResponse.reset method, which according to the Servlet Specification, will reset the buffer of the servlet's response.
However, this method also seems to have a catch, as it will only work if the buffer has not been committed (i.e. sent to the client) by the ServletOutputStream's flush method.

You cannot. The best thing is to write to a buffer (StringWriter / StringBuilder) and then you can replace the written data any time. Only when you know for sure what is the response you can write the buffer's content to the response.
In the same matter, and reason to write the response this way and not to use some view technology for your output such as JSP, Velocity, FreeMarker, etc.?

If you have an immediate problem that you need to solve quickly, you could work around this design problem by increasing the size of the response buffer - you'll have to read your application server's docs to see if this is possible. However, this solution will not scale as you'll soon run into out-of-memory issues if you site traffic peaks.
No view technology will protect you from this issue. You should design your application to figure out what you're going to show the user before you start writing the response. That means doing all your DB access and business logic ahead of time. This is a common issue I've seen with convoluted system designs that use proxy objects that lazily access the database. E.g. ORM with Entity relationships are bad news if accessed from your view layer! There's not much you can do about an exception that happens 3/4 of the way into a rendered page.
Thinking about it, there might be some way to inject a page redirect via AJAX. Anyone ever heard of a solution like that?
Good luck with re-architecting your design!

I know the post is pretty old, but just thought of sharing my views on this.
I suppose you could actually use a Filter and a ServletResponseWrapper to wrap the response and pass it along the chain.
That is, You can have an output stream in the wrapper class and write to it instead of writing into the original response's output stream... you can clear the wrapper's output stream as and when you please and you can finally write to the original response's output stream when you are done with your processing.
For example,
public class MyResponseWrapper extends HttpServletResponseWrapper {
protected ByteArrayOutputStream baos = null;
protected ServletOutputStream stream = null;
protected PrintWriter writer = null;
protected HttpServletResponse origResponse = null;
public MyResponseWrapper( HttpServletResponse response ) {
super( response );
origResponse = response;
}
public ServletOutputStream getOutputStream()
throws IOException {
if( writer != null ) {
throw new IllegalStateException( "getWriter() has already been " +
"called for this response" );
}
if( stream == null ) {
baos = new ByteArrayOutputStream();
stream = new MyServletStream(baos);
}
return stream;
}
public PrintWriter getWriter()
throws IOException {
if( writer != null ) {
return writer;
}
if( stream != null ) {
throw new IllegalStateException( "getOutputStream() has already " +
"been called for this response" );
}
baos = new ByteArrayOutputStream();
stream = new MyServletStream(baos);
writer = new PrintWriter( stream );
return writer;
}
public void commitToResponse() {
origResponse.getOutputStream().write(baos.toByteArray());
origResponse.flush();
}
private static class MyServletStream extends ServletOutputStream {
ByteArrayOutputStream baos;
MyServletStream(ByteArrayOutputStream baos) {
this.baos = baos;
}
public void write(int param) throws IOException {
baos.write(param);
}
}
//other methods you want to implement
}