Develop Reference

Thread problem with input stream from servlet request

I have brief understanding on multi threading in Java web apps. However, I came across an issue while developing our android app which communicates with the server via REST.
Our web application is based on Apache Wicket 8.6 and contains additional HttpServlet endpoints. One endpoint is applied while uploading images from the app via post. This works fine as long as I am uploading only one image at a time. If I perform multiple upload requests in my android app in a quick succession (some milliseconds), only the last upload is performed successfully (it is working fine when I put a second break between the uploads). It seems, as if all requests except the last one are missing the image content read from the input stream of the servlet request. Thus, I am guessing, that I am having a threading problem with my servlet. I would appreciate, if someone could guide me in the right direction to solve this issue.
public class MyServlet extends HttpServlet{
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
boolean proceed =true;
Map<String,String[]> parameters = req.getParameterMap();
if(parameters!=null){
//read some parameters which specify the request
}
if(proceed) {
InputStream is = req.getInputStream();
if(is!=null) {
//The result of this output is 0 for the first requests only for the last request it is 1
System.err.println("Stream size: "+is.available());
//do something
}
}
//do something....
}
}
I could, of course, write the images in my android app in one request using multipart but still I'd like to make the servlet thread safe for the occasion that it gets two requests at the same time.
I appreciate your help.

So, after some additional research I discovered, that the input stream is not empty even is.available() returns 0
My problem was something different. I save the uploaded image in a ModeShape repository. The repository session is stored in the wicket application instance. THus, only one ModeSHape repository session exists. When writing the image, it seems as if there was a problem with the ModeShape session. Thus, I put the ModeShape session in a synchronized block and now everything runs fine.
public class MyServlet extends HttpServlet{
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
boolean proceed =true;
Map<String,String[]> parameters = req.getParameterMap();
if(parameters!=null){
//read some parameters which specify the request
}
if(proceed) {
String fileName = req.getHeader("fileName");
if(!StringUtils.isEmpty(fileName)) {
InputStream is = req.getInputStream();
if(is!=null) {
//The result of this output is 0 for the first requests only for the last request it is 1
System.err.println("Stream size: "+is.available());
WicketApplication app=null;
try {
app = (WicketApplication)Application.get("wicket");
} catch (Exception e) {
LOG.error("Error while generating WicketApplication object for project "+project.getName(),e);
}
if(app!=null) {
final Session repoSession = app.getRepoSession();
synchronized (repoSession) {
//write image to repository
}
}
}
}
}
//do something....
}
}

Why can't I execute different Matlab functions from Java concurrently?

I have two Java Servlets: DataFetcherServlet and UploaderServlet. Both servlets call 2 different Java methods which in turn call their corresponding Matlab functions through JNI, and each of which was compiled into a separate Java jar file to use as a library. The application is powered by AJAX to create a Desktop like feel. For the UploaderServlet, the users can upload an excel file to this servlet, the parsed data then get passed to a Java method which then calls the compiled Matlab function to generate and save alot of images (currently over 5000 images), because this will take alot of time, I use an ExecutorService to execute it in the background. But new requests sent the DataFetcherServlet which will also call another compiled Matlab function is blocked until the image generation part is completed. I don't know why it is blocking new requests even though the requests are sent to a different servlet.
DataFetcherServlet.java
public class DataFetcherServlet extends HttpServlet {
#Inject
private CdfReader reader; // An EJB to get a data array from Matlab
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
try {
String filePath = "path/to/file";
Object[] result = reader.read(filePath); // reader.read() is just a wrapper around the method in the jar file mentioned above that actually calls the matlab function to return an array of number
MWNumericArray array = (MWNumericArray)result[0] // This will block while the other Matlab function is generating the images.
.
.
.
} catch (MWException ex) {
Logger.getLogger(DataFetcherServlet.class.getName()).log(Level.SEVERE, null, ex);
}
}
UploaderServlet.java
public class UploaderServlet extends HttpServlet {
#Inject
private ExcelIonImageGenerator generator; // An EJB to call Matlab to generate the images
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
try {
String dir = "path/to/parent/directory";
Path excel = Paths.get(dir+ "excel", part.getSubmittedFileName()); // Path to where the uploaded excel file is stored
if (!Files.exists(excel))
Files.copy(part.getInputStream(), excel);
// ExcelExtractor is a helper class to parse the excel file.
Double[][] ranges = ExcelExtractor.extractSheet(WorkbookFactory.create(excel.toFile()));
// This will call a Java library method which in turns call the Matlab function
// to generate the images (over 5000 in this case)
// See the code for this method below.
generator.generate(dir+ "images" + File.separator, ranges);
} catch (MWException | InvalidFormatException ex) {
Logger.getLogger(UploaderServlet.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
ExcelIonImageGenerator.java
import com.mathworks.toolbox.javabuilder.*; // Matlab SDK needed to integrate with Java
import java.util.concurrent.*;
import java.util.logging.*;
import javax.annotation.PreDestroy;
import javax.ejb.Stateless;
import save_ion_image_for_all_ranges_in_spreadsheet.Class1; // The jar file which contains code to call Matlab code through JNI
#Stateless
public class ExcelIonImageGenerator {
private final Class1 clazz1;
private ExecutorService pool;
public ExcelIonImageGenerator() throws MWException {
clazz1 = new Class1();
pool = Executors.newFixedThreadPool(1);
}
public void generate(String path, Double[][] ranges) throws MWException {
// Submit this task to the ExecutorService so it can be processed
// in a different thread than the caller thread
pool.submit(() -> generateHelper(path, ranges, clazz1), 1);
}
private void generateHelper(String path, Double[][] ranges, Class1 clazz) {
try {
// This method was generated by Matlab tool, it calls the native
// Matlab code through JNI, and it will block any request that will call
// other Matlab functions until it finishes.
clazz.save_ion_image_for_all_ranges_in_spreadsheet(path, ranges);
} catch (MWException ex) {
Logger.getLogger(ExcelIonImageGenerator.class.getName()).log(Level.SEVERE, null, ex);
}
}
}

You have three options:
Launch multiple processes of your Java application that makes calls to Matlab. Calls from a single process use the same MCR which has a process-wide lock, however calls from different processes would run on separate MCR computational engines.
Use Matlab Production Server, which basically facilitates the use of multiple MCRs. This is a toolkit that requires a separate license and installation.
You don't necessarily have to limit yourself to running MCR / compiled code unless you have very specific performance concerns. You can actually install Matlab itself on the server, launch multiple instances (headless, etc) from the same Java process, and communicate with them e.g. through MatlabControl or the new official MATLAB Engine API for Java.
There's a very good answer from MathWorks Support Team on MatlabCentral, explaining these limitations of MCR in detail.

Creating HttpURLConnection for URLStreamHandlerFactory, getting Error: protocol doesn't support input

A bit of background, I'm trying to create a URL Stream Handler so I can keep track of how many connections I have active on my webview in my javafx application. Essentially, I'm running an AngularJs app in the WebView, and I'd like to know when it's finished. I can't touch the web site code, so adding a js notifier is not on the table. So, no matter what I put together, the setup always errors with 'protocol doesn't support input.' I've tried to override 'getDoInput' with a method that only returns false, but I still get the error. Any ideas?
Here is something close to what I'm doing:
public class MyUrlStreamHandlerFactory implements URLStreamHandlerFactory {
public URLStreamHandler createURLStreamHandler(String protocol) {
if (protocol.equalsIgnoreCase("http") || protocol.equalsIgnoreCase("https")) {
return new URLStreamHandler() {
#Override
protected URLConnection openConnection(URL url) throws IOException {
return new HttpURLConnection(url) {
#Override
public void connect() throws IOException {
}
#Override
public void disconnect() {
}
#Override
public boolean usingProxy() {
return false;
}
#Override
public boolean getDoInput() {
return false;
}
};
}
};
}
return null;
}
}
I'm installing it with:
URL.setURLStreamHandlerFactory(new MyUrlStreamHandlerFactory());

I understand what you're trying to accomplish however, I think this is the wrong way to go about it.
From: Java Network Programming by Elliotte Rusty Harold
Only abstract URLConnection classes are present in the java.net package. The concrete subclasses are hidden inside the sun.net package hierarchy. It is rare to instantiate URLConnection objects directly in your source code; instead, the runtime environment creates these objects as needed, depending on the protocol in use. The class (which is unknown at compile time) is then instantiated using the forName() and newInstance() methods of the java.lang.Class class.
For example, the connect() method of sun.net.www.protocol.http.HttpURLConnection creates a sun.net.www.http.HttpClient object, which is responsible for connecting to the server.
So unless you want to write your own http protocol handler and an HttpClient, I would suggest exploring other avenues.
Other Things
The only method, that I could find, that throws an UnknownServiceException with the message being "protocol doesn't support input" is:
java.net.URLConnection#getInputStream
/**
* Returns an input stream that reads from this open connection.
*
* #return an input stream that reads from this open connection.
* #exception IOException if an I/O error occurs while
* creating the input stream.
* #exception UnknownServiceException if the protocol does not support
* input.
*/
public InputStream getInputStream() throws IOException {
throw new UnknownServiceException("protocol doesn't support input");
}
Overriding getDoInput
You should not override getDoInput to only return false. Instead you should use setDoInput(false). However, you don't want to set doInput to false. You always want to read something, for instance the response code.

Calling application.getRealPath() from within a class in JSP

I am writing a class in JSP to retrieve a bunch of config values from an XML file. My plan is to have a class "XMLConfig" that loads in the values from the file, and then uses access methods to get at the values in the config object.
My problem is that i cannot seem to call application.getRealPath() from within the class, since eclipse tells me that "application cannot be resolved". I suspect that I must change "application" to something else but I am unsure what.
My code for the class:
<%!
//Config object
public class XMLConfig {
public boolean loadConfigFile(String strName) {
String XMLfileName = application.getRealPath(strName);
try {
DocumentBuilder db = DocumentBuilderFactory.newInstance().newDocumentBuilder();
Document doc = null;
doc = db.parse(XMLFileName);
}catch(Exception e)
{
System.out.println(e.getMessage());
return false;
}
return true;
}
}
%>

application isn't a global var. If you want to use it in your method then you'll need to pass it as a parameter.
Not sure why you're defining the class within the jsp though instead of just creating a 'normal' java class.

That's a job for a servlet instead of JSP. Create a class which extends HttpServlet and implement the doGet() method as follows:
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String strName = getOrDefineItSomehow();
Document doc = loadConfigFile(getServletContext().getRealPath(strName));
// Do whatever you want with it and then display JSP page.
request.getRequestDispatcher("/WEB-INF/config.jsp").forward(request, response);
}
Map this servlet in web.xml on an url-pattern of for example /config and invoke it by for example http://example.com/context/config. It'll run the code in doGet().
See also:
Beginning and intermediate JSP/Servlet tutorials
How to avoid Java code in JSP?
Hidden features of JSP/Servlet

Java/JSP Image upload. Where to keep these image files?

I am writing a simple application that lets a user upload images. After the upload, the user can tag them or remove them.
I figured out how to upload the files and save them once the files are uploaded. I am keeping tracking of a global path where images are kept. In the database I keep the meta data about the images like file name, tags, etc.
I am using Java/JSP (specifically Stripes framework but my problem is generic).
My question is where do I keep these image files once they are uploaded?
Right now I have two web applications deployed on a Tomcat server. One main web application and other one is the where I upload the images.
But this does not work as I can not see the uploaded images in the main application until I redeploy/restart Tomcat.
It seems like Tomcat does not pick newly uploaded images automatically.
Does anyone have any solutions?
This is a simple project, so I do not want to store them in a database or use Apache for images. That is all just too complicated for this small project.
Thanks.

Definitely don't store the images in the database, but you will want to store the image path in the database. This will allow you to store the image just about anywhere.
Since you are using two tomcat applications, your best bet may be to store the images outside of either app and stream the image back to the user instead of letting tomcat manage the files. Otherwise, I would ask why you are trying to do this with two web apps.

However, storing uploaded images inside the web-app directory is not a wise thing to do, and you know it.
By the way, you might want to look this stackoverflow thread, lately discussed where to store the images. It might not solve your issue, surely will give you more confidence on what you are doing.

I've solved this in different ways.
First, the non-portable way, is that Glassfish (and I do believe Tomcat as well) allows you to map an external directory in to the webapps hierarchy. This works really well and does exactly what you want. It lets you store your images in an external directory away from your webapp, yet still serve them up.
However, this technique is not portable.
The way to I've done it portably is by creating a filter.
You place the filter someplace obvious, say "/images".
What the filter does is this:
it checks for the image (or anything, it works with any static resource) in a special directory within the webapp. For our example we'll use the url "/webapp/images".
if the file DOES NOT exist, we copy the file from your external location in to the appropriate spot within the webapp. So, let's say the reqyest url is "/images/banner.gif". And that your files are stored on disk at "/home/app/images". So, our source file is "/home/app/images/banner.gif". We then copy it to where we want it in the webapp tree. We use "ServletContext.getRealPath" for this. So, the destination will be "ServletContext.get RealPath("/webapp/images/banner.gif"). Just copy the source to the destination.
if the file already existed, or now exists, simply forward to the actual image at /webapp/images/banner.gif.
Effectively you end up having a file cache within your webapps deployment tree. The down side is that it's a cache, so it needs to be maintained (i.e. you should check if the original is newer than your cache, make sure you delete if the source is deleted, etc.). Also, it duplicates your resources, so your images will consume, eventually, twice as much disk space. Finally, there's the initial copy cost at start up.
However, it DOES work, and it prevents you from having to serve up static resources using your own code. (Which is the 3rd solution, map a filter/servlet to intercept the URLs and simply stream it your self.)
I would look at the construct within Tomcat (assuming it exists) to do the mapping for you. I know it exists in Glassfish. (Google alternatedocroot for Glassfish to see how it works.)

I was using two web applications to avoid over writing the uploaded images in case I redeploy a new main application war file.
But as you mention there is no other option but to stream them through a Servlet or something I guess I can keep them outside tomcat directory.
I wanted to avoid writing this Streaming Servlet. Just too small project to deal with all the mess (like proper content type, 404, etc.) while writing the streaming servlet.

import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
/**
* Image streaming Servlet.
*/
public class ImageDisplayServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
/**
* #see HttpServlet#HttpServlet()
*/
public ImageDisplayServlet() {
super();
}
/**
* #see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String relativePath = trimToEmpty(request.getPathInfo());
// Make sure no one try to screw with us.
// This is important as user can literally access any file if we are not careful
if(isXSSAttack(relativePath) == false) {
String pathToFile = this.getServletContext().getRealPath(request.getPathInfo());
File file = new File(pathToFile);
System.out.println("Looking for file " + file.getAbsolutePath());
// show a 404 page
if(!file.exists() || !file.isFile()) {
httpError(404, response);
} else {
try {
streamImageFile(file, response);
} catch(Exception e) {
// Tell the user there was some internal server error.\
// 500 - Internal server error.
httpError(500, response);
e.printStackTrace();
}
}
} else {
// what to do if i think it is a XSS attack ?!?
}
}
private void streamImageFile(File file, HttpServletResponse response) {
// find the right MIME type and set it as content type
response.setContentType(getContentType(file));
BufferedInputStream bis = null;
BufferedOutputStream bos = null;
try {
response.setContentLength((int) file.length());
// Use Buffered Stream for reading/writing.
bis = new BufferedInputStream(new FileInputStream(file));
bos = new BufferedOutputStream(response.getOutputStream());
byte[] buff = new byte[(int) file.length()];
int bytesRead;
// Simple read/write loop.
while (-1 != (bytesRead = bis.read(buff, 0, buff.length))) {
bos.write(buff, 0, bytesRead);
}
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
if (bis != null) {
try {
bis.close();
} catch (IOException e) {
e.printStackTrace();
// To late to do anything about it now, we may have already sent some data to user.
}
}
if (bos != null) {
try {
bos.close();
} catch (IOException e) {
e.printStackTrace();
// To late to do anything about it now, we may have already sent some data to user.
}
}
}
}
private String getContentType(File file) {
if(file.getName().length() > 0) {
String[] parts = file.getName().split("\\.");
if(parts.length > 0) {
// only last part interests me
String extention = parts[parts.length - 1];
if(extention.equalsIgnoreCase("jpg")) {
return "image/jpg";
} else if(extention.equalsIgnoreCase("gif")) {
return "image/gif";
} else if(extention.equalsIgnoreCase("png")) {
return "image/png";
}
}
}
throw new RuntimeException("Can not find content type for the file " + file.getAbsolutePath());
}
private String trimToEmpty(String pathInfo) {
if(pathInfo == null) {
return "";
} else {
return pathInfo.trim();
}
}
private void httpError(int statusCode, HttpServletResponse response) {
try {
response.setStatus(statusCode);
response.setContentType("text/html");
PrintWriter writer = response.getWriter();
writer.append("<html><body><h1>Error Code: " + statusCode + "</h1><body></html>");
writer.flush();
} catch (IOException e) {
e.printStackTrace();
}
}
private boolean isXSSAttack(String path) {
boolean xss = false;
// Split on the bases of know file separator
String[] parts = path.split("/|\\\\");
// Now verify that no part contains anything harmful
for(String part : parts) {
// No double dots ..
// No colons :
// No semicolons ;
if(part.trim().contains("..") || part.trim().contains(":") || part.trim().contains(";")) {
// Fire in the hole!
xss = true;
break;
}
}
return xss;
}
/**
* #see HttpServlet#doPost(Ht/promotions/some.jpgtpServletRequest request, HttpServletResponse response)
*/
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
doGet(request, response);
}
}
Ok Here is a Servlet that I quickly wrote that can stream images:
Here is the List of limitations and know issues:
May have XSS vulnerability use with care
Not production ready use as reference
Images need to in the web application directory. Can be easily change but I too lazy (it is not worth it the project is too small)
Only stream jpg,gif or png files.
Usage:
Let say you deploy this web application called images as separate application.
http://www.example.com/images/promotions/promo.jpg
means there should be a directory in "promotions" with image "promo.jpg" with in this images web application.
PS: Do not ask why I am doing this Servlet Container only solution that sucks big time.

<servlet>
<description></description>
<display-name>ImageDisplayServlet</display-name>
<servlet-name>ImageDisplayServlet</servlet-name>
<servlet-class>com.example.images.ImageDisplayServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>ImageDisplayServlet</servlet-name>
<url-pattern>/*</url-pattern>
</servlet-mapping>
Oh ya configure your Servlet like above for best results :P