I am writing a simple application that lets a user upload images. After the upload, the user can tag them or remove them.
I figured out how to upload the files and save them once the files are uploaded. I am keeping tracking of a global path where images are kept. In the database I keep the meta data about the images like file name, tags, etc.
I am using Java/JSP (specifically Stripes framework but my problem is generic).
My question is where do I keep these image files once they are uploaded?
Right now I have two web applications deployed on a Tomcat server. One main web application and other one is the where I upload the images.
But this does not work as I can not see the uploaded images in the main application until I redeploy/restart Tomcat.
It seems like Tomcat does not pick newly uploaded images automatically.
Does anyone have any solutions?
This is a simple project, so I do not want to store them in a database or use Apache for images. That is all just too complicated for this small project.
Thanks.
Definitely don't store the images in the database, but you will want to store the image path in the database. This will allow you to store the image just about anywhere.
Since you are using two tomcat applications, your best bet may be to store the images outside of either app and stream the image back to the user instead of letting tomcat manage the files. Otherwise, I would ask why you are trying to do this with two web apps.
However, storing uploaded images inside the web-app directory is not a wise thing to do, and you know it.
By the way, you might want to look this stackoverflow thread, lately discussed where to store the images. It might not solve your issue, surely will give you more confidence on what you are doing.
I've solved this in different ways.
First, the non-portable way, is that Glassfish (and I do believe Tomcat as well) allows you to map an external directory in to the webapps hierarchy. This works really well and does exactly what you want. It lets you store your images in an external directory away from your webapp, yet still serve them up.
However, this technique is not portable.
The way to I've done it portably is by creating a filter.
You place the filter someplace obvious, say "/images".
What the filter does is this:
it checks for the image (or anything, it works with any static resource) in a special directory within the webapp. For our example we'll use the url "/webapp/images".
if the file DOES NOT exist, we copy the file from your external location in to the appropriate spot within the webapp. So, let's say the reqyest url is "/images/banner.gif". And that your files are stored on disk at "/home/app/images". So, our source file is "/home/app/images/banner.gif". We then copy it to where we want it in the webapp tree. We use "ServletContext.getRealPath" for this. So, the destination will be "ServletContext.get RealPath("/webapp/images/banner.gif"). Just copy the source to the destination.
if the file already existed, or now exists, simply forward to the actual image at /webapp/images/banner.gif.
Effectively you end up having a file cache within your webapps deployment tree. The down side is that it's a cache, so it needs to be maintained (i.e. you should check if the original is newer than your cache, make sure you delete if the source is deleted, etc.). Also, it duplicates your resources, so your images will consume, eventually, twice as much disk space. Finally, there's the initial copy cost at start up.
However, it DOES work, and it prevents you from having to serve up static resources using your own code. (Which is the 3rd solution, map a filter/servlet to intercept the URLs and simply stream it your self.)
I would look at the construct within Tomcat (assuming it exists) to do the mapping for you. I know it exists in Glassfish. (Google alternatedocroot for Glassfish to see how it works.)
I was using two web applications to avoid over writing the uploaded images in case I redeploy a new main application war file.
But as you mention there is no other option but to stream them through a Servlet or something I guess I can keep them outside tomcat directory.
I wanted to avoid writing this Streaming Servlet. Just too small project to deal with all the mess (like proper content type, 404, etc.) while writing the streaming servlet.
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
/**
* Image streaming Servlet.
*/
public class ImageDisplayServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
/**
* #see HttpServlet#HttpServlet()
*/
public ImageDisplayServlet() {
super();
}
/**
* #see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String relativePath = trimToEmpty(request.getPathInfo());
// Make sure no one try to screw with us.
// This is important as user can literally access any file if we are not careful
if(isXSSAttack(relativePath) == false) {
String pathToFile = this.getServletContext().getRealPath(request.getPathInfo());
File file = new File(pathToFile);
System.out.println("Looking for file " + file.getAbsolutePath());
// show a 404 page
if(!file.exists() || !file.isFile()) {
httpError(404, response);
} else {
try {
streamImageFile(file, response);
} catch(Exception e) {
// Tell the user there was some internal server error.\
// 500 - Internal server error.
httpError(500, response);
e.printStackTrace();
}
}
} else {
// what to do if i think it is a XSS attack ?!?
}
}
private void streamImageFile(File file, HttpServletResponse response) {
// find the right MIME type and set it as content type
response.setContentType(getContentType(file));
BufferedInputStream bis = null;
BufferedOutputStream bos = null;
try {
response.setContentLength((int) file.length());
// Use Buffered Stream for reading/writing.
bis = new BufferedInputStream(new FileInputStream(file));
bos = new BufferedOutputStream(response.getOutputStream());
byte[] buff = new byte[(int) file.length()];
int bytesRead;
// Simple read/write loop.
while (-1 != (bytesRead = bis.read(buff, 0, buff.length))) {
bos.write(buff, 0, bytesRead);
}
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
if (bis != null) {
try {
bis.close();
} catch (IOException e) {
e.printStackTrace();
// To late to do anything about it now, we may have already sent some data to user.
}
}
if (bos != null) {
try {
bos.close();
} catch (IOException e) {
e.printStackTrace();
// To late to do anything about it now, we may have already sent some data to user.
}
}
}
}
private String getContentType(File file) {
if(file.getName().length() > 0) {
String[] parts = file.getName().split("\\.");
if(parts.length > 0) {
// only last part interests me
String extention = parts[parts.length - 1];
if(extention.equalsIgnoreCase("jpg")) {
return "image/jpg";
} else if(extention.equalsIgnoreCase("gif")) {
return "image/gif";
} else if(extention.equalsIgnoreCase("png")) {
return "image/png";
}
}
}
throw new RuntimeException("Can not find content type for the file " + file.getAbsolutePath());
}
private String trimToEmpty(String pathInfo) {
if(pathInfo == null) {
return "";
} else {
return pathInfo.trim();
}
}
private void httpError(int statusCode, HttpServletResponse response) {
try {
response.setStatus(statusCode);
response.setContentType("text/html");
PrintWriter writer = response.getWriter();
writer.append("<html><body><h1>Error Code: " + statusCode + "</h1><body></html>");
writer.flush();
} catch (IOException e) {
e.printStackTrace();
}
}
private boolean isXSSAttack(String path) {
boolean xss = false;
// Split on the bases of know file separator
String[] parts = path.split("/|\\\\");
// Now verify that no part contains anything harmful
for(String part : parts) {
// No double dots ..
// No colons :
// No semicolons ;
if(part.trim().contains("..") || part.trim().contains(":") || part.trim().contains(";")) {
// Fire in the hole!
xss = true;
break;
}
}
return xss;
}
/**
* #see HttpServlet#doPost(Ht/promotions/some.jpgtpServletRequest request, HttpServletResponse response)
*/
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
doGet(request, response);
}
}
Ok Here is a Servlet that I quickly wrote that can stream images:
Here is the List of limitations and know issues:
May have XSS vulnerability use with care
Not production ready use as reference
Images need to in the web application directory. Can be easily change but I too lazy (it is not worth it the project is too small)
Only stream jpg,gif or png files.
Usage:
Let say you deploy this web application called images as separate application.
http://www.example.com/images/promotions/promo.jpg
means there should be a directory in "promotions" with image "promo.jpg" with in this images web application.
PS: Do not ask why I am doing this Servlet Container only solution that sucks big time.
<servlet>
<description></description>
<display-name>ImageDisplayServlet</display-name>
<servlet-name>ImageDisplayServlet</servlet-name>
<servlet-class>com.example.images.ImageDisplayServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>ImageDisplayServlet</servlet-name>
<url-pattern>/*</url-pattern>
</servlet-mapping>
Oh ya configure your Servlet like above for best results :P
Related
I have brief understanding on multi threading in Java web apps. However, I came across an issue while developing our android app which communicates with the server via REST.
Our web application is based on Apache Wicket 8.6 and contains additional HttpServlet endpoints. One endpoint is applied while uploading images from the app via post. This works fine as long as I am uploading only one image at a time. If I perform multiple upload requests in my android app in a quick succession (some milliseconds), only the last upload is performed successfully (it is working fine when I put a second break between the uploads). It seems, as if all requests except the last one are missing the image content read from the input stream of the servlet request. Thus, I am guessing, that I am having a threading problem with my servlet. I would appreciate, if someone could guide me in the right direction to solve this issue.
public class MyServlet extends HttpServlet{
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
boolean proceed =true;
Map<String,String[]> parameters = req.getParameterMap();
if(parameters!=null){
//read some parameters which specify the request
}
if(proceed) {
InputStream is = req.getInputStream();
if(is!=null) {
//The result of this output is 0 for the first requests only for the last request it is 1
System.err.println("Stream size: "+is.available());
//do something
}
}
//do something....
}
}
I could, of course, write the images in my android app in one request using multipart but still I'd like to make the servlet thread safe for the occasion that it gets two requests at the same time.
I appreciate your help.
So, after some additional research I discovered, that the input stream is not empty even is.available() returns 0
My problem was something different. I save the uploaded image in a ModeShape repository. The repository session is stored in the wicket application instance. THus, only one ModeSHape repository session exists. When writing the image, it seems as if there was a problem with the ModeShape session. Thus, I put the ModeShape session in a synchronized block and now everything runs fine.
public class MyServlet extends HttpServlet{
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
boolean proceed =true;
Map<String,String[]> parameters = req.getParameterMap();
if(parameters!=null){
//read some parameters which specify the request
}
if(proceed) {
String fileName = req.getHeader("fileName");
if(!StringUtils.isEmpty(fileName)) {
InputStream is = req.getInputStream();
if(is!=null) {
//The result of this output is 0 for the first requests only for the last request it is 1
System.err.println("Stream size: "+is.available());
WicketApplication app=null;
try {
app = (WicketApplication)Application.get("wicket");
} catch (Exception e) {
LOG.error("Error while generating WicketApplication object for project "+project.getName(),e);
}
if(app!=null) {
final Session repoSession = app.getRepoSession();
synchronized (repoSession) {
//write image to repository
}
}
}
}
}
//do something....
}
}
I have two Java Servlets: DataFetcherServlet and UploaderServlet. Both servlets call 2 different Java methods which in turn call their corresponding Matlab functions through JNI, and each of which was compiled into a separate Java jar file to use as a library. The application is powered by AJAX to create a Desktop like feel. For the UploaderServlet, the users can upload an excel file to this servlet, the parsed data then get passed to a Java method which then calls the compiled Matlab function to generate and save alot of images (currently over 5000 images), because this will take alot of time, I use an ExecutorService to execute it in the background. But new requests sent the DataFetcherServlet which will also call another compiled Matlab function is blocked until the image generation part is completed. I don't know why it is blocking new requests even though the requests are sent to a different servlet.
DataFetcherServlet.java
public class DataFetcherServlet extends HttpServlet {
#Inject
private CdfReader reader; // An EJB to get a data array from Matlab
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
try {
String filePath = "path/to/file";
Object[] result = reader.read(filePath); // reader.read() is just a wrapper around the method in the jar file mentioned above that actually calls the matlab function to return an array of number
MWNumericArray array = (MWNumericArray)result[0] // This will block while the other Matlab function is generating the images.
.
.
.
} catch (MWException ex) {
Logger.getLogger(DataFetcherServlet.class.getName()).log(Level.SEVERE, null, ex);
}
}
UploaderServlet.java
public class UploaderServlet extends HttpServlet {
#Inject
private ExcelIonImageGenerator generator; // An EJB to call Matlab to generate the images
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
try {
String dir = "path/to/parent/directory";
Path excel = Paths.get(dir+ "excel", part.getSubmittedFileName()); // Path to where the uploaded excel file is stored
if (!Files.exists(excel))
Files.copy(part.getInputStream(), excel);
// ExcelExtractor is a helper class to parse the excel file.
Double[][] ranges = ExcelExtractor.extractSheet(WorkbookFactory.create(excel.toFile()));
// This will call a Java library method which in turns call the Matlab function
// to generate the images (over 5000 in this case)
// See the code for this method below.
generator.generate(dir+ "images" + File.separator, ranges);
} catch (MWException | InvalidFormatException ex) {
Logger.getLogger(UploaderServlet.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
ExcelIonImageGenerator.java
import com.mathworks.toolbox.javabuilder.*; // Matlab SDK needed to integrate with Java
import java.util.concurrent.*;
import java.util.logging.*;
import javax.annotation.PreDestroy;
import javax.ejb.Stateless;
import save_ion_image_for_all_ranges_in_spreadsheet.Class1; // The jar file which contains code to call Matlab code through JNI
#Stateless
public class ExcelIonImageGenerator {
private final Class1 clazz1;
private ExecutorService pool;
public ExcelIonImageGenerator() throws MWException {
clazz1 = new Class1();
pool = Executors.newFixedThreadPool(1);
}
public void generate(String path, Double[][] ranges) throws MWException {
// Submit this task to the ExecutorService so it can be processed
// in a different thread than the caller thread
pool.submit(() -> generateHelper(path, ranges, clazz1), 1);
}
private void generateHelper(String path, Double[][] ranges, Class1 clazz) {
try {
// This method was generated by Matlab tool, it calls the native
// Matlab code through JNI, and it will block any request that will call
// other Matlab functions until it finishes.
clazz.save_ion_image_for_all_ranges_in_spreadsheet(path, ranges);
} catch (MWException ex) {
Logger.getLogger(ExcelIonImageGenerator.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
You have three options:
Launch multiple processes of your Java application that makes calls to Matlab. Calls from a single process use the same MCR which has a process-wide lock, however calls from different processes would run on separate MCR computational engines.
Use Matlab Production Server, which basically facilitates the use of multiple MCRs. This is a toolkit that requires a separate license and installation.
You don't necessarily have to limit yourself to running MCR / compiled code unless you have very specific performance concerns. You can actually install Matlab itself on the server, launch multiple instances (headless, etc) from the same Java process, and communicate with them e.g. through MatlabControl or the new official MATLAB Engine API for Java.
There's a very good answer from MathWorks Support Team on MatlabCentral, explaining these limitations of MCR in detail.
I have a simple plugin system, that loads external JAR plugins into main application. I am using Mountainblade Modular to do so. Not sure how they do it "under the hood" but probably it's something standard.
This works fine, I instantiate classes from external jar and it all works. Except that some plugins come with icons/images. I am a bit unsure on how do I load/refer to images from that external JAR (with code inside that external JAR, as it is ran in context of the main JAR, kind of)
How should I approach this?
This issue is not as straightforward as it seems to be.
When you load classes from external jar, they are "loaded" into JVM. By "loading" into JVM I mean that JVM is responsible for their storage in memory. Usually it is done like this:
ClassLoader myClassLoader = new MyClassLoader(jarFileContent);
Class myExtClass = myClassLoader.loadClass(myClassName);
Resources from classpath jars can be accessed easily with
InputStream resourceStream = myClass.getResourceAsStream("/myFile.txt");
You can do that, because these jars are in classpath, I mean their location is known. These files are not stored in memory. When resource is accessed, JVM can search for it in classpath jars (for example on file system).
But for external jars it is completely different: jar comes from nowhere, is once processed and forgotten. JVM does not load resources from it in memory. In order to access these files, you have to manually organize their storage. I've done this once so I can share the code. It will help you to understand the basic idea (but probably won't help you with your specific library).
// Method from custom UrlClassLoader class.
// jarContent here is byte array of loaded jar file.
// important notes:
// resources can be accesed only with this custom class loader
// resource content is provided with the help of custom URLStreamHandler
#Override
protected URL findResource(String name) {
JarInputStream jarInputStream;
try {
jarInputStream = new JarInputStream(new ByteArrayInputStream(jarContent));
JarEntry jarEntry;
while (true) {
jarEntry = jarInputStream.getNextJarEntry();
if (jarEntry == null) {
break;
}
if (name.equals(jarEntry.getName())) {
final byte[] bytes = IOUtils.toByteArray(jarInputStream);
return new URL(null, "in-memory-bytes", new URLStreamHandler() {
#Override
protected URLConnection openConnection(URL u) throws IOException {
return new URLConnection(u) {
#Override
public void connect() throws IOException {
// nothing to do here
}
#Override
public InputStream getInputStream() throws IOException {
return new ByteArrayInputStream(bytes);
}
};
}
});
}
}
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
I'm running an embedded Jetty 8 server that loads a few *.war files at startup:
for (File aWebAppDirectory : aWebAppDirectories) {
if (aWebAppDirectory.exists() && aWebAppDirectory.isDirectory()) {
for (File warFile : aWebAppDirectory.listFiles(new WarFileFilter())) { String basename = warFile.getName().replaceFirst("\\.war$", "");
fHandlers.addHandler(new WebAppContext(warFile.getAbsolutePath(), "/" + basename));
}
}
}
These war-files have some dependencies on a few classes that may or may not exist in the classpath.
Right now if one of my servlets is missing a dependency, my entire embedded Jetty service fails. (Because of NoClassDefFoundExceptions)
I need a method that allows me to catch exceptions for failing servlets and simply doesn't activate them. I'm looking for the same thing that TomCat does when a servlet fails to load: It still loads the rest.
I haven't found any solutions after some time searching on Google.
Anyone know how I can tackle this problem using embedded Jetty 8?
If anyone is curious how I fixed this, I simply made sure that all my servlets have a wrapper servlet that basically has no dependencies. The wrapper tries to initialize a delegate with dependencies and explicitly checks for NoClassDefFountException. If this happens, the delegate is set to null, and all calls to the wrapper interface will result in an exception.
So on a high level:
public class ServletWrapper extends HttpServlet{
private ServletDelegate fDelegate;
//If this is false, the delegate does not work, and we should not forward anything to it.
private boolean fAvailable = false;
public ServletWrapper(){
try{
fDelegate = new ServletDelegate();
fAvailable = true;
} catch (NoClassDefFoundError e) {
fAvailable = false;
}
}
#Override
protected void doPost( HttpServletRequest request, HttpServletResponse response )
throws ServletException, IOException {
if ( !fAvailable || fDelegate==null ) {
response.sendError( HttpServletResponse.SC_SERVICE_UNAVAILABLE, LSP_MISSING_ERROR_MESSAGE );
return;
}
fDelegate.doPost(request,response);
}
}
It's simple enough, and it works.
I need to upload a file in GWT in the background, ie. not from a visual widget.
For this I've found the RequestBuilder class, but little documentation on how to use it for upload purposes.
The file content I need to upload is 100% plaintext.
My code so far looks like this:
final String filename = UUID.randomUUID().toString() + ".txt";
RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, "http://localhost/upload");
rb.setRequestData(selected.getBody()); // getBody() is plain-text
rb.setHeader("Content-Type", "application/x-www-form-urlencodeddata");
rb.setCallback(new RequestCallback() {
#Override
public void onResponseReceived(Request request, Response response) {
w.setUrl("http://localhost/magic.html?hide=status&open=" + filename);
w.show();
w.maximize();
}
#Override
public void onError(Request request, Throwable exception) {
exception.printStackTrace();
}
});
// Checked Exceptions sucks *sigh*
try {
rb.send();
}
catch (RequestException e) {
e.printStackTrace();
}
I hope someone can give me some pointers towards solving this issue, thanks!
I think it's not possible to upload files to the server without user interaction using JavaScript only. I think it's blocked by the browser, because it would mean anybody could upload any file from your system when you visit a site, which would be a major security problem. I don't know why you would want to do this, but I guess you need to look at another approach to what you are trying to do.
It seems that you can upload files with gears, so in the worst case you could implement something like this with javascript: link text