I have a Java Servlet/JSP application which requires the user to upload an archive file (either .rar or .zip). This archive file is then extracted, and the extracted files are parsed. After parsing the files, the data in them is added to the database and the files are deleted again.
On my local machine, this works perfectly, since you just use the filesystem provided by the OS. But now I'd like to run this application on Heroku and I'm unsure on how to do the file uploads.
Since these files are user specific, and not permanent, my initial thoughts were that I could just use the ephemeral file system provided by Heroku and I do not require the use of S3.
At the moment, my application runs on only 1 web dyno and no worker dyno's but in the future this may get scaled to multiple web dyno's, depending on the amount of users that are going to use it.
Can I use the ephemeral file system for my specific use case, and will it scale properly?
I am currently writing using ServletFileUpload, and am writing to java.io.File; Can I just change the path of my java.io.File to a path in the ephemeral file system? What would be an example of such path?
I guess you can use the ephemeral fileystem in your specific case, as it's just a temporary usage for parsing the file.
You can use the /tmp directory but keep in mind that the file will be destroyed after the request is complete.
This is discussed in this post https://stackoverflow.com/a/12416923/476782
Related
I have a JAVA application that needs to store profile pictures that user uploaded.
My project already finish and works fine.
//in my LOCALHOST i use this path:
File file = new File("C:/myProject/uploads/profile_images");
So, now I want deploy this project, i'm using jelastic environment and the question is:
Where should these files (pictures) be stored in our Jelastic ?
I already tried in the same code but doens't work.
I already tried save the files in WebContent folder, work, but when I expand a new .war file, the files that user has uploaded are overrides.
I read about save files in mySql, is a good idea?
Thank for your atention.
Local filesystem is persistent, but make sure to define in 'volumes' if your node has it to ensure files are kept during image redeploy. You can also use Jelastic storage node, but only worthwhile if you're using multiple application nodes.
See https://docs.jelastic.com/docker-volumes for details about how to use the volumes feature - if your node doesn't have this feature it is not based on Docker (not all node types were converted yet). In that case you can write to the filesystem without any risk of those files going missing (i.e. it will behave the same as a dedicated server or your local dev machine).
We are running our test environment on AWS and we use NFS (EBS file system) mounts across all the servers to access our files. Multiple java processes access files on this file system. Application server uses different consumers running on other systems to get files processed. And one such consumer converts or files to images, but the issue is that the application server can't access these files (file.exist() is false). Now the strange thing is that it can't access the first image of the file page (files with several pages), but it can access the second page and so on. Many time I observe, odd numbered page images are not accessible, but I can see that the image exists there. So, I think the image conversion process is still accessing the image while application server tries to access it, but the same issue doesn't surface to the even numbered page images. These images reside inside a nested folder, so the conversion process also creates the folder structure and then writes them into those folders. Any idea what may be the issue here? Anyone facing this issue with NFS shares on AWS?
It depends on the expected behaviour of your application
NFS works great with readonly systems or with programs that have a low demand on the timely updating of data
See this question for more details Alternative to File.exists() in Java
You might like to consider an alternative to NFS such as AWS EFS or the open source Gluster
I'm writing a Java application which requires a number of resource files (there will be about 100 files of 20-40K each). They are not edited by the user, but will require periodic updates (the application will have a function to check for changes to the resource files and download them). Ideally, the application should be cross-platform.
Allowing write access to a subdirectory of the program directory is generally frowned upon. If I was doing it as a Windows application I might put them in Application Data, but that's not going to fly cross-platform. What would be the best place to put them?
I would typically create a directory (name starting with a period ".") in user's home directory (System.getProperty("user.home") if I am not mistaken) and use that for application specific storage. Alternatively, you could take the directory name from user at the time of application installation.
Have a directory you use to keep these files in. Put that information in a properties configuration file. When you start up load the configuration file from your application install directory. From that properties file it tells you where to find your file directory. When your installer runs it can write out this configuration file for the platform you are installing on, and that can be OS specific.
Provide a configurable location, but default to a directory in the user's home directory, or in an OS-specific location.
You'll have to deal with this in a platform-specific way no matter what. You have a few options under OS X, though. For unix-like systems either a home directory, or perhaps something under /var.
That said, I don't believe a program managing its own data in its own directory is a bad thing; consider a program with an embedded database or similar. It's much more reliable to use an app home directory.
I uploaded files to the server using BLOB db type on play framework. In the application.conf file I have
attachments.path=home/dotcloud/uploads
But I couldn't find the files on the server.
The issue is that if I restart my www service, then I lose all my files, I only have the db records.
I believe that there are two issues in here. First your path lack a initial '/' to be a full path (I'm assuming that was your intention):
attachments.path=/home/dotcloud/uploads
Second, I'm not sure that your Play server will have rights to write to that folder, as it's outside the application context path. Default folder is local to the application and Play can write it, not so sure about other folders though. You should double check that.
I am facing a problem with file upload.I have used Apache Commons
servlet file upload for uploading the file. Though the file is getting
uploaded and the data is getting stored on the local server(http://
127.0.0.1:8888/_ah/admin/datastore) but it is not going to the Google
App Engine datastore.
What I am doing is loading
the file as a stream and immediately parsing the stream and creating
datastore entities with the data. I am not actually trying to save the
file. In the local server it works. It even works when I try to access
the local server from another machine. However it does not work when I
deploy it to Appengine using the Google Pluggin for Eclipse. My parsing
code depends on resource files which are under the web-inf directory.
Is it possible these resource files are not getting uploaded and is
there a way to check what files are uploaded on Appengine?
Whatever's in your .war is going up into AppEngine. I don't see how parts of it will be selectively excluded. What's more likely is that your application is depending on stuff that is lurking SOMEwhere on your PC but not included in that .war file.
However, shouldn't your application be checking for those resources and throwing exceptions if they are not found? If it's failing silently, I'd consider that a design flaw.
Logging a lot may help you debug the problem. You can look at your program's logs via the AppEngine console. I recommend more error checking and logging.
Something else to check for is to not be running the version of your software you think you are. There's a kind of versioning mechanism that allows you to deploy different concurrent versions of your and only one will be actually accessible. One of the things you should be logging and/or making otherwise accessible is some version information (perhaps even including a build timestamp) for your app's build.
The files in the .war folder are executed in the app engine and the others are uploaded. What you need to verify is the path you have set and the path of you source java file and the file you are reading. You cannot use the local file system path in app engine. You need to include the file in your project