I'm in the processing of designing a Java based web application (Spring based to be specific). One of the key requirement is that, this application has to accept many files of various formats (pdf, jpeg, dwg, png etc.) uploaded by the user. Also, to be able to download back to user's local computer. There will be thousands of files being uploaded/downloaded.
I am thinking of two approaches:
Upload the documents to the same box where server is running. Mostly all the documents will be uploaded to, and downloaded from box where Tomcat is running. I'm worried that, as the documents grow in number, this may impact overall performance.
Upload/download documents to another server dedicated for storing/retrieving of documents.
If 2nd approach is taken, how Spring application can upload/download files to/from remote server? Or which approach is being used in the similar applications.
Or could you suggest any other optimal way of handling this requirement.
Thanks in advance.
Ganesh
Many modern applications built like this are going to use an external storage system like Amazon S3 to store these files, which buys you all kinds of nice features - high availability for downloads, an effectively unlimited pool of disk space, data replication, and so on.
There's a tutorial available for integrating spring with Amazon S3. You should check that out. Regardless of whether you choose S3 or something else, the approach will be similar.
Have you thought about using a DB. You could store those files as BLOBs. Here is a tutorial for this: link
"This tutorial walks you through the steps of building an interesting Spring MVC web application that allows the user to upload files from her computer to the server. The files are stored in database using Hibernate."
As to two approaches you consider:
Either way you will have to manage those files, back them up and check if there is enough space to store more files. Also this may cause some security issues as you accept all files.
I recommend you using a JCR like Apache Sling or Apache Jackrabbit.
Apache Sling™ is a framework for RESTful web-applications based on an extensible content tree with the following feautures:
Content resolution that maps a request URL to a content node in the content repository
Servlet resolution that maps a content node and a request method to a Servlet handling the request
Default servlets supporting WebDAV, content creation from web forms and JSON representation
A Javascript client library, allowing access to the content repository through AJAX
Support for server-side scripting with Javascript, JSP, Ruby, Velocity and Scala
OSGi-based extensibility through Apache Felix – the Felix Web Console was originally developed by the Apache Sling project
Related
I have application where various files are uploaded in server. And the application has various users. Now, I want to implement authentication so that only privileged users can access to those uploaded documents. So, is there any document manager api in java to manage these documents. I have looked into alfresco, knowledgetree but those are separate applications. I want an api to implement in my application.
We do not have enough information to suggest a solution, but I think that you can still think to use Alfresco if you embed it in your application.
Look at the following url: Deployment options that shows some deployment options.
Of course, this makes sense only if you need all the power of Alfresco.
We have multiple web applications for different functional areas. There is no overlap of functionality between these apps and hence they are fairly independent. All these apps generate content like PDF and XML data. Currently all these apps are storing these documents in a path relative to their web root. The documents are accessed using url relative to the app specific web root.
Now we want to move to a design where these apps store the data/files in one central location and these documents can be accessed thru a URL outside of the specific application web root. Also we want these documents to be available even if the specific application is down.
We experimented with Apache Jackrabbit etc, but most of these are CMS tools that provide a lot more than what we want. We don't need full CMC capabilities since we don't really intend to do any web publishing, editing etc. We just need a simple way for multiple apps to store files in one single location and later access them thru a URL. Some thing probably like cloud storage.
Are there any tools out there that could help us implement this? Or Design pattern?
We need
beans from multiple apps to be able to save files in one central location (we can't use fixed disk drive location)
common url based access to these resources
We use: Java web apps on Tomcat 7 using JSF/Myfaces
Use an Apache web server (or other web server). Save the files in a folder published through HTTP by the server. To save them you can use any protocol that allows file transfer (FTP, SCP...).
How do you decide which protocol you've to use to transfer/move/copy files across servers?
I'm writing a service to copy files from a Windows server file system to a oracle-based database.
How about the File Transfer Protocol for transferring files.
Since Oracle doesn't store files, I suspect you need more than just transferring files. I suspect you will need to use JDBC in any case.
I am not sure it's clear what you are building and why is it it a java question?
Do you need to implement a custom protocol in Java to transfer files for a web service you are building? Or can you use an off the shelf product and a bunch of shell scripts to take care of the file transfer?
If it's a custom thing you need to build in Java (the question is tagged with java) then more questions:
Is it a web app that provides end-users with GUI allowing them to submit files?
- you need to take care of file transfer using http post. Apache httpclient library can help you there.
Is it a SOAP Web Service that saves incoming data into oracle?
- again then SOAP implementations provide answer to that.
Is it a command-line tool running on Windows machine that talk to an Oracle database ?
-then protocol is irrelevant, as you can just read the files using file.io libs and use jdbc to save them into oracle.
Is it a custom server with a yet-to-be built protocol and subsequently a custom client that will use that protocol?
- then I would use binary over https for my implementation of the custom server/service.
4.If you are doing everything using COTS, FTP is simple and fast, but it's not secure and it's not reliable. SFTP will almost always going to be better.
http://geekswithblogs.net/bvamsi/archive/2006/03/23/73147.aspx
5.If your question is really about how to save files into Oracle database using jdbc, then you need to find a way to stream them (or you will run out of memory).
E.g. this may be helpful http://www.coderanch.com/t/415625/JDBC/databases/save-files-oracle-database-java
I'm planning a web service providing file download service for handheld devices.
The requirement I have is use HTTP or FTP Protocol to build a server, use database, store files and provide Command Line Interface for devices to download requested files. No platforms and languages limit.
I'm a student and I'm learning Java Web(practicing Structs2, haven't started learnning Spring or Hibernate), and this is just a simulate project which may be required in a E-Book company. So I'm feeling like to use Java to build the service.
I'm now doing the requirement analysis, please give me some suggestions. About how to design the structure, book files stored in database or filesystem, any open source library to refer to... About any aspect is OK.
I just want to know what can I do to make it better and where I should be paying attention to.
You could implement a simple Servlet, which returns files based on request arguments.
Then, use your command line client to call the Servlet with the correct parameters, depending on what the user typed.
EDIT: more info on Java Servlet technology here. If you are learning Struts, you could implement your servlet there, but it might be a bit of overkill.
What is an efficient way to implement a multiple file client upload service?
Are there any popular libraries for that?
Basically I'm looking at a Web view, served a client, that would allow them to upload files: i.e browse their file system and choose say up to 6 files.
Is implementing the above service better than a single file upload-service. i.e allowing only one upload at a time?
If so what is an efficient way to implement that or again what popular libraries are there to use.
I'm building a Java Dynamic Web to be deployed on Tomcat.
Thanks
If you're looking to do multiple file uploads on the web, I would strongly recommend SWFUpload. It's totally customizable, and it can use whatever back end technology you'd like. Take a look at some of the demos on the site for some use cases involving multiple files at once. I've only used it in one project so far, but it beats some of the other third party upload controls I've used hands down.
If your clients can use modern builds of browsers, you can use multiple file upload field support that mozilla/webkit offers. Otherwise, YUIUpload from latest YUI3. Much cleaner/faster/easier than SWFUpload.