I've been tasked with implementing large (2gb+) file uploads via a web browser. After evaluating various technologies, java applets seem to be the way forward (only one's which provide proper access to the local disk). I was wondering if anyone can recommend a 3rd party file upload app we can use as a base? requirements are
Decent UI, ideally we want something similar to facebooks photo uploader
Can handle large (2gb+) files
Resumable uploads
We beed the source to extend it to our needs (dont mind paying extra)
You're probably looking for JUpload.
Update: not sure if it has as nice of UI as you're hoping, but unless you want to build a custom solution like I have it's your best option.
Just a tip, maybe it is obvious, i don't know :P
It is nice to send the big file in chunks like 2mb, and on the server side you just append the bytes to the target file. The server knows what bytes it needs, and if a upload is aborted and continued later, the server can just send a message about from what bytes to start uploading the file again. Then we get resumability (is it a word? :P) and safety of large HTTP-uploads (since, in fact, we are sending many smuller uploads, and each upload is checked to be of the correct size on the server).
We wrote an implementation like this once with a Java-applet as the client and PHP on the server, I'll see if I can dig it out as a reference for you :p
Not really a solution : from experience you may bump into the following issues:
problems when uploading over HTTPs
problems uploading through proxies
Just wanted to make you aware of these two cases, for you to test when evaluating a solution.
Hope, you will get solutions for your prob over here.. http://jupload.sourceforge.net/
Related
I am writing an applet that I eventually want to put online so that my friends/family can use it. I have the applet running now locally, but in order to work properly it needs to read a .ser file in when the applet opens, and update that same file when the applet closes. The file is quite large (~180 MB), though I am working on paring it down.
What would be the fastest/most effective way to read/write this file in java? There is a lot of information out there on this and I have never done anything like it before, so it's a bit overwhelming. The class HTTPURLConnection seems like an option to read it, but not write it. Any free web hosting that I have seen will not allow a file that big to be uploaded.
The size of the file should hopefully go down substantially, it is a list of 2.8 million musical artists, many of which I'm sure nobody using the program will ever encounter, but if this program is to be effective, many artists will have to be stored, so the problem most likely remains the same.
Thanks in advance for any help
It sounds like it would be wise to keep this large data and the processing of it on your server instead of making the applet operate on it. That's because you would avoid each user downloading a large file and processing it. If you had a server side piece that the applet could call to get useful information from, then only your server would have to load it, write it, and process it. You could implement a Java servlet, or a PHP program to respond to http requests from your applet in a format that suits the data. I'm assuming that your server can handle either servlets or custom PHP (most can).
I need to allow for csv-file downloads on my page and I was going to try ngCsv (from Angular) but for browser support this seems fairly limited. I've seen quite a few examples of this being done with vanilla Javascript. And after a discussion with a colleague of "backend vs. frontend" I'm feeling more and more unsure of what to do.
Are there any true optimization/efficiency reasons why I should avoid doing this on the client side (assuming the files are no more than 100MB each download)?
Are there any true optimization/efficiency reasons why I should avoid
doing this on the client side (assuming the files are no more than
100MB each download)?
If the data on the .csv would be the same for each user, and only updated every now and then, I would suggest you have your server create / update a static .csv. It wouldn't be resource-intensive, and you wouldn't have to worry about browser compatibility / user resources.
If, however, the data you need to create a .csv for is different on a per-user basis, then you should consider creating the file client-side. If you can help it, you don't want your server having to dynamically generate 100MB .csv files each time a user clicks the link.
You could write a script that only generates the .csv client-side if the browser is not mobile and there is web-worker support. If either of those conditions are not met, you could fall back to having your server do it.
Ultimately, your answer is going to really depend on the requirements / context of this project. Try to cache the results where possible, and use common sense. Good luck :)
I want to prototype a server component that accepts and handles file uploads from a mobile application (should not matter what platform the phone's running). The files in question are most likely just photos, so the file size should not be that big.
I'm looking for technology/tool/library recommendations or pointers. I'm not sure where to start. I haven't set up anything on the server-side yet so I'm not sure what I don't know. Something Java-related would definitely ease me into setting up the prototype quickly.
Thanks in advance.
You can use servlet on server-side to receive files, to generate unique names for these files and keep them preferable on file system (not on database). For example you can see
http://www.servletworld.com/servlet-tutorials/servlet-file-upload-example.html
I am trying to upload a lot of files from a browser to a webserver that I have total control of. I've tried to use SWFUpload, but the problem is that my browser hangs when I am uploading a file. The files are supposed to be of any length (up to 2GB due to other restrictions, of course).
While the upload is in progress the browser hangs and the GUI is not updated. When the file is finally uploaded the GUI responds again. In firefox the entire browser stalls and in Chrome just the tab.
I thought it might have something to do with flash trying to inject the entire file in to the memory first, but that does not seem to be the case (although different test cases yielded different results, but I'm not sure that it's because of that reason).
An applet is also an option, although flash is preferred. The reason I don't want to use a form is because you can select multiple in flash (also possible with Java, I assume) which is a nice feature that I'm going for.
Any ideas? Thought?
(math|tob)ias
If you're using linux, this is a known problem. For Windows users this doesn't happen. Otherwise, you'll have to stick to ajax (the best way is with the jQuery forms plugin).
http://www.plupload.com/, this one looks interesting, but I haven't looked into it. You may want to give it a try.
Try an AJAX one:
http://sourceforge.net/projects/uber-uploader/
Lighter on the resources and more platform-independent.
client-side I like this jquery plugin:
http://valums.com/ajax-upload/
if you need examples of server-side handling:
http://php.net/manual/en/features.file-upload.php
I've use this for my projects... http://jupload.sourceforge.net/ I found I had better results though when I used it in FTP mode. I had issues with http uploads. It does support multiple files which is nice. You can upload the files into an "incoming directory" and then process them later when the transfer is finished.
Check SWFUpload swfupload.org
and there is a descussion about this point:
http://swfupload.org/forum/generaldiscussion/173
I'm working on a web application. There is one place where the user can upload files with the HTTP protocol. There is a choice between the classic HTML file upload control and a Java applet to upload the files.
The classic HTML file upload isn't great because you can only select one file at a time, and it's quite hard to get any progress indication during the actual upload (I finally got it using a timer refreshing a progress indicator with data fetched from the server via an AJAX call). The advantage: it's always working.
With the Java applet I can do more things: select multiple files at once (even a folder), compress the files, get a real progress bar, drag'n'drop files on the applet, etc...
BUT there are a few drawbacks:
it's a nightmare to get it to work properly on Mac Safari and Mac Firefox (Thanks Liveconnect)
the UI isn't exactly the native UI and some people notice that
the applet isn't as responsive as it should (could be my fault, but everything looks ok to me)
there are bugs in the Java UrlConnection class with HTTPS, so I use the Apache common HTTP client to do the actual HTTP upload. It's quite big a package and slows down the download of the .jar file
the Apache common HTTP client has sometimes trouble going through proxies
the Java runtime is quite big
I've been maintaining this Java applet for a while but now I'm fed up with all the drawbacks, and considering writing/buying a completely new component to upload theses files.
Question
If you had the following requirements:
upload multiple files easily from a browser, through HTTP or HTTPS
compress the files to reduce the upload time
upload should work on any platform, with native UI
must be able to upload huge files, up to 2gb at least
you have carte blanche on the technology
What technology/compontent would you use?
Edit :
Drag'n'Drop of files on the component would be a great plus.
It looks like there are a lot of issues related to bugs with the Flash Player (swfupload known issues). Proper Mac support and upload through proxies with authentication are options I can not do without. This would probably rule out all Flash-based options :-( .
I rule out all HTML/Javascript-only options because you can't select more than one file at a time with the classic HTML control. It's a pain to click n-times the "browse" button when you want to select multiple files in a folder.
I implemented something very recently in Silverlight.
Basically uses HttpWebRequest to send a chunk of data to a GenericHandler.
On the first post, 4KB of data is sent. On the 2nd chunk, I send another 4K chunk.
When the 2nd chunk is received, I calculate the round trip it took between first and 2nd chunk and so now
the 3rd chunk when sent will know to increase speed.
Using this method I can upload files of ANY size and I can resume.
Each post I send along this info:
[PARAMETERS]
[FILEDATA]
Here, parameters contain the following:
[Chunk #]
[Filename]
[Session ID]
After each chunk is received, I send a response back to my Silverlight saying how fast it took so that it can now send a larger
chunk.
Hard to put my explaination without code but that's basically how I did it.
At some point I will put together a quick writeup on how I did this.
I've never used it with files of 2GB in size, but the YUI File Uploader worked pretty well on a previous project. You may also be interested in this jQuery Plugin.
That said, I still think the Java Applet is the way to go. I think you'll end up with less portability and UI issues than you expect and Drag/Drop works great. For the record, Box.net uses a Java Applet for their multi-file quick uploads.
OK this is my take on this
I did some testing with swfupload, and I have my previous experience with Java, and my conclusion is that whatever technology is used there is no perfect solution to do uploads on the browser : you'll always end up with bugs when uploading huge files, going through proxies, with ssl, etc...
BUT :
a flash uploader (a la swfupload) is really lightweight, doesn't need authorization from the user and has a native interface which is REALLY cool, me thinks
a java uploader needs authorization but you can do whatever you want with the files selected by the user (aka compression if needed), and drag and drop works well. Be prepared for some epic bugs debuggin' though.
I didn't get a change to play with Silverlight as long as I'd like maybe that's the real answer, though the technology is still quite young so ... I'll edit this post if I get a chance to fiddle a bit with Silverlight
Thanks for all the answers !!
There are a number of free flash components that exist with nice multiple file upload capability. They make use of ActionScripts FileReference class with a PHP (or whatever) receiver on the server side. Some have recently broken with the launch of FP10 but I know for certain that swfupload will work :)
Hope this helps!
What about these two
Jupload
http://jupload.sourceforge.net/
and
jumploader
http://jumploader.com/
Both are java applets but they are also both really easy to use and implement.
what about google gears?
There are HTTP/HTTPS upload controls that allow multi-file upload. Here is one from Telerik, which I have found to be solid and reliable. The latest version looks to have most if not all of your feature requirements.
You can upload multiple files with HTTP forms as well, as Dave already pointed out, but if you're set on using something beyond what HTTP and Javascript offers I would heavily consider Flash. There are even some pre-existing solutions for it such as MultiPowUpload and it offers many of the features you're looking for. It's also easier to obtain progress information using a Flash client than with AJAX calls from Javascript since you have a little more flexibility.
You may check the Apache Commons FileUpload package. It allows you to upload multiple files, monitor the progress of the upload, and more. You can find more information here:
http://commons.apache.org/fileupload/
http://commons.apache.org/fileupload/using.html
Good luck