In my web application I have a link which, when clicked, invokes an external web service to retrieve a download URL for a file.
I need to send back to client the file which is beyond this URL, instead of the download URL retrieved from the web service. If possible, I would also like to do it without having to download the file on my server beforehand.
I've found this question about a similar task, but which used PHP with the readfile() function.
Is there a similar way to do this in Java 8?
If you doesn't even want to handle that file you should answer the request with a redirect (eg HTTP 301 or 302). If you want to handle the file you should read the file in a byte buffer and send it to the client which would make the transfer slower.
Without seeing your implementation so far, this is my best suggest.
I have a question regarding file upload, which is more related to how it works rather than a code issue. I looked on the internet, but I couldn't find a proper answer.
I have a web application running on tomcat, which handles file uploads (through a servlet). Let's say I want now to upload huge files (> 1 Gb). My understading was that the multipart content of the HTTP request was available in my servlet once the whole file was actually transfered.
My question is where the content of the request is actually stored ? When one calls HttpServletRequest.getParts() an InputStream is available on the Part object. However, where is the stream reading from ? Does Tomcat store it somewhere ?
I guess this might not be clear enough, so I'll update the post according to your comments, if any.
Thanks
Tomcat stores Parts in "X:\some\path\Tomcat 7.0\temp" (/some/path/apache-tomcat-7.0.x/temp) directory.
when a multipart request is parsed, if the size of a single part exceed a threshold, a temporary file is created for that part.
your servlet/jsp will be invoked when transfer of all parts has been completed.
when the request is destroyed all temporary files are deleted as well.
if you are interested in the multipart parse phase, take a look at apache commons-fileupload (specifically ServletFileUpload.parseRequest()), tomcat is based on a variant of that
UPDATE
you can configure it as a java arg, ie in windows:
The InputStream will typically read from a temporary file which is created by the multipart framework during the request. The temp file is normally stored in the application server's temporary area - as specified by the servlet context attribute javax.servlet.context.tempdir. In Tomcat this is somewhere beneath $CATALINA_HOME/work. The file will be deleted once the request completes.
For small file sizes, the multipart framework may keep the whole upload in memory - in which case the InputStream will be reading directly from memory.
If you're using Spring's CommonsMultipartResolver then you can set the maximum upload size allowed in memory via the maxInMemorySize property. If an upload is bigger than this, then it will be stored as a temp file on disk.
I think we should step back for a moment and give a thought on the web infrastructure. First of all the HTTP transmits text data, so binary information encoded in base 64 so that data won't get messed up. This ends up leading to large amouts of data and this gives birth to the multipart form, which breaks datum into parts of encoded text with special markers that allow the server to assembly everything together. But to use this data we have to decode it first, and to do that I have to use the multiple parts of the form.
[a break so we can breath]
Continuing, so the browser needs to send lots of datum (1GB as you mentioned in your example), this datum is encoded with base64 and then separated into pieces (the multipart form) with its markers, then the browser starts to send the pieces to the server, but the server only returns the HTTP RESPONSE once it has finished receiving and processing the HTTP REQUEST (or if a timeout occurs, which incurs in an error on the browser screen).
What can assume here is that Tomcat could (I didn't check the internals) start decoding each part of the multipart that has already arrieved (either from the temp file or from memory) passing the inputstream to the user, since the inputstrem reading is a blocking operation the server would wait for the next piece of data to pass to Tomcat, which in turn would pass it to the program that is processing the data.
Once all data has reached the server the program would prepare the response that Tomcat would return to the browser completing the HTTP Request-Response cycle and closing the connection (since HTTP is a connectionless protocol).
Hope it helps :)
Tomcat follows the Servlet 3.0 specification which allows you to specify things such as how large of a multipart "part" can be before it gets stored (temporarily) on the disk, where temporary files will be written, what the maximum size of a file is, and what the maximum size of the whole request can be. You can find all kinds of good information about configuring multipart uploads (in Tomcat or any other spec-3.0-compliant server) here and here.
Tomcat's implementation specifics aren't terribly relevant: it adheres to the spec. If the file to be uploaded is smaller than the threshold set, then you should be able to read the bytes of the file from memory (i.e. no disk involved). If the file is larger, then it will be written to disk, first (in its entirety) and then you can get the bytes from the container.
So if you want to receive a 1GiB file and don't have that kind of memory available (I wouldn't recommend allowing clients to fill-up your heap with 1GiB of uploaded data for each upload... easy DoS if you just start several simultaneous 1GiB uploads and you are toast), then Tomcat (or whatever container you are using) will read the file (again, in its entirety) onto the disk, and when your servlet gets control, you can read the bytes back from that file.
Note that the container must process the entire multipart request before any of your code really runs. That's to prevent you from breaking anything by partially-reading the request's InputStream or anything like that. Processing multipart requests is non-trivial, and it's easy to break things.
If you want to be able to stream large files for processing (e.g. huge XML files that can be processed serially), then you are going to want to handle the multipart parsing yourself. That way, you don't need a huge amount of heap to buffer the file and you don't need to store the file on the disk before you start processing it. (If this is your use-case, I suggest using HTTP PUT or HTTP POST and not using multipart requests.)
(It's worth mentioning that base64 encoding is not even mentioned in any specification for multipart processing. A few folks have mentioned base64 here, but I've never seen a standard web client use base64 for uploading a file using multipart/form-data. HTTP handles binary uploads just fine, thanks.)
Here is it
User's browser composes http multiple parts request
Tcp/ip stack of user's OS slices them into packets
Routers over the internet pass those packet to your server
Tcp/ip stack of your server's OS get back payloads and passes them
to tcp port listener
Tomcat http connector decodes http post request from tcp data
(source code is
https://github.com/apache/tomcat/tree/trunk/java/org/apache/coyote )
Tomcat http connector wrap a Http Request and eventually forwards to
your servlet (https://github.com/apache/tomcat/blob/trunk/java/org/apache/catalina/connector/Request.java)
Before and while your code reading the content of Http Request, tomcat will buffer the http request body internally
Tomcat will not parse multiple parts body before you call request.getParts() (https://github.com/apache/tomcat/blob/trunk/java/org/apache/catalina/connector/Request.java#L2561), thus no temp file for parts before calling.
Tomcat stores files uploaded into location pointing by #MultipartConfig annotation in your servlet code, unless your code doesn't provide it and allowCasualMultipartParsing is set (http://tomcat.apache.org/tomcat-7.0-doc/config/context.html#Common_Attributes)
Considering allowCasualMultipartParsing is false by default, you should not worry about where tomcat stores file though it is easy to dig out.
I mention 1~5 because it is important to understand the stream returns by request.getInputStream() which is required before Servlet 3.x request.getParts() feature. Typically, tomcat will deliver the request to web app very soon, it is not necessary to wait client side to finish uploading, thus tomcat need not buffer a lot of data. I have left java server side for some years, before JSR-000315 is approved :-)
I want to make a system where java client programs send images to a central server. The central server saves them and runs a website that uses these images.
How should I send the images and how should I receive them? Can I use the same webserver for receiving and displaying the website?
You need 3 things:
Upload client Need to know how to do multipart upload. See here
Upload Server There are a couple of ways. Apache Commons Upload is my pet.
Displaying File It's easy. If the files are uploaded somewhere under your web-app directory outside of WEB-INF directory. Just give the path like http://your/apps/base/url/folderName and the listing will come-up for download. There are ways to secure that. But I donot think you need to know that at this stage. By the way this may help.
And yes, same server can be used to upload and display (download) the images/files.
Hope this helps.
I dont have any experience for web services. please give me some suggestion about the task below.
the task is:
users will send a txt file (size should less than 20K ) from a .NET application, I need to write a web services which runs by jboss 5.x to read this file and edit this file and send the file back to .NET UI to display the edited version.
question is that if the txt file is just text string or binary string, are there any restriction of the string length? if it's binary string, can I need to use BinaryReader class to read it? or not need special reader to read it? (this could be a dumn question :P)
what if the .NET application can save the file on either the .NET application server or some shared server location, send a download URL to web serivces, can web services download it and read it? JBoss will be run on Linux sever.
After edit the file, how do I send it back?
Thanks for your help!
It's hard to find reliable information about limitations in size, this depends on the libraries you use.
I would rather use an attachment than a single string containing the whole file Handling attachments in SOAP
For .Net see Add Attachments to a SOAP Message by Using DIME
After edit the file, how do I send it
back?
Request processFile(File)
Response UniqueId
Request getProcessedFile(UniqueId)
Response Edited File
Is there any way of just getting the content of the browsed file without any upload/file transfer operations? I currently use ICEFaces inputFile component but I do not need the default uploading operation of the file.
Thanks.
That's not possible. The client needs to send (upload) the file content along the request body to the server side whenever you want to have the file content at the server side.
If you'd expect that you can solve this by passing only the file path around and use the usual java.io.File stuff and so on, then you're on the wrong track. Imagine that I am the client and I have a c:/passwords.txt, how would you as being the server at the other end of the network ever get its content by java.io.File?
I don't thnik this is possible. Browsers do not allow any file transfer from the client to the server without user interaction.
Tough, if you do not stick to IceFaces, it may be possible to achieve this by writing an applet, wich is granted the necessary permissions.