Is it possible to download large files (>=1Gb) from a servlet to an applet using HttpClient? And what servlet-side lib is useful in this case? Is there another way to approach this?
Any server-side lib that allows you access to the raw output stream should be just fine.
Servlets or JAX-RS for example.
Get the output stream, get the input stream of your file, use a nice big buffer (4k maybe) and pump the bytes from input to output.
On the client side, your applet needs access to the file system. I assume you don't want to keep the 1GB in memory. (maybe we want to stream it to the screen, in which case you don't need elevated access).
Avoid client libraries that try to fully materialize the returned content before handing it to.
Example code here:
Streaming large files in a java servlet
Related
I am developing a Java application through which I need to read the log files present on a server and perform operations depending on the content of the logs.
Files range from 3GB up to 9GB.
Here on stack I have already read the discussion about reading large files with java, I am attaching the link:
Java reading large file discussion
In the discussion, the files are read locally,
in my case i have to retrieve and read the file on the server, is there an efficient way to achieve this?
I would like to avoid having to download files given their size.
I had thought about using URL Reader to retrieve the files, but I have doubts about the speed of execution.
The files I need to recover are under the path C:\production\LOG\file.log
Do you have any suggestions or advice?
I dont know how to do this, or whether is possible or wise, so any form of answer that points me to a library, example or reasoning will be helpful.
I need to upload and process some Java XML files (actually, XSLT files - XML Excel files).
I dont want to store the file on the server and then invoke processing on it. Instead, I want to stream the file in, and process it as a stream.
I also want to be able to process multipart file uploads, but still process that as an input stream.
I am expressly trying to avoid creating a file on disk for this.
I have a file which I need to upload to a service, and parse into relevant data. The parser and the uploader both require an InputStream. Ought I to open the file twice? I could save the file to a String, but having many of these files in memory is concerning.
EDIT: Thought I should make it clear that the parsing and uploading are entirely separate processes.
Since you are parsing it already it would be most efficient to load the file into a string. Parse it into indexes to the string, you will save memory and can just upload the string whenever you want to. This would be the most effective way, with memory but maybe not processing time.
A reply to one of the comments above.
Separate processes does not mean different threads or processes, just they do not need each other to operate.
I am writing a client-server program in JAVA in which I am sending a file from server to client.As the file size may be quite high therefore I decided to divide the file in 5 parts and then send it to the same client in 5 different Threads.
My Algorithm is to use Java Zip API and create a zip file of the file to be sent,then I will divide the Zip file into 5 parts.
The problem is that there is not method in [ZIP API][2] that could divide the file.
This is the tutorial that I am referring for sending files through Thread.
Anyone who can guide me is there anything wrong with my Algorithm Or do I have to do with different strategy?
You should separate the zipping part from the splitting part. If you have to send these to a client, you probably don't want to keep the complete zip file in memory while you wait for the client to request the next chunk... so the simplest approach would be to zip to disk first, and then serve that file in chunks. At that point, it really doesn't matter that it's a zip file at all - and indeed for certain files types (e.g. images, sound, video) you may not want to go via a zip file at all.
I would suggest you tell the client the file name and size, and then let the client request whatever section of the file it wants. It can then decide what chunk size to use: you just need to seek to the right bit of the file and serve as much data as the client has requested.
Breaking up the file isn't a ZIP function. You could create multiple byte arrays from the resulting zip file (by segmenting the array) and sending each segment in a different thread. This would be similar to what download managers of yesteryear would do.
The client would then have code to re-assemble the byte array in the correct order. You'd probably need to add some additional information to each segment like the correct sequence, the filename to be restored, and the number of segments expected.
Basically, I want to be able execute a byte stream as a file, without writing said bytes to a file.
I want to do this in a windows environment.
You could say I want a to create a file, copy the stream to the file, and open the file as an executable, but I want the file to have no physical manifestation on the disk.
Thanks
This is not possible to do using the standard Java API. You will have to rely on some OS specific native implementations, either that do the whole thing for you, or that allow you to create some sort of RAM-disk on which you can place your temporary data and execute it.
Possibly related:
write file in memory with java.nio?
Supposing you bytestream is the one of a standard exe, and you don't want to call a whole virtualisation environnement, the short answer is no because internal adresses in the code wouldn't map.
Here's a more detailled answer :
process.start() embedded exe without extracting to file first c#