I'm currently trying to create a download manager (DLManager) that you can submit links to online files and it keeps track of the progress of all the downloads. I works great for normal files but I wanted to add the ability to download and decompress zip files. I have gotten it to work with two basic steps: Initialization, and then Download. The initializations process sets up a zipInputStream and gathers the file information for the archive using FileEntry.getName and .getSize. Using this information I can calculate the total download size of the queue and any previous progress for resumed downloads. When the DLManager queue's up the link it then starts the download step by setting up another zipInputStream and using the .read method to output the files to disk.
The problem that I am having is that when I initialize the files it take a long time. I can download over three large video files in the time it takes to just initialize one zip file. I'm confused as to how the zipInputStream is working. Is it just streaming the header information first and then waiting for the .read command to download the rest or is it actually downloading the entire archive before returning the header information? If it does download everything at once is there a way to reuse the zipInputStream (return to the first entry)? I tried going as far to read the source for the zipInputStream but got lost. I would be thankful to anyone that can shed some light on this problem. Thanks.
Related
I'm downloading data from a download link (the link that e.g. a download button contains on a website). This through HttpsURLConnection class.
It is working fine so far just that I have no idea how to skip bytes when the file is for example already half downloaded.
I don't mean the InputStream.skip() method because this doesn't prevent internet use (it just skips bytes while downloading).
What I want is a way to tell the server/input stream to start downloading at a certain byte of the file.
Greetings,
Lucas
I'm working on a Java program that will allow me to view images in a zip/rar file without unzipping it to a folder on my hdd. I'd like to be able to flip through them like on a normal image viewer, possibly able to zoom in/out if needed.
From what I've looked into, it'll have to be extracted even if it's just to a temporary folder, which I'm fine with as long as the program can delete it on its own after. I believe ZipFile would be something I should do more research in? Most of what I've seen deal with text documents rather than image files, so I'm not sure how to proceed in my research.
I'm looking to see if I'm on the right track or if there are any good resources/specific apis I could look into to help as I haven't done any coding in months (save for light php and html) or any java in about a year, and I've had this on the back-burner for long enough.
Thanks in advance. :)
You're on the right track with ZipFile, and I don't think you need to extract to disk before viewing the images.
A ZipFile object will give you a list of its contents with entries(). You can iterate this collection of ZipEntry objects to present a choice of which file to view, and of course filter it to known extensions if you desire.
Strangely enough it's the ZipFile object and not the the individual ZipEntry objects that will give you an InputStream for the given entry. You can read this object into a byte[] in memory and send it to the component that will be responsible for displaying the image.
One caveat is that with zip files, in order to get to the last file stored in the zip it will basically have to decompress the whole archive which can be time consuming. So it may make sense to cache files on disk or an in-memory LRU cache depending on the usage pattern.
You didn't mention if this is for a Swing application, but if it is this might be helpful for displaying the images:
http://docs.oracle.com/javase/tutorial/uiswing/components/icon.html
I developing an application like Internet Download Manager for Android.
I want to know how to download different parts of file in Android as it is done in IDM.
How can I get the metadata of file before download and how to download files in parts?
There is no username-password or any restrictions in downloading... just simple download by url.
Assuming you're using HTTP for the download, you'll want to use the HEAD http verb and RANGE http header.
HEAD will give you the filesize (if available), and then RANGE lets you download a byte range.
Once you have the filesize, divide it into roughly equal sized chunks and spawn download thread for each chunk. Once all are done, write the file chunks in the correct order.
EDIT:
If you don't know how to use the RANGE header, here's another SO answer that explains how: https://stackoverflow.com/a/6323043/1355166
I am writing a client-server program in JAVA in which I am sending a file from server to client.As the file size may be quite high therefore I decided to divide the file in 5 parts and then send it to the same client in 5 different Threads.
My Algorithm is to use Java Zip API and create a zip file of the file to be sent,then I will divide the Zip file into 5 parts.
The problem is that there is not method in [ZIP API][2] that could divide the file.
This is the tutorial that I am referring for sending files through Thread.
Anyone who can guide me is there anything wrong with my Algorithm Or do I have to do with different strategy?
You should separate the zipping part from the splitting part. If you have to send these to a client, you probably don't want to keep the complete zip file in memory while you wait for the client to request the next chunk... so the simplest approach would be to zip to disk first, and then serve that file in chunks. At that point, it really doesn't matter that it's a zip file at all - and indeed for certain files types (e.g. images, sound, video) you may not want to go via a zip file at all.
I would suggest you tell the client the file name and size, and then let the client request whatever section of the file it wants. It can then decide what chunk size to use: you just need to seek to the right bit of the file and serve as much data as the client has requested.
Breaking up the file isn't a ZIP function. You could create multiple byte arrays from the resulting zip file (by segmenting the array) and sending each segment in a different thread. This would be similar to what download managers of yesteryear would do.
The client would then have code to re-assemble the byte array in the correct order. You'd probably need to add some additional information to each segment like the correct sequence, the filename to be restored, and the number of segments expected.
There are 2 servers that are geographically very far from each other.
One server does file processing, then saves the processed file in a directory:
c:\processed\
Files can be 100-1GB in size.
The 2nd server is to download these files.
What techniques can I use to check if the file correctly downloaded?
Is a checksum all I need to do? will it hash according to the contents of the file or just the file attributes? (or what is best practise)
If the file is 1GB, will creating the checksum take a long time?
Checksum is fine to make sure that the downloaded data matches the source data. For a discussion of making it fast, see What is the fastest way to create a checksum for large files in C#.