Develop Reference

Stream file from URL to File without storing it in the memory

I want to download a file from a URL and store it into the file system. However I have memory limitation and I don't want to store it in the memory before. I am not a java expert and I am a bit lost with all the class InputStream, BufferedReader, FileOutputStream, etc. Could you help me please ?
For now I have:
URLConnection ucon = url.openConnection();
ucon.connect();
InputStream is = ucon.getInputStream();
// Create a reader for the input stream.
BufferedReader br = new BufferedReader(isr);
// ?
FileOutputStream fos = context.openFileOutput(FILENAME, Context.MODE_PRIVATE);
// Here the content can be too big for the memory...
fos.write(content.getBytes());
fos.close();
Please, could you give me some clue ? I was thinking also to read it chunk by chunk, but I am not sure what would be the easiest with java...

you can use apache commons
org.apache.commons.io.FileUtils.copyURLToFile(URL, File)
I guess it may not work on android
I use this code
InputStream input = connection.getInputStream();
byte[] buffer = new byte[4096];
int cnt = - 1;
OutputStream output = new FileOutputStream(file);
while ( (cnt = input.read(buffer)) != -1)
{
output.write(buffer, 0, cnt);
}
output.close();

Downloading files using servlets

hi i have tried the following java codes which works fine if i use them as a java application but when i use the same code in my servlet page they dont work means i am not able to download the files. Please suggest what changes should i do so that i can download the file using Servlets.
a.
java.io.BufferedInputStream in = new java.io.BufferedInputStream(new java.net.URL("http://169.254.174.150:8084/WebApplication1/files/check.txt").openStream());
File f1 = new File("D:\\a.txt");
java.io.FileOutputStream fos = new java.io.FileOutputStream(f1);
java.io.BufferedOutputStream bout = new BufferedOutputStream(fos, 1024);
byte data[] = new byte[1024];
while (in.read(data, 0, 1024) >= 0) {
bout.write(data);
}
bout.close();
in.close();
}
b. http://www.javabeat.net/examples/2012/04/13/download-file-from-http-https-server-using-java/

One of the older JavaBeat examples like the one you specified can be found here
I found other solutions too but this seems to be the most comprehensive.

Couple of things, insetad of writing it to a file try wrting the data directly to the responce. Before writing data you will have to set the following parameters to the responce
//byte[] filedata = ; intialize your file contents
String filename = "a.txt";
// set the header information in the response.
res.setHeader("Content-Disposition", "attachment; filename=\"" + filename + "\";");
res.setContentType("application/x-unknown");
ByteArrayInputStream byteStream = new ByteArrayInputStream(filedata);
BufferedInputStream bufStream = new BufferedInputStream(byteStream);
ServletOutputStream responseOutputStream = res.getOutputStream();
int data = bufStream.read();
while (data != -1)
{
responseOutputStream.write(data);
data = bufStream.read();
}
bufStream.close();
responseOutputStream.close();
where res is a HttpServletResponse object. After this you can write data to responseOutputStream.

Java servlet and IO: Create a file without saving to disk and sending it to the user

I`m hoping can help me out with a file creation/response question.
I know how to create and save a file. I know how to send that file back to the user via a ServletOutputStream.
But what I need is to create a file, without saving it on the disk, and then send that file via the ServletOutputStream.
The code above explains the parts that I have. Any help appreciated. Thanks in Advance.
// This Creates a file
//
String text = "These days run away like horses over the hill";
File file = new File("MyFile.txt");
Writer writer = new BufferedWriter(new FileWriter(file));
writer.write(text);
writer.close();
// Missing link goes here
//
// This sends file to browser
//
InputStream inputStream = null;
inputStream = new FileInputStream("C:\\MyFile.txt");
byte[] buffer = new byte[8192];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int bytesRead;
while ( (bytesRead = inputStream.read(buffer)) != -1)
baos.write(buffer, 0, bytesRead);
response.setContentType("text/html");
response.addHeader("Content-Disposition", "attachment; filename=Invoice.txt");
byte[] outBuf = baos.toByteArray();
stream = response.getOutputStream();
stream.write(outBuf);

You don't need to save off a file, just use a ByteArray stream, try something like this:
inputStream = new ByteArrayInputStream(text.getBytes());
Or, even simpler, just do:
stream.write(text.getBytes());
As cHao suggests, use text.getBytes("UTF-8") or something similar to specify a charset other than the system default. The list of available charsets is available in the API docs for Charset.

URL Connection (FTP) in Java - Simple Question

I have a simple question. I'm trying to upload a file to my ftp server in Java.
I have a file on my computer, and I want to make a copy of that file and upload it. I tried manually writing each byte of the file to the output stream, but that doesn't work for complicated files, like zip files or pdf files.
File file = some file on my computer;
String name = file.getName();
URL url = new URL("ftp://user:password#domain.com/" + name +";type=i");
URLConnection urlc = url.openConnection();
OutputStream os = urlc.getOutputStream();
//then what do I do?
Just for kicks, here is what I tried to do:
OutputStream os = urlc.getOutputStream();
BufferedReader br = new BufferedReader(new FileReader(file));
String line = br.readLine();
while(line != null && (!line.equals(""))) {
os.write(line.getBytes());
os.write("\n".getBytes());
line = br.readLine();
}
os.close();
For example, when I do this with a pdf and then try and open the pdf that I run with this program, it says an error occurred when trying to open the pdf. I'm guessing because I am writing a "\n" to the file? How do I copy the file without doing this?

Do not use any of the Reader or Writer classes when you're trying to copy the byte-for-byte exact contents of a binary file. Use these only for plain text! Instead, use the InputStream and OutputStream classes; they do not interpret the data at all, while the Reader and Writer classes interpret the data as characters. For example
OutputStream os = urlc.getOutputStream();
FileInputStreamReader fis = new FileInputStream(file);
byte[] buffer = new byte[1000];
int count = 0;
while((count = fis.read(buffer)) > 0) {
os.write(buffer, 0, count);
}
Whether your URLConnection usage is correct here, I don't know; using Apache Commons FTP (as suggested elsewhere) would be an excellent idea. Regardless, this would be the way to read the file.

Use a BufferedInputStream to read and BufferedOutputStream to write. Take a look at this post: http://www.ajaxapp.com/2009/02/21/a-simple-java-ftp-connection-file-download-and-upload/
InputStream is = new FileInputStream(localfilename);
BufferedInputStream bis = new BufferedInputStream(is);
OutputStream os =m_client.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
byte[] buffer = new byte[1024];
int readCount;
while( (readCount = bis.read(buffer)) > 0) {
bos.write(buffer, 0, readCount);
}
bos.close();

FTP usually opens another connection for data transfer.
So I am not convinced that this approach with URLConnection is going
to work.
I highly recommend that you use specialized ftp client. Apache commons
may have one.
Check this out
http://commons.apache.org/net/api/org/apache/commons/net/ftp/FTPClient.html

How do I get the InputStream of decompressed data from an InputStream of GZIPed data?

I call a service which returns a gzipped file. I have the data as an InputStream (courtesy of javax.activation.DataHandler.getInputStream();) from the response.
What I would like to do is, without writing anything to disk, get an InputStream of the decompressed data in the file that is in the archive. The compressed file in this case is an xml document that I am trying to unmarshal using javax.xml.bind.Unmarshaller which takes an InputStream.
I'm currently trying to write the InputStream to an OutputStream (decompressing the data) and then I'll need to write it back to an InputStream. It's not working yet so I thought I would see if there was a better (I would hope so) approach.
I can write the initial InputStream to disk and get a gz file, and then read that file, get the compressed file out of it and go from there but I'd rather keep it all in memory is possible.
Update 1: Here is my current (not working - get a "Not in GZIP format" exception):
ByteArrayInputStream xmlInput = null;
try {
InputStream in = dh.getInputStream(); //dh is a javax.activation.DataHandler
BufferedInputStream bis = new BufferedInputStream(in);
ByteArrayOutputStream bo = new ByteArrayOutputStream();
int bytes_read = 0;
byte[] dataBuf = new byte[4096];
while ((bytes_read = bis.read(dataBuf)) != -1) {
bo.write(dataBuf, 0, bytes_read);
}
ByteArrayInputStream bin = new ByteArrayInputStream(bo.toByteArray());
GZIPInputStream gzipInput = new GZIPInputStream(bin);
ByteArrayOutputStream out = new ByteArrayOutputStream();
dataBuf = new byte[4096];;
bytes_read = 0;
while ((bytes_read = gzipInput.read(dataBuf)) > 0) {
out.write(dataBuf, 0, bytes_read);
}
xmlInput = new ByteArrayInputStream(out.toByteArray());
If instead of writing to a ByteArrayOutputStream I write to a FileOutputStream the first time around I get a compressed file (which I can manually open to get the xml file within) and the service (eBay) says it should be a gzip file so I'm not sure why I get a "Not in GZIP format" error.
Update 2: I tried something a little different - same error ("Not in GZIP format"). Wow, I just tried to end that parenthesis with a semi-colon. Anyways, here is my second attempt, which still does not work:
ByteArrayInputStream xmlInput = null;
try {
GZIPInputStream gzipInput = new GZIPInputStream(dh.getInputStream());
ByteArrayOutputStream bo = new ByteArrayOutputStream();
int bytes_read = 0;
byte[] dataBuf = new byte[4096];
while ((bytes_read = gzipInput.read(dataBuf)) != -1) {
bo.write(dataBuf, 0, bytes_read);
}
xmlInput = new ByteArrayInputStream(bo.toByteArray());

Decorate the input stream with a GZIPInputStream.
InputStream decompressed = new GZIPInputStream(compressed);

The following code should work. Keep in mind you'll have to handle exceptions properly.
OutputStream out = null;
InputStream in = null;
try {
out = /* some output stream */;
in = new java.util.GZIPInputStream(/*some stream*/);
byte[] buffer = new byte[4096];
int c = 0;
while (( c = in.read(buffer, 0, 4096)) > 0) {
out.write(buffer, 0, c);
}
} finally {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}

Take a look at GZIPInputStream. Here's an example; the class handles this very transparently, it's almost no work to use.