Im trying to stream a zip file as output for the user to download. I create a zipOutputStream, and have tried to send it into the vertx http response.
However this download a corrupt zip file or hangs
public void download(RoutingContext routingContext) {
try (ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(baos)) {
for (int i = 0; i < 2; i++) {
String fileName = "test" + i + ".csv";
File tempFile = createTempFile("test" + i);
tempFile.deleteOnExit();
InputStream is = new FileInputStream(tempFile);
byte[] buffer = new byte[is.available()];
is.read(buffer);
ZipEntry entry = new ZipEntry(fileName);
zos.putNextEntry(entry);
try {
zos.write(buffer, 0, buffer.length);
zos.closeEntry();
String b64String = Base64.encodeBase64String(baos.toByteArray());
routingContext.response()
.setChunked(true)
.putHeader(HttpHeaders.CONTENT_TYPE, "application/zip")
.putHeader("Content-Disposition", "attachment; filename=\"test.zip\"")
.putHeader(HttpHeaders.TRANSFER_ENCODING, "chunked")
.putHeader(HttpHeaders.CONTENT_LENGTH, Long.toString(b64String.length()));
routingContext.response().write(b64String);
routingContext.response().end();
routingContext.response().close();
} finally {
baos.close();
}
}
}
catch(Exception ex) {
}
}
I worked out the solution. I wrapped the vertx httpResponse in an outputStream, and pushed that into zipOutputStream. And it worked, by setting chunked to true.
For very big files, you can use streaming to avoid loading them completely into RAM:
AsyncInputStream fileContent = new AsyncInputStream(routingContext.vertx(), fileInputStream);
fileContent.endHandler((ev) -> {
routingContext.response().end();
});
Pump.pump(fileContent, routingContext.response()).start();
Unfortunately, AsyncInputStream (wrapper for InputStream into ReadStream) is not yet part of VertX framework so you have to implement it yourself or find some open source implementation.
Related
I am trying to download a PDF file with HttpClient, it is downloading the PDF file but pages are blank. I can see the bytes on console from response if I print them. But when I try to write it to file it is producing a blank file.
FileUtils.writeByteArrayToFile(new File(outputFilePath), bytes);
However the file is showing correct size of 103KB and 297KB as expected but its just blank!!
I tried with Output stream as well like:
FileOutputStream fileOutputStream = new FileOutputStream(outFile);
fileOutputStream.write(bytes);
Also tried to write with UTF-8 coding like:
Writer out = new BufferedWriter( new OutputStreamWriter(
new FileOutputStream(outFile), "UTF-8"));
String str = new String(bytes, StandardCharsets.UTF_8);
try {
out.write(str);
} finally {
out.close();
}
Nothing is working for me. Any suggestion is highly appreciated..
Update: I am using DefaultHttpClient.
HttpGet httpget = new HttpGet(targetURI);
HttpResponse response = null;
String htmlContents = null;
try {
httpget = new HttpGet(url);
response = httpclient.execute(httpget);
InputStreamReader dataStream=new InputStreamReader(response.getEntity().getContent());
byte[] bytes = IOUtils.toByteArray(dataStream);
...
You do
InputStreamReader dataStream=new InputStreamReader(response.getEntity().getContent());
byte[] bytes = IOUtils.toByteArray(dataStream);
As has already been mentioned in comments, using a Reader class can damage binary data, e.g. PDF files. Thus, you should not wrap your content in an InputStreamReader.
As your content can be used to construct an InputStreamReader, though, I assume response.getEntity().getContent() returns an InputStream. Such an InputStream usually can be directly used as IOUtils.toByteArray argument.
So:
InputStream dataStream=response.getEntity().getContent();
byte[] bytes = IOUtils.toByteArray(dataStream);
should already work for you!
Here is a method I use to download a PDF file from a specific URL. The method requires two string arguments, an url string (example: "https://www.ibm.com/support/knowledgecenter/SSWRCJ_4.1.0/com.ibm.safos.doc_4.1/Planning_and_Installation.pdf") and a destination folder path to download the PDF file (or whatever) into. If the destination path does not exist within the local file system then it is automatically created:
public boolean downloadFile(String urlString, String destinationFolderPath) {
boolean result = false; // will turn to true if download is successful
if (!destinationFolderPath.endsWith("/") && !destinationFolderPath.endsWith("\\")) {
destinationFolderPath+= "/";
}
// If the destination path does not exist then create it.
File foldersToMake = new File(destinationFolderPath);
if (!foldersToMake.exists()) {
foldersToMake.mkdirs();
}
try {
// Open Connection
URL url = new URL(urlString);
// Get just the file Name from URL
String fileName = new File(url.getPath()).getName();
// Try with Resources....
try (InputStream in = url.openStream(); FileOutputStream outStream =
new FileOutputStream(new File(destinationFolderPath + fileName))) {
// Read from resource and write to file...
int length = -1;
byte[] buffer = new byte[1024]; // buffer for portion of data from connection
while ((length = in.read(buffer)) > -1) {
outStream.write(buffer, 0, length);
}
}
// File Successfully Downloaded");
result = true;
}
catch (MalformedURLException ex) { ex.printStackTrace(); }
catch (IOException ex) { ex.printStackTrace(); }
return result;
}
I use NanoHTTPD as web server in my Android APP, I hope to compress some files and create a InputStream in server side, and I download the InputStream in client side using Code A.
I have read Code B at How to zip and unzip the files?, but how to create a ZIP InputStream in Android without creating a ZIP file first?
BTW, I don't think Code C is good way, because it make ZIP file first, then convert ZIP file to FileInputStream , I hope to create a ZIP InputStream directly!
Code A
private Response ActionDownloadSingleFile(InputStream fis) {
Response response = null;
response = newChunkedResponse(Response.Status.OK, "application/octet-stream",fis);
response.addHeader("Content-Disposition", "attachment; filename="+"my.zip");
return response;
}
Code B
public static void zip(String[] files, String zipFile) throws IOException {
BufferedInputStream origin = null;
ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(new FileOutputStream(zipFile)));
try {
byte data[] = new byte[BUFFER_SIZE];
for (int i = 0; i < files.length; i++) {
FileInputStream fi = new FileInputStream(files[i]);
origin = new BufferedInputStream(fi, BUFFER_SIZE);
try {
ZipEntry entry = new ZipEntry(files[i].substring(files[i].lastIndexOf("/") + 1));
out.putNextEntry(entry);
int count;
while ((count = origin.read(data, 0, BUFFER_SIZE)) != -1) {
out.write(data, 0, count);
}
}
finally {
origin.close();
}
}
}
finally {
out.close();
}
}
Code C
File file= new File("my.zip");
FileInputStream fis = null;
try
{
fis = new FileInputStream(file);
} catch (FileNotFoundException ex)
{
}
ZipInputStream as per the documentation ZipInputStream
ZipInputStream is an input stream filter for reading files in the ZIP file format. Includes support for both compressed and uncompressed entries.
Earlier I answered to this question in a way that it is not possible using ZipInputStream. I am Sorry.
But after investing some time I found that it is possible as per the below code
It is very much obvious that since you are sending files in zip format
over the network.
//Create proper background thread pool. Not best but just for solution
new Thread(new Runnable() {
#Override
public void run() {
// Moves the current Thread into the background
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_BACKGROUND);
HttpURLConnection httpURLConnection = null;
byte[] buffer = new byte[2048];
try {
//Your http connection
httpURLConnection = (HttpURLConnection) new URL("https://s3-ap-southeast-1.amazonaws.com/uploads-ap.hipchat.com/107225/1251522/SFSCjI8ZRB7FjV9/zvsd.zip").openConnection();
//Change below path to Environment.getExternalStorageDirectory() or something of your
// own by creating storage utils
File outputFilePath = new File ("/mnt/sdcard/Android/data/somedirectory/");
ZipInputStream zipInputStream = new ZipInputStream(new BufferedInputStream(httpURLConnection.getInputStream()));
ZipEntry zipEntry = zipInputStream.getNextEntry();
int readLength;
while(zipEntry != null){
File newFile = new File(outputFilePath, zipEntry.getName());
if (!zipEntry.isDirectory()) {
FileOutputStream fos = new FileOutputStream(newFile);
while ((readLength = zipInputStream.read(buffer)) > 0) {
fos.write(buffer, 0, readLength);
}
fos.close();
} else {
newFile.mkdirs();
}
Log.i("zip file path = ", newFile.getPath());
zipInputStream.closeEntry();
zipEntry = zipInputStream.getNextEntry();
}
// Close Stream and disconnect HTTP connection. Move to finally
zipInputStream.closeEntry();
zipInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}finally {
// Close Stream and disconnect HTTP connection.
if (httpURLConnection != null) {
httpURLConnection.disconnect();
}
}
}
}).start();
I am trying to unzip a response from a .net middleware. The response has been ziped using GZipStream.
GZipStream zipStream = new GZipStream(fileStream, CompressionMode.Compress, true);
when I used GZIPInputStream in java to unzip the file. I am getting an IOException with message "not in zip format" in the following code.
GZIPInputStream gzin = new GZIPInputStream(response);
I tried this too.
ByteArrayInputStream memstream = new ByteArrayInputStream(buffer2);
GZIPInputStream gzin = new GZIPInputStream(memstream);
Any help or suggestions are welcomed.
Thanks in advance
Try something like this
GZIPInputStream gis = null;
FileOutputStream fos = null;
try {
gis = new GZIPInputStream(new FileInputStream("pathOfTheGZipFile"));
fos = new FileOutputStream("pathOfDecompressedFile");
byte[] buffer = new byte[gis.available()];
int len;
while((len = gis.read(buffer)) != -1){
fos.write(buffer, 0, len);
}
} catch (IOException e) {
e.printStackTrace();
}
finally {
fos.close();
gis.close();
}
I have finally figured out the solution, In the response returned by the server first few bytes were not zipped so if any one is facing the same issue you just need to check the bytes. After I removed those bytes from response. It started working.
I am sending a request XML to the URL and receiving a zip file to the given path.
Sometimes I'm facing troubles when the bandwidth is low this zip file, most likely 120MB size is not getting downloaded properly. And getting an error when extracting the zip file. Extracting happens from the code as well. When I download in high bandwidth this file gets download without issue.
I'm looking for a solution without making the bandwidth high, from program level are there any ways to download this zip file, may be part by part or something like that? Or anyother solution that you all are having is highly appreciated.
Downloading :
url = new URL(_URL);
sc = (HttpURLConnection) url.openConnection();
sc.setDoInput(true);
sc.setDoOutput(true);
sc.setRequestMethod("POST");
sc.connect();
OutputStream mOstr = sc.getOutputStream();
mOstr.write(request.getBytes());
InputStream in = sc.getInputStream();
FileOutputStream out = new FileOutputStream(path);
int count;
byte[] buffer = new byte[86384];
while ((count = in.read(buffer,0,buffer.length)) > 0)
out.write(buffer, 0, count);
out.close();
Extracting :
try {
ZipFile zipFile = new ZipFile(path+zFile);
Enumeration<?> enu = zipFile.entries();
while (enu.hasMoreElements()) {
ZipEntry zipEntry = (ZipEntry) enu.nextElement();
String name = path+"/data_FILES/"+zipEntry.getName();
long size = zipEntry.getSize();
long compressedSize = zipEntry.getCompressedSize();
System.out.printf("name: %-20s | size: %6d | compressed size: %6d\n", name, size, compressedSize);
File file = new File(name);
if (name.endsWith("/")) {
file.mkdirs();
continue;
}
File parent = file.getParentFile();
if (parent != null) {
parent.mkdirs();
}
InputStream is = zipFile.getInputStream(zipEntry);
FileOutputStream fos = new FileOutputStream(file);
byte[] bytes = new byte[86384];
int length;
while ((length = is.read(bytes)) >= 0) {
fos.write(bytes, 0, length);
}
is.close();
fos.close();
}
zipFile.close();
} catch (Exception e) {
log("Error in extracting zip file ");
e.printStackTrace();
}
I have an application where Service A will provide a zipped data to Service B. And service B needs to unzip it.
Service A has an exposes method getStream and it gives ByteArrayInputStream as output and the data init is zipped data.
However passing that to GzipInputStream gives Not in Gzip format exception.
InputStream ins = method.getInputStream();
GZIPInputStream gis = new GZIPInputStream(ins);
This gives an exception. When the file is dumped in Service A the data is zipped. So getInputStream gives the zipped data.
How to process it ans pass it to the GzipInputStream?
Regards
Dheeraj Joshi
If it zipped, then you must use ZipInputstream.
It does depend on the "zip" format. There are multiple formats that have the zip name (zip, gzip, bzip2, lzip) and different formats call for different parsers.
http://en.wikipedia.org/wiki/List_of_archive_formats
http://www.codeguru.com/java/tij/tij0115.shtml
http://docstore.mik.ua/orelly/java-ent/jnut/ch25_01.htm
If you are using zip then try this code:
public void doUnzip(InputStream is, String destinationDirectory) throws IOException {
int BUFFER = 2048;
// make destination folder
File unzipDestinationDirectory = new File(destinationDirectory);
unzipDestinationDirectory.mkdir();
ZipInputStream zis = new ZipInputStream(is);
// Process each entry
for (ZipEntry entry = zis.getNextEntry(); entry != null; entry = zis
.getNextEntry()) {
File destFile = new File(unzipDestinationDirectory, entry.getName());
// create the parent directory structure if needed
destFile.getParentFile().mkdirs();
try {
// extract file if not a directory
if (!entry.isDirectory()) {
// establish buffer for writing file
byte data[] = new byte[BUFFER];
// write the current file to disk
FileOutputStream fos = new FileOutputStream(destFile);
BufferedOutputStream dest = new BufferedOutputStream(fos,
BUFFER);
// read and write until last byte is encountered
for (int bytesRead; (bytesRead = zis.read(data, 0, BUFFER)) != -1;) {
dest.write(data, 0, bytesRead);
}
dest.flush();
dest.close();
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
is.close();
}
public static void main(String[] args) {
UnzipInputStream unzip = new UnzipInputStream();
try {
InputStream fis = new FileInputStream(new File("test.zip"));
unzip.doUnzip(fis, "output");
} catch (IOException e) {
e.printStackTrace();
}
}