Please have a look at the below code.
public void startCompress(String path,String fileName,String outputLocation,int compressType,int filSize) throws Exception
{
System.out.println("Input Location: "+path);
System.out.println("Output Location: "+outputLocation);
System.out.println(compressType);
byte[] bs=new byte[filSize];
System.out.println(filSize);
FileOutputStream fos=new FileOutputStream(outputLocation+"/test.zip");
System.out.println(fos.toString());
ZipOutputStream zos=new ZipOutputStream(fos);
ZipEntry ze = new ZipEntry(fileName);
zos.putNextEntry(ze);
FileInputStream inputStream=new FileInputStream(path);
int len;
while((len=inputStream.read(bs))>0){
zos.write(bs, 0, len);
}
inputStream.close();
zos.closeEntry();
zos.close();
}
In above code, we compress a file using java.util.zip package. But we have an issue. That is, if we select multiple files then only one file is being compressed. If we select a folder, the compression simply won't work.
How can I fix this to compress either a file, files, folder, folders, or even nested folders? Java zip package does support .zip, .tar, .tarGz and tarZ. So the solution should not be something which is limited to .zip extension as well.
Here is my solution that uses the new java.nio package. Just call zipDir giving it the path to the directory. It will create a zip file in the same location but called <directory>.zip.
private static Path buildPath(final Path root, final Path child) {
if (root == null) {
return child;
} else {
return Paths.get(root.toString(), child.toString());
}
}
private static void addZipDir(final ZipOutputStream out, final Path root, final Path dir) throws IOException {
try (DirectoryStream<Path> stream = Files.newDirectoryStream(dir)) {
for (Path child : stream) {
Path entry = buildPath(root, child.getFileName());
if (Files.isDirectory(child)) {
addZipDir(out, entry, child);
} else {
out.putNextEntry(new ZipEntry(entry.toString()));
Files.copy(child, out);
out.closeEntry();
}
}
}
}
public static void zipDir(final Path path) throws IOException {
if (!Files.isDirectory(path)) {
throw new IllegalArgumentException("Path must be a directory.");
}
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(path.toString() + ".zip"));
try (ZipOutputStream out = new ZipOutputStream(bos)) {
addZipDir(out, path.getFileName(), path);
}
}
The zip libraries for java cannot be used to compress folders in simpler way like - compress this folder.
You need to do the test if the input is folder or file by yourself. If it is a file - add it to the zip. If it is a folder - iterate the folder and add each file to the zip. For the subfolders to the same. To add more than one file to the Zip you need to create ZipEntry for each file.
You can try this code which works for me:
public static void zip(File directory, File zipfile) throws IOException {
URI base = directory.toURI();
Deque<File> queue = new LinkedList<File>();
queue.push(directory);
OutputStream out = new FileOutputStream(zipfile);
Closeable res = out;
try {
ZipOutputStream zout = new ZipOutputStream(out);
res = zout;
while (!queue.isEmpty()) {
directory = queue.pop();
for (File kid : directory.listFiles()) {
String name = base.relativize(kid.toURI()).getPath();
if (kid.isDirectory()) {
queue.push(kid);
name = name.endsWith("/") ? name : name + "/";
zout.putNextEntry(new ZipEntry(name));
} else {
zout.putNextEntry(new ZipEntry(name));
copy(kid, zout);
zout.closeEntry();
}
}
}
} finally {
res.close();
}
}
Updated from this answer, which fixes issue with each file been added to it's own directory. Also better supports Windows explorer.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
public class Test {
public static void main(String agrs[]) {
ZipUtils appZip = new ZipUtils();
appZip.zipIt(new File(source directory), new File(dest zip));
}
public static class ZipUtils {
private final List<File> fileList;
private List<String> paths;
public ZipUtils() {
fileList = new ArrayList<>();
paths = new ArrayList<>(25);
}
public void zipIt(File sourceFile, File zipFile) {
if (sourceFile.isDirectory()) {
byte[] buffer = new byte[1024];
FileOutputStream fos = null;
ZipOutputStream zos = null;
try {
// This ensures that the zipped files are placed
// into a folder, within the zip file
// which is the same as the one been zipped
String sourcePath = sourceFile.getParentFile().getPath();
generateFileList(sourceFile);
fos = new FileOutputStream(zipFile);
zos = new ZipOutputStream(fos);
System.out.println("Output to Zip : " + zipFile);
FileInputStream in = null;
for (File file : this.fileList) {
String path = file.getParent().trim();
path = path.substring(sourcePath.length());
if (path.startsWith(File.separator)) {
path = path.substring(1);
}
if (path.length() > 0) {
if (!paths.contains(path)) {
paths.add(path);
ZipEntry ze = new ZipEntry(path + "/");
zos.putNextEntry(ze);
zos.closeEntry();
}
path += "/";
}
String entryName = path + file.getName();
System.out.println("File Added : " + entryName);
ZipEntry ze = new ZipEntry(entryName);
zos.putNextEntry(ze);
try {
in = new FileInputStream(file);
int len;
while ((len = in.read(buffer)) > 0) {
zos.write(buffer, 0, len);
}
} finally {
in.close();
}
}
zos.closeEntry();
System.out.println("Folder successfully compressed");
} catch (IOException ex) {
ex.printStackTrace();
} finally {
try {
zos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
protected void generateFileList(File node) {
// add file only
if (node.isFile()) {
fileList.add(node);
}
if (node.isDirectory()) {
File[] subNote = node.listFiles();
for (File filename : subNote) {
generateFileList(filename);
}
}
}
}
}
Related
I am wondering if any one can help implementing Parallel Zip Creation using ScatterZipOutputStream . I have searched a lot but no where I am finding an example for the same.
https://commons.apache.org/proper/commons-compress/zip.html
I have tried making Zip, Zipping a directory etc with ZipArchiveOutputStream . Now, I am trying to do that in parallel.
public static void makeZip(String filename) throws IOException,
ArchiveException {
File sourceFile = new File(filename);
final OutputStream out = new FileOutputStream(filename.substring(0, filename.lastIndexOf('.')) + ".zip");
ZipArchiveOutputStream os = new ZipArchiveOutputStream(out);
os.setUseZip64(Zip64Mode.AsNeeded);
os.putArchiveEntry(new ZipArchiveEntry(sourceFile.getName()));
IOUtils.copy(new FileInputStream(sourceFile), os);
os.closeArchiveEntry();
os.close();
}
It should be able to process individual files as thread and then combine it to write the result zip.
Following is the working code of both zip and unzip:
1. Change path for sourceFolder and zipFilePath
2. Zipping only *.text type of files it can be any type or all the files
3. Unzipped files at sourceFolder/unzip/
Import following dependency in build.gradle or in pom.xml
implementation("org.apache.commons:commons-compress:1.18")
implementation("commons-io:commons-io:2.6")
Ref: https://mvnrepository.com/artifact/org.apache.commons/commons-compress/1.18
https://mvnrepository.com/artifact/commons-io/commons-io/2.6
//code
import org.apache.commons.compress.archivers.zip.*;
import org.apache.commons.compress.parallel.InputStreamSupplier;
import org.apache.commons.io.FileUtils;
import java.io.*;
import java.nio.file.Files;
import java.util.Iterator;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
public class ZipMain {
static ParallelScatterZipCreator scatterZipCreator = new ParallelScatterZipCreator();
static ScatterZipOutputStream dirs;
static {
try {
dirs = ScatterZipOutputStream.fileBased(File.createTempFile("java-zip-dirs", "tmp"));
} catch (IOException e) {
e.printStackTrace();
}
}
public static void main(String[] args) throws IOException {
String sourceFolder = "/Users/<user>/Desktop/";
String zipFilePath = "/Users/<user>/Desktop/Desk.zip";
String fileTypesToBeAddedToZip = "txt";
zip(sourceFolder, zipFilePath, fileTypesToBeAddedToZip);
unzip(zipFilePath, sourceFolder + "/unzip/");
}
private static void zip(String sourceFolder, String zipFilePath, String fileTypesToBeAddedToZip) throws IOException {
OutputStream outputStream = null;
ZipArchiveOutputStream zipArchiveOutputStream = null;
try {
File srcFolder = new File(sourceFolder);
if (srcFolder.isDirectory()) {
// uncomment following code if you want to add all files under srcFolder
//Iterator<File> fileIterator = Arrays.asList(srcFolder.listFiles()).iterator();
Iterator<File> fileIterator = FileUtils.iterateFiles(srcFolder, new String[]{fileTypesToBeAddedToZip}, true);
File zipFile = new File(zipFilePath);
zipFile.delete();
outputStream = new FileOutputStream(zipFile);
zipArchiveOutputStream = new ZipArchiveOutputStream(outputStream);
zipArchiveOutputStream.setUseZip64(Zip64Mode.AsNeeded);
int srcFolderLength = srcFolder.getAbsolutePath().length() + 1; // +1 to remove the last file separator
while (fileIterator.hasNext()) {
File file = fileIterator.next();
// uncomment following code if you want to add all files under srcFolder
//if (file.isDirectory()) {
// continue;
// }
String relativePath = file.getAbsolutePath().substring(srcFolderLength);
InputStreamSupplier streamSupplier = () -> {
InputStream is = null;
try {
is = Files.newInputStream(file.toPath());
} catch (IOException e) {
e.printStackTrace();
}
return is;
};
ZipArchiveEntry zipArchiveEntry = new ZipArchiveEntry(relativePath);
zipArchiveEntry.setMethod(ZipEntry.DEFLATED);
scatterZipCreator.addArchiveEntry(zipArchiveEntry, streamSupplier);
}
scatterZipCreator.writeTo(zipArchiveOutputStream);
}
if (zipArchiveOutputStream != null) {
zipArchiveOutputStream.close();
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (outputStream != null) {
outputStream.close();
}
}
}
private static void unzip(String zipFilePath, String destDir) {
File dir = new File(destDir);
// create output directory if it doesn't exist
if (!dir.exists()) {
dir.mkdirs();
} else {
dir.delete();
}
FileInputStream fis;
//buffer for read and write data to file
byte[] buffer = new byte[1024];
try {
fis = new FileInputStream(zipFilePath);
ZipInputStream zis = new ZipInputStream(fis);
ZipEntry ze = zis.getNextEntry();
while (ze != null) {
String fileName = ze.getName();
File newFile = new File(destDir + File.separator + fileName);
System.out.println("Unzipping to " + newFile.getAbsolutePath());
//create directories for sub directories in zip
String parentFolder = newFile.getParent();
File folder = new File(parentFolder);
folder.mkdirs();
FileOutputStream fos = new FileOutputStream(newFile);
int len;
while ((len = zis.read(buffer)) > 0) {
fos.write(buffer, 0, len);
}
fos.close();
//close this ZipEntry
zis.closeEntry();
ze = zis.getNextEntry();
}
//close last ZipEntry
zis.closeEntry();
zis.close();
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Ref: Fast zipping folder using java ParallelScatterZipCreator
Here i have Bookfolder in that few more folders(english,hindi,japanese).Converting english,hindi,japanese to english.zip,hindi.zip and japanese.zip.Everything is working fine and i'm keeping zip files and folders inside Bookfolder,this thing i'm doing using with java.But when i'm unzipping manually the zip file ex:english.zip ,right click on that extract here then showing error as UNEXPECTED END OF ARCHIVE.This is my code.
public void foldertToZip(File zipDeleteFile) {
//System.out.println(zipDeleteFile);
File directoryToZip = zipDeleteFile;
List<File> fileList = new ArrayList<>();
//System.out.println("---Getting references to all files in: " + directoryToZip.getCanonicalPath());
getAllFiles(directoryToZip, fileList);
//System.out.println("---Creating zip file");
writeZipFile(directoryToZip, fileList);
//System.out.println("---Done");
}
public static void getAllFiles(File dir, List<File> fileList) {
try {
File[] files = dir.listFiles();
for (File file : files) {
fileList.add(file);
if (file.isDirectory()) {
System.out.println("directory:" + file.getCanonicalPath());
getAllFiles(file, fileList);
} else {
System.out.println("file:" + file.getCanonicalPath());
}
}
} catch (IOException e) {
}
}
public static void writeZipFile(File directoryToZip, List<File> fileList) {
try {
//try (FileOutputStream fos = new FileOutputStream(directoryToZip.getName() + ".zip"); ZipOutputStream zos = new ZipOutputStream(fos)) {
File path = directoryToZip.getParentFile();
File zipFile = new File(path, directoryToZip.getName() + ".zip");
try (FileOutputStream fos = new FileOutputStream(zipFile)) {
ZipOutputStream zos = new ZipOutputStream(fos);
for (File file : fileList) {
if (!file.isDirectory()) { // we only zip files, not directories
addToZip(directoryToZip, file,zos);
}
}
}
} catch (FileNotFoundException e) {
} catch (IOException e) {
}
}
public static void addToZip(File directoryToZip, File file, ZipOutputStream zos) throws FileNotFoundException,
IOException {
try (FileInputStream fis = new FileInputStream(file)) {
String zipFilePath = file.getCanonicalPath().substring(directoryToZip.getCanonicalPath().length() + 1,
file.getCanonicalPath().length());
System.out.println("Writing '" + zipFilePath + "' to zip file");
ZipEntry zipEntry = new ZipEntry(zipFilePath);
zos.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while ((length = fis.read(bytes)) >= 0) {
zos.write(bytes, 0, length);
}
zos.closeEntry();
}
}`
while i'm extracting new zip file(eg:english.zip) it showing error as unexpected end of archieve (i think not zipping exactly)'
You need to close ZipOutputStream in writeZipFile() method;
for (File file : fileList) {
if (!file.isDirectory()) { // we only zip files, not directories
addToZip(directoryToZip, file,zos);
}
}
//here close zos
zos.close();
I was missing media type on ResponseEntity object and therefore I got this error information during extracting zip archive.
That means when calling rest api to download zip archive with files then needed information needs to be passed within ResponseEntity (media type application/zip, header info, cache info, etc.).
Another observation
is that I was missing correct return response type from angular side. There must be responseType as blob and not as text. That caused me the main issue.
Angular side:
getZIPData(id: number) {
const path = resolveBase() + 'rest-api-path' + id
return this._http.get(path, { observe: 'response', responseType: 'blob' });
}
It might help somebody.
Here i have folder(Books)structure inside of Books folder i have folders called physics,chemistry,science,english.I'm passing Books folder as zipDeleteFile but inside all folder has to convert in the same folder(Books)as physics.zip,chemistry.zip,science.zip,english.zip.But this code is not working.
'
public void foldertToZip(File zipDeleteFile) {
//System.out.println(zipDeleteFile);
File directoryToZip = zipDeleteFile;
List<File> fileList = new ArrayList<File>();
//System.out.println("---Getting references to all files in: " + directoryToZip.getCanonicalPath());
getAllFiles(directoryToZip, fileList);
//System.out.println("---Creating zip file");
writeZipFile(directoryToZip, fileList);
//System.out.println("---Done");
}
public static void getAllFiles(File dir, List<File> fileList) {
try {
File[] files = dir.listFiles();
for (File file : files) {
fileList.add(file);
if (file.isDirectory()) {
System.out.println("directory:" + file.getCanonicalPath());
getAllFiles(file, fileList);
} else {
System.out.println(" file:" + file.getCanonicalPath());
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
public static void writeZipFile(File directoryToZip, List<File> fileList) {
try {
try (FileOutputStream fos = new FileOutputStream(directoryToZip.getName() + ".zip")) {
ZipOutputStream zos = new ZipOutputStream(fos);
for (File file : fileList) {
if (!file.isDirectory()) { // we only zip files, not directories
addToZip(directoryToZip, file, zos);
}
}
zos.close();
}
} catch (FileNotFoundException e) {
} catch (IOException e) {
}
}
public static void addToZip(File directoryToZip, File file, ZipOutputStream zos) throws FileNotFoundException,
IOException {
try (FileInputStream fis = new FileInputStream(file)) {
String zipFilePath = file.getCanonicalPath().substring(directoryToZip.getCanonicalPath().length() + 1,
file.getCanonicalPath().length());
System.out.println("Writing '" + zipFilePath + "' to zip file");
ZipEntry zipEntry = new ZipEntry(zipFilePath);
zos.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while ((length = fis.read(bytes)) >= 0) {
zos.write(bytes, 0, length);
}
zos.closeEntry();
}
}`'
Initially i'm passing zipDeleteFile as C:\Books inside Books i have all physics,english,science folder those folders has to convert into zip files in the same root folder(Books).
So, basically, you want to zip each of the directories in the Books folder into their own zip file. There is a couple of ways you could do this, but the eaiest might be to change the way you are calling foldertToZip
So, instead of (something like)...
foldertToZip(new File("C:\\Books"));
You could do something like...
for (File file : new File("C:\\Books").listFiles()) {
if (file.isDirectory()) {
foldertToZip(file);
}
}
This will result in each directory within Books been added to it's own zip file, which will reside within Books
One other change you might need to make is...
public static void writeZipFile(File directoryToZip, List<File> fileList) {
try {
//try (FileOutputStream fos = new FileOutputStream(directoryToZip.getName() + ".zip")) {
File path = directoryToZip.getParentFile();
File zipFile = new File(path, directoryToZip.getName() + ".zip");
try (FileOutputStream fos = new FileOutputStream(zipFile)) {
This will create the zip file within the parent directory of the directory to be zipped (ie, the Books directory)
I'm running into a problem using the commons compress library to create a tar.gz of a directory. I have a directory structure that is as follows.
parent/
child/
file1.raw
fileN.raw
I'm using the following code to do the compression. It runs fine without exceptions. However, when I try to decompress that tar.gz, I get a single file with the name "childDirToCompress". Its the correct size so the files have clearly been appended to each other in the tarring process. The desired output would be a directory. I can't figure out what I'm doing wrong. Can any wise commons compresser set me upon the correct path?
CreateTarGZ() throws CompressorException, FileNotFoundException, ArchiveException, IOException {
File f = new File("parent");
File f2 = new File("parent/childDirToCompress");
File outFile = new File(f2.getAbsolutePath() + ".tar.gz");
if(!outFile.exists()){
outFile.createNewFile();
}
FileOutputStream fos = new FileOutputStream(outFile);
TarArchiveOutputStream taos = new TarArchiveOutputStream(new GZIPOutputStream(new BufferedOutputStream(fos)));
taos.setBigNumberMode(TarArchiveOutputStream.BIGNUMBER_STAR);
taos.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU);
addFilesToCompression(taos, f2, ".");
taos.close();
}
private static void addFilesToCompression(TarArchiveOutputStream taos, File file, String dir) throws IOException{
taos.putArchiveEntry(new TarArchiveEntry(file, dir));
if (file.isFile()) {
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
IOUtils.copy(bis, taos);
taos.closeArchiveEntry();
bis.close();
}
else if(file.isDirectory()) {
taos.closeArchiveEntry();
for (File childFile : file.listFiles()) {
addFilesToCompression(taos, childFile, file.getName());
}
}
}
I followed this solution and it worked until I was processing a larger set of files and it randomly crashes after processing 15000 - 16000 files. the following line is leaking file handlers:
IOUtils.copy(new FileInputStream(f), tOut);
and the code crashed with a "Too many open files" error at the OS level
The following minor change fix the problem:
FileInputStream in = new FileInputStream(f);
IOUtils.copy(in, tOut);
in.close();
I haven't figured out what exactly was going wrong but a scouring of google caches I found a working example. Sorry for the tumbleweed!
public void CreateTarGZ()
throws FileNotFoundException, IOException
{
try {
System.out.println(new File(".").getAbsolutePath());
dirPath = "parent/childDirToCompress/";
tarGzPath = "archive.tar.gz";
fOut = new FileOutputStream(new File(tarGzPath));
bOut = new BufferedOutputStream(fOut);
gzOut = new GzipCompressorOutputStream(bOut);
tOut = new TarArchiveOutputStream(gzOut);
addFileToTarGz(tOut, dirPath, "");
} finally {
tOut.finish();
tOut.close();
gzOut.close();
bOut.close();
fOut.close();
}
}
private void addFileToTarGz(TarArchiveOutputStream tOut, String path, String base)
throws IOException
{
File f = new File(path);
System.out.println(f.exists());
String entryName = base + f.getName();
TarArchiveEntry tarEntry = new TarArchiveEntry(f, entryName);
tOut.putArchiveEntry(tarEntry);
if (f.isFile()) {
IOUtils.copy(new FileInputStream(f), tOut);
tOut.closeArchiveEntry();
} else {
tOut.closeArchiveEntry();
File[] children = f.listFiles();
if (children != null) {
for (File child : children) {
System.out.println(child.getName());
addFileToTarGz(tOut, child.getAbsolutePath(), entryName + "/");
}
}
}
}
I ended up doing the following:
public URL createTarGzip() throws IOException {
Path inputDirectoryPath = ...
File outputFile = new File("/path/to/filename.tar.gz");
try (FileOutputStream fileOutputStream = new FileOutputStream(outputFile);
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
GzipCompressorOutputStream gzipOutputStream = new GzipCompressorOutputStream(bufferedOutputStream);
TarArchiveOutputStream tarArchiveOutputStream = new TarArchiveOutputStream(gzipOutputStream)) {
tarArchiveOutputStream.setBigNumberMode(TarArchiveOutputStream.BIGNUMBER_POSIX);
tarArchiveOutputStream.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU);
List<File> files = new ArrayList<>(FileUtils.listFiles(
inputDirectoryPath,
new RegexFileFilter("^(.*?)"),
DirectoryFileFilter.DIRECTORY
));
for (int i = 0; i < files.size(); i++) {
File currentFile = files.get(i);
String relativeFilePath = new File(inputDirectoryPath.toUri()).toURI().relativize(
new File(currentFile.getAbsolutePath()).toURI()).getPath();
TarArchiveEntry tarEntry = new TarArchiveEntry(currentFile, relativeFilePath);
tarEntry.setSize(currentFile.length());
tarArchiveOutputStream.putArchiveEntry(tarEntry);
tarArchiveOutputStream.write(IOUtils.toByteArray(new FileInputStream(currentFile)));
tarArchiveOutputStream.closeArchiveEntry();
}
tarArchiveOutputStream.close();
return outputFile.toURI().toURL();
}
}
This takes care of the some of the edge cases that come up in the other solutions.
I had to make some adjustments to #merrick solution to get it to work related to the path. Perhaps with the latest maven dependencies. The currently accepted solution didn't work for me.
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream;
import org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.apache.commons.io.filefilter.DirectoryFileFilter;
import org.apache.commons.io.filefilter.RegexFileFilter;
public class TAR {
public static void CreateTarGZ(String inputDirectoryPath, String outputPath) throws IOException {
File inputFile = new File(inputDirectoryPath);
File outputFile = new File(outputPath);
try (FileOutputStream fileOutputStream = new FileOutputStream(outputFile);
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
GzipCompressorOutputStream gzipOutputStream = new GzipCompressorOutputStream(bufferedOutputStream);
TarArchiveOutputStream tarArchiveOutputStream = new TarArchiveOutputStream(gzipOutputStream)) {
tarArchiveOutputStream.setBigNumberMode(TarArchiveOutputStream.BIGNUMBER_POSIX);
tarArchiveOutputStream.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU);
List<File> files = new ArrayList<>(FileUtils.listFiles(
inputFile,
new RegexFileFilter("^(.*?)"),
DirectoryFileFilter.DIRECTORY
));
for (int i = 0; i < files.size(); i++) {
File currentFile = files.get(i);
String relativeFilePath = inputFile.toURI().relativize(
new File(currentFile.getAbsolutePath()).toURI()).getPath();
TarArchiveEntry tarEntry = new TarArchiveEntry(currentFile, relativeFilePath);
tarEntry.setSize(currentFile.length());
tarArchiveOutputStream.putArchiveEntry(tarEntry);
tarArchiveOutputStream.write(IOUtils.toByteArray(new FileInputStream(currentFile)));
tarArchiveOutputStream.closeArchiveEntry();
}
tarArchiveOutputStream.close();
}
}
}
Maven
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.18</version>
</dependency>
Something I use (via Files.walk API), you can chain gzip(tar(youFile));
public static File gzip(File fileToCompress) throws IOException {
final File gzipFile = new File(fileToCompress.toPath().getParent().toFile(),
fileToCompress.getName() + ".gz");
final byte[] buffer = new byte[1024];
try (FileInputStream in = new FileInputStream(fileToCompress);
GZIPOutputStream out = new GZIPOutputStream(
new FileOutputStream(gzipFile))) {
int len;
while ((len = in.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
}
return gzipFile;
}
public static File tar(File folderToCompress) throws IOException, ArchiveException {
final File tarFile = Files.createTempFile(null, ".tar").toFile();
try (TarArchiveOutputStream out = (TarArchiveOutputStream) new ArchiveStreamFactory()
.createArchiveOutputStream(ArchiveStreamFactory.TAR,
new FileOutputStream(tarFile))) {
out.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU);
Files.walk(folderToCompress.toPath()) //
.forEach(source -> {
if (source.toFile().isFile()) {
final String relatifSourcePath = StringUtils.substringAfter(
source.toString(), folderToCompress.getPath());
final TarArchiveEntry entry = new TarArchiveEntry(
source.toFile(), relatifSourcePath);
try (InputStream in = new FileInputStream(source.toFile())){
out.putArchiveEntry(entry);
IOUtils.copy(in, out);
out.closeArchiveEntry();
}
catch (IOException e) {
// Handle this better than bellow...
throw new RuntimeException(e);
}
}
});
}
return tarFile;
}
Check below for Apache commons-compress and File walker examples.
This example tar.gz a directory.
public static void createTarGzipFolder(Path source) throws IOException {
if (!Files.isDirectory(source)) {
throw new IOException("Please provide a directory.");
}
// get folder name as zip file name
String tarFileName = source.getFileName().toString() + ".tar.gz";
try (OutputStream fOut = Files.newOutputStream(Paths.get(tarFileName));
BufferedOutputStream buffOut = new BufferedOutputStream(fOut);
GzipCompressorOutputStream gzOut = new GzipCompressorOutputStream(buffOut);
TarArchiveOutputStream tOut = new TarArchiveOutputStream(gzOut)) {
Files.walkFileTree(source, new SimpleFileVisitor<>() {
#Override
public FileVisitResult visitFile(Path file,
BasicFileAttributes attributes) {
// only copy files, no symbolic links
if (attributes.isSymbolicLink()) {
return FileVisitResult.CONTINUE;
}
// get filename
Path targetFile = source.relativize(file);
try {
TarArchiveEntry tarEntry = new TarArchiveEntry(
file.toFile(), targetFile.toString());
tOut.putArchiveEntry(tarEntry);
Files.copy(file, tOut);
tOut.closeArchiveEntry();
System.out.printf("file : %s%n", file);
} catch (IOException e) {
System.err.printf("Unable to tar.gz : %s%n%s%n", file, e);
}
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFileFailed(Path file, IOException exc) {
System.err.printf("Unable to tar.gz : %s%n%s%n", file, exc);
return FileVisitResult.CONTINUE;
}
});
tOut.finish();
}
}
This example extracts a tar.gz, and checks zip slip attack.
public static void decompressTarGzipFile(Path source, Path target)
throws IOException {
if (Files.notExists(source)) {
throw new IOException("File doesn't exists!");
}
try (InputStream fi = Files.newInputStream(source);
BufferedInputStream bi = new BufferedInputStream(fi);
GzipCompressorInputStream gzi = new GzipCompressorInputStream(bi);
TarArchiveInputStream ti = new TarArchiveInputStream(gzi)) {
ArchiveEntry entry;
while ((entry = ti.getNextEntry()) != null) {
Path newPath = zipSlipProtect(entry, target);
if (entry.isDirectory()) {
Files.createDirectories(newPath);
} else {
// check parent folder again
Path parent = newPath.getParent();
if (parent != null) {
if (Files.notExists(parent)) {
Files.createDirectories(parent);
}
}
// copy TarArchiveInputStream to Path newPath
Files.copy(ti, newPath, StandardCopyOption.REPLACE_EXISTING);
}
}
}
}
private static Path zipSlipProtect(ArchiveEntry entry, Path targetDir)
throws IOException {
Path targetDirResolved = targetDir.resolve(entry.getName());
Path normalizePath = targetDirResolved.normalize();
if (!normalizePath.startsWith(targetDir)) {
throw new IOException("Bad entry: " + entry.getName());
}
return normalizePath;
}
References
https://mkyong.com/java/how-to-create-tar-gz-in-java/
https://commons.apache.org/proper/commons-compress/examples.html
I have a dynamic text file that picks content from a database according to the user's query. I have to write this content into a text file and zip it in a folder in a servlet. How should I do this?
Look at this example:
StringBuilder sb = new StringBuilder();
sb.append("Test String");
File f = new File("d:\\test.zip");
ZipOutputStream out = new ZipOutputStream(new FileOutputStream(f));
ZipEntry e = new ZipEntry("mytext.txt");
out.putNextEntry(e);
byte[] data = sb.toString().getBytes();
out.write(data, 0, data.length);
out.closeEntry();
out.close();
This will create a zip in the root of D: named test.zip which will contain one single file called mytext.txt. Of course you can add more zip entries and also specify a subdirectory like this:
ZipEntry e = new ZipEntry("folderName/mytext.txt");
You can find more information about compression with Java here.
Java 7 has ZipFileSystem built in, that can be used to create, write and read file from zip file.
Java Doc: ZipFileSystem Provider
Map<String, String> env = new HashMap<>();
// Create the zip file if it doesn't exist
env.put("create", "true");
URI uri = URI.create("jar:file:/codeSamples/zipfs/zipfstest.zip");
try (FileSystem zipfs = FileSystems.newFileSystem(uri, env)) {
Path externalTxtFile = Paths.get("/codeSamples/zipfs/SomeTextFile.txt");
Path pathInZipfile = zipfs.getPath("/SomeTextFile.txt");
// Copy a file into the zip file
Files.copy(externalTxtFile, pathInZipfile, StandardCopyOption.REPLACE_EXISTING);
}
To write a ZIP file, you use a ZipOutputStream. For each entry that you want to place into the ZIP file, you create a ZipEntry object. You pass the file name to the ZipEntry constructor; it sets the other parameters such as file date and decompression method. You can override these settings if you like. Then, you call the putNextEntry method of the ZipOutputStream to begin writing a new file. Send the file data to the ZIP stream. When you are done, call closeEntry. Repeat for all the files you want to store. Here is a code skeleton:
FileOutputStream fout = new FileOutputStream("test.zip");
ZipOutputStream zout = new ZipOutputStream(fout);
for all files
{
ZipEntry ze = new ZipEntry(filename);
zout.putNextEntry(ze);
send data to zout;
zout.closeEntry();
}
zout.close();
Here is an example code to compress a Whole Directory(including sub files and sub directories), it's using the walk file tree feature of Java NIO.
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
public class ZipCompress {
public static void compress(String dirPath) {
final Path sourceDir = Paths.get(dirPath);
String zipFileName = dirPath.concat(".zip");
try {
final ZipOutputStream outputStream = new ZipOutputStream(new FileOutputStream(zipFileName));
Files.walkFileTree(sourceDir, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attributes) {
try {
Path targetFile = sourceDir.relativize(file);
outputStream.putNextEntry(new ZipEntry(targetFile.toString()));
byte[] bytes = Files.readAllBytes(file);
outputStream.write(bytes, 0, bytes.length);
outputStream.closeEntry();
} catch (IOException e) {
e.printStackTrace();
}
return FileVisitResult.CONTINUE;
}
});
outputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
To use this, just call
ZipCompress.compress("target/directoryToCompress");
and you'll get a zip file directoryToCompress.zip
Single file:
String filePath = "/absolute/path/file1.txt";
String zipPath = "/absolute/path/output.zip";
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipPath))) {
File fileToZip = new File(filePath);
zipOut.putNextEntry(new ZipEntry(fileToZip.getName()));
Files.copy(fileToZip.toPath(), zipOut);
}
Multiple files:
List<String> filePaths = Arrays.asList("/absolute/path/file1.txt", "/absolute/path/file2.txt");
String zipPath = "/absolute/path/output.zip";
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipPath))) {
for (String filePath : filePaths) {
File fileToZip = new File(filePath);
zipOut.putNextEntry(new ZipEntry(fileToZip.getName()));
Files.copy(fileToZip.toPath(), zipOut);
}
}
Spring boot controller, zip the files in a directory, and can be downloaded.
#RequestMapping(value = "/files.zip")
#ResponseBody
byte[] filesZip() throws IOException {
File dir = new File("./");
File[] filesArray = dir.listFiles();
if (filesArray == null || filesArray.length == 0)
System.out.println(dir.getAbsolutePath() + " have no file!");
ByteArrayOutputStream bo = new ByteArrayOutputStream();
ZipOutputStream zipOut= new ZipOutputStream(bo);
for(File xlsFile:filesArray){
if(!xlsFile.isFile())continue;
ZipEntry zipEntry = new ZipEntry(xlsFile.getName());
zipOut.putNextEntry(zipEntry);
zipOut.write(IOUtils.toByteArray(new FileInputStream(xlsFile)));
zipOut.closeEntry();
}
zipOut.close();
return bo.toByteArray();
}
This is how you create a zip file from a source file:
String srcFilename = "C:/myfile.txt";
String zipFile = "C:/myfile.zip";
try {
byte[] buffer = new byte[1024];
FileOutputStream fos = new FileOutputStream(zipFile);
ZipOutputStream zos = new ZipOutputStream(fos);
File srcFile = new File(srcFilename);
FileInputStream fis = new FileInputStream(srcFile);
zos.putNextEntry(new ZipEntry(srcFile.getName()));
int length;
while ((length = fis.read(buffer)) > 0) {
zos.write(buffer, 0, length);
}
zos.closeEntry();
fis.close();
zos.close();
}
catch (IOException ioe) {
System.out.println("Error creating zip file" + ioe);
}
public static void main(String args[])
{
omtZip("res/", "omt.zip");
}
public static void omtZip(String path,String outputFile)
{
final int BUFFER = 2048;
boolean isEntry = false;
ArrayList<String> directoryList = new ArrayList<String>();
File f = new File(path);
if(f.exists())
{
try {
FileOutputStream fos = new FileOutputStream(outputFile);
ZipOutputStream zos = new ZipOutputStream(new BufferedOutputStream(fos));
byte data[] = new byte[BUFFER];
if(f.isDirectory())
{
//This is Directory
do{
String directoryName = "";
if(directoryList.size() > 0)
{
directoryName = directoryList.get(0);
System.out.println("Directory Name At 0 :"+directoryName);
}
String fullPath = path+directoryName;
File fileList = null;
if(directoryList.size() == 0)
{
//Main path (Root Directory)
fileList = f;
}else
{
//Child Directory
fileList = new File(fullPath);
}
String[] filesName = fileList.list();
int totalFiles = filesName.length;
for(int i = 0 ; i < totalFiles ; i++)
{
String name = filesName[i];
File filesOrDir = new File(fullPath+name);
if(filesOrDir.isDirectory())
{
System.out.println("New Directory Entry :"+directoryName+name+"/");
ZipEntry entry = new ZipEntry(directoryName+name+"/");
zos.putNextEntry(entry);
isEntry = true;
directoryList.add(directoryName+name+"/");
}else
{
System.out.println("New File Entry :"+directoryName+name);
ZipEntry entry = new ZipEntry(directoryName+name);
zos.putNextEntry(entry);
isEntry = true;
FileInputStream fileInputStream = new FileInputStream(filesOrDir);
BufferedInputStream bufferedInputStream = new BufferedInputStream(fileInputStream, BUFFER);
int size = -1;
while( (size = bufferedInputStream.read(data, 0, BUFFER)) != -1 )
{
zos.write(data, 0, size);
}
bufferedInputStream.close();
}
}
if(directoryList.size() > 0 && directoryName.trim().length() > 0)
{
System.out.println("Directory removed :"+directoryName);
directoryList.remove(0);
}
}while(directoryList.size() > 0);
}else
{
//This is File
//Zip this file
System.out.println("Zip this file :"+f.getPath());
FileInputStream fis = new FileInputStream(f);
BufferedInputStream bis = new BufferedInputStream(fis,BUFFER);
ZipEntry entry = new ZipEntry(f.getName());
zos.putNextEntry(entry);
isEntry = true;
int size = -1 ;
while(( size = bis.read(data,0,BUFFER)) != -1)
{
zos.write(data, 0, size);
}
}
//CHECK IS THERE ANY ENTRY IN ZIP ? ----START
if(isEntry)
{
zos.close();
}else
{
zos = null;
System.out.println("No Entry Found in Zip");
}
//CHECK IS THERE ANY ENTRY IN ZIP ? ----START
}catch(Exception e)
{
e.printStackTrace();
}
}else
{
System.out.println("File or Directory not found");
}
}
}
Given exportPath and queryResults as String variables, the following block creates a results.zip file under exportPath and writes the content of queryResults to a results.txt file inside the zip.
URI uri = URI.create("jar:file:" + exportPath + "/results.zip");
Map<String, String> env = Collections.singletonMap("create", "true");
try (FileSystem zipfs = FileSystems.newFileSystem(uri, env)) {
Path filePath = zipfs.getPath("/results.txt");
byte[] fileContent = queryResults.getBytes();
Files.write(filePath, fileContent, StandardOpenOption.CREATE);
}
You have mainly to create two functions. First is writeToZipFile() and second is createZipfileForOutPut .... and then call the createZipfileForOutPut('file name of .zip')` …
public static void writeToZipFile(String path, ZipOutputStream zipStream)
throws FileNotFoundException, IOException {
System.out.println("Writing file : '" + path + "' to zip file");
File aFile = new File(path);
FileInputStream fis = new FileInputStream(aFile);
ZipEntry zipEntry = new ZipEntry(path);
zipStream.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while ((length = fis.read(bytes)) >= 0) {
zipStream.write(bytes, 0, length);
}
zipStream.closeEntry();
fis.close();
}
public static void createZipfileForOutPut(String filename) {
String home = System.getProperty("user.home");
// File directory = new File(home + "/Documents/" + "AutomationReport");
File directory = new File("AutomationReport");
if (!directory.exists()) {
directory.mkdir();
}
try {
FileOutputStream fos = new FileOutputStream("Path to your destination" + filename + ".zip");
ZipOutputStream zos = new ZipOutputStream(fos);
writeToZipFile("Path to file which you want to compress / zip", zos);
zos.close();
fos.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
There is another option by using zip4j at https://github.com/srikanth-lingala/zip4j
Creating a zip file with single file in it / Adding single file to an existing zip
new ZipFile("filename.zip").addFile("filename.ext");
Or
new ZipFile("filename.zip").addFile(new File("filename.ext"));
Creating a zip file with multiple files / Adding multiple files to an existing zip
new ZipFile("filename.zip").addFiles(Arrays.asList(new File("first_file"), new File("second_file")));
Creating a zip file by adding a folder to it / Adding a folder to an existing zip
new ZipFile("filename.zip").addFolder(new File("/user/myuser/folder_to_add"));
Creating a zip file from stream / Adding a stream to an existing zip
new ZipFile("filename.zip").addStream(inputStream, new ZipParameters());
I know this question is answered but if you have a list of strings and you want to create a separate file for each string in the archive, you can use the snippet below.
public void zipFileTest() throws IOException {
Map<String, String> map = Map.ofEntries(
new AbstractMap.SimpleEntry<String, String>("File1.txt", "File1 Content"),
new AbstractMap.SimpleEntry<String, String>("File2.txt", "File2 Content"),
new AbstractMap.SimpleEntry<String, String>("File3.txt", "File3 Content")
);
createZipFileFromStringContents(map, "archive.zip");
}
public void createZipFileFromStringContents(Map<String, String> map, String zipfilePath) throws IOException {
FileOutputStream fout = new FileOutputStream(zipfilePath);
ZipOutputStream zout = new ZipOutputStream(fout);
for (Map.Entry<String, String> entry : map.entrySet()) {
String fileName = entry.getKey();
ZipEntry zipFile = new ZipEntry(fileName);
zout.putNextEntry(zipFile);
String fileContent = entry.getValue();
zout.write(fileContent.getBytes(), 0, fileContent.getBytes().length);
zout.closeEntry();
}
zout.close();
}
It will create a zip file with the structure as in the below image:
Here is my working solution:
public static byte[] createZipFile(Map<String, FileData> files) throws IOException {
try(ByteArrayOutputStream tZipFile = new ByteArrayOutputStream()) {
try (ZipOutputStream tZipFileOut = new ZipOutputStream(tZipFile)) {
for (Map.Entry<String, FileData> file : files.entrySet()) {
ZipEntry zipEntry = new ZipEntry(file.getValue().getFileName());
tZipFileOut.putNextEntry(zipEntry);
tZipFileOut.write(file.getValue().getBytes());
}
}
return tZipFile.toByteArray();
}
}
public class FileData {
private String fileName;
private byte[] bytes;
public String getFileName() {
return this.fileName;
}
public byte[] getBytes() {
return this.bytes;
}
}
This will create byte[] of ZIP file which contains one or more compressed files. I've used this method inside controller method and write bytes[] of ZIP file into response to download ZIP file(s) from server.
Since it took me a while to figure it out, I thought it would be helpful to post my solution using Java 7+ ZipFileSystem
openZip(runFile);
addToZip(filepath); //loop construct;
zipfs.close();
private void openZip(File runFile) throws IOException {
Map<String, String> env = new HashMap<>();
env.put("create", "true");
env.put("encoding", "UTF-8");
Files.deleteIfExists(runFile.toPath());
zipfs = FileSystems.newFileSystem(URI.create("jar:" + runFile.toURI().toString()), env);
}
private void addToZip(String filename) throws IOException {
Path externalTxtFile = Paths.get(filename).toAbsolutePath();
Path pathInZipfile = zipfs.getPath(filename.substring(filename.lastIndexOf("results"))); //all files to be stored have a common base folder, results/ in my case
if (Files.isDirectory(externalTxtFile)) {
Files.createDirectories(pathInZipfile);
try (DirectoryStream<Path> ds = Files.newDirectoryStream(externalTxtFile)) {
for (Path child : ds) {
addToZip(child.normalize().toString()); //recursive call
}
}
} else {
// copy file to zip file
Files.copy(externalTxtFile, pathInZipfile, StandardCopyOption.REPLACE_EXISTING);
}
}
public static void zipFromTxt(String zipFilePath, String txtFilePath) {
Assert.notNull(zipFilePath, "Zip file path is required");
Assert.notNull(txtFilePath, "Txt file path is required");
zipFromTxt(new File(zipFilePath), new File(txtFilePath));
}
public static void zipFromTxt(File zipFile, File txtFile) {
ZipOutputStream out = null;
FileInputStream in = null;
try {
Assert.notNull(zipFile, "Zip file is required");
Assert.notNull(txtFile, "Txt file is required");
out = new ZipOutputStream(new FileOutputStream(zipFile));
in = new FileInputStream(txtFile);
out.putNextEntry(new ZipEntry(txtFile.getName()));
int len;
byte[] buffer = new byte[1024];
while ((len = in.read(buffer)) > 0) {
out.write(buffer, 0, len);
out.flush();
}
} catch (Exception e) {
log.info("Zip from txt occur error,Detail message:{}", e.toString());
} finally {
try {
if (in != null) in.close();
if (out != null) {
out.closeEntry();
out.close();
}
} catch (Exception e) {
log.info("Zip from txt close error,Detail message:{}", e.toString());
}
}
}
Using Jeka https://jeka.dev JkPathTree, it's quite straightforward.
Path wholeDirToZip = Paths.get("dir/to/zip");
Path zipFile = Paths.get("file.zip");
JkPathTree.of(wholeDirToZip).zipTo(zipFile);
If you want decompress without software better use this code. Other code with pdf files sends error on manually decompress
byte[] buffer = new byte[1024];
try {
FileOutputStream fos = new FileOutputStream("123.zip");
ZipOutputStream zos = new ZipOutputStream(fos);
ZipEntry ze= new ZipEntry("file.pdf");
zos.putNextEntry(ze);
FileInputStream in = new FileInputStream("file.pdf");
int len;
while ((len = in.read(buffer)) > 0) {
zos.write(buffer, 0, len);
}
in.close();
zos.closeEntry();
zos.close();
} catch(IOException ex) {
ex.printStackTrace();
}