Develop Reference

Java Basics - Looping through folder

I am very new to java and coming from a js background. I am attempting to loop through a folder full of files and zipping it. Currently, I have done the zipping part successfully, but doing by statically adding the files. The answer is obviously a loop from a programming perspective. I am having trouble looping a list and making it equal to the zipping method below. Online Resources are not making sense much sense to me due to my beginner skill.
package zipFile;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
public class ZipFiles {
public static void main(String[] args) {
try {
FileOutputStream fos = new FileOutputStream("atest.zip");
ZipOutputStream zos = new ZipOutputStream(fos);
String file1Name = "src/resources/text1";
String file2Name = "src/resources/text2";
String file3Name = "src/resources/text3";
String file4Name = "src/resources/text4";
String file5Name = "src/resources/text5";
String file6Name = "src/resources/text6";
addToZipFile(file1Name, zos);
addToZipFile(file2Name, zos);
addToZipFile(file3Name, zos);
addToZipFile(file4Name, zos);
addToZipFile(file5Name, zos);
addToZipFile(file6Name, zos);
zos.close();
fos.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public static void addToZipFile(String fileName, ZipOutputStream zos) throws FileNotFoundException, IOException {
System.out.println("Writing '" + fileName + "' to zip file");
File file = new File(fileName);
FileInputStream fis = new FileInputStream(file);
ZipEntry zipEntry = new ZipEntry(fileName);
zos.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while ((length = fis.read(bytes)) >= 0) {
zos.write(bytes, 0, length);
}
zos.closeEntry();
fis.close();
}
}

The answer is in this article: http://www.baeldung.com/java-compress-and-uncompress
This code zips multiple files (Very similar to your code but slightly changed):
public class ZipMultipleFiles {
public static void main(String[] args) throws IOException {
List<String> srcFiles = Arrays.asList("test1.txt", "test2.txt");
FileOutputStream fos = new FileOutputStream("multiCompressed.zip");
ZipOutputStream zipOut = new ZipOutputStream(fos);
for (String srcFile : srcFiles) {
File fileToZip = new File(srcFile);
FileInputStream fis = new FileInputStream(fileToZip);
ZipEntry zipEntry = new ZipEntry(fileToZip.getName());
zipOut.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while((length = fis.read(bytes)) >= 0) {
zipOut.write(bytes, 0, length);
}
fis.close();
}
zipOut.close();
fos.close();
}
}
EDIT:
This line in the code creates an array that is easy to go through in a while loop:
List<String> srcFiles = Arrays.asList("test1.txt", "test2.txt");

basically used finding children of a folder method thanks to Elliotk link. I am making the string equal to the path of the parent folder - >checking if whether if its a directory - > list its files -> get the names and while loop to write all of them to a zipped folder
here is my whole code
package zipfolder2;
import java.io.*;
import java.util.*;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
public class zipfolders2 {
public static void main(String[] args) {
try {
String sourceFile = "src/resources";
FileOutputStream fos = new FileOutputStream("zippedfiles.zip");
ZipOutputStream zipOut = new ZipOutputStream(fos);
File fileToZip = new File(sourceFile);
zipFile(fileToZip, fileToZip.getName(), zipOut);
zipOut.close();
fos.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
private static void zipFile(File fileToZip, String fileName, ZipOutputStream zipOut) throws IOException {
if (fileToZip.isHidden()) {
return;
}
if (fileToZip.isDirectory()) {
File[] children = fileToZip.listFiles();
for (File childFile : children) {
zipFile(childFile, fileName + "/" + childFile.getName(), zipOut);
}
return;
}
FileInputStream fis = new FileInputStream(fileToZip);
ZipEntry zipEntry = new ZipEntry(fileName);
zipOut.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while ((length = fis.read(bytes)) >= 0) {
zipOut.write(bytes, 0, length);
}
fis.close();
}
}

Mule Zip File and send zipped file towards FTP server

I know Mule has great support for gzip compression of data using the element. However the client now wants zip compression since the file has to be placed on an FTP as a zip compressed file :(
I encounter difficulties in mule with following scenario:
I created a Spring bean where a file comes in. I want to compress this file using the ZipOutputStream class and pass it towards our ftp.
This is my flow configuration:
<flow name="testFlow" initialState="stopped">
<file:inbound-endpoint path="${home.dir}/out" moveToDirectory="${hip.dir}/out/hist" fileAge="10000" responseTimeout="10000" connector-ref="input"/>
<component>
<spring-object bean="zipCompressor"/>
</component>
<set-variable value="#[message.inboundProperties.originalFilename]" variableName="originalFilename" />
<ftp:outbound-endpoint host="${ftp.host}" port="${ftp.port}" user="${ftp.username}" password="${ftp.password}" path="${ftp.root.out}" outputPattern="#[flowVars['originalFilename']].zip" />
</flow>
This is the code of my zipCompressor:
#Component
public class ZipCompressor implements Callable {
private static final Logger LOG = LogManager.getLogger(ZipCompressor.class.getName());
#Override
#Transactional
public Object onCall(MuleEventContext eventContext) throws Exception {
if (eventContext.getMessage().getPayload() instanceof File) {
final File srcFile = (File) eventContext.getMessage().getPayload();
final String fileName = srcFile.getName();
final File zipFile = new File(fileName + ".zip");
try {
// create byte buffer
byte[] buffer = new byte[1024];
FileOutputStream fos = new FileOutputStream(zipFile);
ZipOutputStream zos = new ZipOutputStream(fos);
FileInputStream fis = new FileInputStream(srcFile);
// begin writing a new ZIP entry, positions the stream to the start of the entry data
zos.putNextEntry(new ZipEntry(srcFile.getName()));
int length;
while ((length = fis.read(buffer)) > 0) {
zos.write(buffer, 0, length);
}
zos.closeEntry();
// close the InputStream
fis.close();
// close the ZipOutputStream
zos.close();
}
catch (IOException ioe) {
LOG.error("Error creating zip file" + ioe);
}
eventContext.getMessage().setPayload(zipFile);
}
return eventContext.getMessage();
}
}
I wrote a unit test and the compression works great. A file is indeed transferred to the FTP with the correct name, but the zip file is invalid and by opening it in NotePad++, it contains just the original file name.
I think I'm doing something wrong with passing the zip file back to the mule flow, but I'm stuck at the moment so any help would be greatly appreciated!

I have implemented the transformer for this
package com.test.transformer;
import java.io.IOException;
import java.io.InputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.commons.io.IOUtils;
import org.apache.commons.io.output.ByteArrayOutputStream;
import org.mule.api.MuleMessage;
import org.mule.api.transformer.TransformerException;
import org.mule.transformer.AbstractMessageTransformer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ZipTransformer
extends AbstractMessageTransformer
{
private static final Logger log = LoggerFactory.getLogger(ZipTransformer.class);
public static final int DEFAULT_BUFFER_SIZE = 32768;
public static byte[] MAGIC = { 'P', 'K', 0x3, 0x4 };
public ZipTransformer()
{
registerSourceType(InputStream.class);
registerSourceType(byte[].class);
}
public Object transformMessage(MuleMessage message, String outputEncoding)
throws TransformerException
{
Object payload = message.getPayload();
try{
byte[] data;
if (payload instanceof byte[])
{
data = (byte[]) payload;
}
else if (payload instanceof InputStream) {
data = IOUtils.toByteArray((InputStream)payload);
}
else if (payload instanceof String)
{
data = ((String) payload).getBytes(outputEncoding);
}
else
{
data = muleContext.getObjectSerializer().serialize(payload);
}
return compressByteArray(data);
}catch (Exception ioex)
{
throw new TransformerException(this, ioex);
}
}
public Object compressByteArray(byte[] bytes) throws IOException
{
if (bytes == null || isCompressed(bytes))
{
if (logger.isDebugEnabled())
{
logger.debug("Data already compressed; doing nothing");
}
return bytes;
}
if (logger.isDebugEnabled())
{
logger.debug("Compressing message of size: " + bytes.length);
}
ByteArrayOutputStream baos = null;
ZipOutputStream zos = null;
try
{
baos = new ByteArrayOutputStream(DEFAULT_BUFFER_SIZE);
zos = new ZipOutputStream(baos);
zos.putNextEntry(new ZipEntry("test.txt"));
zos.write(bytes, 0, bytes.length);
zos.finish();
zos.close();
byte[] compressedByteArray = baos.toByteArray();
baos.close();
if (logger.isDebugEnabled())
{
logger.debug("Compressed message to size: " + compressedByteArray.length);
}
return compressedByteArray;
}
catch (IOException ioex)
{
throw ioex;
}
finally
{
IOUtils.closeQuietly(zos);
IOUtils.closeQuietly(baos);
}
}
public boolean isCompressed(byte[] bytes) throws IOException
{
if ((bytes == null) || (bytes.length < 4 ))
{
return false;
}
else
{
for (int i = 0; i < MAGIC.length; i++) {
if (bytes[i] != MAGIC[i]) {
return false;
}
}
return true;
}
}
}
Used it as
<custom-transformer class="com.test.transformer.ZipTransformer" doc:name="file zip transformer"/>
As of now sets file name as test.txt. you can change is using any property or variable.
Hope this helps.

A simpler way to do it is to use the gzip transformer in mule to compress the file. Note that you have to do it through the xml.
<gzip-compress-transformer/>

In the ZipTransformer constructor, the following is deprecated.
registerSourceType(InputStream.class);
registerSourceType(byte[].class);
Use this instead:
registerSourceType(DataTypeFactory.create(InputStream.class));
registerSourceType(DataTypeFactory.create(byte[].class));

how to create zip inside the other zip.i got the fileNotFoundException

How to create zip inside zip file
enter image description here
got bellow Error:
D:\sagar\my work\Package Maker\DirectoryStruct>java zipStructure
java.io.FileNotFoundException: Additional_Sub_Folder\Additional_file.zip (The sy
stem cannot find the path specified)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:110)
at zipStructure.main(zipStructure.java:22)
D:\sagar\my work\Package Maker\DirectoryStruct> that is Additional_sub_folder and Digital_sub_folder. inside Additional_sub_folder 1 zip file created(Additional_file.zip),inside that zip 2 folder are created like xml folder and pdf folder
and inside Digital_sub_folder create Artical_sub_folder,and inside Artical_sub_folder 3 new folder are created that is xml folder,pdf folder and Graphics folder. i try it be below java code but not works properly,please help to create the this structure.
import java.io.*;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.*;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
public class zipStructure {
public static void main(String[] args) {
try {
FileOutputStream fos = new FileOutputStream("main.zip");
ZipOutputStream zos = new ZipOutputStream(fos);
zos.putNextEntry(new ZipEntry("Additional_Sub_Folder/"));
zos.putNextEntry(new ZipEntry("Digital_Sub_Folder/"));
FileOutputStream fos1 = new FileOutputStream("Additional_Sub_Folder/Additional_file.zip");
ZipOutputStream zos1 = new ZipOutputStream(fos1);
/*zos1.putNextEntry(new ZipEntry("Additional_file.zip/xml/"));
zos1.putNextEntry(new ZipEntry("Additional_file.zip/pdf/"));*/
zos1.putNextEntry(new ZipEntry("Additional_file1.zip/xml/"));
zos1.putNextEntry(new ZipEntry("Additional_file1.zip/pdf/"));
zos1.close();
fos1.close();
zos.close();
fos.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public static void addToZipFile(String fileName, ZipOutputStream zos) throws FileNotFoundException, IOException {
System.out.println("Writing '" + fileName + "' to zip file");
File file = new File(fileName);
FileInputStream fis = new FileInputStream(file);
ZipEntry zipEntry = new ZipEntry(fileName);
zos.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while ((length = fis.read(bytes)) >= 0) {
zos.write(bytes, 0, length);
}
zos.closeEntry();
fis.close();
}
}
[1]: http://i.stack.imgur.com/TNA8b.jpg

I did not have the problem with the zip-I/O, I had the FileNotFoundException with every type of I/O that uses the fileSystem.
What helped for me was, if you're working with eclipse, that there is function like "refresh project" or something like this. Everytime I had problems with I/O, that worked.

How to zip files and folders in Java?

Please have a look at the below code.
public void startCompress(String path,String fileName,String outputLocation,int compressType,int filSize) throws Exception
{
System.out.println("Input Location: "+path);
System.out.println("Output Location: "+outputLocation);
System.out.println(compressType);
byte[] bs=new byte[filSize];
System.out.println(filSize);
FileOutputStream fos=new FileOutputStream(outputLocation+"/test.zip");
System.out.println(fos.toString());
ZipOutputStream zos=new ZipOutputStream(fos);
ZipEntry ze = new ZipEntry(fileName);
zos.putNextEntry(ze);
FileInputStream inputStream=new FileInputStream(path);
int len;
while((len=inputStream.read(bs))>0){
zos.write(bs, 0, len);
}
inputStream.close();
zos.closeEntry();
zos.close();
}
In above code, we compress a file using java.util.zip package. But we have an issue. That is, if we select multiple files then only one file is being compressed. If we select a folder, the compression simply won't work.
How can I fix this to compress either a file, files, folder, folders, or even nested folders? Java zip package does support .zip, .tar, .tarGz and tarZ. So the solution should not be something which is limited to .zip extension as well.

Here is my solution that uses the new java.nio package. Just call zipDir giving it the path to the directory. It will create a zip file in the same location but called <directory>.zip.
private static Path buildPath(final Path root, final Path child) {
if (root == null) {
return child;
} else {
return Paths.get(root.toString(), child.toString());
}
}
private static void addZipDir(final ZipOutputStream out, final Path root, final Path dir) throws IOException {
try (DirectoryStream<Path> stream = Files.newDirectoryStream(dir)) {
for (Path child : stream) {
Path entry = buildPath(root, child.getFileName());
if (Files.isDirectory(child)) {
addZipDir(out, entry, child);
} else {
out.putNextEntry(new ZipEntry(entry.toString()));
Files.copy(child, out);
out.closeEntry();
}
}
}
}
public static void zipDir(final Path path) throws IOException {
if (!Files.isDirectory(path)) {
throw new IllegalArgumentException("Path must be a directory.");
}
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(path.toString() + ".zip"));
try (ZipOutputStream out = new ZipOutputStream(bos)) {
addZipDir(out, path.getFileName(), path);
}
}

The zip libraries for java cannot be used to compress folders in simpler way like - compress this folder.
You need to do the test if the input is folder or file by yourself. If it is a file - add it to the zip. If it is a folder - iterate the folder and add each file to the zip. For the subfolders to the same. To add more than one file to the Zip you need to create ZipEntry for each file.
You can try this code which works for me:
public static void zip(File directory, File zipfile) throws IOException {
URI base = directory.toURI();
Deque<File> queue = new LinkedList<File>();
queue.push(directory);
OutputStream out = new FileOutputStream(zipfile);
Closeable res = out;
try {
ZipOutputStream zout = new ZipOutputStream(out);
res = zout;
while (!queue.isEmpty()) {
directory = queue.pop();
for (File kid : directory.listFiles()) {
String name = base.relativize(kid.toURI()).getPath();
if (kid.isDirectory()) {
queue.push(kid);
name = name.endsWith("/") ? name : name + "/";
zout.putNextEntry(new ZipEntry(name));
} else {
zout.putNextEntry(new ZipEntry(name));
copy(kid, zout);
zout.closeEntry();
}
}
}
} finally {
res.close();
}
}

Updated from this answer, which fixes issue with each file been added to it's own directory. Also better supports Windows explorer.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
public class Test {
public static void main(String agrs[]) {
ZipUtils appZip = new ZipUtils();
appZip.zipIt(new File(source directory), new File(dest zip));
}
public static class ZipUtils {
private final List<File> fileList;
private List<String> paths;
public ZipUtils() {
fileList = new ArrayList<>();
paths = new ArrayList<>(25);
}
public void zipIt(File sourceFile, File zipFile) {
if (sourceFile.isDirectory()) {
byte[] buffer = new byte[1024];
FileOutputStream fos = null;
ZipOutputStream zos = null;
try {
// This ensures that the zipped files are placed
// into a folder, within the zip file
// which is the same as the one been zipped
String sourcePath = sourceFile.getParentFile().getPath();
generateFileList(sourceFile);
fos = new FileOutputStream(zipFile);
zos = new ZipOutputStream(fos);
System.out.println("Output to Zip : " + zipFile);
FileInputStream in = null;
for (File file : this.fileList) {
String path = file.getParent().trim();
path = path.substring(sourcePath.length());
if (path.startsWith(File.separator)) {
path = path.substring(1);
}
if (path.length() > 0) {
if (!paths.contains(path)) {
paths.add(path);
ZipEntry ze = new ZipEntry(path + "/");
zos.putNextEntry(ze);
zos.closeEntry();
}
path += "/";
}
String entryName = path + file.getName();
System.out.println("File Added : " + entryName);
ZipEntry ze = new ZipEntry(entryName);
zos.putNextEntry(ze);
try {
in = new FileInputStream(file);
int len;
while ((len = in.read(buffer)) > 0) {
zos.write(buffer, 0, len);
}
} finally {
in.close();
}
}
zos.closeEntry();
System.out.println("Folder successfully compressed");
} catch (IOException ex) {
ex.printStackTrace();
} finally {
try {
zos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
protected void generateFileList(File node) {
// add file only
if (node.isFile()) {
fileList.add(node);
}
if (node.isDirectory()) {
File[] subNote = node.listFiles();
for (File filename : subNote) {
generateFileList(filename);
}
}
}
}
}

Generate .tar.gz file in java

I generated .tar.gz file in java.
When I tried to open that,I m getting exception as "Unable to read header of that file" and it asks for some over writing also.
It is also throwing error as
cannot open file:it does not appear to be a valid archive.
My code is as follows
import java.io.File;
import java.io.FileInputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
public class Tarzlatest {
public static void main(String[] args) {
File file =new File(path);
String gzipFile = path;
String newFile = path ;
compressGzipFile(file, gzipFile);
decompressGzipFile(gzipFile, newFile);
}
private static void decompressGzipFile(String gzipFile, String newFile) {
try {
FileInputStream fis = new FileInputStream(gzipFile);
GZIPInputStream gis = new GZIPInputStream(fis);
FileOutputStream fos = new FileOutputStream(newFile);
byte[] buffer = new byte[1024];
int len;
while((len = gis.read(buffer)) != -1){
fos.write(buffer, 0, len);
}
//close resources
fos.close();
gis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
private static void compressGzipFile(File dir, String gzipFile) {
try {
File[] flist = dir.listFiles();
for(int i=0; i<flist.length; i++)
{
FileInputStream fis = new FileInputStream(flist[i]);
FileOutputStream fos = new FileOutputStream(gzipFile);
GZIPOutputStream gzipOS = new GZIPOutputStream(fos);
byte[] buffer = new byte[1024];
int len;
while((len=fis.read(buffer)) != -1){
gzipOS.write(buffer, 0, len);
}
//close resources
gzipOS.close();
fos.close();
fis.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}

It means that you generated your .tar.gz incorrectly. Simple as that.