I am compressing in java using the following and the LZ4 library. If I try to call this method again on the same file name, it overwrites with the new contents instead of appending. Is there a way to append using LZ4? I just want to add another file to the existing zip archive at a later time.
public void zipFile(File[] fileToZip, String outputFileName, boolean activeZip)
{
try (FileOutputStream fos = new FileOutputStream(new File(outputFileName));
LZ4FrameOutputStream lz4fos = new LZ4FrameOutputStream(fos);)
{
for (File a : fileToZip)
{
try (FileInputStream fis = new FileInputStream(a))
{
byte[] buf = new byte[bufferSizeZip];
int length;
while ((length = fis.read(buf)) > 0)
{
lz4fos.write(buf, 0, length);
}
}
}
}
catch (Exception e)
{
LOG.error("Zipping file failed ", e);
}
}
The only way I could figure out how to do this is to send
new FileOutputStream(new File(outputFileName),false)
in the try-with-resources
Related
I am currently trying to create some code which requires me to rename a text file. During the same execution I must access the new file that has been created. I am using the code below:
public static void updateUkRepFile(String fileName) throws IOException {
CommonVariables.newFileName = "UK_REP_"+CommonMiscFunctions.getCurrentMst("MMddyyyy.hhmmss")+".txt";
System.out.println(CommonVariables.newFileName+"NEW FILE NAME");
File origFile = new File(fileName);
File destFile = new File("src/test/resources/testDataFiles/"+CommonVariables.newFileName);
InputStream is = null;
OutputStream os = null;
try{
is = new FileInputStream(origFile);
os = new FileOutputStream(destFile);
byte[] buffer = new byte[1024];
int length;
while((length = is.read(buffer)) >0 ){
os.write(buffer, 0, length);
}
}finally{
is.close();
os.close();
}
}
This doesn't work for me because the new file isn't available until the END of the execution so I get an error. How can I rename and copy the file AND make it available instantly to use throughout my program?
You need to create the destination file using the method createNewFile() present in java.io.File class
Update the code to this :
public static void updateUkRepFile(String fileName) throws IOException {
CommonVariables.newFileName = "UK_REP_"+CommonMiscFunctions.getCurrentMst("MMddyyyy.hhmmss")+".txt";
System.out.println(CommonVariables.newFileName+"NEW FILE NAME");
File origFile = new File(fileName);
File destFile = new File("src/test/resources/testDataFiles/"+CommonVariables.newFileName);
destFile.createNewFile(); // this creates the file in the location.
InputStream is = null;
OutputStream os = null;
try{
is = new FileInputStream(origFile);
os = new FileOutputStream(destFile);
byte[] buffer = new byte[1024];
int length;
while((length = is.read(buffer)) >0 ){
os.write(buffer, 0, length);
}
}finally{
is.close();
os.close();
}
}
I ended up using the Files.copy method to meet my requirements`
public static void updateUkRepFile(String fileName) throws IOException {
File fileToUpload = new File(String.valueOf(fileName));
System.out.println("original file to upload "+ fileToUpload);
String tdFileName = fileName.substring(fileName.lastIndexOf("/")+1);
String fileExtn = tdFileName.substring(tdFileName.lastIndexOf(".")+1);
long ut1 = Instant.now().getEpochSecond();
String newFileName = tdFileName.substring(0,tdFileName.lastIndexOf("."))+ut1+"_UK_REP"+"."+fileExtn;
String dirName = System.getProperty("user.dir")+"/src/test/resources/testDataFiles/UKRep/";
Files.copy(Paths.get(fileName), Paths.get(dirName, newFileName), StandardCopyOption.REPLACE_EXISTING);
CommonVariables.newFileName=newFileName;
System.out.println("NEW FILE NAME: "+newFileName);
}`
Add a thread sleep to pause the execution of the current thread after renaming the file and add InterruptedException to updateUkRepFile method signature
Thread.sleep(1000);
I have been working on a task to decompress from different types of file format such as "zip,tar,tbz,tgz". I am able to do for all except tbz because apache common compress library provides BZIP2 compressors. But I need to decompress a old BZIP not BZIP2. Is there any way to do it java. I have added the code I have done so far for extracting different tar file archives using apache commons library below.
public List<ArchiveFile> processTarFiles(String compressedFilePath, String fileType) throws IOException {
List<ArchiveFile> extractedFileList = null;
TarArchiveInputStream is = null;
FileOutputStream fos = null;
BufferedOutputStream dest = null;
try {
if(fileType.equalsIgnoreCase("tar"))
{
is = new TarArchiveInputStream(new FileInputStream(new File(compressedFilePath)));
}
else if(fileType.equalsIgnoreCase("tbz")||fileType.equalsIgnoreCase("bz"))
{
is = new TarArchiveInputStream(new BZip2CompressorInputStream(new FileInputStream(new File(compressedFilePath))));
}
else if(fileType.equalsIgnoreCase("tgz")||fileType.equalsIgnoreCase("gz"))
{
is = new TarArchiveInputStream(new GzipCompressorInputStream(new FileInputStream(new File(compressedFilePath))));
}
TarArchiveEntry entry = is.getNextTarEntry();
extractedFileList = new ArrayList<>();
while (entry != null) {
// grab a zip file entry
String currentEntry = entry.getName();
if (!entry.isDirectory()) {
File destFile = new File(Constants.DEFAULT_ZIPOUTPUTPATH, currentEntry);
File destinationParent = destFile.getParentFile();
// create the parent directory structure if needed
destinationParent.mkdirs();
ArchiveFile archiveFile = new ArchiveFile();
int currentByte;
// establish buffer for writing file
byte data[] = new byte[(int) entry.getSize()];
// write the current file to disk
fos = new FileOutputStream(destFile);
dest = new BufferedOutputStream(fos, (int) entry.getSize());
// read and write until last byte is encountered
while ((currentByte = is.read(data, 0, (int) entry.getSize())) != -1) {
dest.write(data, 0, currentByte);
}
dest.flush();
dest.close();
archiveFile.setExtractedFilePath(destFile.getAbsolutePath());
archiveFile.setFormat(destFile.getName().split("\\.")[1]);
extractedFileList.add(archiveFile);
entry = is.getNextTarEntry();
} else {
new File(Constants.DEFAULT_ZIPOUTPUTPATH, currentEntry).mkdirs();
entry = is.getNextTarEntry();
}
}
} catch (IOException e) {
System.out.println(("ERROR: " + e.getMessage()));
} catch (Exception e) {
System.out.println(("ERROR: " + e.getMessage()));
} finally {
is.close();
dest.flush();
dest.close();
}
return extractedFileList;
}
The original Bzip was supposedly using a patented algorithm so Bzip2 was born using algorithms and techniques that were not patented.
That might be the reason why it's no longer in widespread use and open source libraries ignore it.
There's some C code for decompressing Bzip files shown here (gist.github.com mirror).
You might want to read and rewrite that in Java.
Using Java I have to read text files which are inside gz file which is in another .tar.gz
gz_ltm_logs.tar.gz is the filename. It then has files ltm.1.gz, ltm.2.gz inside it and then these files have text files in them.
I wanted to do it using java.util.zip.* only but if it is impossible then I can look at other libraries.
I thought I will be able to do it using java.util.zip. But doesn't seem straightforward
Here's some code to give you an idea. This method will try to extract a given tar.gz file to outputFolder.
public static void extract(File input, File outputFolder) throws IOException {
byte[] buffer = new byte[1024];
GZIPInputStream gzipFile = new GZIPInputStream(new FileInputStream(input));
ByteOutputStream tarStream = new ByteOutputStream();
int gzipLengthRead;
while ((gzipLengthRead = gzipFile.read(buffer)) > 0){
tarStream.write(buffer, 0, gzipLengthRead);
}
gzipFile.close();
org.apache.tools.tar.TarInputStream tarFile = null;
// files inside the tar
OutputStream out = null;
try {
tarFile = new org.apache.tools.tar.TarInputStream(tarStream.newInputStream());
tarStream.close();
TarEntry entry = null;
while ((entry = tarFile.getNextEntry()) != null) {
String outFilename = entry.getName();
if (entry.isDirectory()) {
File directory = new File(outputFolder, outFilename);
directory.mkdirs();
} else {
File outputFile = new File(outputFolder, outFilename);
File outputDirectory = outputFile.getParentFile();
if (!outputDirectory.exists()) {
outputDirectory.mkdirs();
}
out = new FileOutputStream(outputFile);
// Transfer bytes from the tarFile to the output file
int innerLen;
while ((innerLen = tarFile.read(buffer)) > 0) {
out.write(buffer, 0, innerLen);
}
out.close();
}
}
} finally {
if (tarFile != null) {
tarFile.close();
}
if (out != null) {
out.close();
}
}
}
given a zip file with multiple nested directory structure, how do I unzip it into the same tree structure?
does ZipFile.entries() provide the enumeration in any order?
This is mine.
In file you specify the file you want to expand
in target dir you have to specify the target location as "new File("/tmp/foo/bar")". If you want to extract in the current directory you can specify targetDir = new File(".")
public static void unzip(File file, File targetDir) throws ZipException,
IOException {
targetDir.mkdirs();
ZipFile zipFile = new ZipFile(file);
try {
Enumeration<? extends ZipEntry> entries = zipFile.entries();
while (entries.hasMoreElements()) {
ZipEntry entry = entries.nextElement();
File targetFile = new File(targetDir, entry.getName());
if (entry.isDirectory()) {
targetFile.mkdirs();
} else {
InputStream input = zipFile.getInputStream(entry);
try {
OutputStream output = new FileOutputStream(targetFile);
try {
copy(input, output);
} finally {
output.close();
}
} finally {
input.close();
}
}
}
} finally {
zipFile.close();
}
}
private static void copy(InputStream input, OutputStream output)
throws IOException {
byte[] buffer = new byte[4096];
int size;
while ((size = input.read(buffer)) != -1)
output.write(buffer, 0, size);
}
Worked for me. Good luck.
Here's the one I use all the times. It should directly work after a copy/paste and in any circumstances.
public static File unzip(File inFile, File outFolder)
{ final int BUFFER = 2048;
try
{
BufferedOutputStream out = null;
ZipInputStream in = new ZipInputStream(
new BufferedInputStream(
new FileInputStream(inFile)));
ZipEntry entry;
while((entry = in.getNextEntry()) != null)
{
//System.out.println("Extracting: " + entry);
int count;
byte data[] = new byte[BUFFER];
//We will try to reconstruct the entry directories
File entrySupposedPath = new File(outFolder.getAbsolutePath()+File.separator+entry.getName());
//Does the parent folder exist?
if (!entrySupposedPath.getParentFile().exists()){
entrySupposedPath.getParentFile().mkdirs();
}
// write the files to the disk
out = new BufferedOutputStream(
new FileOutputStream(outFolder.getPath() + "/" + entry.getName()),BUFFER);
while ((count = in.read(data,0,BUFFER)) != -1)
{
out.write(data,0,count);
}
out.flush();
out.close();
}
in.close();
return outFolder;
}
catch(Exception e)
{
e.printStackTrace();
return inFile;
}
}
Zip doesn't offer directory structure per se. The tree alike structure is built by having full path of each entry. ZipFile enumerates the entries in the same way they have been added to the file.
Note: java.util.ZipEntry.isDirectory() just tests if the last character of the name is '/', that's how it works.
What you need to extract the files into the same directory. Parse then name like that:
for(ZipEntry zipEntry : java.util.Collections.list(zipFile.entries())){//lazislav
String name = zipEntry.getName();
int idx = name.lastIndexOf('/');
if (idx>=0) name=name.substring(idx)
if (name.length()==0) continue;
File f = new File(targetDir, name);
}
That shall do it more or less (you still need to take care of duplicate file names, etc)
ZipFile zipFile = new ZipFile("archive.zip");
try {
for (Enumeration<? extends ZipEntry> entries = zipFile.entries(); entries.hasMoreElements();) {
ZipEntry entry = entries.nextElement();
if (entry.isDirectory()) {
new File(entry.getName()).mkdirs();
} else {
InputStream in = zipFile.getInputStream(entry);
try {
OutputStream out = new BufferedOutputStream(new FileOutputStream(entry.getName()));
try {
// this util class is taken from apache commons io (see http://commons.apache.org/io/)
IOUtils.copy(in, out);
} finally {
out.close();
}
} finally {
in.close();
}
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
zipFile.close();
}
Why do you care about order?
If the ZipFile entry has a name /a/b/c/file.txt, then you can work out the directory name /a/b/c and then create a directory in your tree called a/b/c.
I get the following IOException :
java.io.IOException: Access is denied
at java.io.WinNTFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:850)
at zipUnzipper.main(zipUnzipper.java:41)
When trying to run the following piece of code :
public class zipUnzipper {
public zipUnzipper() {
}
public static void main(String[] args){
//Unzip to temp folder. Add all files to mFiles. Print names of all files in mFfiles.
File file = new File("C:\\aZipFile.zip");
String filename = file.getName();
String filePathName = new String();
int o = filename.lastIndexOf('.');
filename = filename.substring(0,o);
try {
ZipFile zipFile = new ZipFile (file.getAbsoluteFile());
Enumeration entries = zipFile.entries();
while(entries.hasMoreElements()) {
ZipEntry zipEntry = (ZipEntry) entries.nextElement();
System.out.println("Unzipping: " + zipEntry.getName());
BufferedInputStream bis = new BufferedInputStream(zipFile.getInputStream(zipEntry));
byte[] buffer = new byte[2048];
filePathName = "C:\\TEMP\\"+filename+"\\";
File fileToWrite = new File(filePathName+ zipEntry.getName());
fileToWrite.mkdirs();
fileToWrite.createNewFile();
FileOutputStream fos = new FileOutputStream(fileToWrite);
BufferedOutputStream bos = new BufferedOutputStream( fos , buffer.length );
int size;
while ((size = bis.read(buffer, 0, buffer.length)) != -1) {
bos.write(buffer, 0, size);
}
bos.flush();
bos.close();
bis.close();
}
zipFile.close();
File folder = new File (filePathName);
File [] mFiles = folder.listFiles();
for (int x=0; x<mFiles.length; x++) {
System.out.println(mFiles[x].getAbsolutePath());
}
} catch (ZipException ze) {
ze.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
It seems to me that for some reason the JVM can't create a new file. The code runs perfectly well if the files already exist. Is there some kind of access file which dictates whether the JVM can create a new file or am I simply doing something wrong?
Any help is much appreciated :-)
I'm running Java 1.4 and have been testing in JDeveloper in Windows XP.
The issue is that these calls step on each other:
fileToWrite.mkdirs(); //creates a directory e.g. C:\temp\foo\x
fileToWrite.createNewFile(); //attempts to create a file C:\temp\foo\x
The create operation fails because you just created a directory with the same name than the file you want to create.
What you want to do instead is:
fileToWrite.getParentFile().mkdirs()
And also, the call to createNewFile() is unnecessary.
Based on your code. The following "unzips" a zip file:
import java.io.*;
import java.util.zip.ZipFile;
import java.util.zip.ZipEntry;
import java.util.Enumeration;
public class Unzipper {
public static void main(String[] args)
throws IOException {
final File file = new File(args[0]);
final ZipFile zipFile = new ZipFile(file);
final byte[] buffer = new byte[2048];
final File tmpDir = new File(System.getProperty("java.io.tmpdir"), zipFile.getName());
if(!tmpDir.mkdir() && tmpDir.exists()) {
System.err.println("Cannot create: " + tmpDir);
System.exit(0);
}
for(final Enumeration entries = zipFile.entries(); entries.hasMoreElements();) {
final ZipEntry zipEntry = (ZipEntry) entries.nextElement();
System.out.println("Unzipping: " + zipEntry.getName());
final InputStream is = zipFile.getInputStream(zipEntry);
final File fileToWrite = new File(tmpDir, zipEntry.getName());
final File folder = fileToWrite.getParentFile();
if(!folder.mkdirs() && !folder.exists()) {
System.err.println("Cannot create: " + folder);
System.exit(0);
}
if(!zipEntry.isDirectory()) {
//No need to use buffered streams since we're doing our own buffering
final FileOutputStream fos = new FileOutputStream(fileToWrite);
int size;
while ((size = is.read(buffer)) != -1) {
fos.write(buffer, 0, size);
}
fos.close();
is.close();
}
}
zipFile.close();
}
}
Disclaimer: I haven't tested it beyond the very basics.
Why are you calling createNewFile()? Just create the FileOutputStream.
It also could be that in context where you are launching the application you haven't access rights to the place where you are trying to create the file. Launch the app as admin or create the file in the project folder.