I need to add a config file to an existing tar file. I am using apache.commons.compress library. The following code snippet adds the entry correctly but overwrites the existing entries of the tar file.
public static void injectFileToTar () throws IOException, ArchiveException {
String agentSourceFilePath = "C:\\Work\\tar.gz\\";
String fileToBeAdded = "activeSensor.cfg";
String unzippedFileName = "sample.tar";
File f2 = new File(agentSourceFilePath+unzippedFileName); // Refers to the .tar file
File f3 = new File(agentSourceFilePath+fileToBeAdded); // The new entry to be added to the .tar file
// Injecting an entry in the tar
OutputStream tarOut = new FileOutputStream(f2);
TarArchiveOutputStream aos = (TarArchiveOutputStream) new ArchiveStreamFactory().createArchiveOutputStream("tar", tarOut);
TarArchiveEntry entry = new TarArchiveEntry(fileToBeAdded);
entry.setMode(0100000);
entry.setSize(f3.length());
aos.putArchiveEntry(entry);
FileInputStream fis = new FileInputStream(f3);
IOUtils.copy(fis, aos);
fis.close();
aos.closeArchiveEntry();
aos.finish();
aos.close();
tarOut.close();
}
On checking the tar, only "activeSensor.cfg" file is found and the initial content of the tar is found missing. Is the "mode" not set correctly ?
The problem is that the TarArchiveOutputStream does not automatically read in the existing archive, which is something that you'd need to do. Something along the lines of:
CompressorStreamFactory csf = new CompressorStreamFactory();
ArchiveStreamFactory asf = new ArchiveStreamFactory();
String tarFilename = "test.tgz";
String toAddFilename = "activeSensor.cfg";
File toAddFile = new File(toAddFilename);
File tempFile = File.createTempFile("updateTar", "tgz");
File tarFile = new File(tarFilename);
FileInputStream fis = new FileInputStream(tarFile);
CompressorInputStream cis = csf.createCompressorInputStream(CompressorStreamFactory.GZIP, fis);
ArchiveInputStream ais = asf.createArchiveInputStream(ArchiveStreamFactory.TAR, cis);
FileOutputStream fos = new FileOutputStream(tempFile);
CompressorOutputStream cos = csf.createCompressorOutputStream(CompressorStreamFactory.GZIP, fos);
ArchiveOutputStream aos = asf.createArchiveOutputStream(ArchiveStreamFactory.TAR, cos);
// copy the existing entries
ArchiveEntry nextEntry;
while ((nextEntry = ais.getNextEntry()) != null) {
aos.putArchiveEntry(nextEntry);
IOUtils.copy(ais, aos, (int)nextEntry.getSize());
aos.closeArchiveEntry();
}
// create the new entry
TarArchiveEntry entry = new TarArchiveEntry(toAddFilename);
entry.setSize(toAddFile.length());
aos.putArchiveEntry(entry);
IOUtils.copy(new FileInputStream(toAddFile), aos, (int)toAddFile.length());
aos.closeArchiveEntry();
aos.finish();
ais.close();
aos.close();
// copies the new file over the old
tarFile.delete();
tempFile.renameTo(tarFile);
A couple of notes:
This code does not include any exception handling (please add the appropriate try-catch-finally blocks)
This code does not handle files with a size over 2147483647 (Integer.MAX_VALUE) as it only reads file sizes to integer precision bytes (see the cast to int). However, that's not a problem as Apache Compress does not handle files over 2 GB anyway.
Try changing
OutputStream tarOut = new FileOutputStream(f2);
to
OutputStream tarOut = new FileOutputStream(f2, true); //Set append to true
Related
I am storing a TAR file in Google Cloud Storage. The file can be successfully downloaded via gsutil and extracted in my computer using macOS Archive Utility. However, the Java program that I implement always encounter java.io.IOException: Corrupted TAR archive upon accessing the file. I have tried several ways and all of them are utilizing the org.apache.commons:commons-compress library. Can you give me insight on how to fix this problem or something that I can try on?
Here are the implementations that I have tried:
Blob blob = storage.get(BUCKET_NAME, FILE_PATH);
blob.downloadTo(Paths.get("filename.tar"));
String contentType = blob.getContentType(); // application/x-tar
InputStream is = Channels.newInputStream(blob.reader());
String mime = URLConnection.guessContentTypeFromStream(is); // null
TarArchiveInputStream ais = new TarArchiveInputStream(is);
ais.getNextEntry(); // raise java.io.IOException: Corrupted TAR archive
InputStream is2 = new ByteArrayInputStream(blob.getContent());
String mime2 = URLConnection.guessContentTypeFromStream(is2); // null
TarArchiveInputStream ais2 = new TarArchiveInputStream(is2);
ais2.getNextEntry(); // raise java.io.IOException: Corrupted TAR archive
InputStream is3 = new FileInputStream("filename.tar");
String mime3 = URLConnection.guessContentTypeFromStream(is3); // null
TarArchiveInputStream ais3 = new TarArchiveInputStream(is3);
ais3.getNextEntry(); // raise java.io.IOException: Corrupted TAR archive
TarFile file = new TarFile(blob.getContent()); // raise java.io.IOException: Corrupted TAR archive
TarFile tarFile = new TarFile(Paths.get("filename.tar")); // raise java.io.IOException: Corrupted TAR archive
Addition: I have tried to parse a JSON from GCS and it's working fine.
Blob blob = storage.get(BUCKET_NAME, FILE_PATH);
JSONTokener jt = new JSONTokener(Channels.newInputStream(blob.reader()));
JSONObject jo = new JSONObject(jt);
The problem is that your tar is compressed, it is a tgz file.
For that reason, you need to decompress the file when processing your tar contents.
Please, consider the following example; note the use of the common compress builtin GzipCompressorInputStream class:
public static void main(String... args) {
final File archiveFile = new File("latest.tar");
try (
FileInputStream in = new FileInputStream(archiveFile);
GzipCompressorInputStream gzIn = new GzipCompressorInputStream(in);
TarArchiveInputStream tarIn = new TarArchiveInputStream(gzIn)
) {
TarArchiveEntry tarEntry = tarIn.getNextTarEntry();
while (tarEntry != null) {
final File path = new File("/tmp/" + File.separator + tarEntry.getName());
if (!path.getParentFile().exists()) {
path.getParentFile().mkdirs();
}
if (!tarEntry.isDirectory()) {
try (OutputStream out = new FileOutputStream(path)){
IOUtils.copy(tarIn, out);
}
}
tarEntry = tarIn.getNextTarEntry();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
I wanted to zip a directory with files and subdirectories in it. I did this and worked fine but I am getting and unusual and curious file structure (At least I see it that way).
This is the created file: When I click on it, I see an "empty" directory like this: but when I unzip this I see this file structure (Not all the names are exacly as they are showed in the image below):
|mantenimiento
|Carpeta_A
|File1.txt
|File2.txt
|Carpeta_B
|Sub_carpetaB
|SubfileB.txt
|Subfile1B.txt
|Subfile2B.txt
|File12.txt
My problem somehow is that the folder "mantenimiento" is where I am zippping from (the directory which I want to zip) and I dont want it to be there, so when I unzip the just created .zip file I want it with this file structure (which are the files and directories inside "mantenimiento" directory): and the other thing is when I click on the .zip file I want to see the files and directories just like the image showed above.
I dont know what's wrong with my code, I have searched but haven't found a reference to what my problem might be.
Here's my code:
private void zipFiles( List<File> files, String directory) throws IOException
{
ZipOutputStream zos = null;
ZipEntry zipEntry = null;
FileInputStream fin = null;
FileOutputStream fos = null;
BufferedInputStream in = null;
String zipFileName = getZipFileName();
try
{
fos = new FileOutputStream( File.separatorChar + zipFileName + EXTENSION );
zos = new ZipOutputStream(fos);
byte[] buf = new byte[1024];
int len;
for(File file : files)
{
zipEntry = new ZipEntry(file.toString());
fin = new FileInputStream(file);
in = new BufferedInputStream(fin);
zos.putNextEntry(zipEntry);
while ((len = in.read(buf)) >= 0)
{
zos.write(buf, 0, len);
}
}
}
catch(Exception e)
{
System.err.println("No fue posible zipear los archivos");
e.printStackTrace();
}
finally
{
in.close();
zos.closeEntry();
zos.close();
}
}
Hope you guys can give me a hint about what I am doing wrong or what I am missing.
Thanks a lot.
Btw, the directory i am giving to the method is never used. The other parameter i am giving is a list of files which contains all the files and directories from the C:\mantenimiento directory.
I once had a problem with windows and zip files, where the created zip did not contain the entries for the folders (i.e. /, /Carpeta_A etc) only the file entries. Try adding ZipEntries for the folders without streaming content.
But as alternative to the somewhat bulky Zip API of Java you could use Filesystem (since Java7) instead. The following example is for Java8 (lambda):
//Path pathToZip = Paths.get("path/to/your/folder");
//Path zipFile = Paths.get("file.zip");
public Path zipPath(Path pathToZip, Path zipFile) {
Map<String, String> env = new HashMap<String, String>() {{
put("create", "true");
}};
try (FileSystem zipFs = FileSystems.newFileSystem(URI.create("jar:" + zipFile.toUri()), env)) {
Path root = zipFs.getPath("/");
Files.walk(pathToZip).forEach(path -> zip(root, path));
}
}
private static void zip(final Path zipRoot, final Path currentPath) {
Path entryPath = zipRoot.resolve(currentPath.toString());
try {
Files.createDirectories(entryPath.getParent());
Files.copy(currentPath, entryPath);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
So I have written the parser for parsing an individual file but can i read each file within the archive without having to actually extract the archive to disk
Following the examples in http://commons.apache.org/proper/commons-compress/examples.html you have to wrap one InputStream with another
// 1st InputStream from your compressed file
FileInputStream in = new FileInputStream(tarbz2File);
// wrap in a 2nd InputStream that deals with compression
BZip2CompressorInputStream bzIn = new BZip2CompressorInputStream(in);
// wrap in a 3rd InputStream that deals with tar
TarArchiveInputStream tarIn = new TarArchiveInputStream(bzIn);
ArchiveEntry entry = null;
while (null != (entry = tarIn.getNextEntry())){
if (entry.getSize() < 1){
continue;
}
// use your parser here, the tar inputStream deals with the size of the current entry
parser.parse(tarIn);
}
tarIn.close();
NOTE: This is a followup to my question here.
I have a program that takes the contents of a directory and bundles everything into a JAR file. The code I use to do this is here:
try
{
FileOutputStream stream = new FileOutputStream(target);
JarOutputStream jOS = new JarOutputStream(stream);
LinkedList<File> fileList = new LinkedList<File>();
buildList(directory, fileList);
JarEntry jarAdd;
String basePath = directory.getAbsolutePath();
byte[] buffer = new byte[4096];
for(File file : fileList)
{
String path = file.getPath().substring(basePath.length() + 1);
path.replaceAll("\\\\", "/");
jarAdd = new JarEntry(path);
jarAdd.setTime(file.lastModified());
jOS.putNextEntry(jarAdd);
FileInputStream in = new FileInputStream(file);
while(true)
{
int nRead = in.read(buffer, 0, buffer.length);
if(nRead <= 0)
break;
jOS.write(buffer, 0, nRead);
}
in.close();
}
jOS.close();
stream.close();
So, all is well and good and the jar gets created, and when I explore its contents with 7-zip it has all the files I need. However, when I try to access the contents of the Jar via a URLClassLoader (the jar is not on the classpath and I don't intend it to be), I get null pointer exceptions.
The odd thing is, when I use a Jar that I've exported from Eclipse, I can access the contents of it in the way I want. This leads me to believe that I'm somehow not creating the Jar correctly, and am leaving something out. Is there anything missing from the method up above?
I figured it out based on this question - the problem was me not properly handling backslashes.
Fixed code is here:
FileOutputStream stream = new FileOutputStream(target);
JarOutputStream jOS = new JarOutputStream(stream);
LinkedList<File> fileList = new LinkedList<File>();
buildList(directory, fileList);
JarEntry entry;
String basePath = directory.getAbsolutePath();
byte[] buffer = new byte[4096];
for(File file : fileList)
{
String path = file.getPath().substring(basePath.length() + 1);
path = path.replace("\\", "/");
entry = new JarEntry(path);
entry.setTime(file.lastModified());
jOS.putNextEntry(entry);
FileInputStream in = new FileInputStream(file);
while(true)
{
int nRead = in.read(buffer, 0, buffer.length);
if(nRead <= 0)
break;
jOS.write(buffer, 0, nRead);
}
in.close();
jOS.closeEntry();
}
jOS.close();
stream.close();
I have to create a file zip with apache-commons-compress-1.x API.
I have used the following code:
File fileZip = new File("D:\\file.zip");
ZipEncoding zipEncoding = ZipEncodingHelper.getZipEncoding("UTF8");
ZipArchiveOutputStream zipOut = new ZipArchiveOutputStream(fileZip);
zipOut.setEncoding("UTF-8");
File entryFile = new File("D:\\attività.jpg");
String entryName = entryFile.getName();
entryName = new String(entryName.getBytes("UTF-8"), "UTF-8");
ZipArchiveEntry entry = new ZipArchiveEntry(entryName);
entry.setSize(entryFile.length());
FileInputStream fInputStream = new FileInputStream(entryFile);
zipOut.setUseLanguageEncodingFlag(true);
zipOut.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS);
zipOut.putArchiveEntry(entry);
zipOut.write(IOUtils.toByteArray(fInputStream));
zipOut.closeArchiveEntry();
zipOut.flush();
zipOut.close();
The zip entry file name has a encoding error. If I open the zipped file with zip manager built windows xp, the filename is attivit+á.jpg.
Help me, please.
Do
entryName = zipEncoding.encode("attivit\u00E0.jpg");