Develop Reference

Zip entry stream not writing anything [duplicate]

I am currently extracting the contents of a war file and then adding some new files to the directory structure and then creating a new war file.
This is all done programatically from Java - but I am wondering if it wouldn't be more efficient to copy the war file and then just append the files - then I wouldn't have to wait so long as the war expands and then has to be compressed again.
I can't seem to find a way to do this in the documentation though or any online examples.
Anyone can give some tips or pointers?
UPDATE:
TrueZip as mentioned in one of the answers seems to be a very good java library to append to a zip file (despite other answers that say it is not possible to do this).
Anyone have experience or feedback on TrueZip or can recommend other similar libaries?

In Java 7 we got Zip File System that allows adding and changing files in zip (jar, war) without manual repackaging.
We can directly write to files inside zip files as in the following example.
Map<String, String> env = new HashMap<>();
env.put("create", "true");
Path path = Paths.get("test.zip");
URI uri = URI.create("jar:" + path.toUri());
try (FileSystem fs = FileSystems.newFileSystem(uri, env))
{
Path nf = fs.getPath("new.txt");
try (Writer writer = Files.newBufferedWriter(nf, StandardCharsets.UTF_8, StandardOpenOption.CREATE)) {
writer.write("hello");
}
}

As others mentioned, it's not possible to append content to an existing zip (or war). However, it's possible to create a new zip on the fly without temporarily writing extracted content to disk. It's hard to guess how much faster this will be, but it's the fastest you can get (at least as far as I know) with standard Java. As mentioned by Carlos Tasada, SevenZipJBindings might squeeze out you some extra seconds, but porting this approach to SevenZipJBindings will still be faster than using temporary files with the same library.
Here's some code that writes the contents of an existing zip (war.zip) and appends an extra file (answer.txt) to a new zip (append.zip). All it takes is Java 5 or later, no extra libraries needed.
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Enumeration;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import java.util.zip.ZipOutputStream;
public class Main {
// 4MB buffer
private static final byte[] BUFFER = new byte[4096 * 1024];
/**
* copy input to output stream - available in several StreamUtils or Streams classes
*/
public static void copy(InputStream input, OutputStream output) throws IOException {
int bytesRead;
while ((bytesRead = input.read(BUFFER))!= -1) {
output.write(BUFFER, 0, bytesRead);
}
}
public static void main(String[] args) throws Exception {
// read war.zip and write to append.zip
ZipFile war = new ZipFile("war.zip");
ZipOutputStream append = new ZipOutputStream(new FileOutputStream("append.zip"));
// first, copy contents from existing war
Enumeration<? extends ZipEntry> entries = war.entries();
while (entries.hasMoreElements()) {
ZipEntry e = entries.nextElement();
System.out.println("copy: " + e.getName());
append.putNextEntry(e);
if (!e.isDirectory()) {
copy(war.getInputStream(e), append);
}
append.closeEntry();
}
// now append some extra content
ZipEntry e = new ZipEntry("answer.txt");
System.out.println("append: " + e.getName());
append.putNextEntry(e);
append.write("42\n".getBytes());
append.closeEntry();
// close
war.close();
append.close();
}
}

I had a similar requirement sometime back - but it was for reading and writing zip archives (.war format should be similar). I tried doing it with the existing Java Zip streams but found the writing part cumbersome - especially when directories where involved.
I'll recommend you to try out the TrueZIP (open source - apache style licensed) library that exposes any archive as a virtual file system into which you can read and write like a normal filesystem. It worked like a charm for me and greatly simplified my development.

You could use this bit of code I wrote
public static void addFilesToZip(File source, File[] files)
{
try
{
File tmpZip = File.createTempFile(source.getName(), null);
tmpZip.delete();
if(!source.renameTo(tmpZip))
{
throw new Exception("Could not make temp file (" + source.getName() + ")");
}
byte[] buffer = new byte[1024];
ZipInputStream zin = new ZipInputStream(new FileInputStream(tmpZip));
ZipOutputStream out = new ZipOutputStream(new FileOutputStream(source));
for(int i = 0; i < files.length; i++)
{
InputStream in = new FileInputStream(files[i]);
out.putNextEntry(new ZipEntry(files[i].getName()));
for(int read = in.read(buffer); read > -1; read = in.read(buffer))
{
out.write(buffer, 0, read);
}
out.closeEntry();
in.close();
}
for(ZipEntry ze = zin.getNextEntry(); ze != null; ze = zin.getNextEntry())
{
out.putNextEntry(ze);
for(int read = zin.read(buffer); read > -1; read = zin.read(buffer))
{
out.write(buffer, 0, read);
}
out.closeEntry();
}
out.close();
tmpZip.delete();
}
catch(Exception e)
{
e.printStackTrace();
}
}

I don't know of a Java library that does what you describe. But what you described is practical. You can do it in .NET, using DotNetZip.
Michael Krauklis is correct that you cannot simply "append" data to a war file or zip file, but it is not because there is an "end of file" indication, strictly speaking, in a war file. It is because the war (zip) format includes a directory, which is normally present at the end of the file, that contains metadata for the various entries in the war file. Naively appending to a war file results in no update to the directory, and so you just have a war file with junk appended to it.
What's necessary is an intelligent class that understands the format, and can read+update a war file or zip file, including the directory as appropriate. DotNetZip does this, without uncompressing/recompressing the unchanged entries, just as you described or desired.

As Cheeso says, there's no way of doing it. AFAIK the zip front-ends are doing exactly the same as you internally.
Anyway if you're worried about the speed of extracting/compressing everything, you may want to try the SevenZipJBindings library.
I covered this library in my blog some months ago (sorry for the auto-promotion). Just as an example, extracting a 104MB zip file using the java.util.zip took me 12 seconds, while using this library took 4 seconds.
In both links you can find examples about how to use it.
Hope it helps.

See this bug report.
Using append mode on any kind of
structured data like zip files or tar
files is not something you can really
expect to work. These file formats
have an intrinsic "end of file"
indication built into the data format.
If you really want to skip the intermediate step of un-waring/re-waring, you could read the war file file, get all the zip entries, then write to a new war file "appending" the new entries you wanted to add. Not perfect, but at least a more automated solution.

Yet Another Solution: You may find code below useful in other situations as well. I have used ant this way to compile Java directories, generating jar files, updating zip files,...
public static void antUpdateZip(String zipFilePath, String libsToAddDir) {
Project p = new Project();
p.init();
Target target = new Target();
target.setName("zip");
Zip task = new Zip();
task.init();
task.setDestFile(new File(zipFilePath));
ZipFileSet zipFileSet = new ZipFileSet();
zipFileSet.setPrefix("WEB-INF/lib");
zipFileSet.setDir(new File(libsToAddDir));
task.addFileset(zipFileSet);
task.setUpdate(true);
task.setProject(p);
task.init();
target.addTask(task);
target.setProject(p);
p.addTarget(target);
DefaultLogger consoleLogger = new DefaultLogger();
consoleLogger.setErrorPrintStream(System.err);
consoleLogger.setOutputPrintStream(System.out);
consoleLogger.setMessageOutputLevel(Project.MSG_DEBUG);
p.addBuildListener(consoleLogger);
try {
// p.fireBuildStarted();
// ProjectHelper helper = ProjectHelper.getProjectHelper();
// p.addReference("ant.projectHelper", helper);
// helper.parse(p, buildFile);
p.executeTarget(target.getName());
// p.fireBuildFinished(null);
} catch (BuildException e) {
p.fireBuildFinished(e);
throw new AssertionError(e);
}
}

this a simple code to get a response with using servlet and send a response
myZipPath = bla bla...
byte[] buf = new byte[8192];
String zipName = "myZip.zip";
String zipPath = myzippath+ File.separator+"pdf" + File.separator+ zipName;
File pdfFile = new File("myPdf.pdf");
ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipPath));
ZipEntry zipEntry = new ZipEntry(pdfFile.getName());
out.putNextEntry(zipEntry);
InputStream in = new FileInputStream(pdfFile);
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
out.closeEntry();
in.close();
out.close();
FileInputStream fis = new FileInputStream(zipPath);
response.setContentType("application/zip");
response.addHeader("content-disposition", "attachment;filename=" + zipName);
OutputStream os = response.getOutputStream();
int length = is.read(buffer);
while (length != -1)
{
os.write(buffer, 0, length);
length = is.read(buffer);
}

Here are examples how easily files can be appended to existing zip using TrueVFS:
// append a file to archive under different name
TFile.cp(new File("existingFile.txt"), new TFile("archive.zip", "entry.txt"));
// recusively append a dir to the root of archive
TFile src = new TFile("dirPath", "dirName");
src.cp_r(new TFile("archive.zip", src.getName()));
TrueVFS, the successor of TrueZIP, uses Java 7 NIO 2 features under the hood when appropriate but offers much more features like thread-safe async parallel compression.
Beware also that Java 7 ZipFileSystem by default is vulnerable to OutOfMemoryError on huge inputs.

Here is Java 1.7 version of Liam answer which uses try with resources and Apache Commons IO.
The output is written to a new zip file but it can be easily modified to write to the original file.
/**
* Modifies, adds or deletes file(s) from a existing zip file.
*
* #param zipFile the original zip file
* #param newZipFile the destination zip file
* #param filesToAddOrOverwrite the names of the files to add or modify from the original file
* #param filesToAddOrOverwriteInputStreams the input streams containing the content of the files
* to add or modify from the original file
* #param filesToDelete the names of the files to delete from the original file
* #throws IOException if the new file could not be written
*/
public static void modifyZipFile(File zipFile,
File newZipFile,
String[] filesToAddOrOverwrite,
InputStream[] filesToAddOrOverwriteInputStreams,
String[] filesToDelete) throws IOException {
try (ZipOutputStream out = new ZipOutputStream(new FileOutputStream(newZipFile))) {
// add existing ZIP entry to output stream
try (ZipInputStream zin = new ZipInputStream(new FileInputStream(zipFile))) {
ZipEntry entry = null;
while ((entry = zin.getNextEntry()) != null) {
String name = entry.getName();
// check if the file should be deleted
if (filesToDelete != null) {
boolean ignoreFile = false;
for (String fileToDelete : filesToDelete) {
if (name.equalsIgnoreCase(fileToDelete)) {
ignoreFile = true;
break;
}
}
if (ignoreFile) {
continue;
}
}
// check if the file should be kept as it is
boolean keepFileUnchanged = true;
if (filesToAddOrOverwrite != null) {
for (String fileToAddOrOverwrite : filesToAddOrOverwrite) {
if (name.equalsIgnoreCase(fileToAddOrOverwrite)) {
keepFileUnchanged = false;
}
}
}
if (keepFileUnchanged) {
// copy the file as it is
out.putNextEntry(new ZipEntry(name));
IOUtils.copy(zin, out);
}
}
}
// add the modified or added files to the zip file
if (filesToAddOrOverwrite != null) {
for (int i = 0; i < filesToAddOrOverwrite.length; i++) {
String fileToAddOrOverwrite = filesToAddOrOverwrite[i];
try (InputStream in = filesToAddOrOverwriteInputStreams[i]) {
out.putNextEntry(new ZipEntry(fileToAddOrOverwrite));
IOUtils.copy(in, out);
out.closeEntry();
}
}
}
}
}

this works 100% , if you dont want to use extra libs ..
1) first, the class that append files to the zip ..
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
public class AddZip {
public void AddZip() {
}
public void addToZipFile(ZipOutputStream zos, String nombreFileAnadir, String nombreDentroZip) {
FileInputStream fis = null;
try {
if (!new File(nombreFileAnadir).exists()) {//NO EXISTE
System.out.println(" No existe el archivo : " + nombreFileAnadir);return;
}
File file = new File(nombreFileAnadir);
System.out.println(" Generando el archivo '" + nombreFileAnadir + "' al ZIP ");
fis = new FileInputStream(file);
ZipEntry zipEntry = new ZipEntry(nombreDentroZip);
zos.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while ((length = fis.read(bytes)) >= 0) {zos.write(bytes, 0, length);}
zos.closeEntry();
fis.close();
} catch (FileNotFoundException ex ) {
Logger.getLogger(AddZip.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(AddZip.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
2) you can call it in your controller ..
//in the top
try {
fos = new FileOutputStream(rutaZip);
zos = new ZipOutputStream(fos);
} catch (FileNotFoundException ex) {
Logger.getLogger(UtilZip.class.getName()).log(Level.SEVERE, null, ex);
}
...
//inside your method
addZip.addToZipFile(zos, pathFolderFileSystemHD() + itemFoto.getNombre(), "foto/" + itemFoto.getNombre());

Based on the answer given by #sfussenegger above, following code is used to append to a jar file and download it:
public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
Resource resourceFile = resourceLoader.getResource("WEB-INF/lib/custom.jar");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ZipOutputStream zos = new ZipOutputStream(baos, StandardCharsets.ISO_8859_1);) {
try (ZipFile zin = new ZipFile(resourceFile.getFile(), StandardCharsets.ISO_8859_1);) {
zin.stream().forEach((entry) -> {
try {
zos.putNextEntry(entry);
if (!entry.isDirectory()) {
zin.getInputStream(entry).transferTo(zos);
}
zos.closeEntry();
} catch (Exception ex) {
ex.printStackTrace();
}
});
}
/* build file records to be appended */
....
for (FileContents record : records) {
zos.putNextEntry(new ZipEntry(record.getFileName()));
zos.write(record.getBytes());
zos.closeEntry();
}
zos.flush();
}
response.setContentType("application/java-archive");
response.setContentLength(baos.size());
response.setHeader(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"custom.jar\"");
try (BufferedOutputStream out = new BufferedOutputStream(response.getOutputStream())) {
baos.writeTo(out);
}
}

how to replacing some files within a certain directory inside a jar file?

I'm having the problem of replacing or updating some files within a certain directory inside a jar file.
I've read a few post already. The code (the JarUpdater Class) given at this link Updating .JAR's contents from code
is being very helpful for me to understand the role and the use of ZipInputStream, ZipOutputStream and ZipEntry, etc..
However, when I run it,
I have an EOF Exception
[EDITED by mk7: I found out the jar file was corrupted after I went through it 20 times or so. So after I replaced the jar file with a new one, the EOF Exception went away. The other two problems below still remains unsolved]
these two new xml files only get copied to the "root directory" of the jar file.
these two new xml files NEVER replaced the two original files inside a directory called /conf.
Which lines of code should I change in order to replace the xml files with the new ones?
With the System.out.println, I did see that the while loop steps through every directory and compare at every file as expected. A new temp jar was also created as expected...
I thought the statement "notInFiles = false" would take care of my need but it's NOT.
How do I step into the /conf and only replace those two files and NOT leave a copy at the root of the jar file?
What am I missing? Thanks for any insight!
Below are the code from that link.
import java.io.*;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
public class JarUpdater {
public static void main(String[] args) {
File[] contents = {new File("abc.xml"),
new File("def.xml")};
File jarFile = new File("xyz.jar");
try {
updateZipFile(jarFile, contents);
} catch (IOException e) {
e.printStackTrace();
}
}
public static void updateZipFile(File jarFile,
File[] contents) throws IOException {
// get a temp file
File tempFile = File.createTempFile(jarFile.getName(), null);
// delete it, otherwise you cannot rename your existing zip to it.
tempFile.delete();
System.out.println("tempFile is " + tempFile);
boolean renameOk=jarFile.renameTo(tempFile);
if (!renameOk)
{
throw new RuntimeException("could not rename the file "+jarFile.getAbsolutePath()+" to "+tempFile.getAbsolutePath());
}
byte[] buf = new byte[1024];
ZipInputStream zin = new ZipInputStream(new FileInputStream(tempFile));
ZipOutputStream out = new ZipOutputStream(new FileOutputStream(jarFile));
ZipEntry entry = zin.getNextEntry();
while (entry != null) {
String name = entry.getName();
boolean notInFiles = true;
for (File f : contents) {
System.out.println("f is " + f);
if (f.getName().equals(name)) {
// that file is already inside the jar file
notInFiles = false;
System.out.println("file already inside the jar file");
break;
}
}
if (notInFiles) {
System.out.println("name is " + name);
System.out.println("entry is " + entry);
// Add ZIP entry to output stream.
out.putNextEntry(new ZipEntry(name));
// Transfer bytes from the ZIP file to the output file
int len;
while ((len = zin.read(buf)) > 0) {
out.write(buf, 0, len);
}
}
entry = zin.getNextEntry();
}
// Close the streams
zin.close();
// Compress the contents
for (int i = 0; i < contents.length; i++) {
InputStream in = new FileInputStream(contents[i]);
// Add ZIP entry to output stream.
out.putNextEntry(new ZipEntry(contents[i].getName()));
// Transfer bytes from the file to the ZIP file
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
// Complete the entry
out.closeEntry();
in.close();
}
// Complete the ZIP file
out.close();
tempFile.delete();
}
}

In your first cycle (while loop) where you copy the entries which you don't want to replace you don't close the entries in the output zip file. Add out.closeEntry(); like this:
// Add ZIP entry to output stream.
out.putNextEntry(new ZipEntry(name));
// Transfer bytes from the ZIP file to the output file
int len;
while ((len = zin.read(buf)) > 0) {
out.write(buf, 0, len);
}
// ADD THIS LINE:
out.closeEntry();
Also when you check if an entry is to be replaced, you should compare it to a full path, not just to the name of the file. For example if you want to replace abc.xml which is in the /conf folder, you should compare the entry name to "/conf/abc.xml" and not to "abc.xml".
To properly check if an entry is to be replaced:
String name = entry.getName();
boolean notInFiles = true;
for (File f : contents) {
System.out.println("f is " + f);
if (name.equals("/conf/" + f.getName()) {
// that file is already inside the jar file
notInFiles = false;
System.out.println("file already inside the jar file");
break;
}
}
And when you add the entries to the output which are the replaced files, you also have to specify the entry name having full path, e.g. "/conf/abc.xml" and not just "abc.xml" because it would put "abc.xml" in the root of the output zip.
To do this, start the entry name with "/conf/" like this:
// Add ZIP entry to output stream.
out.putNextEntry(new ZipEntry("/conf/" + contents[i].getName()));

For URIs with the protocol jar:file: (usable for all zip files), you can use a zip file system.
URI jarUri = new URI("jar:" + jarFile.toURI().toString()); // "jar:file:/C:/../xyz.jar"
Map<String, String> zipProperties = new HashMap<>();
zipProperties.put("encoding", "UTF-8");
try (FileSystem zipFS = FileSystems.newFileSystem(jarUri, zipProperties)) {
for (File file : contents) {
Path updatePath = zipFS.getPath("/" + file.getName());
Files.delete(updatePath);
Files.copy(file.toPath(), updatePath, StandardCopyOption.REPLACE_EXISTING);
}
} // closes.
One way to derive the URI is prefixing "jar:" to a File.toURI().
This is a bit more elegant and abstract, and also allows Files.copy in and out the zip. Something to keep in ones tool chest.

Using java to extract .rar files

I am looking a ways to unzip .rar files using Java and where ever I search i keep ending up with the same tool - JavaUnRar. I have been looking into unzipping .rar files with this but all the ways i seem to find to do this are very long and awkward like in this example
I am currently able to extract .tar, .tar.gz, .zip and .jar files in 20 lines of code or less so there must be a simpler way to extract .rar files, does anybody know?
Just if it helps anybody this is the code that I am using to extract both .zip and .jar files, it works for both
public void getZipFiles(String zipFile, String destFolder) throws IOException {
BufferedOutputStream dest = null;
ZipInputStream zis = new ZipInputStream(
new BufferedInputStream(
new FileInputStream(zipFile)));
ZipEntry entry;
while (( entry = zis.getNextEntry() ) != null) {
System.out.println( "Extracting: " + entry.getName() );
int count;
byte data[] = new byte[BUFFER];
if (entry.isDirectory()) {
new File( destFolder + "/" + entry.getName() ).mkdirs();
continue;
} else {
int di = entry.getName().lastIndexOf( '/' );
if (di != -1) {
new File( destFolder + "/" + entry.getName()
.substring( 0, di ) ).mkdirs();
}
}
FileOutputStream fos = new FileOutputStream( destFolder + "/"
+ entry.getName() );
dest = new BufferedOutputStream( fos );
while (( count = zis.read( data ) ) != -1)
dest.write( data, 0, count );
dest.flush();
dest.close();
}
}

You are able to extract .gz, .zip, .jar files as they use number of compression algorithms built into the Java SDK.
The case with RAR format is a bit different. RAR is a proprietary archive file format. RAR license does not allow to include it into software development tools like Java SDK.
The best way to unrar your files will be using 3rd party libraries such as junrar.
You can find some references to other Java RAR libraries in SO question RAR archives with java. Also SO question How to compress text file to rar format using java program explains more on different workarounds (e.g. using Runtime).

You can use the library junrar
<dependency>
<groupId>com.github.junrar</groupId>
<artifactId>junrar</artifactId>
<version>0.7</version>
</dependency>
Code example:
File f = new File(filename);
Archive archive = new Archive(f);
archive.getMainHeader().print();
FileHeader fh = archive.nextFileHeader();
while(fh!=null){
File fileEntry = new File(fh.getFileNameString().trim());
System.out.println(fileEntry.getAbsolutePath());
FileOutputStream os = new FileOutputStream(fileEntry);
archive.extractFile(fh, os);
os.close();
fh=archive.nextFileHeader();
}

You can use http://sevenzipjbind.sourceforge.net/index.html
In addition to supporting a large number of archive formats, version 16.02-2.01 has full support for RAR5 extraction with:
password protected archives
archives with encrypted headers
archives splitted in volumes
gradle
implementation 'net.sf.sevenzipjbinding:sevenzipjbinding:16.02-2.01'
implementation 'net.sf.sevenzipjbinding:sevenzipjbinding-all-platforms:16.02-2.01'
or maven
<dependency>
<groupId>net.sf.sevenzipjbinding</groupId>
<artifactId>sevenzipjbinding</artifactId>
<version>16.02-2.01</version>
</dependency>
<dependency>
<groupId>net.sf.sevenzipjbinding</groupId>
<artifactId>sevenzipjbinding-all-platforms</artifactId>
<version>16.02-2.01</version>
</dependency>
And code example
import net.sf.sevenzipjbinding.ExtractOperationResult;
import net.sf.sevenzipjbinding.IInArchive;
import net.sf.sevenzipjbinding.SevenZip;
import net.sf.sevenzipjbinding.impl.RandomAccessFileInStream;
import net.sf.sevenzipjbinding.simple.ISimpleInArchiveItem;
import java.io.*;
import java.util.HashMap;
import java.util.Map;
/**
* Responsible for unpacking archives with the RAR extension.
* Support Rar4, Rar4 with password, Rar5, Rar5 with password.
* Determines the type of archive itself.
*/
public class RarExtractor {
/**
* Extracts files from archive. Archive can be encrypted with password
*
* #param filePath path to .rar file
* #param password string password for archive
* #return map of extracted file with file name
* #throws IOException
*/
public Map<InputStream, String> extract(String filePath, String password) throws IOException {
Map<InputStream, String> extractedMap = new HashMap<>();
RandomAccessFile randomAccessFile = new RandomAccessFile(filePath, "r");
RandomAccessFileInStream randomAccessFileStream = new RandomAccessFileInStream(randomAccessFile);
IInArchive inArchive = SevenZip.openInArchive(null, randomAccessFileStream);
for (ISimpleInArchiveItem item : inArchive.getSimpleInterface().getArchiveItems()) {
if (!item.isFolder()) {
ExtractOperationResult result = item.extractSlow(data -> {
extractedMap.put(new BufferedInputStream(new ByteArrayInputStream(data)), item.getPath());
return data.length;
}, password);
if (result != ExtractOperationResult.OK) {
throw new RuntimeException(
String.format("Error extracting archive. Extracting error: %s", result));
}
}
}
return extractedMap;
}
}
P.S.
#BorisBrodski https://github.com/borisbrodski Happy 40th birthday to you! Hope you had a great celebration. Thanks for your work!

you could simply add this maven dependency to you code:
<dependency>
<groupId>com.github.junrar</groupId>
<artifactId>junrar</artifactId>
<version>0.7</version>
</dependency>
and then use this code for extract rar file:
File rar = new File("path_to_rar_file.rar");
File tmpDir = File.createTempFile("bip.",".unrar");
if(!(tmpDir.delete())){
throw new IOException("Could not delete temp file: " + tmpDir.getAbsolutePath());
}
if(!(tmpDir.mkdir())){
throw new IOException("Could not create temp directory: " + tmpDir.getAbsolutePath());
}
System.out.println("tmpDir="+tmpDir.getAbsolutePath());
ExtractArchive extractArchive = new ExtractArchive();
extractArchive.extractArchive(rar, tmpDir);
System.out.println("finished.");

This method helps to extract files to streams from rar(RAR5) file stream if you have input stream. In my case I was processing MimeBodyPart from email.
The example from #Alexey Bril didn't work for me.
Dependencies are the same
Gradle
implementation 'net.sf.sevenzipjbinding:sevenzipjbinding:16.02-2.01'
implementation 'net.sf.sevenzipjbinding:sevenzipjbinding-all-platforms:16.02-2.01'
Maven
<dependency>
<groupId>net.sf.sevenzipjbinding</groupId>
<artifactId>sevenzipjbinding</artifactId>
<version>16.02-2.01</version>
</dependency>
<dependency>
<groupId>net.sf.sevenzipjbinding</groupId>
<artifactId>sevenzipjbinding-all-platforms</artifactId>
<version>16.02-2.01</version>
</dependency>
Code
private List<InputStream> getInputStreamsFromRar5InputStream(InputStream is) throws IOException {
List<InputStream> inputStreams = new ArrayList<>();
File tempFile = File.createTempFile("tempRarArchive-", ".rar", null);
try (FileOutputStream fos = new FileOutputStream(tempFile)) {
fos.write(is.readAllBytes());
fos.flush();
try (RandomAccessFile raf = new RandomAccessFile(tempFile, "r")) {// open for reading
try (IInArchive inArchive = SevenZip.openInArchive(null, // autodetect archive type
new RandomAccessFileInStream(raf))) {
// Getting simple interface of the archive inArchive
ISimpleInArchive simpleInArchive = inArchive.getSimpleInterface();
for (ISimpleInArchiveItem item : simpleInArchive.getArchiveItems()) {
if (!item.isFolder()) {
ExtractOperationResult result;
final InputStream[] IS = new InputStream[1];
final Integer[] sizeArray = new Integer[1];
result = item.extractSlow(new ISequentialOutStream() {
/**
* #param bytes of extracted data
* #return size of extracted data
*/
#Override
public int write(byte[] bytes) {
InputStream is = new ByteArrayInputStream(bytes);
sizeArray[0] = bytes.length;
IS[0] = new BufferedInputStream(is); // Data to write to file
return sizeArray[0];
}
});
if (result == ExtractOperationResult.OK) {
inputStreams.add(IS[0]);
} else {
log.error("Error extracting item: " + result);
}
}
}
}
}
} finally {
tempFile.delete();
}
return inputStreams;
}

How to copy a file on the FTP server to a directory on the same server in Java?

I'm using Apache Commons FTP to upload a file. Before uploading I want to check if the file already exists on the server and make a backup from it to a backup directory on the same server.
Does anyone know how to copy a file from a FTP server to a backup directory on the same server?
public static void uploadWithCommonsFTP(File fileToBeUpload){
FTPClient f = new FTPClient();
FTPFile backupDirectory;
try {
f.connect(server.getServer());
f.login(server.getUsername(), server.getPassword());
FTPFile[] directories = f.listDirectories();
FTPFile[] files = f.listFiles();
for(FTPFile file:directories){
if (!file.getName().equalsIgnoreCase("backup")) {
backupDirectory=file;
} else {
f.makeDirectory("backup");
}
}
for(FTPFile file: files){
if(file.getName().equals(fileToBeUpload.getName())){
//copy file to backupDirectory
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
Edited code: still there is a problem, when i backup zip file, the backup-ed file is corrupted.
Does any body know the reason for it?
public static void backupUploadWithCommonsFTP(File fileToBeUpload) {
FTPClient f = new FTPClient();
boolean backupDirectoryExist = false;
boolean fileToBeUploadExist = false;
FTPFile backupDirectory = null;
try {
f.connect(server.getServer());
f.login(server.getUsername(), server.getPassword());
FTPFile[] directories = f.listDirectories();
// Check for existence of backup directory
for (FTPFile file : directories) {
String filename = file.getName();
if (file.isDirectory() && filename.equalsIgnoreCase("backup")) {
backupDirectory = file;
backupDirectoryExist = true;
break;
}
}
if (!backupDirectoryExist) {
f.makeDirectory("backup");
}
// Check if file already exist on the server
f.changeWorkingDirectory("files");
FTPFile[] files = f.listFiles();
f.changeWorkingDirectory("backup");
String filePathToBeBackup="/home/user/backup/";
String prefix;
String suffix;
String fileNameToBeBackup;
FTPFile fileReadyForBackup = null;
f.setFileType(FTP.BINARY_FILE_TYPE);
f.setFileTransferMode(FTP.BINARY_FILE_TYPE);
for (FTPFile file : files) {
if (file.isFile() && file.getName().equals(fileToBeUpload.getName())) {
prefix = FilenameUtils.getBaseName(file.getName());
suffix = ".".concat(FilenameUtils.getExtension(file.getName()));
fileNameToBeBackup = prefix.concat(Calendar.getInstance().getTime().toString().concat(suffix));
filePathToBeBackup = filePathToBeBackup.concat(fileNameToBeBackup);
fileReadyForBackup = file;
fileToBeUploadExist = true;
break;
}
}
// If file already exist on the server create a backup from it otherwise just upload the file.
if(fileToBeUploadExist){
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
f.retrieveFile(fileReadyForBackup.getName(), outputStream);
InputStream is = new ByteArrayInputStream(outputStream.toByteArray());
if(f.storeUniqueFile(filePathToBeBackup, is)){
JOptionPane.showMessageDialog(null, "Backup succeeded.");
f.changeWorkingDirectory("files");
boolean reply = f.storeFile(fileToBeUpload.getName(), new FileInputStream(fileToBeUpload));
if(reply){
JOptionPane.showMessageDialog(null,"Upload succeeded.");
}else{
JOptionPane.showMessageDialog(null,"Upload failed after backup.");
}
}else{
JOptionPane.showMessageDialog(null,"Backup failed.");
}
}else{
f.changeWorkingDirectory("files");
f.setFileType(FTP.BINARY_FILE_TYPE);
f.enterLocalPassiveMode();
InputStream inputStream = new FileInputStream(fileToBeUpload);
ByteArrayInputStream in = new ByteArrayInputStream(FileUtils.readFileToByteArray(fileToBeUpload));
boolean reply = f.storeFile(fileToBeUpload.getName(), in);
System.out.println("Reply code for storing file to server: " + reply);
if(!f.completePendingCommand()) {
f.logout();
f.disconnect();
System.err.println("File transfer failed.");
System.exit(1);
}
if(reply){
JOptionPane.showMessageDialog(null,"File uploaded successfully without making backup." +
"\nReason: There wasn't any previous version of this file.");
}else{
JOptionPane.showMessageDialog(null,"Upload failed.");
}
}
//Logout and disconnect from server
in.close();
f.logout();
f.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}

If you are using apache commons net FTPClient, there is a direct method to move a file from one location to another location (if the user has proper permissions).
ftpClient.rename(from, to);
or, If you are familiar with ftp commands, you can use something like
ftpClient.sendCommand(FTPCommand.yourCommand, args);
if(FTPReply.isPositiveCompletion(ftpClient.getReplyCode())) {
//command successful;
} else {
//check for reply code, and take appropriate action.
}
If you are using any other client, go through the documentation, There wont be much changes between client implementations.
UPDATE:
Above approach moves the file to to directory, i.e, the file won't be there in from directory anymore. Basically ftp protocol meant to be transfer the files from local <-> remote or remote <-> other remote but not to transfer with in the server.
The work around here, would be simpler, get the complete file to a local InputStream and write it back to the server as a new file in the back up directory.
to get the complete file,
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ftpClient.retrieveFile(fileName, outputStream);
InputStream is = new ByteArrayInputStream(outputStream.toByteArray());
now, store this stream to backup directory. First we need to change working directory to backup directory.
// assuming backup directory is with in current working directory
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);//binary files
ftpClient.changeWorkingDirectory("backup");
//this overwrites the existing file
ftpClient.storeFile(fileName, is);
//if you don't want to overwrite it use storeUniqueFile
Hope this helps you..

Try this way,
I am using apache's library .
ftpClient.rename(from, to) will make it easier, i have mentioned in the code below
where to add ftpClient.rename(from,to).
public void goforIt(){
FTPClient con = null;
try
{
con = new FTPClient();
con.connect("www.ujudgeit.net");
if (con.login("ujud3", "Stevejobs27!!!!"))
{
con.enterLocalPassiveMode(); // important!
con.setFileType(FTP.BINARY_FILE_TYPE);
String data = "/sdcard/prerakm4a.m4a";
ByteArrayInputStream(data.getBytes());
FileInputStream in = new FileInputStream(new File(data));
boolean result = con.storeFile("/Ads/prerakm4a.m4a", in);
in.close();
if (result)
{
Log.v("upload result", "succeeded");
//$$$$$$$$$$$$$$$$$$$$$$$$$$$$$Add the backup Here$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$//
// Now here you can store the file into a backup location
// Use ftpClient.rename(from, to) to place it in backup
//$$$$$$$$$$$$$$$$$$$$$$$$$$$$$Add the backup Here$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$//
}
con.logout();
con.disconnect();
}
}
catch (Exception e)
{
e.printStackTrace();
}
}

There's no standard way to duplicate a remote file over FTP protocol. Some FTP servers support proprietary or non-standard extensions for this though.
So if your are lucky that your server is ProFTPD with mod_copy module, you can use FTP.sendCommand to issue these two commands:
f.sendCommand("CPFR sourcepath");
f.sendCommand("CPTO targetpath");
The second possibility is that your server allows you to execute arbitrary shell commands. This is even less common. If your server supports this you can use SITE EXEC command:
SITE EXEC cp -p sourcepath targetpath
Another workaround is to open a second connection to the FTP server and make the server upload the file to itself by piping a passive mode data connection to an active mode data connection. Implementation of this solution (in PHP though) is shown in FTP copy a file to another place in same FTP.
If neither of this works, all you can do is to download the file to a local temporary location and re-upload it back to the target location. This is that the answer by #RP- shows.
See also FTP copy a file to another place in same FTP.

To backup at same Server (move), can you use:
String source="/home/user/some";
String goal ="/home/user/someOther";
FTPFile[] filesFTP = cliente.listFiles(source);
clientFTP.changeWorkingDirectory(goal); // IMPORTANT change to final directory
for (FTPFile f : archivosFTP)
{
if(f.isFile())
{
cliente.rename(source+"/"+f.getName(), f.getName());
}
}

java: how to get a string representation of a compressed byte array?

I want to put some compressed data into a remote repository.
To put data on this repository I can only use a method that take the name of the resource and its content as a String. (like data.txt + "hello world").
The repository is moking a filesystem but is not, so I can not use File directly.
I want to be able to do the following:
client send to server a file 'data.txt'
server compress 'data.txt' into a compressed file 'data.zip'
server send a string representation of data.zip to the repository
repository store data.zip
client download from repository data.zip and his able to open it with its favorite zip tool
The problem arise at step 3 when I try to get a string representation of my compressed file.
Here is a sample class, using the zip*stream and that emulate the repository showcasing my problem.
The created zip file is working, but after its 'serialization' it's get corrupted.
(the sample class use jakarta commons.io )
Many thanks for your help.
package zip;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
import org.apache.commons.io.FileUtils;
/**
* Date: May 19, 2010 - 6:13:07 PM
*
* #author Guillaume AME.
*/
public class ZipMe {
public static void addOrUpdate(File zipFile, File ... files) throws IOException {
File tempFile = File.createTempFile(zipFile.getName(), null);
// delete it, otherwise you cannot rename your existing zip to it.
tempFile.delete();
boolean renameOk = zipFile.renameTo(tempFile);
if (!renameOk) {
throw new RuntimeException("could not rename the file " + zipFile.getAbsolutePath() + " to " + tempFile.getAbsolutePath());
}
byte[] buf = new byte[1024];
ZipInputStream zin = new ZipInputStream(new FileInputStream(tempFile));
ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipFile));
ZipEntry entry = zin.getNextEntry();
while (entry != null) {
String name = entry.getName();
boolean notInFiles = true;
for (File f : files) {
if (f.getName().equals(name)) {
notInFiles = false;
break;
}
}
if (notInFiles) {
// Add ZIP entry to output stream.
out.putNextEntry(new ZipEntry(name));
// Transfer bytes from the ZIP file to the output file
int len;
while ((len = zin.read(buf)) > 0) {
out.write(buf, 0, len);
}
}
entry = zin.getNextEntry();
}
// Close the streams
zin.close();
// Compress the files
if (files != null) {
for (File file : files) {
InputStream in = new FileInputStream(file);
// Add ZIP entry to output stream.
out.putNextEntry(new ZipEntry(file.getName()));
// Transfer bytes from the file to the ZIP file
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
// Complete the entry
out.closeEntry();
in.close();
}
// Complete the ZIP file
}
tempFile.delete();
out.close();
}
public static void main(String[] args) throws IOException {
final String zipArchivePath = "c:/temp/archive.zip";
final String tempFilePath = "c:/temp/data.txt";
final String resultZipFile = "c:/temp/resultingArchive.zip";
File zipArchive = new File(zipArchivePath);
FileUtils.touch(zipArchive);
File tempFile = new File(tempFilePath);
FileUtils.writeStringToFile(tempFile, "hello world");
addOrUpdate(zipArchive, tempFile);
//archive.zip exists and contains a compressed data.txt that can be read using winrar
//now simulate writing of the zip into a in memory cache
String archiveText = FileUtils.readFileToString(zipArchive);
FileUtils.writeStringToFile(new File(resultZipFile), archiveText);
//resultingArchive.zip exists, contains a compressed data.txt, but it can not
//be read using winrar: CRC failed in data.txt. The file is corrupt
}
}

Zip files are binary. String handling in Java is textual and might be mangling what it sees as CRLFs, zero bytes and EOF markers. When it comes to reading and rewriting the zipfile, I suggest you try with readFileToByteArray and writeByteArrayToFile as an experiment. If that works then I'd suspect the String handling is to blame.

server send a string representation of
data.zip to the repository
So you want to get a string (i.e. textual) representation of a zip (i.e. binary) stream.
Base64 is the most popular way to do this.
One popular Java implementation is from Apache commons (codec component)