Develop Reference

Provide log if the two files are identical and has same contents in Java

I have below code where i am reading the file from particular directory, processing it and once processed i am moving the file to archive directory. This is working fine. I am receiving new file everyday and i am using Control-M scheduler job to run this process.
Now in next run i am reading the new file from that particularly directory again and checking this file with the file in the archive directory and if the content is different then only process the file else dont do anything. There is shell script written to do this job and we dont see any log for this process.
Now i want to produce log message in my java code if the files are identical from the particular directory and in the archive directory then generate log that 'files are identical'. But i dont know exactly how to do this. I dont want to write the the logic to process or move anything in the file ..i just need to check the files are equal and if it is then
produce log message. The file which i recieve are not very big and the max size can be till 10MB.
Below is my code:
for(Path inputFile : pathsToProcess) {
// read in the file:
readFile(inputFile.toAbsolutePath().toString());
// move the file away into the archive:
Path archiveDir = Paths.get(applicationContext.getEnvironment().getProperty(".archive.dir"));
Files.move(inputFile, archiveDir.resolve(inputFile.getFileName()),StandardCopyOption.REPLACE_EXISTING);
}
return true;
}
private void readFile(String inputFile) throws IOException, FileNotFoundException {
log.info("Import " + inputFile);
try (InputStream is = new FileInputStream(inputFile);
Reader underlyingReader = inputFile.endsWith("gz")
? new InputStreamReader(new GZIPInputStream(is), DEFAULT_CHARSET)
: new InputStreamReader(is, DEFAULT_CHARSET);
BufferedReader reader = new BufferedReader(underlyingReader)) {
if (isPxFile(inputFile)) {
Importer.processField(reader, tablenameFromFilename(inputFile));
} else {
Importer.processFile(reader, tablenameFromFilename(inputFile));
}
}
log.info("Import Complete");
}
}

Based on the limited information about the size of file or performance needs, something like this can be done. This may not be 100% optimized, but just an example. You may also have to do some exception handling in the main method, since the new method might throw an IOException:
import org.apache.commons.io.FileUtils; // Add this import statement at the top
// Moved this statement outside the for loop, as it seems there is no need to fetch the archive directory path multiple times.
Path archiveDir = Paths.get(applicationContext.getEnvironment().getProperty("betl..archive.dir"));
for(Path inputFile : pathsToProcess) {
// Added this code
if(checkIfFileMatches(inputFile, archiveDir); {
// Add the logger here.
}
//Added the else condition, so that if the files do not match, only then you read, process in DB and move the file over to the archive.
else {
// read in the file:
readFile(inputFile.toAbsolutePath().toString());
Files.move(inputFile, archiveDir.resolve(inputFile.getFileName()),StandardCopyOption.REPLACE_EXISTING);
}
}
//Added this method to check if the source file and the target file contents are same.
// This will need an import of the FileUtils class. You may change the approach to use any other utility file, or read the data byte by byte and compare. If the files are very large, probably better to use Buffered file reader.
private boolean checkIfFileMatches(Path sourceFilePath, Path targetDirectoryPath) throws IOException {
if (sourceFilePath != null) { // may not need this check
File sourceFile = sourceFilePath.toFile();
String fileName = sourceFile.getName();
File targetFile = new File(targetDirectoryPath + "/" + fileName);
if (targetFile.exists()) {
return FileUtils.contentEquals(sourceFile, targetFile);
}
}
return false;
}

Creating zip with directory containing special characters

I'm trying to create a zip archive with some directories inside. Some of directories has Polish letters in the name like: ą, ę, ł, etc. Everything looks fine except that for any directory with special letter in name there is a another one created in the zip file. What is wrong with the following code:
import java.io.File;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
public class Main {
public static void main(String[] args) throws URISyntaxException, IOException {
Map<String, String> env = new HashMap<>();
env.put("create", "true");
URI fileUri = new File("zipfs.zip").toPath().toUri();
URI zipUri = new URI("jar:" + fileUri.getScheme(), fileUri.getPath(), null);
try (FileSystem zipfs = FileSystems.newFileSystem(zipUri, env)) {
Path directory = zipfs.getPath("ą");
Files.createDirectory(directory);
Path pathInZipfile = directory.resolve("someFile.txt");
Path source = Paths.get("source.txt");
Files.copy(source, pathInZipfile, StandardCopyOption.REPLACE_EXISTING);
}
FileSystem zipFs = FileSystems.newFileSystem(zipUri, Collections.emptyMap());
Path root = zipFs.getPath("/");
Files.walkFileTree(root, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult visitFile(Path path, BasicFileAttributes attrs) throws IOException {
System.out.println(path);
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
System.out.println(dir);
return super.preVisitDirectory(dir, attrs);
}
});
}
}
The output of this program is as expected:
/
/ą/
/ą/someFile.txt
But when you open created zip file there are two directories inside:
Ä?
ą
First one is empty and text file is as it should be in the 'ą' directory.

It seems ZipFileSystem doesn't set the Language encoding flag (EFS) with folders. This flag basically says "this path uses UTF-8".
Let's see with zipdetails (skipping not interesting lines):
0072 CENTRAL HEADER #1 02014B50
007A General Purpose Flag 0000 // <= no EFS flag
00A0 Filename 'ą/'
00AC CENTRAL HEADER #2 02014B50
00B4 General Purpose Flag 0800
[Bits 1-2] 0 'Normal Compression'
[Bit 11] 1 'Language Encoding' // <= EFS flag
00DA Filename 'ą/someFile.txt'
Otherwise, ą/ is correctly encoded in UTF-8.
Without this flag, it's up to the program reading/extracting the zip file to choose an encoding (usually the system default). unzip doesn't work well here:
$ unzip -t zipfs.zip
Archive: zipfs.zip
testing: -à/ OK
testing: ą/someFile.txt OK
No errors detected in compressed data of zipfs.zip.
Note, if you disable the unicode support with -UU, you get -à in both entries.
7z works better here (but only because my system default encoding is UTF-8):
$ 7z l zipfs.zip
...
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2017-01-10 22:51:14 D.... 0 0 ą
2017-01-10 22:51:15 ..... 0 2 ą/someFile.txt
------------------- ----- ------------ ------------ ------------------------
2017-01-10 22:51:15 0 2 1 files, 1 folders
If you can't force the way the zip file is opened (if the zip file is sent to users instead of one of your server for example) or only use ASCII characters in your folders, using a different library looks like the only solution.

Saving file to certain path (Java)

When I run this as a jar, this .properties file is created on the desktop. For the sake of keeping things clean, how can I set the path to save this file somewhere else, like a folder? Or even the jar itself but I was having trouble getting that to work. I plan on giving this to someone and wouldn't want their desktop cluttered in .properties files..
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.Properties;
public class DataFile {
public static void main(String[] args) {
Properties prop = new Properties();
OutputStream output = null;
try {
output = new FileOutputStream("config.properties");
prop.setProperty("prop1", "000");
prop.setProperty("prop2", "000");
prop.setProperty("prop3", "000");
prop.store(output, null);
} catch (IOException io) {
io.printStackTrace();
} finally {
if (output != null) {
try {
output.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}

Since you are using the file name without a path, the file you creates ends in the CWD. That is the Current working directory the process inherits from the OS.
That is is you execute your jar file from the Desktop directory any files that use relative paths will end in the Desktop or any of it sub directories.
To control the absolute location of the file you must use absolute paths.
An absolute path always starts with a slash '/'.
Absolute path:
/etc/config.properties
Relative path:
sub_dir/config.properties
The simplest way is to hard code some path into the file path string.
output = new FileOutputStream("/etc/config.properties");
You can of course setup the path in a property which you can pass using the command line instead of hard coding it. The you concat the path name and the file name together.
String path = "/etc";
String full_path = "/etc" + "/" + "config.properties";
output = new FileOutputStream(full_path);
Please note that windows paths in java use a forward slash instead of back slash.
Check this for more details
file path Windows format to java format

Decompress folder without overwriting new files

I want to decompress a large folder in ZIP format with nested subdirectories in a directory that already exists. The files inside the ZIP folder can exist in the decompressed directory. I need to keep the previous files only when the date of that file is newer than the date of the file in the ZIP folder. If the file in the ZIP is newer, then I want to overwrite it.
There is some good strategy for doing this? I already checked truezip and zip4j, but I can't find the option (the best option for me so far is modifying the zip4j sources, but it should be a better way.
P.S. If I haven't explained this correctly, please feel free to ask. English is not my native language and I could have expressed anything wrong..
Thanks.

With Zip4j, this is how it can be done:
import java.io.File;
import java.util.Date;
import java.util.List;
import net.lingala.zip4j.core.ZipFile;
import net.lingala.zip4j.model.FileHeader;
import net.lingala.zip4j.util.Zip4jUtil;
public class ExtractWithoutOverwriting {
public static void main(String[] args) {
try {
String outputPath = "yourOutputPath";
ZipFile zipFile = new ZipFile(new File("yourZipFile.zip"));
if (zipFile.isEncrypted()) {
zipFile.setPassword("yourPassword".toCharArray());
}
#SuppressWarnings("unchecked")
List<FileHeader> fileHeaders = zipFile.getFileHeaders();
for (FileHeader fileHeader : fileHeaders) {
if (fileHeader.isDirectory()) {
File file = new File(outputPath + System.getProperty("file.separator") + fileHeader.getFileName());
file.mkdirs();
} else {
if (canWrite(outputPath, fileHeader)) {
System.out.println("Writing file: " + fileHeader.getFileName());
zipFile.extractFile(fileHeader, outputPath);
} else {
System.out.println("Not writing file: " + fileHeader.getFileName());
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
private static boolean canWrite(String outputPath, FileHeader fileHeader) {
File file = new File(outputPath + System.getProperty("file.separator") + fileHeader.getFileName());
//time stamps are stored in dos format in a zip file
//convert it to java format
long lastModifiedFromZip = Zip4jUtil.dosToJavaTme(fileHeader.getLastModFileTime());
//If the file exists, it can be overwritten only if the file in the destination path
//is newer than the one in the zip file
return !(file.exists() && isLastModifiedDateFromFileNewer(file.lastModified(), lastModifiedFromZip));
}
public static boolean isLastModifiedDateFromFileNewer(long lastModifiedFromFile, long lastModifiedFromZip) {
Date lastModifiedDateFromFile = new Date(lastModifiedFromFile);
Date lastModifiedDateFromZip = new Date(lastModifiedFromZip);
return lastModifiedDateFromFile.after(lastModifiedDateFromZip);
}
}
What we do here is:
Create a new instance of the ZipFile
If the zip file is encrypted, set a password
Loop over all files in the zip file
Check if a file with this name exists in the output path and if this file's last modification time is "newer" than the one in the zip file. This check is done in the method: canWrite()
This code is not completely tested, but I hope it gives you an idea of a solution.

How to create a file -- including folders -- for a given path?

Am downloading a zip file from web. It contain folders and files. Uncompressing them using ZipInputstream and ZipEntry. Zipentry.getName gives the name of file as htm/css/aaa.htm.
So I am creating new File(zipentry.getName);
But problem it is throwing an exception: File not found. I got that it is creating subfolders htm and css.
My question is: how to create a file including its sub directories, by passing above path?

Use this:
File targetFile = new File("foo/bar/phleem.css");
File parent = targetFile.getParentFile();
if (parent != null && !parent.exists() && !parent.mkdirs()) {
throw new IllegalStateException("Couldn't create dir: " + parent);
}
While you can just do file.getParentFile().mkdirs() without checking the result, it's considered a best practice to check for the return value of the operation. Hence the check for an existing directory first and then the check for successful creation (if it didn't exist yet).
Also, if the path doesn't include any parent directory, parent would be null. Check it for robustness.
Reference:
File.getParentFile()
File.exists()
File.mkdir()
File.mkdirs()

You can use Google's guava library to do it in a couple of lines with Files class:
Files.createParentDirs(file);
Files.touch(file);
https://code.google.com/p/guava-libraries/

Java NIO API Files.createDirectories
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
Path path = Paths.get("/folder1/folder2/folder3");
Files.createDirectories(path);

You need to create subdirectories if necessary, as you loop through the entries in the zip file.
ZipFile zipFile = new ZipFile(myZipFile);
Enumeration e = zipFile.entries();
while(e.hasMoreElements()){
ZipEntry entry = (ZipEntry)e.nextElement();
File destinationFilePath = new File(entry.getName());
destinationFilePath.getParentFile().mkdirs();
if(!entry.isDirectory()){
//code to uncompress the file
}
}

This is how I do it
static void ensureFoldersExist(File folder) {
if (!folder.exists()) {
if (!folder.mkdirs()) {
ensureFoldersExist(folder.getParentFile());
}
}
}

Looks at the file you use the .mkdirs() method on a File object: http://www.roseindia.net/java/beginners/java-create-directory.shtml
isDirectoryCreated = (new File("../path_for_Directory/Directory_Name")).mkdirs();
if (!isDirectoryCreated)
{
// Directory creation failed
}