I currently have a file monitor, which reads an XML file and updates database tables base on the contents. When a file is read it is validated against a specific schema to ensure the contents are correct.
The result of the validation effects the destination of the file to move.
If the schema is successful, there are no issues when moving the file to the processed directory, however, when the schema fails I am unable to rename the file itself.
However, when I debug and explore the file paths to ensure the file is being moved to the right location, it often allows me to rename the file which makes me think this is a timing issue.
File Monitor Class :
// Get a directory listing;
FilterFiles filterFiles = new FilterFiles();
String inbox = GlobalVars.getConfiguration().getInboundInbox();
String [] dir = new File(inbox).list(filterFiles);
// Wait 500 ms before obtaining files (Avoid the ResourceBusy error);
try {
Thread.sleep(500);
} catch (InterruptedException e) {
// Log errors
logger.info("Interrupted Exception " + e.getMessage());
new LogStackTrace(logger, e);
e.printStackTrace();
}
// Iterate through directory;
for ( int a=0; a < dir.length; a++ ) {
// Derive the full file path i.e. folder + path name;
String fullFilePath = GetAbsoloutePath.getFullAbsoluteFilePath(inbox, dir[a]);
logger.info("==========================================================");
logger.info("");
logger.info("Found File : " + fullFilePath);
logger.info("==========================================================");
// Does the file exist & can we read the file at this time;
File file = new File(fullFilePath);
if (file.exists() && file.canRead()) {
// Process the file;
StringBuffer renamedFile = new StringBuffer();
// Rename the file to indicate InProgess;
String fileName = file.getAbsolutePath();
String fileNameInProgress = fileName.trim() + ".InProgress";
// Delete the file if exists where we are moving the file to;
File deleteFile = new File(fileNameInProgress);
if (deleteFile.delete()) logger.info("Existing file deleted: " + fileNameInProgress);
// Ensure file is renamed so we can see it is in progress
if (!file.renameTo(new File(fileNameInProgress)) ) {
// Logging
logger.error("Failed to renamed file :" + fileName + " to " + fileNameInProgress);
break;
}else{
logger.info("Renamed file to : " + fileNameInProgress);
// Pass back name of renamed file;
renamedFile.append(fileNameInProgress);
File renamedFileRef = new File(renamedFile.toString());
// Path to move - could be errors or processed
String pathToMove = "";
// Parse XMobject
ParseInboundXML par = new ParseInboundXML();
// check if parse was succesful
if(par.parseXML(renamedFileRef, con)){
// Path to move XML file
pathToMove = GlobalVars.getConfiguration().getInboundProcessed()+ "\\" + file.getName();
//Logging
logger.info("File parsed and tables updated successfully");
logger.info("Moving file to : " + pathToMove);
}else{
pathToMove = GlobalVars.getConfiguration().getInboundErrors()+ "\\" + file.getName();
//Logging
logger.error("Errors when parsing file and updating tables");
logger.error("Moving file to : " + pathToMove);
}
// Help with garbage collection
par = null;
// New file
File newPath = new File(pathToMove);
// Need to check if this already exists in processed- if so we must override the existing file
// Otherwise the move will not be processed and the same docuemnt will be continously processed
if(newPath.exists())
newPath.delete();
After the above code has been executed, this block of code is executed and this is where the rename fails:
// Rename path so it is placed in processed folder
if(renamedFileRef.renameTo(newPath)){
//Logging
logger.info("File processed successfully");
}else{
// Logging
logger.error("Unable process file");
}
Within the code you may notice the call to :
// Parse XMobject
ParseInboundXML par = new ParseInboundXML();
// check if parse was succesful
if(par.parseXML(renamedFileRef, con)){
This method is where the validation occurs and the parsing of the XML document and returns a true or false value depending on the outcome.
The area in which it fails is at the start of the method :
// Find schema path
String schemaPath = GlobalVars.getConfiguration().getInboundSchemaLocation();
String schemaFileName = "WMS_" + file.getAbsoluteFile().toString().split("#")[2] + ".xsd";
// Schema location
String schemaLocation = schemaPath + "\\" + schemaFileName;
// Create schema factory
SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
// Schema file
Source schemaFile = new StreamSource(new File(schemaLocation));
// Apply schema and validate XML
Schema schema = schemaFactory.newSchema(schemaFile);
Validator validator = schema.newValidator();
Source xmlFile = new StreamSource(file);
validator.validate(xmlFile);
I think the issue may lie around
Source xmlFile = new StreamSource(file);
as this is the file we try to rename, and i think if it fails validation, its possible that this may be keeping the file open and would explain why there are no issues when it passes validation.
EDIT
This is the catch clause for a failed validation:
catch (SAXException e) {
// Write Error Record
CreateErrorMessages.createIW702Message(record.getTrno701TransactionNumber(), IntegrationConstants.SAX_ERROR, "SAX Error", e.toString(), con);
// Logging
logger.info("SAX Exception " + e.getMessage());
// Log stack trace
new LogStackTrace(logger, e);
// Prints to stack trace
e.printStackTrace();
return false;
}
If you want to later rename the file, it has to be closed first. Apparently, your code does not do that.
Closing StreamSource and Moving files after failed validation (Java) suggest to not pass in a File but instead a FileInputStream that you can then close at the end yourself (or use try-with-resource)
try(InputStream in = new FileInputStream (file)) {
Source xmlFile = new StreamSource(fr);
validator.validate(xmlFile);
}
Related
I was tasked to write a program to write some websites on a text file and extract those who have the same top level domains and write it to a separate text file. This is my code:
FileWriter myWriter, comWriter, eduWriter, orgWriter, netWriter;
File newFile = new File("URL.txt");
newFile.createNewFile();
try {
myWriter = new FileWriter("URL.txt");
myWriter.write("""
yahoo.com
google.com
usl.edu
battle.net
CSU.edu
shopee.com
xyz.org
spup.edu
slideshare.net
php.net""");
myWriter.close();
System.out.println("Successfully wrote to the file!");
} catch (IOException e) {
System.out.println("Program didn't write");
}
try {
Scanner scanner = new Scanner(newFile);
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
if (line.endsWith(".com")) {
comWriter = new FileWriter("com.txt");
comWriter.write(line);
comWriter.close();
}
if (line.endsWith(".edu")) {
eduWriter = new FileWriter("edu.txt");
eduWriter.write(line);
eduWriter.close();
}
if (line.endsWith(".net")) {
netWriter = new FileWriter("net.txt");
netWriter.write(line);
netWriter.close();
}
if (line.endsWith(".org")) {
orgWriter = new FileWriter("org.txt");
orgWriter.write(line);
orgWriter.close();
}
}
}
catch (Exception e) {
System.out.println("Error occurred");
}
My program creates new txt files. However, it only prints one line. For example, in my com.txt file it only prints "shopee.com"
Is there something that I'm missing? Improvements? Or wrongs maybe? I hope someone can help me. Thank you.
The way you are opening the files for writing always truncates the output files. If you call the FileWriter constructor with just the file name, then that file will be opened for writing and truncated.
In order to append to the file, you should use the constructor that takes a boolean argument that specifies whether you want to append or not:
comWriter = new FileWriter("com.txt", true);
You can also use Files.write method from NIO.2 instead of FileWriter:
List<String> lines = new ArrayList<>();
lines.add("yahoo.com");
lines.add("google.com");
Files.write(Paths.get("URL.txt"), lines);
Or
Files.write(Paths.get("URL.txt"), lines, StandardOpenOption.APPEND);
if you want to append to existing file.
Optionally you can create a list of lines like this:
List<String> lines = List.of("yahoo.com", "google.com");
The file is closed when all lines have been written automatically.
There are so many ways this sort of thing can be done. Here is one example using Java's NIO:
Creating the URL List file:
List<String> urlList = Arrays.asList(new String[] {"yahoo.com", "google.com", "usl.edu",
"battle.net", "CSU.edu", "shopee.com", "xyz.org", "spup.edu",
"slideshare.net", "php.net"});
try {
java.nio.file.Files.write(java.nio.file.Paths.get("URL.txt"),
urlList, java.nio.charset.StandardCharsets.UTF_8);
System.out.println("Successfully created the URL.txt file!");
}
catch (IOException ex) {
System.err.println("Error creating the URL.txt file!" +
System.lineSeparator() + ex.getMessage());
}
Reading the URL List file and creating Top-Level Domain (TLD) files:
/*
Creates Top-Level Domain (TLD) files that will store respective Domain Name Strings.
Returns a comma (,) delimited string containing the Number of TLD files created and
the number of Domain Name Strings added to the various TLD files created (in that
order).
Output to console can be commented out or removed from this method.
If the supplied source path does not exist then an 'IllegalArgumentexception' is
thrown.
If the supplied destination path does not exist then there will be an attempt to
create it.
*/
public static String createTopLevelFiles(String sourcePath, String destinationPath) {
// Does Source file exist? Throw exception if not.
sourcePath = sourcePath.replace("\\\\",File.separator).replace("\\",File.separator);
if (!java.nio.file.Files.exists(java.nio.file.Paths.get(sourcePath))) {
// No...
throw new IllegalArgumentException("createTopLevelFiles() Method Error!"
+ "Source file can not be found! (" + sourcePath + ")");
}
// Read Source file domains content into a List Interface Oject:
// =============================================================
List<String> urlList = null;
try {
java.nio.file.Path path = java.nio.file.Paths.get(sourcePath);
urlList = java.nio.file.Files.readAllLines(path, java.nio.charset.StandardCharsets.UTF_8);
if (urlList == null || urlList.isEmpty()) {
System.err.println("createTopLevelFiles() Method Warning! No Domains "
+ "available within URL List source file!");
return null; // Exit with null due to failure;
}
}
catch (IOException ex) {
System.err.println("createTopLevelFiles() Method Error!"
+ System.lineSeparator() + ex.getMessage());
return null; // Exit with null due to failure;
}
// =============================================================
// Check Destination Path - Create it if it doesn't already exist:
// =============================================================
destinationPath = destinationPath.replace("\\\\",File.separator).replace("\\",File.separator);
java.nio.file.Path destPath = java.nio.file.Paths.get(destinationPath);
if (!java.nio.file.Files.exists(destPath)) {
try {
// Creating the new directory because it doesn't exist in local system.
Files.createDirectories(destPath);
// System.out.println("Destination Directory Successfully Created!");
}
catch (IOException ex) {
System.err.println("Problem Occured While Creating The Destination "
+ "Directory!" + System.lineSeparator() + ex.getMessage());
return null; // Exit with null due to failure;
}
}
// =============================================================
// Create the Top-Level Domain files:
// =============================================================
int fileCount = 0; // Keep count of TLD file creations.
int failCount = 0; // Keep count of Saved Domain Name failures.
for (String domain : urlList) {
String fileName = domain.substring(domain.lastIndexOf(".") + 1) + ".txt";
String filePath = new StringBuilder(destinationPath)
.append(File.separator).append(fileName)
.toString().replace("\\\\",File.separator)
.replace("\\",File.separator);
try {
java.nio.file.Path fPath = java.nio.file.Paths.get(filePath);
// If the file doesn't already exist then create it...
if (!java.nio.file.Files.exists(fPath)) {
java.nio.file.Files.createFile(fPath);
System.out.println("Successfully created the \"" +
filePath.substring(filePath.lastIndexOf(File.separator) + 1)
+ "\" TLD file!");
fileCount++;
}
// Is the domain name already in file? if so ignore it.
List<String> tmpList = null;
tmpList = java.nio.file.Files.readAllLines(fPath, java.nio.charset.StandardCharsets.UTF_8);
if (tmpList.contains(domain)) {
tmpList.clear();
failCount++;
continue;
}
// Append the domain name String to file...
java.nio.file.Files.write(fPath, (domain + System.lineSeparator()).getBytes(java.nio.charset.StandardCharsets.UTF_8),
java.nio.file.StandardOpenOption.APPEND);
System.out.println("Successfully added the \"" + domain + "\" domain name to the '"
+ filePath.substring(filePath.lastIndexOf(File.separator) + 1)
+ "' TLD file!");
}
catch (IOException ex) {
failCount++;
System.err.println("WARNING - Error creating the " +
filePath.substring(filePath.lastIndexOf(File.separator) + 1) +
" file! | " + ex.getMessage());
}
}
// =============================================================
// Failure results (delete this code if you like).
// =============================================================
if (failCount > 0) {
System.err.println(failCount + " out of " + urlList.size() +
" domain names failed to be added!");
System.err.println("This could be due to the fact that the domain");
System.err.println("name is already stored or, one or more domain");
System.err.println("names were found to be invalid.");
}
// =============================================================
/* Return the total number of TLD files created and
the total number of domain names added to these
various TLD files. */
return new StringBuilder(String.valueOf(fileCount)).append(", ")
.append(String.valueOf((urlList.size() - failCount))).toString();
}
The method above will automatically create the destination path (directory path) within the local file system if it doesn't already exist and it automatically determines the file names required to create into that destination path based on the TLD domain name attached to each domain name string. Domain name strings are appended to their respective file once that file is created. If a Domain Name is already contained within a TLD file then that name is ignored and 1 is added to the failure count.
Read all comments in code.
Usage Example:
String result = createTopLevelFiles("URL.txt", "D:\\Top-Level_Domains");
System.out.println();
System.out.println("Result is: " + result);
Console output example:
Successfully created the "com.txt" TLD file!
Successfully added the "yahoo.com" domain name to the 'com.txt' TLD file!
Successfully added the "google.com" domain name to the 'com.txt' TLD file!
Successfully created the "edu.txt" TLD file!
Successfully added the "usl.edu" domain name to the 'edu.txt' TLD file!
Successfully created the "net.txt" TLD file!
Successfully added the "battle.net" domain name to the 'net.txt' TLD file!
Successfully added the "CSU.edu" domain name to the 'edu.txt' TLD file!
Successfully added the "shopee.com" domain name to the 'com.txt' TLD file!
Successfully created the "org.txt" TLD file!
Successfully added the "xyz.org" domain name to the 'org.txt' TLD file!
Successfully added the "spup.edu" domain name to the 'edu.txt' TLD file!
Successfully added the "slideshare.net" domain name to the 'net.txt' TLD file!
Successfully added the "php.net" domain name to the 'net.txt' TLD file!
Result is: 4, 10
4 TLD files created in destination path and 10 domain names were added to those files.
My goal is to move a file from one directory to another. The source is on a local drive and the destination is on a network drive. It doesn't matter whether I move or I copy then delete source. The file is approx 6GB.
What I've tried:
// C:\path\to\dir\file.bak
File source = new File(localRoot + backup);
// \\192.168.1.100\path\to\dir\file.bak
File dest = new File(storageRoot + "/" + storagePath + "/" + backup);
try {
log("Copying");
// I've tried copyFile as well.
FileUtils.copyFileToDirectory(source, dest);
log("copied");
} catch (Exception e) {
e.printStackTrace();
}
File source = new File(localRoot + backup);
File dest = new File(storageRoot + "/" + storagePath + "/" + backup);
try {
log("Copying");
// I've tried move and creating Paths instead of Files as well.
Files.copy(source.toPath(), dest.toPath());
log("copied")
} catch (Exception e) {
e.printStackTrace();
}
I've tried as well a manual method using Input, OutputStreams and reading bytes.
The results is that a file is created in the destination with the correct filename with 0 bytes, and the source file is rewritten from 6GB to 0 bytes. This happens for all methods I've tried, the only exception is that when I tried move, the source file was deleted rather than rewritten.
All code is in early development, please refrain from commenting on best practices.
Thank you, and what am I missing or what else can I try?
So i'm working on a simple Windows Explorer replacement. I want to add the ability to create Folders and Files. For some reason, it only works when i'm in my root or c:/ folder, but as soon as it's somewhere else (for example C:\Program Files (x86)) it doesn't work. I either get a java.io.IOException: Access Denied when i create a File and when i try to create a folder, no Exception comes up, but no folder is created.
This is my code for a new file:
String location = getPath();
String name = JOptionPane.showInputDialog("Fill in the name of the new file. \nDon't forget to add file type (.txt, .pdf).", null);
if(name == null){
}
else {
File newFile = new File(location + "\\" + name);
boolean flag = false;
try {
flag = newFile.createNewFile();
} catch (IOException Io) {
JFrame messageDialog = new JFrame("Error!");
JOptionPane.showMessageDialog(messageDialog, "File creation failed with the following reason: \n" + Io);
}
}
This is my code for a new Folder:
String location = getPath();
String name = JOptionPane.showInputDialog("Fill in the name of the new folder.", null);
if(name == null){
}
else {
File newFolder = new File(location + "\\" + name);
boolean flag = false;
try {
flag = newFolder.mkdir();
} catch (SecurityException Se) {
JFrame messageDialog = new JFrame("Error!");
JOptionPane.showMessageDialog(messageDialog, "Folder creation failed with the following reason: \n" + Se);
}
}
I'm stuck right now and i have no idea what i'm doing wrong to get rid of the access denied error.
Short explenation of how this program works:
My program shows a list of all folders and files from a selected File.
That File is a field in the class JXploreFile called "currentFile", which behaves almost the same as a File.
When browsing through the folders, the currentFile is set to a new JXploreFile, containing the new folder you are in as File.
When creating a new folder/file, my program ask the path the user is currently browsing in with the method getPath().
Thanks for the help!
Image of my program:
Before you try to make any I/O operation just check if you have the permission
go to the parent directory (your case location)
then do something like
File f = new File(location);
if(f.canWrite()) {
/*your full folder creation code here */
} else {
}
try to put
String location ="c:\\user\<<youruser>>\\my documents"
or a folder with full perission to write
I want to overwrite existing zip file when I create zip file by using Zip4j. When I create zip file by using Zip4j, my file will split according to splitSize. So I can't check it. Here my Code sample ...
File file = new File("C:\\temp\\5.pdf");
ZipParameters parameters = new ZipParameters();
// set compression method to store compression
parameters.setCompressionMethod(Zip4jConstants.COMP_DEFLATE);
// Set the compression level. This value has to be in between 0 to 9
parameters.setCompressionLevel(Zip4jConstants.DEFLATE_LEVEL_NORMAL);
zipFile.createZipFile(file, parameters, true, splitSize);
Whether the file already exists or not, the following code will work:
File file = new File("<your zip file">);
boolean delete = file.delete();
The boolean will be true if the file was deleted, and false if the file did not exist or could not be deleted. Of course if the file could not be deleted for any reason other than "file does not exist", you will not know. If you care about it, you should use the code suggested by Arno_Geismar.
Hope this helps, regards
//step 1
Path p1 = ...; //path to your potentially existing file
Path p2 = ...; //path of your new file;
if (Files.isSameFile(p1, p2)) {
try {
delete(p1) // delete the directory where your duplicate is located
//step 2: insert your saving logic with zip4j ( make sure you create a new subdirectory for each file you zip
//step 3: eat some strawberry ice-cream
} catch (NoSuchFileException x) {
System.err.format("%s: no such" + " file or directory%n", path);
} catch (DirectoryNotEmptyException x) {
System.err.format("%s not empty%n", path);
} catch (IOException x) {
// File permission problems are caught here.
System.err.println(x);
}
}
//use this method to delete the files in the directory before deleting it.
void delete(File f) throws IOException {
if (f.isDirectory()) {
for (File c : f.listFiles())
delete(c);
}
if (!f.delete())
throw new FileNotFoundException("Failed to delete file: " + f);
}
I'm trying to zip a bunch of files using the Zip4j library. I pass a list of the file paths of the files I want to compress and I add them one by one into the zip file. For some reason, the last file does not get added. I checked the indexes of the loop and I'm pretty sure they're correct. I am not getting any exceptions or error messages. Here's my code:
// get the path; paths refers to the list of files to compress
String uuidString = UUID.randomUUID().toString();
String path = "H:/public/ZipFiles/" + uuidString + ".zip";
try {
// create the new zip file
ZipFile zipFile = new ZipFile(path);
File fileToAdd;
String message = "";
ZipParameters parameters = new ZipParameters();
// set compression method to store compression
parameters.setCompressionMethod(Zip4jConstants.COMP_DEFLATE);
// Set the compression level
parameters.setCompressionLevel(Zip4jConstants.DEFLATE_LEVEL_NORMAL);
// add each file to the zipFile
for(int i = 0; i < paths.size(); i++)
{
fileToAdd = new File(paths.get(i));
if(fileToAdd.exists())
{
System.out.println("writing file at " + paths.get(i) + " to the zip file");
zipFile.addFile(fileToAdd, parameters);
}
else
{
message += "File with at path " + paths.get(i) + " was not found.\n";
}
}
} catch (ZipException e) {
e.printStackTrace();
}
All the file paths get printed when they are added. Any ideas?
You're not closing the ZipFile.
I think there is a problem with the jar file from their own website at http://www.lingala.net/zip4j/download.php
But when I downloaded it from the maven repository at https://mvnrepository.com/artifact/net.lingala.zip4j/zip4j/1.3.2 , it is working perfectly.