I want to create a Java code which makes benchmark about importing xml file into SQL table. I tried this:
import java.io.IOException;
import java.nio.file.*;
import java.util.HashMap;
import java.util.List;
public class Application extends SqlUtils {
private static String folderPath = "D:\\EntityImportEversana";
public static void main(final String[] args) throws IOException, InterruptedException {
System.out.println("Running file verifier");
System.out.println("monitoring folder " + folderPath);
SqlUtils sql = new SqlUtils();
WatchService watchService = FileSystems.getDefault().newWatchService();
Path path = Paths.get(folderPath);
path.register(watchService, StandardWatchEventKinds.ENTRY_CREATE, StandardWatchEventKinds.ENTRY_DELETE, StandardWatchEventKinds.ENTRY_MODIFY);
WatchKey key;
while ((key = watchService.take()) != null) {
for (WatchEvent<?> event : key.pollEvents()) {
System.out.println("Event kind:" + event.kind() + ". File affected: " + event.context() + ".");
if(event.kind().equals(StandardWatchEventKinds.ENTRY_DELETE)){
// Here we make a constant pull and we get the status using sql from db.
// make here start of benchmark
HashMap<String, List> map = sql.checkFileImport();
// end here of benchmark
}
}
key.reset();
}
watchService.close();
}
}
Do you know how I can make a benchmark into the above code and trace when the file is updated into db?
You can measure the time like this:
Instant start = Instant.now();
... // do your import here
Instant end = Instant.now();
System.out.println(Duration.between(start,end));
Related
I want to create a Java code which makes benchmark about importing xml file into SQL table. I tried this:
import java.io.IOException;
import java.nio.file.*;
import java.util.HashMap;
import java.util.List;
public class Application extends SqlUtils {
private static String folderPath = "D:\\EntityImportEversana";
public static void main(final String[] args) throws IOException, InterruptedException {
System.out.println("Running file verifier");
System.out.println("monitoring folder " + folderPath);
SqlUtils sql = new SqlUtils();
WatchService watchService = FileSystems.getDefault().newWatchService();
Path path = Paths.get(folderPath);
path.register(watchService, StandardWatchEventKinds.ENTRY_CREATE, StandardWatchEventKinds.ENTRY_DELETE, StandardWatchEventKinds.ENTRY_MODIFY);
WatchKey key;
while ((key = watchService.take()) != null) {
for (WatchEvent<?> event : key.pollEvents()) {
System.out.println("Event kind:" + event.kind() + ". File affected: " + event.context() + ".");
if(event.kind().equals(StandardWatchEventKinds.ENTRY_DELETE)){
Instant start = Instant.now();
// make here call every second to get the status
HashMap<String, List> map = sql.checkFileImport();
Instant end = Instant.now();
System.out.println(Duration.between(start,end));
}
}
key.reset();
}
watchService.close();
}
}
Do you know how I can make a call every second to DB and get the status?
Something like:
while(true) { // repeat code within indefinitely
while ((key = watchService.take()) != null) {
// do stuff
}
Thread.sleep(1000); // get executing thread to sleep for 1000millis - 1s
}
Should work
am trying to monitor a folder/file for any changes and then extract all the data from the file and append it to the database using java.
i have tried the watch service api in java to monitor the file as shown in the code snippet below.
import static java.nio.file.StandardWatchEventKinds.*;
import java.io.*;
import java.util.*;
public class FolderMonitor {
public void fileMonitor() throws IOException {
WatchService watchService = FileSystems.getDefault().newWatchService();
Path path = Paths.get("C:/Users/xxxxx/Desktop/yyyyy");
path.register(watchService, ENTRY_CREATE, ENTRY_MODIFY, ENTRY_DELETE);
}
}
The above snippet monitors the file path and shows that in the console if a main function is added or called in another method.
So now i need to be able to read the data from that file whenever it is created or modified and then append the data to the database
Any help is highly welcome
so tihs is how i achieved this, i created various classes as shown below.
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardWatchEventKinds;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
public class FolderMonitor {
public static void fileMonitor() {
final String FOLDER_NAME = "xxxxxxxxxxx";
final String FILE_NAME = "xxxxxx/filename";
try {System.out.println("Watching directory for changes in the info file");
WatchService watchService = FileSystems.getDefault().newWatchService();
Path directory = Paths.get(FOLDER_NAME);
WatchKey watchKey = directory.register(watchService, StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_MODIFY, StandardWatchEventKinds.ENTRY_DELETE);
while (true) {
for (WatchEvent<?> event : watchKey.pollEvents()) {
#SuppressWarnings("unchecked")
WatchEvent<Path> pathEvent = (WatchEvent<Path>) event;
Path fileName = pathEvent.context();
WatchEvent.Kind<?> kind = event.kind();
if (kind == StandardWatchEventKinds.ENTRY_CREATE) {
System.out.println("Info file has been created : " + fileName);
FileReadingToJson.infoFileReading(FILE_NAME);
}
if (kind == StandardWatchEventKinds.ENTRY_DELETE) {
System.out.println("The info file has been deleted: " + fileName);
}
if (kind == StandardWatchEventKinds.ENTRY_MODIFY) {
FileReadingToJson.infoFileReading(FILE_NAME);
}
}
boolean valid = watchKey.reset();
if (!valid) {
break;
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
And this is how i read the properties file and parsed it to json
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import org.json.JSONObject;
import org.json.Property;
public class FileReadingToJson {
public static void infoFileReading(String filename) {
try (InputStream input = new FileInputStream(filename)) {
Properties infoFilepropeProperties = new Properties();
infoFilepropeProperties.load(input);
JSONObject jsonObject = Property.toJSONObject(infoFilepropeProperties);
String jsonObjectstring = jsonObject.toString();
System.out.println(jsonObject);
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
Currently those snippets do the monitoring and reading of the properties file and parse them to json format.
Now am remaining with using that json data to populate the database whenever there is change in the file.
For my current side project I need to utilize a WatchService to track events in a given directory. My code is yet mainly based off of Oracles WatchService tutorial example
I however need it to be limited to folder only events (e.g ENTRY_CREATE C:\temp\folder_a).
What I'm trying to do is to take an initial Snapshot of the directory's content
and store each contents path into either dirCache or fileCache
If an new event is registered this should be checked:
is event context a file in fileCache or
is event context a new file (-> Files.isRegularFile)
so both new File events should be discarded or events from files that are already in the cache.
But printing out the events produces
ENTRY_DELETE: C:\temp\k.txt
for files but no ENTRY_CREATE or ENTRY_MODIFY.
What am I doing wrong? Am I not checking against the cache correctly or is it something completely different?
Here's the current code base:
public class Main {
public static void main(String[] args) {
try {
new DirectoryWatcher(Paths.get("C:\\temp")).processEvents();
} catch (IOException e) {
e.printStackTrace();
}
}
}
DirectoryWatcher Class
package service;
import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_DELETE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
import static java.nio.file.StandardWatchEventKinds.OVERFLOW;
import java.io.IOException;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.WatchEvent;
import java.nio.file.WatchEvent.Kind;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
import java.util.HashMap;
import java.util.Map;
/**
* Slightly modified version of Oracle
* example file WatchDir.java
* /
public class DirectoryWatcher {
private final Path path;
private final WatchService watcher;
private final Map<WatchKey,Path> keys;
private PathSnapshot pathSnapshot;
private boolean trace = false;
#SuppressWarnings("unchecked")
static <T> WatchEvent<T> cast(WatchEvent<?> event) {
return (WatchEvent<T>)event;
}
/**
* Register the given directory with the WatchService
*/
private void register(Path dir) throws IOException {
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
if (trace) {
Path prev = keys.get(key);
if (prev == null) {
System.out.format("register: %s\n", dir);
} else {
if (!dir.equals(prev)) {
System.out.format("update: %s -> %s\n", prev, dir);
}
}
}
keys.put(key, dir);
}
public DirectoryWatcher(Path dir) throws IOException {
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<WatchKey,Path>();
this.path = dir;
this.pathSnapshot = new PathSnapshot(dir);
register(dir);
// enable trace after initial registration
this.trace = true;
}
/**
* Process all events for keys queued to the watcher
*/
void processEvents() {
for (;;) {
// wait for key to be signaled
WatchKey key;
try {
key = watcher.take();
} catch (InterruptedException x) {
return;
}
Path dir = keys.get(key);
if (dir == null) {
System.err.println("WatchKey not recognized!!");
continue;
}
for (WatchEvent<?> event: key.pollEvents()) {
Kind<?> kind = event.kind();
// TBD - provide example of how OVERFLOW event is handled
if (kind == OVERFLOW) {
continue;
}
// Context for directory entry event is the file name of entry
WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
this.updateDirContent();
/*
* currently: creating file events are neglected
* but deleting a file creates an event which is printed
* TODO: disregard delete event if sent from file
*/
boolean isFile = Files.isRegularFile(child);
if (pathSnapshot.isInFileCache(child)|| isFile) {
//disregard the event if file
event = null;
} else {
// print out event
System.out.format("%s: %s\n", event.kind().name(), child);
}
}
// reset key and remove from set if directory no longer accessible
boolean valid = key.reset();
if (!valid) {
keys.remove(key);
// all directories are inaccessible
if (keys.isEmpty()) {
break;
}
}
}
}
private void updateDirContent() {
this.pathSnapshot = pathSnapshot.updateSnapshot(path);
}
}
PathSnapshot Class
package service;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.stream.Stream;
public class PathSnapshot {
public ArrayList<Path> dirCache = new ArrayList<Path>();
public ArrayList<Path> fileCache = new ArrayList<Path>();
public PathSnapshot(Path dir) {
try {
Stream<Path> rawDirContent = Files.walk(
dir, 1);
Object[] dirContent = rawDirContent.toArray();
rawDirContent.close();
sortIntoCache(dirContent, dir);
} catch (IOException e) {
e.printStackTrace();
}
}
private void sortIntoCache(Object[] dirContent, Path rootdir) {
for (Object object : dirContent) {
//create path from element
Path objectPath = Paths.get(object.toString());
//skip start path / the root directory
if (object.equals(rootdir)) {
continue;
} else if (Files.isRegularFile(objectPath)) {
fileCache.add(objectPath);
} else if (Files.isDirectory(objectPath)) {
dirCache.add(objectPath);
}
}
}
public boolean isInFileCache(Path path) {
if (fileCache.contains(path)) {
return true;
} else {
return false;
}
}
public boolean isInDirCache(Path path) {
if (dirCache.contains(path)) {
return true;
} else {
return false;
}
}
public PathSnapshot updateSnapshot(Path dir){
return new PathSnapshot(dir);
}
}
You are listening to all possible events from the file system so there isn't more to ask for. Java can't do anything if the OS isn't presenting more events and in more detail. Some complex file system operations are just not represented by one event but of a sequence of basic events. So you have to make the best out of the events and have to interpret what a sequence of events actually means.
Setup
Pasted is modified version of an Oracle example
While the program's running, if you now make a new file or directory directory inside of the path directory, you'll see that the path displayed is incorrect
For example, when I run it, it shows the absolute path of a newly-created file in the path directory as /home/hoagy/test/new_file, when the actual path to the file is /home/hoagy/test/path/new_file
Question
How do I get the proper absolute path?
Code
package path.question;
import java.nio.file.*;
import static java.nio.file.StandardWatchEventKinds.*;
import static java.nio.file.LinkOption.*;
import java.nio.file.attribute.*;
import java.io.*;
import java.util.*;
public class WatchDir {
private final WatchService watcher;
private final Map<WatchKey,Path> keys;
#SuppressWarnings("unchecked")
static <T> WatchEvent<T> cast(WatchEvent<?> event) {
return (WatchEvent<T>)event;
}
private void register(Path dir) throws IOException {
WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
keys.put(key, dir);
}
WatchDir(Path dir) throws IOException {
this.watcher = FileSystems.getDefault().newWatchService();
this.keys = new HashMap<WatchKey,Path>();
register(dir);
}
void processEvents() {
for (;;) {
WatchKey key;
try { key = watcher.take(); }
catch (InterruptedException x) { return; }
Path dir = keys.get(key);
if (dir == null) {
continue;
}
for (WatchEvent<?> event: key.pollEvents()) {
WatchEvent.Kind kind = event.kind();
if (kind == OVERFLOW) { continue; }
WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
System.out.println("Full path: " + name.toAbsolutePath().toString());
}
key.reset();
}
}
public static void main(String[] args) throws IOException {
Path dir = Paths.get("./path");
new WatchDir(dir).processEvents();
}
}
Replace :
//prints the "absolute path" of the name of the file
System.out.println("Full path: " + name.toAbsolutePath().toString());
With
//prints the "absolute path" of the "child"
System.out.println("child " + child.toAbsolutePath().toString());
So I have to make a program in java that automatically runs in the background and looks for a new .dat file and when it sees the new .dat file it then runs a .bat file to load data into a database. So far I have a program that watches for new file creation, modification, and deletion. I also have a script that runs the .bat file and loads the data into the database now i just need to connect the two but I am not sure how to go about this, If someone could point me in the right direction I would greatly appreciate it.
Below is the code I have so far.
import static java.nio.file.LinkOption.NOFOLLOW_LINKS;
import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
import static java.nio.file.StandardWatchEventKinds.OVERFLOW;
import static java.nio.file.StandardWatchEventKinds.ENTRY_DELETE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
import java.io.*;
import java.util.*;
import java.io.File;
import java.io.IOException;
import java.nio.file.FileSystem;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.WatchEvent;
import java.nio.file.WatchEvent.Kind;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
public class Order_Processing {
public static void watchDirectoryPath(Path path)
{
try {
Boolean isFolder = (Boolean) Files.getAttribute(path,
"basic:isDirectory", NOFOLLOW_LINKS);
if (!isFolder)
{
throw new IllegalArgumentException("Path: " + path
+ " is not a folder");
}
}
catch (IOException ioe)
{
ioe.printStackTrace();
}
System.out.println("Watching path: "+ path);
FileSystem fs = path.getFileSystem();
try (WatchService service = fs.newWatchService())
{
path.register(service, ENTRY_CREATE, ENTRY_MODIFY, ENTRY_DELETE);
WatchKey key = null;
while (true)
{
key = service.take();
Kind<?> kind = null;
for (WatchEvent<?> watchEvent : key.pollEvents())
{
kind = watchEvent.kind();
if (OVERFLOW == kind)
{
continue;
}
else if (ENTRY_CREATE == kind)
{
Path newPath = ((WatchEvent<Path>) watchEvent)
.context();
System.out.println("New Path Created: " + newPath);
}
else if (ENTRY_MODIFY == kind)
{
Path newPath = ((WatchEvent<Path>) watchEvent)
.context();
System.out.println("New path modified: "+ newPath);
}
else if (ENTRY_DELETE == kind)
{
Path newPath = ((WatchEvent<Path>) watchEvent)
.context();
System.out.println("New path deleted: "+ newPath);
}
}
if (!key.reset())
{
break;
}
}
}
catch (IOException ioe)
{
ioe.printStackTrace();
}
catch (InterruptedException ie)
{
ie.printStackTrace();
}
}
public static void main(String[] args)
throws FileNotFoundException
{
File dir = new File("C:\\Paradigm");
watchDirectoryPath(dir.toPath());
//below is the script that runs the .bat file and it works if by itself
//with out all the other watch code.
try {
String[] command = {"cmd.exe", "/C", "Start", "C:\\Try.bat"};
Process p = Runtime.getRuntime().exec(command);
}
catch (IOException ex) {
}
}
}
This doesn't work because you have a while (true). This makes sense because you are listening and want the to happen continuously; however, the bat call will never be executed because watchDirectory(...) will never terminate. To solve this, pull the rest of the main out into its own function like so
public static void executeBat() {
try {
String[] command = {"cmd.exe", "/C", "Start", "C:\\Try.bat"};
Process p = Runtime.getRuntime().exec(command);
}
catch (IOException ex) {
// You should do something with this.
// DON'T JUST IGNORE FAILURES
}
so that upon file creation, you can call that bat script
...
else if (ENTRY_CREATE == kind)
{
Path newPath = ((WatchEvent<Path>) watchEvent).context();
executeBat();
}
...