Develop Reference

Java NIO WatchService

I have Java project that we need to keep listening some path to detect if there is any new XML file and if true, we need to process it through few other rules.
The WatchService is doing very well to discover if I have a new file there to be processed, but I wasnt able to manage it so I could read the file, I just gives me back the file name, from the event.
Is there any way to that with WatchService? If not, whats another sugestion to reach?
Thanks.

Sure, you can easely to that:
Create the watch service:
WatchService watchService = null;
try {
watchService = FileSystems.getDefault().newWatchService();
Path path = get(pathToSee);
path.register(watchService, new WatchEvent.Kind[]{ENTRY_MODIFY, ENTRY_CREATE}, SensitivityWatchEventModifier.HIGH);
watchEvent(watchService, path);
log.info("Watch Service has ben created!");
} catch (IOException e) {
log.error("Exception has ben throw when the service have tried to createWatchService()", e);
}
Note: If you have a large amount of files to be added, you can put:
SensitivityWatchEventModifier.HIGH
To increase sensitivity.
Watch if have changes in you directory:
WatchKey key;
while (true) {
try {
if ((key = watchService.take()) == null) break;
for (WatchEvent<?> event : key.pollEvents()) {
log.info("Event kind:" + event.kind()
+ ". File affected: " + event.context() + ".");
String fileName = event.context().toString();
File directory = path.toFile();
yourService.readContent(directory, fileName);
}
key.reset();
} catch (InterruptedException | IOException e) {
log.error("InterruptedException when try watchEvent()" + e);
}
}
And finally, you can do what you want to do with that infos:
try (BufferedReader br = new BufferedReader(new FileReader(directory + "/" + fileName))) {
String strLine;
while ((strLine = br.readLine()) != null) {
}
}
}
}
Tips:
You can create a separete thread to do that, or use Spring #Async to create a separete thread to process this information and increase concurrency in your application.
You can use Apache Commons you to do that too!
e.g:
public void getAll() throws Exception {
FileAlterationObserver observer = new FileAlterationObserver(pathToSee);
observer.addListener(new FileAlterationListenerAdaptor() {
#SneakyThrows
#Override
public void onFileCreate(File file) {
}
#Override
public void onFileDelete(File file) {
}
});
}

Multi-threaded read from file and execute runnable

WHAT?
I am trying to build a tool that will reads a text file and publishes the text, after doing some string transformation.
HOW?
The tool reads the file line by line and populates a LinkedBlockingQueue. At the same time I initiate multiple threads that will then take a message each from the LBQ, do some processing and publish them.
Main
private static LinkedBlockingQueue<String> lbQueue = new LinkedBlockingQueue<>();
private static Boolean keepPublisherActive = Boolean.TRUE;
public static void main(String[] args) {
try {
tool.initMessagePublish();
tool.searchUsingScanner();
} catch (Exception ex) {
logger.error("Exception in Tool Main() " + ex.toString());
throw ex;
}
}
File Reader
private void searchUsingScanner() {
Scanner scanner = null;
try {
scanner = new Scanner(new File(LOG_FILE_PATH));
while (scanner.hasNextLine()) {
String line = scanner.nextLine().trim();
if (StringUtils.isNotBlank(line)) {
lbQueue.offer(line);
}
}
} catch (Exception e) {
logger.error("Error while processing file: " + e.toString());
} finally {
try {
if (scanner != null) {
scanner.close();
}
// end thread execution
keepPublisherActive = false;
} catch (Exception e) {
logger.error("Exception while closing file scanner " + e.toString());
throw e;
}
}
}
Multi-threaded Publisher
private void initMessagePublish() throws InterruptedException {
ExecutorService service = Executors.newFixedThreadPool(6);
try {
while (keepPublisherActive || lbQueue.getSize() > 0) {
service.execute(messagePublisher); // messagePublisher implements Runnable
}
} catch (Exception ex) {
logger.error("Multi threaded message publish failed " + ex.toString());
throw ex;
} finally {
service.shutdown();
}
}
THE PROBLEM
The intention behind calling initMessagePublish() fist is that the publisher need not wait for all lines to be read from the file before starting to publish. It should start publishing as soon as something becomes available in the LBQ.
But with the current implementation, the control never comes out of the initMessagePublish and start searchUsingScanner. How do I solve this? Basically, the two methods should execute parallely.

Just start messagePublisher in a new Thread (Line no #5 in Main class):
new Thread(()->tool.initMessagePublish()).start();
It should solve your problem.

Running python script from java and sending input/output

I am writing a java program that will need to run a python script.
The script will print output which will java need to read to know the progress of the script.
To be able to pause the script while running I want it to ask for input once in a while, only when java give it input the script will keep going.
Here is my Java method:
private static void sevenTry(String[] strCommands) throws IOException {
Object oLock1 = new Object();
Object oLock2 = new Object();
ProcessBuilder pBuilder = new ProcessBuilder(strCommands);
pBuilder.redirectErrorStream(true);
Process proc = pBuilder.start();
Thread tReader = new Thread() {
#Override
public void run() {
System.out.println("~~tReader starting~~");
BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getInputStream()));
synchronized (oLock1) {
try {
String line = reader.readLine();
while (line != null && !line.trim().equals("--EOF--")) {
System.out.println("Stdout: " + line);
if (line.trim().equals("--INPUT--")) {
synchronized (oLock2) {
oLock2.notify();
}
oLock1.wait();
}
line = reader.readLine();
}
} catch (IOException e) {
System.out.println("tReader: " + e.getMessage());
} catch (InterruptedException e) {
System.out.println("tReader: " + e.getMessage());
} catch (Exception e) {
System.out.println("tReader: " + e.getMessage());
}
}
System.out.println("~~tReader end~~");
synchronized (oLock2) {
oLock2.notify();
}
}
};
Thread tWriter = new Thread() {
#Override
public void run() {
System.out.println("~~tWriter starting~~");
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(proc.getOutputStream()));
String line, input;
Scanner scan = new Scanner(System.in);
synchronized (oLock2) {
try {
oLock2.wait();
} catch (InterruptedException e1) {
System.out.println("tWriter: " + e1.getMessage());
}
}
while (tReader.isAlive()) {
synchronized (oLock1) {
System.out.println("Java: insert input");
scan.hasNext();
input = scan.nextLine();
try {
writer.write(input + "\n");
writer.flush();
} catch (IOException e) {
System.out.println("tWriter: " + e.getMessage());
}
oLock1.notify();
}
try {
Thread.sleep(2000);
} catch (InterruptedException e1) {
System.out.println("tWriter: " + e1.getMessage());
}
}
System.out.println("~~tWriter end~~");
}
};
tReader.start();
tWriter.start();
System.out.println("~~everything submitted~~");
try {
tReader.join();
tWriter.join();
System.out.println("~~finish~~");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
This is my python script:
# coding=utf-8
import sys
print '1'
print '--INPUT--'
inum = sys.stdin.readline()
print '2'
print '--EOF--'
I tried running my code
sevenTry("python", "C:\\Testing.py");
but on java side it get stuck inside tReader at line:
String line = reader.readLine();
The program does work if i take out the input line from the python file
inum = sys.stdin.readline()
Using
inum = raw_input()
still bring up the same problem (im using python 2.7)
The most confusing part here that i even tried to test this with a java file (instead of python)
sevenTry("java", "-classpath", "C:\\class", "CheckCMD");
and it worked even with the input lines
import java.util.Scanner;
public class CheckCMD {
public static void main(String[] args) {
Scanner in = new Scanner(System.in);
String line;
System.out.println("1");
System.out.println("--INPUT--");
in.hasNext();
line = in.nextLine();
System.out.println("2");
System.out.println("--EOF--");
}
}

As you may have noticed, this is a problem related to Python.
As described in https://unix.stackexchange.com/questions/182537/write-python-stdout-to-file-immediately,
" when process STDOUT is redirected to something other than a terminal, then the output is buffered into some OS-specific-sized buffer (perhaps 4k or 8k in many cases)."
So, you need to call sys.stdout.flush() after each invoke to print.
Or, as a better option, you can change the default behaviour for the process, using the -u param, to get unbuffered output.

Command line terminal executing on process and input interaction from this process [duplicate]

This question already has answers here:
run interactive command line application from java
(2 answers)
Closed 6 years ago.
Basically, I have a process which runs when I press a button on my java application.
And this process executes a command to the terminal of the OS.
But sometimes this command needs to have an interaction with the user.
And I would like to know if this was possible to have the interaction from the process to the user when needed?
My code:
File marsSimulator = new File("resources/mars_simulator/Mars4_5.jar");
if(marsSimulator.exists() && temp.exists()){
String res="";
try {
Process p = Runtime.getRuntime().exec(new String[]{"java","-jar",marsSimulator.getAbsolutePath(),tempAssembly.getAbsolutePath()});
p.waitFor();
InputStream is = p.getInputStream();
byte b[] = new byte[is.available()];
is.read(b, 0, b.length); // probably try b.length-1 or -2 to remove "new-line(s)"
res = new String(b);
} catch (Exception ex) {
ex.printStackTrace();
}
}
Also, I forgot to say that the application is made with SWING and that the output of the process is shown onto a TextArea... Should I change anything ?
Notice that the process blocks when there is an interaction with the user. If there isn't, the process doesn't block !
What do I need to do in this case (which I don't know how to do it ) ?
When the process needs the interaction. I need to know when the process wants some interaction.
I need to get the output generated of the process interactively (line by line).
P.S.: For people who wanna understand the process line, I am using the Mars Simulator (http://courses.missouristate.edu/KenVollmar/MARS/) and I am sending the jar application into a process with a mips assembly code associated.
This next pieces of code is working with my project
Hope it will help for the next adventurers!
And thank you to Nicolas Filotto for helping me.
My class ObservableStream:
class ObservableStream extends Observable {
private final Queue<String> lines = new ConcurrentLinkedQueue<>();
public void addLine(String line) {
lines.add(line);
setChanged();
notifyObservers();
}
public String nextLine() {
return lines.poll();
}
public String getLine(){return lines.peek();}
}
And the other part of the code:
Process p = Runtime.getRuntime().exec(new String[]{"java","-jar",marsSimulator.getAbsolutePath(),tempAssembly.getAbsolutePath()});
//This code does the interaction from the process with the GUI ! Implied, input interaction+output interaction from the process
ObservableStream out = new ObservableStream();
// Observer that simply sends to my external process line by line what we put in
// the variable output
PrintWriter writer = new PrintWriter(p.getOutputStream(), true);
out.addObserver(
(o, arg) -> {
ObservableStream stream = (ObservableStream) o;
String line;
while ((line = stream.nextLine()) != null) {
writer.println(line);
}
}
);
ObservableStream input = new ObservableStream();
input.addObserver(
(o, arg) -> {
ObservableStream stream = (ObservableStream) o;
String line;
while ((line = stream.nextLine()) != null) {
outputTextArea.appendText(line+"\n");
}
}
);
// The thread that reads the standard output stream of the external process
// and put the lines into my variable input
new Thread(
() -> {
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(p.getInputStream()))
) {
String line;
while ((line = reader.readLine()) != null) {
input.addLine(line);
}
} catch (IOException e1) {
e1.printStackTrace();
}
}
).start();
new Thread(
()->{
while(p.isAlive()){
String res = input.getLine();
if(res!=null && res.equals("Enter integer value:")) {
boolean integerIsRequested=true;
Thread t=null;
while(integerIsRequested){
if(t==null) {
t = new Thread(new Runnable() {
public void run() {
String test1 = JOptionPane.showInputDialog("Enter Integer value:");
while(!test1.matches("^\\d+$")){
test1 = JOptionPane.showInputDialog("Error: Not a valid Integer.\nEnter a correct Integer value:");
}
Integer i = Integer.valueOf(test1);
if (i != null) {
out.addLine(test1);
}
}
});
t.start();
}
if(!t.isAlive()){
integerIsRequested=false;
}
}
}
}
outputTextArea.appendText("Program executed\n");
}
).start();
By the way, this post is unique Jarrod ;)

To implement such use case I would personally use:
An Observable object to notify my UI when a new line has been provided by the external process
An Observable object to which I add new lines provided by my UI
An Observer of #1 that will refresh the data of my UI
An Observer of #2 that will send the lines provided by my UI to my external process
A Thread that will check if a new line has been provided by my external process and if so it will provide those lines to #1
So as I don't have your full env, I will show you how it will work with mock objects:
First my fake external application that only does an Echo of what he receives:
public class Echo {
public static void main(String[] args) {
Scanner scanner = new Scanner(System.in);
while (true) {
String line = scanner.nextLine();
System.out.printf("echo > %s%n", line);
}
}
}
If this class receives foo, it will print into the standard output stream echo > foo
Then my Observable class
public class ObservableStream extends Observable {
private final Queue<String> lines = new ConcurrentLinkedQueue<>();
public void addLine(String line) {
lines.add(line);
setChanged();
notifyObservers();
}
public String nextLine() {
return lines.poll();
}
}
NB: The class ObservableStream (as it is implemented so far) is meant to have only one Observer no more which is enough according to your needs. Indeed is only used to decouple your UI from how the data is retrieved or published
Then finally the main code:
Process p = Runtime.getRuntime().exec(
new String[]{"java", "-cp", "/my/path/to/my/classes", "Echo"}
);
// The Observable object allowing to get the input lines from my external process
ObservableStream input = new ObservableStream();
// A mock observer that simply prints the lines provided by the external process
// but in your case you will update your text area instead
input.addObserver(
(o, arg) -> {
ObservableStream stream = (ObservableStream) o;
String line;
while ((line = stream.nextLine()) != null) {
System.out.printf("Line Received from the external process: %s%n", line);
}
}
);
// The thread that reads the standard output stream of the external process
// and put the lines into my variable input
new Thread(
() -> {
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(p.getInputStream()))
) {
String line;
while ((line = reader.readLine()) != null) {
input.addLine(line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
).start();
// The Observable object allowing to send the input lines to my external process
ObservableStream output = new ObservableStream();
// Observer that simply sends to my external process line by line what we put in
// the variable output
PrintWriter writer = new PrintWriter(p.getOutputStream(), true);
output.addObserver(
(o, arg) -> {
ObservableStream stream = (ObservableStream) o;
String line;
while ((line = stream.nextLine()) != null) {
writer.println(line);
}
}
);
// A simple scanner used to send new messages to my external process
Scanner scanner = new Scanner(System.in);
while (true) {
output.addLine(scanner.nextLine());
}
If this code receives foo, it will print into the standard output stream Line Received from the external process: echo > foo

I hope it answers your question... subProcessStuff "emulates" that sub process. It can be anything - but this way we have all in place. It requires 2 params passed into console. String and Integer. Gobbler got Callback which is an interface, with anonymous implementation - and there are checks for params. To answer if subprocess waits we simply track what is says - just like if a user would operate with it.
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintStream;
import java.util.Scanner;
class Test1 {
public static void main(String[] args) {
for (String arg : args)
System.out.println("arg: " + arg);
for (String arg : args)
if (arg.equals("-test")) {
subProcessStuff();
return;
}
mainProcess();
}
public static void subProcessStuff() {
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
try {
System.out.println("Enter String");
String s = br.readLine();
System.out.println("Enered String: " + s);
System.out.println("Enter Integer:");
int i = Integer.parseInt(br.readLine());
System.out.println("Entered Integer: " + i);
} catch (IOException e) {
System.err.println("io error - " + e.getMessage());
} catch (NumberFormatException nfe) {
System.err.println("Invalid Format!");
}
}
private static PrintStream out;
public static void mainProcess() {
String[] commands = { "ls", "-alt" };
ProcessBuilder builder = new ProcessBuilder("java", "Test1", "-test");
// builder.inheritIO(); // I avoid this. It was messing me up.
try {
Process proc = builder.start();
InputStream errStream = proc.getErrorStream();
InputStream inStream = proc.getInputStream();
OutputStream outStream = proc.getOutputStream();
new Thread(new StreamGobbler("err", out, errStream)).start();
out = new PrintStream(new BufferedOutputStream(outStream));
Callback cb = new Callback() {
#Override
public void onNextLine(String line) {
if (line.equals("Enter String")) {
out.println("aaaaa");
out.flush();
}
if (line.equals("Enter Integer:")) {
out.println("123");
out.flush();
}
}
};
new Thread(new StreamGobbler("in", out, inStream, cb)).start();
int errorCode = proc.waitFor();
System.out.println("error code: " + errorCode);
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
if (out != null) {
out.close();
}
}
}
}
interface Callback {
void onNextLine(String line);
}
class StreamGobbler implements Runnable {
private PrintStream out;
private Scanner inScanner;
private String name;
private Callback cb;
public StreamGobbler(String name, PrintStream out, InputStream inStream) {
this.name = name;
this.out = out;
inScanner = new Scanner(new BufferedInputStream(inStream));
}
public StreamGobbler(String name, PrintStream out, InputStream inStream, Callback cb) {
this.name = name;
this.out = out;
inScanner = new Scanner(new BufferedInputStream(inStream));
this.cb = cb;
}
#Override
public void run() {
while (inScanner.hasNextLine()) {
String line = inScanner.nextLine();
if (cb != null)
cb.onNextLine(line);
System.out.printf("%s: %s%n", name, line);
}
}
}

I don't think you can check the state of the process from the Java. However you can do it by using some Linux command. (Of course if you're using Linux)
If your Java process has access to the /proc directory then you can read the status file for the process.
For example for a process with process id 12280
/proc/12280/status
Here's the relevant output of the status file
Name: java
State: S (sleeping)
Tgid: 12280
Pid: 12280
PPid: 12279
...
Second line gives the state of the process. You'll need to run a thread to continuously poll this file to read the status.

Line by Line The Code i use to interract with a different jar which is a speechRecognizer.I think you want to achieve something like this.
Example:
The jar i am interracting(speechRecognizer) is executing different commands and run some other Threads.Every time it has to interract with the main jar it prints something that i need.For example (user said:How are you),so you can have a same logic and when external jar need interraction with user it prints something and you read it into the main app.So:
// About Process
private Process process;
private BufferedReader bufferedReader;
private boolean stopped = true;
Thread processChecker;
//Running it in a Thread so the app don't lags
new Thread(() -> {
try {
stopped = false;
//Starting the external jar..
ProcessBuilder builder = new ProcessBuilder("java", "-jar", System.getProperty("user.home")
+ File.separator + "Desktop" + File.separator + "speechRecognizer.jar", "BITCH_PLEASE");
//Redirecting the ErrorStream
builder.redirectErrorStream(true);
process = builder.start();
bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
//Check continusly if the process is still alive
//i case of crash i should do something..
processChecker = new Thread(() -> {
while (process.isAlive()) {
try {
Thread.sleep(1200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
stopSpeechReader(false);
});
processChecker.start();
// Continuesly Read Output of external process
while (!stopped) {
while ((line = bufferedReader.readLine()) != null && !line.isEmpty()) {
System.out.println(line);
checkSpeechResult(line);
}
}
// Interrupt the mf Thread if is Alive
if (processChecker.isAlive())
processChecker.interrupt();
System.out.println("SpeechReader Stopped! Process is alive:" + process.isAlive() + " >Exit Value:"
+ process.exitValue());
} catch (Exception e) {
e.printStackTrace();
}
}).start();

Interfacing Python and Java - Multithreaded Input and Output Streams

I have a Python app and a Java app running simultaneously.
I want Java to start the Python process and then communicate using the normal STDIN/STDOUT streams with Python.
I have started the process correctly and have two threads to handle the two I/O streams.
OUTPUT THREAD:
class output2 extends Thread {
Process process;
OutputStream stdin;
BufferedWriter writer;
Scanner in = new Scanner(System.in);
output2(Process p) {
try {
process = p;
stdin = process.getOutputStream();
writer = new BufferedWriter(new OutputStreamWriter(stdin));
} catch (Exception e) {
System.out.println("ERROR output2(): " + e);
}
}
#Override
public void run() {
System.out.println("Starting OUTPUT THREAD");
try {
while (true) {
String input = in.nextLine();
writer.write(input);
writer.flush();
}
} catch (Exception e) {
System.out.println("ERROR output2_run(): " + e);
}
System.out.println("Ending OUTPUT THREAD");
}
}
INPUT THREAD :
class input2 extends Thread {
Process process;
InputStream stdout;
BufferedReader reader;
input2(Process p) {
try {
process = p;
stdout = process.getInputStream();
reader = new BufferedReader(new InputStreamReader(stdout));
} catch (Exception e) {
System.out.println("ERROR input2(): " + e);
}
}
#Override
public void run() {
System.out.println("Started INPUT THREAD");
try {
while (true) {
System.out.println(Thread.currentThread().getName() + " is executing");
if (reader.readLine() != null) {
System.out.println("Stdout: " + reader.readLine());
}
Thread.sleep(1000);
System.out.println(Thread.currentThread().getName() + " stopped executing");
}
} catch (Exception e) {
System.out.println("ERROR input2_run(): " + e);
}
System.out.println("Ending INPUT THREAD");
}
}
MAIN :
public class My_Java {
public static void main(String args[]) {
File file = new File("C:\\Location\\");
try {
Process process = Runtime.getRuntime().exec("C:\\Python27\\python.exe chat_from_file.py", null, file);
input2 input = new input2(process);
output2 output = new output2(process);
input.setName("INPUT THREAD");
output.setName("OUTPUT THREAD");
input.start();
output.start();
} catch (Exception e) {
System.out.println("ERROR main(): " + e);
}
}
}
This doesn't seem to give any response at all.
It starts both threads, says INPUT THREAD is executing but nothing after that.
Where am I going wrong?

First of all, after calling if (reader.readLine() != null) { in your input class, you effectively have read the line and the next call will return null.
Use ready to check for non-blocking read possibility. Don't read upfront.
However, I'm pretty sure that you process exists abnormally, with something like python: can't open file 'chat_from_file.py': [Errno 2] No such file or directory or, throws a stacktrace and exits.
Use getErrorStream to check what the process is outputting if an error exists. This will put you on the correct path to solve your issue.
Also, just in case, make sure there's actually something to be read. Make sure your Python application is outputting enough data for buffers to be flushed (or flushing its writes).
And don't forget to join and exit cleanly and correctly. Good luck.