WHAT?
I am trying to build a tool that will reads a text file and publishes the text, after doing some string transformation.
HOW?
The tool reads the file line by line and populates a LinkedBlockingQueue. At the same time I initiate multiple threads that will then take a message each from the LBQ, do some processing and publish them.
Main
private static LinkedBlockingQueue<String> lbQueue = new LinkedBlockingQueue<>();
private static Boolean keepPublisherActive = Boolean.TRUE;
public static void main(String[] args) {
try {
tool.initMessagePublish();
tool.searchUsingScanner();
} catch (Exception ex) {
logger.error("Exception in Tool Main() " + ex.toString());
throw ex;
}
}
File Reader
private void searchUsingScanner() {
Scanner scanner = null;
try {
scanner = new Scanner(new File(LOG_FILE_PATH));
while (scanner.hasNextLine()) {
String line = scanner.nextLine().trim();
if (StringUtils.isNotBlank(line)) {
lbQueue.offer(line);
}
}
} catch (Exception e) {
logger.error("Error while processing file: " + e.toString());
} finally {
try {
if (scanner != null) {
scanner.close();
}
// end thread execution
keepPublisherActive = false;
} catch (Exception e) {
logger.error("Exception while closing file scanner " + e.toString());
throw e;
}
}
}
Multi-threaded Publisher
private void initMessagePublish() throws InterruptedException {
ExecutorService service = Executors.newFixedThreadPool(6);
try {
while (keepPublisherActive || lbQueue.getSize() > 0) {
service.execute(messagePublisher); // messagePublisher implements Runnable
}
} catch (Exception ex) {
logger.error("Multi threaded message publish failed " + ex.toString());
throw ex;
} finally {
service.shutdown();
}
}
THE PROBLEM
The intention behind calling initMessagePublish() fist is that the publisher need not wait for all lines to be read from the file before starting to publish. It should start publishing as soon as something becomes available in the LBQ.
But with the current implementation, the control never comes out of the initMessagePublish and start searchUsingScanner. How do I solve this? Basically, the two methods should execute parallely.
Just start messagePublisher in a new Thread (Line no #5 in Main class):
new Thread(()->tool.initMessagePublish()).start();
It should solve your problem.
I am writing a java program that will need to run a python script.
The script will print output which will java need to read to know the progress of the script.
To be able to pause the script while running I want it to ask for input once in a while, only when java give it input the script will keep going.
Here is my Java method:
private static void sevenTry(String[] strCommands) throws IOException {
Object oLock1 = new Object();
Object oLock2 = new Object();
ProcessBuilder pBuilder = new ProcessBuilder(strCommands);
pBuilder.redirectErrorStream(true);
Process proc = pBuilder.start();
Thread tReader = new Thread() {
#Override
public void run() {
System.out.println("~~tReader starting~~");
BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getInputStream()));
synchronized (oLock1) {
try {
String line = reader.readLine();
while (line != null && !line.trim().equals("--EOF--")) {
System.out.println("Stdout: " + line);
if (line.trim().equals("--INPUT--")) {
synchronized (oLock2) {
oLock2.notify();
}
oLock1.wait();
}
line = reader.readLine();
}
} catch (IOException e) {
System.out.println("tReader: " + e.getMessage());
} catch (InterruptedException e) {
System.out.println("tReader: " + e.getMessage());
} catch (Exception e) {
System.out.println("tReader: " + e.getMessage());
}
}
System.out.println("~~tReader end~~");
synchronized (oLock2) {
oLock2.notify();
}
}
};
Thread tWriter = new Thread() {
#Override
public void run() {
System.out.println("~~tWriter starting~~");
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(proc.getOutputStream()));
String line, input;
Scanner scan = new Scanner(System.in);
synchronized (oLock2) {
try {
oLock2.wait();
} catch (InterruptedException e1) {
System.out.println("tWriter: " + e1.getMessage());
}
}
while (tReader.isAlive()) {
synchronized (oLock1) {
System.out.println("Java: insert input");
scan.hasNext();
input = scan.nextLine();
try {
writer.write(input + "\n");
writer.flush();
} catch (IOException e) {
System.out.println("tWriter: " + e.getMessage());
}
oLock1.notify();
}
try {
Thread.sleep(2000);
} catch (InterruptedException e1) {
System.out.println("tWriter: " + e1.getMessage());
}
}
System.out.println("~~tWriter end~~");
}
};
tReader.start();
tWriter.start();
System.out.println("~~everything submitted~~");
try {
tReader.join();
tWriter.join();
System.out.println("~~finish~~");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
This is my python script:
# coding=utf-8
import sys
print '1'
print '--INPUT--'
inum = sys.stdin.readline()
print '2'
print '--EOF--'
I tried running my code
sevenTry("python", "C:\\Testing.py");
but on java side it get stuck inside tReader at line:
String line = reader.readLine();
The program does work if i take out the input line from the python file
inum = sys.stdin.readline()
Using
inum = raw_input()
still bring up the same problem (im using python 2.7)
The most confusing part here that i even tried to test this with a java file (instead of python)
sevenTry("java", "-classpath", "C:\\class", "CheckCMD");
and it worked even with the input lines
import java.util.Scanner;
public class CheckCMD {
public static void main(String[] args) {
Scanner in = new Scanner(System.in);
String line;
System.out.println("1");
System.out.println("--INPUT--");
in.hasNext();
line = in.nextLine();
System.out.println("2");
System.out.println("--EOF--");
}
}
As you may have noticed, this is a problem related to Python.
As described in https://unix.stackexchange.com/questions/182537/write-python-stdout-to-file-immediately,
" when process STDOUT is redirected to something other than a terminal, then the output is buffered into some OS-specific-sized buffer (perhaps 4k or 8k in many cases)."
So, you need to call sys.stdout.flush() after each invoke to print.
Or, as a better option, you can change the default behaviour for the process, using the -u param, to get unbuffered output.
when I tried to get console input from both main thread and thread I created, the console input can only be retrieved by one thread, either main thread or new thread. code as follow:
public static void main(String[] args)
{
try
{
//start a new thread to accept user input
Thread thread = new Thread(new Runnable()
{
#Override
public void run()
{
BufferedReader stdIn = new BufferedReader(new InputStreamReader(System.in));
try
{
while(true)
{
String input = stdIn.readLine();
System.out.println(Thread.currentThread().getName() + ":" + input);
}
}
catch (IOException e)
{
e.printStackTrace();
}
}
}, "Owen");
thread.start();
//accept user input in main thread
BufferedReader stdIn = new BufferedReader(new InputStreamReader(System.in));
while(true)
{
String input = stdIn.readLine();
System.out.println(Thread.currentThread().getName() + ":" + input);
}
}
catch (IOException e)
{
e.printStackTrace();
}
}
possible result is as follow:
always new thread:
main
Owen:main
hello
Owen:hello
what
Owen:what
how could I know current input is going to be retrieved by which thread? or Is there any way to start two consoles(or even more) for each thread?
This question already has answers here:
run interactive command line application from java
(2 answers)
Closed 6 years ago.
Basically, I have a process which runs when I press a button on my java application.
And this process executes a command to the terminal of the OS.
But sometimes this command needs to have an interaction with the user.
And I would like to know if this was possible to have the interaction from the process to the user when needed?
My code:
File marsSimulator = new File("resources/mars_simulator/Mars4_5.jar");
if(marsSimulator.exists() && temp.exists()){
String res="";
try {
Process p = Runtime.getRuntime().exec(new String[]{"java","-jar",marsSimulator.getAbsolutePath(),tempAssembly.getAbsolutePath()});
p.waitFor();
InputStream is = p.getInputStream();
byte b[] = new byte[is.available()];
is.read(b, 0, b.length); // probably try b.length-1 or -2 to remove "new-line(s)"
res = new String(b);
} catch (Exception ex) {
ex.printStackTrace();
}
}
Also, I forgot to say that the application is made with SWING and that the output of the process is shown onto a TextArea... Should I change anything ?
Notice that the process blocks when there is an interaction with the user. If there isn't, the process doesn't block !
What do I need to do in this case (which I don't know how to do it ) ?
When the process needs the interaction. I need to know when the process wants some interaction.
I need to get the output generated of the process interactively (line by line).
P.S.: For people who wanna understand the process line, I am using the Mars Simulator (http://courses.missouristate.edu/KenVollmar/MARS/) and I am sending the jar application into a process with a mips assembly code associated.
This next pieces of code is working with my project
Hope it will help for the next adventurers!
And thank you to Nicolas Filotto for helping me.
My class ObservableStream:
class ObservableStream extends Observable {
private final Queue<String> lines = new ConcurrentLinkedQueue<>();
public void addLine(String line) {
lines.add(line);
setChanged();
notifyObservers();
}
public String nextLine() {
return lines.poll();
}
public String getLine(){return lines.peek();}
}
And the other part of the code:
Process p = Runtime.getRuntime().exec(new String[]{"java","-jar",marsSimulator.getAbsolutePath(),tempAssembly.getAbsolutePath()});
//This code does the interaction from the process with the GUI ! Implied, input interaction+output interaction from the process
ObservableStream out = new ObservableStream();
// Observer that simply sends to my external process line by line what we put in
// the variable output
PrintWriter writer = new PrintWriter(p.getOutputStream(), true);
out.addObserver(
(o, arg) -> {
ObservableStream stream = (ObservableStream) o;
String line;
while ((line = stream.nextLine()) != null) {
writer.println(line);
}
}
);
ObservableStream input = new ObservableStream();
input.addObserver(
(o, arg) -> {
ObservableStream stream = (ObservableStream) o;
String line;
while ((line = stream.nextLine()) != null) {
outputTextArea.appendText(line+"\n");
}
}
);
// The thread that reads the standard output stream of the external process
// and put the lines into my variable input
new Thread(
() -> {
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(p.getInputStream()))
) {
String line;
while ((line = reader.readLine()) != null) {
input.addLine(line);
}
} catch (IOException e1) {
e1.printStackTrace();
}
}
).start();
new Thread(
()->{
while(p.isAlive()){
String res = input.getLine();
if(res!=null && res.equals("Enter integer value:")) {
boolean integerIsRequested=true;
Thread t=null;
while(integerIsRequested){
if(t==null) {
t = new Thread(new Runnable() {
public void run() {
String test1 = JOptionPane.showInputDialog("Enter Integer value:");
while(!test1.matches("^\\d+$")){
test1 = JOptionPane.showInputDialog("Error: Not a valid Integer.\nEnter a correct Integer value:");
}
Integer i = Integer.valueOf(test1);
if (i != null) {
out.addLine(test1);
}
}
});
t.start();
}
if(!t.isAlive()){
integerIsRequested=false;
}
}
}
}
outputTextArea.appendText("Program executed\n");
}
).start();
By the way, this post is unique Jarrod ;)
To implement such use case I would personally use:
An Observable object to notify my UI when a new line has been provided by the external process
An Observable object to which I add new lines provided by my UI
An Observer of #1 that will refresh the data of my UI
An Observer of #2 that will send the lines provided by my UI to my external process
A Thread that will check if a new line has been provided by my external process and if so it will provide those lines to #1
So as I don't have your full env, I will show you how it will work with mock objects:
First my fake external application that only does an Echo of what he receives:
public class Echo {
public static void main(String[] args) {
Scanner scanner = new Scanner(System.in);
while (true) {
String line = scanner.nextLine();
System.out.printf("echo > %s%n", line);
}
}
}
If this class receives foo, it will print into the standard output stream echo > foo
Then my Observable class
public class ObservableStream extends Observable {
private final Queue<String> lines = new ConcurrentLinkedQueue<>();
public void addLine(String line) {
lines.add(line);
setChanged();
notifyObservers();
}
public String nextLine() {
return lines.poll();
}
}
NB: The class ObservableStream (as it is implemented so far) is meant to have only one Observer no more which is enough according to your needs. Indeed is only used to decouple your UI from how the data is retrieved or published
Then finally the main code:
Process p = Runtime.getRuntime().exec(
new String[]{"java", "-cp", "/my/path/to/my/classes", "Echo"}
);
// The Observable object allowing to get the input lines from my external process
ObservableStream input = new ObservableStream();
// A mock observer that simply prints the lines provided by the external process
// but in your case you will update your text area instead
input.addObserver(
(o, arg) -> {
ObservableStream stream = (ObservableStream) o;
String line;
while ((line = stream.nextLine()) != null) {
System.out.printf("Line Received from the external process: %s%n", line);
}
}
);
// The thread that reads the standard output stream of the external process
// and put the lines into my variable input
new Thread(
() -> {
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(p.getInputStream()))
) {
String line;
while ((line = reader.readLine()) != null) {
input.addLine(line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
).start();
// The Observable object allowing to send the input lines to my external process
ObservableStream output = new ObservableStream();
// Observer that simply sends to my external process line by line what we put in
// the variable output
PrintWriter writer = new PrintWriter(p.getOutputStream(), true);
output.addObserver(
(o, arg) -> {
ObservableStream stream = (ObservableStream) o;
String line;
while ((line = stream.nextLine()) != null) {
writer.println(line);
}
}
);
// A simple scanner used to send new messages to my external process
Scanner scanner = new Scanner(System.in);
while (true) {
output.addLine(scanner.nextLine());
}
If this code receives foo, it will print into the standard output stream Line Received from the external process: echo > foo
I hope it answers your question... subProcessStuff "emulates" that sub process. It can be anything - but this way we have all in place. It requires 2 params passed into console. String and Integer. Gobbler got Callback which is an interface, with anonymous implementation - and there are checks for params. To answer if subprocess waits we simply track what is says - just like if a user would operate with it.
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintStream;
import java.util.Scanner;
class Test1 {
public static void main(String[] args) {
for (String arg : args)
System.out.println("arg: " + arg);
for (String arg : args)
if (arg.equals("-test")) {
subProcessStuff();
return;
}
mainProcess();
}
public static void subProcessStuff() {
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
try {
System.out.println("Enter String");
String s = br.readLine();
System.out.println("Enered String: " + s);
System.out.println("Enter Integer:");
int i = Integer.parseInt(br.readLine());
System.out.println("Entered Integer: " + i);
} catch (IOException e) {
System.err.println("io error - " + e.getMessage());
} catch (NumberFormatException nfe) {
System.err.println("Invalid Format!");
}
}
private static PrintStream out;
public static void mainProcess() {
String[] commands = { "ls", "-alt" };
ProcessBuilder builder = new ProcessBuilder("java", "Test1", "-test");
// builder.inheritIO(); // I avoid this. It was messing me up.
try {
Process proc = builder.start();
InputStream errStream = proc.getErrorStream();
InputStream inStream = proc.getInputStream();
OutputStream outStream = proc.getOutputStream();
new Thread(new StreamGobbler("err", out, errStream)).start();
out = new PrintStream(new BufferedOutputStream(outStream));
Callback cb = new Callback() {
#Override
public void onNextLine(String line) {
if (line.equals("Enter String")) {
out.println("aaaaa");
out.flush();
}
if (line.equals("Enter Integer:")) {
out.println("123");
out.flush();
}
}
};
new Thread(new StreamGobbler("in", out, inStream, cb)).start();
int errorCode = proc.waitFor();
System.out.println("error code: " + errorCode);
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
if (out != null) {
out.close();
}
}
}
}
interface Callback {
void onNextLine(String line);
}
class StreamGobbler implements Runnable {
private PrintStream out;
private Scanner inScanner;
private String name;
private Callback cb;
public StreamGobbler(String name, PrintStream out, InputStream inStream) {
this.name = name;
this.out = out;
inScanner = new Scanner(new BufferedInputStream(inStream));
}
public StreamGobbler(String name, PrintStream out, InputStream inStream, Callback cb) {
this.name = name;
this.out = out;
inScanner = new Scanner(new BufferedInputStream(inStream));
this.cb = cb;
}
#Override
public void run() {
while (inScanner.hasNextLine()) {
String line = inScanner.nextLine();
if (cb != null)
cb.onNextLine(line);
System.out.printf("%s: %s%n", name, line);
}
}
}
I don't think you can check the state of the process from the Java. However you can do it by using some Linux command. (Of course if you're using Linux)
If your Java process has access to the /proc directory then you can read the status file for the process.
For example for a process with process id 12280
/proc/12280/status
Here's the relevant output of the status file
Name: java
State: S (sleeping)
Tgid: 12280
Pid: 12280
PPid: 12279
...
Second line gives the state of the process. You'll need to run a thread to continuously poll this file to read the status.
Line by Line The Code i use to interract with a different jar which is a speechRecognizer.I think you want to achieve something like this.
Example:
The jar i am interracting(speechRecognizer) is executing different commands and run some other Threads.Every time it has to interract with the main jar it prints something that i need.For example (user said:How are you),so you can have a same logic and when external jar need interraction with user it prints something and you read it into the main app.So:
// About Process
private Process process;
private BufferedReader bufferedReader;
private boolean stopped = true;
Thread processChecker;
//Running it in a Thread so the app don't lags
new Thread(() -> {
try {
stopped = false;
//Starting the external jar..
ProcessBuilder builder = new ProcessBuilder("java", "-jar", System.getProperty("user.home")
+ File.separator + "Desktop" + File.separator + "speechRecognizer.jar", "BITCH_PLEASE");
//Redirecting the ErrorStream
builder.redirectErrorStream(true);
process = builder.start();
bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
//Check continusly if the process is still alive
//i case of crash i should do something..
processChecker = new Thread(() -> {
while (process.isAlive()) {
try {
Thread.sleep(1200);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
stopSpeechReader(false);
});
processChecker.start();
// Continuesly Read Output of external process
while (!stopped) {
while ((line = bufferedReader.readLine()) != null && !line.isEmpty()) {
System.out.println(line);
checkSpeechResult(line);
}
}
// Interrupt the mf Thread if is Alive
if (processChecker.isAlive())
processChecker.interrupt();
System.out.println("SpeechReader Stopped! Process is alive:" + process.isAlive() + " >Exit Value:"
+ process.exitValue());
} catch (Exception e) {
e.printStackTrace();
}
}).start();
Hello i am trying to make a chat that use sockets and when i am trying to send data to the client it sends it once but then its not receiving anything then i noticed that its printwriter messed up i tried other ways but it does the same it only works once even i tried to copy everything from a tutorial online it does the SAME! does not work like it should soo whats the problem here? the printwriter should be able to send data more than once. I am developing this in JavaFX.
This is the printwriter function code that execute on button press:
public void testfunction(ActionEvent event){
//new Thread(new ListenerThread()).start();
//play();
//ObservableList<String> list = FXCollections.observableArrayList("Hello, hello?");
//TabPaneTabSlavesSlavesList.setItems(list);
/*System.out.println("Pc index: " + connectedClients.get(0).getLocalAddress().getLocalHost().getHostAddress());
System.out.println("Pc index: " + connectedClients.get(0).getLocalAddress().getLocalHost().getCanonicalHostName());
System.out.println("Pc index: " + connectedClients.get(0).getLocalAddress().getLocalHost().getAddress());
System.out.println("Pc index: " + connectedClients.get(0).getLocalAddress().getLocalHost().getHostName());*/
// Neina gauti ip ar kanors tokio su sitom funkcijomis FIX: Paimti informatcija ir persiusti su stream.
try {
Random rand = new Random();
int myrand = rand.nextInt(50) +1;
PrintWriter pw = new PrintWriter(connectedClients.get(0).getOutputStream());
pw.println("PINGiamconnected: " + Integer.toString(myrand));
pw.flush();
} catch (Exception e) {
e.printStackTrace();
}
}
This is the listener code that is working on a thread:
#Override
public void run() {
// TODO Auto-generated method stub
try {
Socket clientSocket = MainController.serverSocket.accept();
MainController.connectedClients.add(clientSocket);
MainController.NewClientConnected = true;
} catch (Exception e) {
e.printStackTrace();
}
}
And here is the reader on the client that is also working on a thread:
#Override
public void run() {
try {
BufferedReader in = new BufferedReader(new InputStreamReader(MainController.socket.getInputStream()));
//DataInputStream in = new DataInputStream(MainController.socket.getInputStream());
String line = in.readLine();
System.out.println(line);
} catch (Exception e) {
e.printStackTrace();
}
}
I think thats all you guys need i think if you need something more tell me. Thanks.
I fixed it the problem was that the thread executed only 1 time and then stopped it did work all that time – GrimReaper