I am able to launch Process with the help of below command and after launching multiple processes I want to control how many processes I want to keep at some point.
For example:
Initiate a Process inside a for loop of range 0 to 50
Pause the for loop once total active processes are 5
Resume for loop once it drop from 5 to 4 or 3 ...
I tried below code, but I am missing something.
public class OpenTerminal {
public static void main(String[] args) throws Exception {
int counter = 0;
for (int i = 0; i < 50; i++) {
while (counter < 5) {
if (runTheProc().isAlive()) {
counter = counter + 1;
}else if(!runTheProc().isAlive()) {
counter = counter-1;
}
}
}
}
private static Process runTheProc() throws Exception {
return Runtime.getRuntime().exec("cmd /c start cmd.exe /c \"dir && ping localhost\"");
}
}
Also, how to find out how many process are active? So that I can control active processes at a time.
You can use thread pool with fixed size.
For example:
public static void main(String[] args) throws Exception {
ExecutorService threadPool = Executors.newFixedThreadPool(5);
for (int i = 0; i < 50; i++) {
threadPool.submit(runTheProc);
}
}
private static final Runnable runTheProc = () -> {
Process process;
try {
process = Runtime.getRuntime().exec("cmd /c start cmd.exe /c \"dir && ping localhost\"");
} catch (Exception e) {
throw new RuntimeException(e);
}
while (process.isAlive()) { }
};
Related
I'm currently writing a program and I have this problem where I want to move the console's cursor to a specific location on the screen.
I quickly found out that this isn't possible in java so I wrote a C# script that would do this for me, but I only can run program in a separate process.
Is there a way to solve this?
Also I'm trying not to use any extra libraries like jline.
Here are some code snippets:
C#
using System;
namespace setCursor
{
public class program
{
static void Main(string[] args)
{
int x = Convert.ToInt16(args[0]);
int y = Convert.ToInt16(args[1]);
Console.SetCursorPosition(x ,y);
}
}
}
java
try
{
ProcessBuilder pb = new ProcessBuilder("setCursor", "0", "0");
Process p = pb.start();
p.waitFor();
for(int i = 0; i < 30; i++)
{
for(int j = 0; j < 120; j++)
{
Thread.sleep(1);
System.out.print(ContentOnTheScreen[i][j]);
}
}
}
catch (Exception e)
{
System.out.println(e);
}
Windows Terminal / console in recent version of Windows supports ANSI / VT codes so you could achieve movement of character position with System.out.print if your terminal is compatible. You will be able to tell by running this:
public class SetCursor {
private static String CSI = "\u001b[";
private static String at(int row, int col) {
return CSI+row+";"+col+"H";
}
public static void main(String[] args) throws InterruptedException {
System.out.println("HELLO");
System.out.print(at(1,1) + "ABCD");
System.out.print(at(10,5) + "EFGH");
System.out.println("WORLD");
for (int i = 0; i <= 100; i++) {
System.out.print(at(30,20) + " Progress: "+i+"%");
Thread.sleep(100);
}
}
}
It will either print the different values around the screen (running from a Windows Terminal Command Prompt), or if VT codes not supported (such as when running via IDE) the output might look strange:
HELLO
[1;1HABCD[10;5HEFGHWORLD
...
I've asked the question on unix.stackexchange as well...
Can the openjdk 8 (on ubuntu) run more than 4 threads
(i.e.: use the 8 cores available) on a system?
Edit: I have 12 Thread objects which run simultaneously (they all produce output) but the linux top program indicates there are only 4 cores occupied with the java process (when scaling from 1 Thread to 2, 3 and 4 I didn't have this problem)...
Edit2: top output:
%Cpu(s): 41.8 us
6982 ubuntu 20 0 8766844 1.7g 18684 S 352.0 11.6 26:07.56 java
lscpu output:
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
Edit3:
Agent[] agentz = new Agent[4]; //implements Runnable
agentz[0] = new Agent(botMovez[0], playerIndex);
for (int i = 1; i < 4; i++) {
agentz[i] = new Agent(botMovez[i], playerIndex, boardCopies[i - 1]);
}
Thread[] threadz = new Thread[4];
for (int i = 0; i < 4; i++) {
threadz[i] = new Thread(agentz[i]);
threadz[i].start();
}
for (int i = 0; i < 4; i++) {
try {
threadz[i].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
-> this code gets executed in a Runnable which is started like:
(new Thread(new Calculator(this, performedMoves))).start();
Edit4:
public class Test {
public static void main(String[] args) {
Test test = new Test();
TestAgent[] testAgentz = new TestAgent[8];
Thread[] threadz = new Thread[8];
for (int i = 0; i < 8; i++) {
testAgentz[i] = test.new TestAgent();
threadz[i] = new Thread(testAgentz[i]);
threadz[i].start();
}
System.out.println("waiting to join...");
for (int i = 0; i < 8; i++) {
try {
threadz[i].join();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.out.println("unreachable code");
}
private class TestAgent implements Runnable {
#Override
public void run() {
while (true) {
}
}
}
}
-> This code does occupy 800% cpu
So it must be either Spring boot or Tomcat...
I'm going to ask another question because this one is getting to cluttered...
I see the output of lscpu give 2 threads per core. You don't have an 8 core machine. Try running something like cat /dev/urandom > /dev/null 8 times. Probably you sill don't go over 400%.
I have a Output class which just prints everything that it gets to print.
public class Output {
private static List<String> textList = new ArrayList<>();
private static Output output = null;
private Output() {
Runnable task = () -> {
int lastIndex = 0;
while (true) {
while (lastIndex < textList.size()) {
System.out.println(lastIndex + " - " + textList.size() + ": " + textList.get(lastIndex));
outputText(textList.get(lastIndex));
lastIndex ++;
}
}
};
new Thread(task).start();
}
private static void outputText(String text) {
synchronized (System.out) {
System.out.println(text);
}
}
public static void say(String text) {
if (output == null) {
output = new Output();
}
textList.add(text);
}
}
When I add something to print, everything works fine:
for (int i = 0; i < 10; i++) {
Output.say("" + i);
}
But when I add a Thread.sleep to the loop it stops on the first output:
for (int i = 0; i < 10; i++) {
Output.say("" + i);
Thread.sleep(100);
}
How can I prevent it? I mean, I'm stopping with sleep just the main thread and not the separate thread.
When you don’t synchronize threads correctly, there is no guaranty that threads see updates made by other threads. They may either completely miss updates or see only parts of them, creating an entirely inconsistent result. Sometimes they may even appear to do the right thing. Without proper synchronization (in the sense of any valid construct specified to be thread safe), this is entirely unpredictable.
Sometimes, the chances of seeing a particular behavior are higher, like in your example. In most runs, the loop without sleep will complete before the other thread even starts its work, whereas inserting sleep raises the chance of lost updates after the second thread has seen values. Once the second thread has seen a value for textList.size(), it might reuse the value forever, evaluating lastIndex < textList.size() to false and executing the equivalent of while(true) { }.
It’s funny that the only place where you inserted a construct for thread safety, is the method outputText that is called by a single thread only (and printing to System.out is synchronized internally in most environments anyway).
Besides, it’s not clear why you are creating an object of type Output that has no relevance here, as all fields and methods are static.
Your code can be corrected and simplified to
public static void main(String[] args) throws InterruptedException {
List<String> textList = new ArrayList<>();
new Thread( () -> {
int index=0;
while(true) synchronized(textList) {
for(; index<textList.size(); index++)
System.out.println(textList.get(index));
}
}).start();
for (int i = 0; i < 10; i++) {
synchronized(textList) {
textList.add(""+i);
}
Thread.sleep(100);
}
}
though it still contains the issues of you original code of never terminating due to the infinite second thread and also burning the CPU with a polling loop. You should let the second thread wait for new items and add a termination condition:
public static void main(String[] args) throws InterruptedException {
List<String> textList = new ArrayList<>();
new Thread( () -> {
synchronized(textList) {
for(int index=0; ; index++) {
while(index>=textList.size()) try {
textList.wait();
} catch(InterruptedException ex) { return; }
final String item = textList.get(index);
if(item==null) break;
System.out.println(item);
}
}
}).start();
for (int i = 0; i < 10; i++) {
synchronized(textList) {
textList.add(""+i);
textList.notify();
}
Thread.sleep(100);
}
synchronized(textList) {
textList.add(null);
textList.notify();
}
}
This is still only an academic example that you shouldn’t use in real life code. There are classes for thread safe data exchange provided by the Java API removing the burden of implementing such things yourself.
public static void main(String[] args) throws InterruptedException {
ArrayBlockingQueue<String> queue = new ArrayBlockingQueue<>(10);
String endMarker = "END-OF-QUEUE"; // the queue does not allow null
new Thread( () -> {
for(;;) try {
String item = queue.take();
if(item == endMarker) break;// don't use == for ordinary strings
System.out.println(item);
} catch(InterruptedException ex) { return; }
}).start();
for (int i = 0; i < 10; i++) {
queue.put(""+i);
Thread.sleep(100);
}
queue.put(endMarker);
}
This is for a custom UDTF in a hive query, CreateLogTable is the UDTF class which I am using as a temp for testing. I am creating one thread per file to be downloaded from Amazon S3 and waiting until another thread becomes available before allocating another file to the thread.
Main Test logic:
CreateLogTable CLT = new CreateLogTable();
int numThreads = 2;
int index = 0;
DownloadFileThread[] dlThreads = new DownloadFileThread[numThreads];
for (S3ObjectSummary oSummary : bucketKeys.getObjectSummaries()) {
while (dlThreads[index] != null && dlThreads[index].isAlive()) {
index += 1;
index = index % numThreads;
}
dlThreads[index] = new DownloadFileThread(CLT , getBucket(oSummary.getBucketName() + "/"
+ oSummary.getKey()), getFile(oSummary.getKey()), index);
dlThreads[index].start();
index += 1;
index = index % numThreads;
}
Thread class (run() method):
try {
System.out.println("Creating thread " + this.threadnum);
this.fileObj = this.S3CLIENT.getObject(new GetObjectRequest(this.filePath, this.fileName));
this.fileIn = new Scanner(new GZIPInputStream(this.fileObj.getObjectContent()));
while (this.fileIn.hasNext()) {
this.parent.forwardToTable(fileIn.nextLine());
}
System.out.println("Finished " + this.threadnum);
} catch (Throwable e) {
System.out.println("Downloading of " + this.fileName + " failed.");
}
The while loop before the thread creation should be looping until it finds a null thread or a dead thread until it exits the loop, in which case a new thread will be created and started. Since I included logging to console, I am able to observe this process, but the output is unexpected:
Creating thread 0
Creating thread 1
Creating thread 0
Creating thread 1
Creating thread 0
Creating thread 1
Creating thread 0
...
Creating thread 1
Creating thread 0
Creating thread 1
Finished 0
Finished 1
Finished 1
Finished 0
Finished 1
Finished 1
...
Finished 0
Finished 1
Finished 0
Finished 1
The above is only the first few lines of output. The issue is that more than two threads are created before any threads complete their tasks.
Why is this happening and how can I fix this?
I reduced your code to this test case:
public class ThreadTest {
private static class SleepThread extends Thread {
private final int index;
SleepThread(int ii) { index = ii; }
#Override
public void run() {
System.out.println("Creating thread " + this.index);
try {
Thread.sleep(5_000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Finished " + this.index);
}
}
public static void main(String[] args) {
int numThreads = 2;
int index = 0;
SleepThread[] dlThreads = new SleepThread[numThreads];
for (int ii = 0; ii < 10; ++ii) {
while (dlThreads[index] != null && dlThreads[index].isAlive()) {
index += 1;
index = index % numThreads;
}
dlThreads[index] = new SleepThread(index);
dlThreads[index].start();
index += 1;
index = index % numThreads;
}
}
}
Using Sun JDK 1.7.0_75, running this produces the result that you'd expect--two threads start, they exit after five seconds, two more threads start, and so on.
The next thing I'd suspect is that your JVM's implementation of Thread.isAlive() isn't returning true for threads immediately after they are started, although that seems contrary to the documentation for the Thread class.
Try to see this example:
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
Runnable worker = new WorkerThread("" + i);
executor.execute(worker);
}
executor.shutdown();
while (!executor.isTerminated()) {
}
System.out.println("Finished all threads");
}
It's a thread pool using Java 8. A very simple and esay way to make it using the Executors. Very staraight forward way to make it.
The reason why the above code wasn't working was because of something wacky going on with the call to isAlive().
For some reason, no matter what state a thread is in, isAlive() will always return false for me, causing the creation of more and more threads, which replace the old ones in the array, dlThreads.
I solved the issue by creating a custom isWorking() method which simply returns a boolean of whether or not the thread's run() method has completed. Here is what the Thread class looks like now:
//this.isWorking initialized to true during instantiation
#Override
public void run() {
try {
System.out.println("Creating thread " + this.threadnum + " for " + filePath + "/" + fileName);
this.fileObj = this.S3CLIENT.getObject(new GetObjectRequest(this.filePath, this.fileName));
this.fileIn = new Scanner(new GZIPInputStream(this.fileObj.getObjectContent()));
while (this.fileIn.hasNext()) {
this.parent.forwardToTable(fileIn.nextLine());
}
System.out.println("Finished " + this.threadnum);
this.isWorking = false;
} catch (Throwable e) {
System.out.println("Downloading of " + this.fileName + " failed.");
e.printStackTrace();
this.isWorking = false;
}
}
public boolean isWorking(){
return this.isWorking;
}
However, after implementing this and being satisfied that my multithreaded script works, I switched over to using an Executor, as suggested by other users, which slightly improved performance and made the code much cleaner.
I am using ThreadPoolExecutor in my multithreading program, I want each thread should have particular range of ID's if ThreadSize is set as 10 and Start = 1 and End = 1000 then each thread would have range of 100 id's(basically by dividing end range with thread size) that it can use without stepping on other threads.
Thread1 will use 1 to 100 (id's)
Thread2 will use 101 to 200 (id's)
Thread3 will use 201 to 300 (id's)
-----
-----
Thread10 will use 901 to 1000
I know the logic basically, the logic can be like this-
Each thread gets `N = (End - Start + 1) / ThreadSize` numbers.
Thread number `i` gets range `(Start + i*N) - (Start + i*N + N - 1)`.
As I am working with ThreadPoolExecutor for the first time, so I am not sure where should I use this logic in my code so that each Thread is Using a predefined ID's without stepping on other threads. Any suggestions will be appreciated.
public class CommandExecutor {
private List<Command> commands;
ExecutorService executorService;
private static int noOfThreads = 3;
// Singleton
private static CommandExecutor instance;
public static synchronized CommandExecutor getInstance() {
if (instance == null) {
instance = new CommandExecutor();
}
return instance;
}
private CommandExecutor() {
try {
executorService = Executors.newFixedThreadPool(noOfThreads);
} catch(Exception e) {
System.out.println(e);
}
}
// Get the next command to execute based on percentages
private synchronized Command getNextCommandToExecute() {
}
// Runs the next command
public synchronized void runNextCommand() {
// If there are any free threads in the thread pool
if (!(((ThreadPoolExecutor) executorService).getActiveCount() < noOfThreads))
return;
// Get command to execute
Command nextCommand = getNextCommandToExecute();
// Create a runnable wrapping that command
Task nextCommandExecutorRunnable = new Task(nextCommand);
executorService.submit(nextCommandExecutorRunnable); // Submit it for execution
}
// Implementation of runnable (the real unit level command executor)
private static final class Task implements Runnable {
private Command command;
public Task(Command command) {
this.command = command;
}
public void run() {
// Run the command
command.run();
}
}
// A wrapper class that invoked at every certain frequency, asks CommandExecutor to execute next command (if any free threads are available)
private static final class CoreTask implements Runnable {
public void run() {
CommandExecutor commandExecutor = CommandExecutor.getInstance();
commandExecutor.runNextCommand();
}
}
// Main Method
public static void main(String args[]) {
// Scheduling the execution of any command every 10 milli-seconds
Runnable coreTask = new CoreTask();
ScheduledFuture<?> scheduledFuture = Executors.newScheduledThreadPool(1).scheduleWithFixedDelay(coreTask, 0, 10, TimeUnit.MILLISECONDS);
}
}
Whether this is a good idea or not I will leave it for you to decide. But to give you a hand, I wrote a little program that does what you want... in my case I am just summing over the "ids".
Here is the code:
public class Driver {
private static final int N = 5;
public static void main(String args[]) throws InterruptedException, ExecutionException{
int startId = 1;
int endId = 1000;
int range = (1 + endId - startId) / N;
ExecutorService ex = Executors.newFixedThreadPool(N);
List<Future<Integer>> futures = new ArrayList<Future<Integer>>(N);
// submit all the N threads
for (int i = startId; i < endId; i += range) {
futures.add(ex.submit(new SumCallable(i, range+i-1)));
}
// get all the results
int result = 0;
for (int i = 0; i < futures.size(); i++) {
result += futures.get(i).get();
}
System.out.println("Result of summing over everything is : " + result);
}
private static class SumCallable implements Callable<Integer> {
private int from, to, count;
private static int countInstance = 1;
public SumCallable(int from, int to) {
this.from = from;
this.to = to;
this.count = countInstance;
System.out.println("Thread " + countInstance++ + " will use " + from + " to " + to);
}
// example implementation: sums over all integers between from and to, inclusive.
#Override
public Integer call() throws Exception {
int result = 0;
for (int i = from; i <= to; i++) {
result += i;
}
System.out.println("Thread " + count + " got result : " + result);
return result;
}
}
}
which produces the following output (notice that in true multi-thread fashion, you have print statements in random order, as the threads are executed in whatever order the system decides):
Thread 1 will use 1 to 200
Thread 2 will use 201 to 400
Thread 1 got result : 20100
Thread 3 will use 401 to 600
Thread 2 got result : 60100
Thread 4 will use 601 to 800
Thread 3 got result : 100100
Thread 5 will use 801 to 1000
Thread 4 got result : 140100
Thread 5 got result : 180100
Result of summing over everything is : 500500