Develop Reference

How to get the execution results of ExecutorService without blocking the current code path?

I have a service which adds a bunch of requests to Callables and then prints the results of the executions. Currently the service request is blocked until I print all the Future results from the execution. However I want to return 200 to the requestor and run these requests in parallel without blocking the request. How can I achieve this? Below is my code.
Below is my code to run parallel code.
public void runParallelFunctions(Callable<Map<String, String>> invokerTask) {
List<Callable<Map<String, String>>> myTasks = new ArrayList<>();
for (int i = 0; i < invocationCount; i++) {
myTasks.add(invokerTask);
}
List<Future<Map<String, String>>> results = null;
try {
results = executorService.invokeAll(myTasks);
} catch (InterruptedException e) {
}
this.printResultsFromParallelInvocations(results);
}
Below is how I print the results from the Futures.
private void printResultsFromParallelInvocations(List<Future<Map<String, String>>> results) {
results.forEach(executionResults -> {
try {
executionResults.get().entrySet().forEach(entry -> {
LOGGER.info(entry.getKey() + ": " + entry.getValue());
});
} catch (InterruptedException e) {
} catch (ExecutionException e) {
}
});
}
Below is how I'm invoking the above methods when someone places a request to the service.
String documentToBeIndexed = GSON.toJson(indexDocument);
int documentId = indexMyDocument(documentToBeIndexed);
createAdditionalCandidatesForFuture(someInput);
return true;
In the above code, I call the createAdditionalCandidatesForFuture and then return true. But the code still waits for the printResultsFromParallelInvocations method to complete. How can I make the code return after invoking createAdditionalCandidatesForFuture without waiting for the results to print? Do I have to print the results using another executor thread or is there another way? Any help would be much appreciated

The answer is CompletableFuture.
Updated runParallelFunctions:
public void runParallelFunctions(Callable<Map<String, String>> invokerTask) {
// write a wrapper to handle exception outside CompletableFuture
Supplier<Map<String, String>> taskSupplier = () -> {
try {
// some task that takes a long time
Thread.sleep(4000);
return invokerTask.call();
} catch (Exception e) {
System.out.println(e);
}
// return default value on error
return new HashMap<>();
};
for (int i = 0; i < 5; i++) {
CompletableFuture.supplyAsync(taskSupplier, executorService)
.thenAccept(this::printResultsFromParallelInvocations);
}
// main thread immediately comes here after running through the loop
System.out.println("Doing other work....");
}
And, printResultsFromParallelInvocations may look like:
private void printResultsFromParallelInvocations(Map<String, String> result) {
result.forEach((key, value) -> System.out.println(key + ": " + value));
}
Output:
Doing other work....
// 4 secs wait
key:value

Calling get on a Future will block the thread until the task is completed, so yes, you will have to move the printing of the results to another thread/Executor service.
Another option is that each task prints its results upon completion, provided they are supplied with the necessary tools to do so (Access to the logger, etc). Or putting it in another way, each task is divided into two consecutive steps: execution and printing.

concurrent modification on arraylist

There are a lot of concurrent mod exception questions, but I'm unable to find an answer that has helped me resolve my issue. If you find an answer that does, please supply a link instead of just down voting.
So I originally got a concurrent mod error when attempting to search through an arraylist and remove elements. For a while, I had it resolved by creating a second arraylist, adding the discovered elements to it, then using removeAll() outside the for loop. This seemed to work, but as I used the for loop to import data from multiple files I started getting concurrent modification exceptions again, but intermittently for some reason. Any help would be greatly appreciated.
Here's the specific method having the problem (as well as the other methods it calls...):
public static void removeData(ServiceRequest r) {
readData();
ArrayList<ServiceRequest> targets = new ArrayList<ServiceRequest>();
for (ServiceRequest s : serviceQueue) {
//ConcurrentModification Exception triggered on previous line
if (
s.getClient().getSms() == r.getClient().getSms() &&
s.getTech().getName().equals(r.getTech().getName()) &&
s.getDate().equals(r.getDate())) {
JOptionPane.showMessageDialog(null, s.getClient().getSms() + "'s Service Request with " + s.getTech().getName() + " on " + s.getDate().toString() + " has been removed!");
targets.add(s);
System.out.print("targetted"); }
}
if (targets.isEmpty()) { System.out.print("*"); }
else {
System.out.print("removed");
serviceQueue.removeAll(targets);
writeData(); }
}
public static void addData(ServiceRequest r) {
readData();
removeData(r);
if (r.getClient().getStatus().equals("MEMBER") || r.getClient().getStatus().equals("ALISTER")) {
serviceQueue.add(r); }
else if (r.getClient().getStatus().equals("BANNED") || r.getClient().getStatus().equals("UNKNOWN")) {
JOptionPane.showMessageDialog(null, "New Request failed: " + r.getClient().getSms() + " is " + r.getClient().getStatus() + "!", "ERROR: " + r.getClient().getSms(), JOptionPane.WARNING_MESSAGE);
}
else {
int response = JOptionPane.showConfirmDialog(null, r.getClient().getSms() + " is " + r.getClient().getStatus() + "...", "Manually Overide?", JOptionPane.OK_CANCEL_OPTION);
if (response == JOptionPane.OK_OPTION) {
serviceQueue.add(r); }
}
writeData(); }
public static void readData() {
try {
Boolean complete = false;
FileReader reader = new FileReader(f);
ObjectInputStream in = xstream.createObjectInputStream(reader);
serviceQueue.clear();
while(complete != true) {
ServiceRequest test = (ServiceRequest)in.readObject();
if(test != null && test.getDate().isAfter(LocalDate.now().minusDays(180))) {
serviceQueue.add(test); }
else { complete = true; }
}
in.close(); }
catch (IOException | ClassNotFoundException e) { e.printStackTrace(); }
}
public static void writeData() {
if(serviceQueue.isEmpty()) { serviceQueue.add(new ServiceRequest()); }
try {
FileWriter writer = new FileWriter(f);
ObjectOutputStream out = xstream.createObjectOutputStream(writer);
for(ServiceRequest r : serviceQueue) { out.writeObject(r); }
out.writeObject(null);
out.close(); }
catch (IOException e) { e.printStackTrace(); }
}
EDIT
The changes cause the concurrent mod to trigger every time rather than intermittently, which I guess means the removal code is better but the error now triggers at it.remove();
public static void removeData(ServiceRequest r) {
readData();
for(Iterator<ServiceRequest> it = serviceQueue.iterator(); it.hasNext();) {
ServiceRequest s = it.next();
if (
s.getClient().getSms() == r.getClient().getSms() &&
s.getTech().getName().equals(r.getTech().getName()) &&
s.getDate().equals(r.getDate())) {
JOptionPane.showMessageDialog(null, s.getClient().getSms() + "'s Service Request with " + s.getTech().getName() + " on " + s.getDate().toString() + " has been removed!");
it.remove(); //Triggers here (line 195)
System.out.print("targetted"); }
}
writeData(); }
Exception in thread "AWT-EventQueue-0" java.util.ConcurrentModificatio
nException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
at java.util.ArrayList$Itr.next(ArrayList.java:851)
at data.ServiceRequest.removeData(ServiceRequest.java:195)
at data.ServiceRequest.addData(ServiceRequest.java:209) <...>
EDIT
After some more searching, I've switch the for loop to:
Iterator<ServiceRequest> it = serviceQueue.iterator();
while(it.hasNext()) {
and it's back to intermittently triggering. By that I mean the first time I attempt to import data (the removeData method is being triggered from the addData method) it triggers the concurrent mod exception, but the next try it pushes past the failure and moves on to another file. I know there's a lot of these concurrent mod questions, but I'm not finding anything that helps in my situation so links to other answers are more than welcome...

This is not how to do it, to remove elements while going through a List you use an iterator. Like that :
List<ServiceRequest> targets = new ArrayList<ServiceRequest>();
for(Iterator<ServiceRequest> it = targets.iterator(); it.hasNext();) {
ServiceRequest currentServReq = it.next();
if(someCondition) {
it.remove();
}
}
And you will not get ConcurrentModificationException this way if you only have one thread.
If there is multiple threads involved in your code, you may still get ConcurrentModificationException. One way to solve this, is to use Collections.synchronizedCollection(...) on your collection (serviceQueue) and as a result you will get a synchronized collection that will not produce ConcurrentModificationException. But, you code may become very slow.

Multithreading ArrayList iteration

Greets!
I am writing a simple server monitoring application in Java(JavaFX8). The current implementation is able to ping target machines one by one, and graph them onto a JavaFX LineChart. Each machine is a "Target" object, which is held in an ArrayList (Observable). My problem is the "one by one" part. The code to ping a target is a Callable that returns the ping. I, somehow, need to multithread the process so that I can ping the targets at least four at a time. Past attempts resulted in quirks such as four threads pinging the same target at the same time, resulting in a very pointless and processor intense redundancy. Heres my current loop...
public void beginPing() {
ExecutorService exec = Executors.newCachedThreadPool();
Runnable r = new Runnable() {
#Override
public void run() {
while (true) {
for (Target t : targets) {
String ping = null;
if (t.flagsProperty().get().contains("A")) {
try {
Callable c = new Pinger(t);
ping = c.call().toString();
switch (ping) {
case "TIME_OUT":
for (XYChart.Series s : lineChart.getData()) {
if (s.getName().equals(t.nameProperty().get())) {
addToChart(s, cycle, 00.00);
}
}
t.setStatus("TIME OUT");
t.setLastrtt("TIME_OUT");
t.setTimeouts(t.timeoutsProperty().get() + 1);
logUtil.log(LogUtil.INFO, t.nameProperty().get() + " - timed out!");
break;
case "UNKNOWN_HOST":
t.setStatus("ERROR");
t.setLastrtt("UNKNOWN HOST");
logUtil.log(LogUtil.WARNING, t.nameProperty().get() + " - unknown host!");
break;
case "UNREACHABLE":
t.setStatus("ERROR");
t.setLastrtt("UNREACHABLE HOST");
logUtil.log(LogUtil.WARNING, t.nameProperty().get() + " - is unreachable!");
break;
default:
t.setLastrtt(ping);
t.setStatus("ACTIVE");
for (XYChart.Series s : lineChart.getData()) {
if (s.getName().equals(t.nameProperty().get())) {
addToChart(s, cycle, Double.valueOf(ping));
}
}
break;
}
} catch (Exception e) {
logUtil.log(LogUtil.CRITICAL, e.getMessage() + ", "+ e.getCause());
e.printStackTrace();
}
}
}
cycle++;
rangeChart(cycle);
updateInfo();
}
}
};
exec.execute(r);
}

My impression is that you misuse your Callable class Pinger like a regular class, although it is only an interface that does not implement any multithreading services.
The thing you want to do should look more like this:
//init
Future<String> futures = new Future[targets.length];
String results = new String[targets.length];
ExecutorService service = Executors.newCachedThreadPool();
//start Threads
for (int i = 0; i<targets.length; i++){
Pinger pinger= new Pinger(targets[i]);
future[i] = service.submit(pinger);
}
//wait for Threads to finish and get results
for(int i = 0; i<futures.length; i++)
results[i] = futures[i].get()
Your Pinger should look like this:
public class Pinger implements Callable<String>{
Pinger(Target target){ ... }
public String call(){ ... }
}
Here you find a fully implemented Example for Callables. In your code you only submit one Runnable to the ExecutorService, so there will be only two threads (Main and your Runnable). You never call the method call(), this is done by the ExecutorService. Compare this to the Runnable Interface you have to execute the Thread calling start or submitting it to a ExecutorService instead of calling run(); You use the Future that is returned during the submit(). Just try to understand the concept of Callable and then you will be able to write everything you want. ;-)

So, heres the current working implementation...
public void beginPing() {
safeTargets = new ArrayList<>(); //thread-safe collection
for (Target t : targets) {
safeTargets.add(t);
}
safeTargets = Collections.synchronizedList(targets);
exec = Executors.newCachedThreadPool();
for (int i = 0; i < 4; i++) { //number of threads
exec.execute(new Runnable() {
#Override
public void run() {
while (true) {
for (Target t : safeTargets) {
String ping = null;
if (t.isActive() && !t.isIsBeingPinged()) { //checks if target is already being pinged by another thread and if it flagged as active and wishes to be pinged.
t.setIsBeingPinged(true);
t.setPinged(t.getPinged() + 1); //just to see how many times it has been pinged
t.setStatus("PINGING");
try {
Callable c = new Pinger(t);
ping = c.call().toString();
switch (ping) {
case "TIME_OUT":
t.setStatus("TIME OUT");
t.setLastrtt("TIME_OUT");
t.setTimeouts(t.timeoutsProperty().get() + 1);
logUtil.log(LogUtil.INFO, t.nameProperty().get() + " - timed out!");
t.setIsBeingPinged(false);
break;
case "UNKNOWN_HOST":
t.setStatus("ERROR");
t.setLastrtt("UNKNOWN HOST");
logUtil.log(LogUtil.WARNING, t.nameProperty().get() + " - unknown host!");
t.setIsBeingPinged(false);
break;
case "UNREACHABLE":
t.setStatus("ERROR");
t.setLastrtt("UNREACHABLE HOST");
logUtil.log(LogUtil.WARNING, t.nameProperty().get() + " - is unreachable!");
t.setIsBeingPinged(false);
break;
default:
t.setLastrtt(ping);
t.setStatus("ACTIVE");
t.setIsBeingPinged(false);
break;
}
System.out.println("C=" + t.getPinged() + " - " + t.nameProperty().get());
} catch (Exception e) {
logUtil.log(LogUtil.CRITICAL, e.getMessage() + ", " + e.getCause());
e.printStackTrace();
}
}
}
}
}
});
}
}
I had to get rid of the immediate addition to the chart after all.
The Target list objects get added to a thread-safe synchronizedList (as suggested by kleopatra).
A boolean variable was added to the Target model to determine if it is currently being pinged by one of the threads. (t.isIsBeingPinged())
The data gets added to the chart using a new Runnable in the same pool, which iterates the target list and adds the last RTT to the chart every second, to avoid Targets with higher pings from falling behind on the chart.
Thanks for the very quick responses!

ZeroMQ - jzmq .recvZeroCopy() fails to get any message while .recv() works

So I am writing my own piece of stuff using jzmq GIT master branch and ZeroMQ 3.2.3.
After installation I tried to test the following simple PUB/SUB program, where a publisher and a subscriber talk in a single process. Since the test is under Windows, I used TCP.
public class ZMQReadynessTest {
private ZMQ.Context context;
#Before
public void setUp() {
context = ZMQ.context(1);
}
#Test
public void testSimpleMessage() {
String topic = "tcp://127.0.0.1:31216";
final AtomicInteger counter = new AtomicInteger();
// _____________________________________ create a simple subscriber
final ZMQ.Socket subscribeSocket = context.socket(ZMQ.SUB);
subscribeSocket.connect(topic);
subscribeSocket.subscribe("TestTopic".getBytes());
Thread subThread = new Thread() {
#Override
public void run() {
while (true) {
String value = null;
// This would result in trouble /\/\/\/\/\/\/\/\/\
{
ByteBuffer buffer = ByteBuffer.allocateDirect(100);
if (subscribeSocket.recvZeroCopy( buffer,
buffer.remaining(),
ZMQ.DONTWAIT
) > 0 ) {
buffer.flip();
value = buffer.asCharBuffer().toString();
System.out.println(buffer.asCharBuffer().toString());
}
}
// This works perfectly + + + + + + + + + + + + +
/*
{
byte[] bytes = subscribeSocket.recv(ZMQ.DONTWAIT);
if (bytes == null || bytes.length == 0) {
continue;
}
value = new String(bytes);
}
*/
if (value != null && value.length() > 0) {
counter.incrementAndGet();
System.out.println(value);
break;
}
}
}
};
subThread.start();
// _____________________________ create a simple publisher
ZMQ.Socket publishSocket = context.socket(ZMQ.PUB);
publishSocket.bind("tcp://*:31216");
try {
Thread.sleep(3000); // + wait 3 sec to make sure its ready
} catch (InterruptedException e) {
e.printStackTrace();
fail();
}
// publish a sample message
try {
publishSocket.send("TestTopic".getBytes(), ZMQ.SNDMORE);
publishSocket.send("This is test string".getBytes(), 0);
subThread.join(100);
} catch (InterruptedException e) {
e.printStackTrace();
fail();
}
assertTrue(counter.get() > 0);
System.out.println(counter.get());
}
}
Now as you can see, in the subscriber if I use a simple .recv(ZMQ.DONTWAIT) method, it works perfectly. However, if I am using the direct byte buffer I got nothing returned - and I got the following exception, seems like on program exit:
Exception in thread "Thread-0" org.zeromq.ZMQException: Resource temporarily unavailable(0xb)
at org.zeromq.ZMQ$Socket.recvZeroCopy(Native Method)
at ZMQReadynessTest$1.run(ZMQReadynessTest.java:48)
I also tried to use a simple ByteBuffer (not a direct buffer), which doesn't throw the exception above; but also return me nothing.
Does anybody know how to resolve the above?
I don't want to create byte[] objects all around, as I am doing some high performance system. If this cannot be resolved, I might simply use Unsafe instead. But I really want to work in the "supposed way".
Thanks in advance.
Alex

Issue with java boolean

I'm attempting to create a chat client, I'm returning a message from the javaSpace, Then setting newMessage = true; So the client can see that there is a new message that needs to be read.
public void notify(RemoteEvent ev)
{
try
{
messageRead = false;
QueueItem qiTemplate = new QueueItem();
newMessage = (QueueItem)space.take(qiTemplate,null,Long.MAX_VALUE);
System.out.println(newMessage.getSender().getName()+ ": " + newMessage.getContent());
}
catch (Exception e)
{
e.printStackTrace();
}
}
Then for the client,
while(true)
{
try
{
boolean temp = _manager.messageRead;
//System.out.println(temp);
if(!temp)
{
QueueItem nextJob = _manager.newMessage;
String nextJobNumber = nextJob.getSender().getName();
String nextJobName = nextJob.getContent();
System.out.println(nextJob.getSender().getName()+ ": " + nextJob.getContent());
jobList.append( nextJobNumber + " : " + nextJobName + "\n" );
_manager.messageRead = true;
}
} catch ( Exception e)
{
e.printStackTrace();
}
}
That right now will ALWAYS return _messager.messageRead to be true, even though I've just set it too false. If I uncomment //System.out.println(temp); the boolean will then for some reason be updated and it will equal what its meant too.
I've never come across this error before and its extremely strange to me, So I'm hoping someone can help.

You don't seem to have synchronized accesses to your boolean flag messageRead. println happens to do that for you hence what you observe.
You could probably fix your issue by decalring the flag volatile:
private volatile boolean messageRead;
That will ensure that changes you make in one thread are visible from another thread without needing to synchronize your code.