I am trying to open a set of links in different WebChrome drivers (atcThread[]) as fast as possible. I tried implementing ExecutorService but realized that the initial execution of the threads are sequential. Is there a way I could open up the links in parallel to be more faster. Thanks!
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class ATC {
private ExecutorService executor = Executors.newFixedThreadPool(15);
public void startThreads() {
for (int i = 0; i < Captcha.x; i++) {
executor.execute(new ATCpool(i, Generator.links[i]));
}
}
private final class ATCpool implements Runnable{
public ATCpool(int x, String link){
Generator.atcThread[x].get(link);
}
#Override
public void run() {
// TODO Auto-generated method stub
}
}
}
For me you get this behavior because you call the method get(String url) in the constructor of your class ATCpool instead of calling it into the run method to delegate its call to the thread pool as you expect.
Simply change your class ATCpool for something like this:
private final class ATCpool implements Runnable {
private final int x;
private final String link;
ATCpool(int x, String link){
this.x = x;
this.link = link;
}
#Override
public void run() {
// Will be called asynchronously by a thread of the thread pool
Generator.atcThread[x].get(link);
}
}
Related
I've created a class for multi threading in a Java application.
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
public class AppThreads {
private static final Object LOCK = new Object();
private static AppThreads sInstance;
private final Executor diskThread;
private final Executor uiThread;
private final Executor networkThread;
private AppExecutors(Executor diskThread, Executor networkThread, Executor uiThread) {
this.diskThread = diskThread;
this.networkThread = networkThread;
this.uiThread = uiThread;
}
public static AppExecutors getInstance() {
if (sInstance == null) {
synchronized (LOCK) {
sInstance = new AppExecutors(Executors.newSingleThreadExecutor(), Executors.newFixedThreadPool(4),
new MainThreadExecutor());
}
}
return sInstance;
}
public Executor diskThread() {
return diskThread;
}
public Executor networkThread() {
return networkThread;
}
private static class MainThreadExecutor implements Executor {
#Override
public void execute(Runnable command) {
command.run();
}
}
}
I am initiating a different thread as
public void getUsers(AppThreads executors) {
executors.networkThread().execute(() -> {
//Some DB operations
//getting server response code
HttpURLConnection con = (HttpURLConnection) url.openConnection();
...
...
..
int response=con.getResponseCode();
}
}
How uiThread will know the value of int response which is being executed in networkThread?
One simple solution: create some kind of callback, for example:
public interface Callback
{
void done(int response);
}
Pass the callback to your getUsers method. When you get your response code, you are able to call callback.done(response).
An alternative would be to create some kind of event / listener as #Jure mentionend.
I am recently introduced to the LMAX Disruptor and decided to give it a try. Thanks to the developers, the setup was quick and hassle free. But I think I am running into an issue if someone can help me with it.
The issue:
I was told that when the producer publish the event, it should block until the consumer had a chance to retrieve it before wrapping around. I have a sequence barrier on the consumer side and I can confirm that if there is no data published by the producer, the consumer's waitFor call will block. But, producer doesn't seem to be regulated in any way and will just wraparound and overwrite unprocessed data in the ring buffer.
I have a producer as a runnable object running on separate thread.
public class Producer implements Runnable {
private final RingBuffer<Event> ringbuffer;
public Producer(RingBuffer<Event> rb) {
ringbuffer = rb;
}
public void run() {
long next = 0L;
while(true) {
try {
next = ringbuffer.next();
Event e = ringbuffer.get(next);
... do stuff...
e.set(... stuff...);
}
finally {
ringbuffer.publish(next);
}
}
}
}
I have a consumer running on the main thread.
public class Consumer {
private final ExecutorService exec;
private final Disruptor<Event> disruptor;
private final RingBuffer<Event> ringbuffer;
private final SequenceBarrier seqbar;
private long seq = 0L;
public Consumer() {
exec = Executors.newCachedThreadPool();
disruptor = new Disruptor<>(Event.EVENT_FACTORY, 1024, Executors.defaultThreadFactory());
ringbuffer = disruptor.start();
seqbar = ringbuffer.newBarrier();
Producer producer = new Producer(ringbuffer);
exec.submit(producer);
}
public Data getData() {
seqbar.waitFor(seq);
Event e = ringbuffer.get(seq);
seq++;
return e.get();
}
}
Finally, I run the code like so:
public class DisruptorTest {
public static void main(String[] args){
Consumer c = new Consumer();
while (true) {
c.getData();
... Do stuff ...
}
}
You need to add a gating sequence (com.lmax.disruptor.Sequence) to the ringBuffer, this sequence must be updated on what point your consumer is.
You can implement your event handling with EventHandler interface and using the provided BatchEventProcessor(com.lmax.disruptor.BatchEventProcessor.BatchEventProcessor) which comes with builtin sequence
Here's a fully working example
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import com.lmax.disruptor.BatchEventProcessor;
import com.lmax.disruptor.EventHandler;
import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.SequenceBarrier;
import com.lmax.disruptor.dsl.Disruptor;
public class Main {
static class Event {
int id;
}
static class Producer implements Runnable {
private final RingBuffer<Event> ringbuffer;
public Producer(RingBuffer<Event> rb) {
ringbuffer = rb;
}
#Override
public void run() {
long next = 0L;
int id = 0;
while (true) {
try {
next = ringbuffer.next();
Event e = ringbuffer.get(next);
e.id = id++;
} finally {
ringbuffer.publish(next);
}
}
}
}
static class Consumer {
private final ExecutorService exec;
private final Disruptor<Event> disruptor;
private final RingBuffer<Event> ringbuffer;
private final SequenceBarrier seqbar;
private BatchEventProcessor<Event> processor;
public Consumer() {
exec = Executors.newCachedThreadPool();
disruptor = new Disruptor<>(() -> new Event(), 1024, Executors.defaultThreadFactory());
ringbuffer = disruptor.start();
seqbar = ringbuffer.newBarrier();
processor = new BatchEventProcessor<Main.Event>(
ringbuffer, seqbar, new Handler());
ringbuffer.addGatingSequences(processor.getSequence());
Producer producer = new Producer(ringbuffer);
exec.submit(producer);
}
}
static class Handler implements EventHandler<Event> {
#Override
public void onEvent(Event event, long sequence, boolean endOfBatch) throws Exception {
System.out.println("Handling event " + event.id);
}
}
public static void main(String[] args) throws Exception {
Consumer c = new Consumer();
while (true) {
c.processor.run();
}
}
}
I have a problem about my java GUI management. I'm making an updater but when i'm trying to edit some JText or JProgressbar from an ExecutorService Thread, nothing..
Why is it broken ? This is my ControllerManager:
public class UpdateManager extends AppManager {
private final List<Controller> controllers;
#Getter private final ExecutorService worker;
#Getter private final UserInterface form;
private boolean isEnd;
public UpdateManager() {
this.controllers = new ArrayList<>();
this.form = new Form(this);
this.worker = Executors.newCachedThreadPool();
}
#Override
public void start(Controller... controllers) {
form.initialize();
this.controllers.addAll(Arrays.asList(controllers));
for(Controller controller: controllers)
controller.start();
}
#Override
public void end(boolean dispose) {
if(isEnd) return;
for(Controller controller: controllers)
controller.end();
if(dispose) form.dispose();
isEnd = true;
}
}
And here is an example when i'm trying to change some labels:
/**
* Created by romain on 17/05/2015.
*/
public class ReleaseController implements Controller {
private final AppManager manager;
#Getter private final LinkedBlockingDeque<URL> files;
private Future<?> future;
private final SerializedObject<SerializedReleases> serializedReleases;
private final SerializedObject<Integer> serializedRelease, serializedTimestamp;
public ReleaseController(AppManager manager) {
this.manager = manager;
this.files = new LinkedBlockingDeque<>();
this.serializedReleases = SerializedObjectImpl.create(FileUtils.path("releases", "releases.dat"), true, null);
this.serializedRelease = SerializedObjectImpl.create(FileUtils.path("swtour", "release.int"), false, 0);
this.serializedTimestamp = SerializedObjectImpl.create(FileUtils.path("swtour", "timestamp.int"), false, 0);
}
/**
* TODO: checking local files
*/
#Override
public void start() {
this.future = manager.getWorker().submit(new Runnable() {
#Override
public void run() {
int release = serializedRelease.get();
int serverRelease = serializedReleases.get().lastRelease(AppUtils.OS);
int result = serverRelease - release;
if(result == 0 || result < 0) {
manager.getForm().alreadyUpdated(); //HERE
return;
}
for(int i=release+1;i<serverRelease;i++) {
try {
files.addLast(new URL(
FileUtils.path(Main.SERVER, "releases", AppUtils.OS.toString(), i + ".zip")));
} catch(Exception e) {
System.out.println(e.getMessage());
}
}
serializedRelease.setObject(serverRelease).write();
serializedTimestamp.setObject((int)System.currentTimeMillis()).write();
}
});
}
#Override
public void end() {
if(future != null && !future.isCancelled())
future.cancel(true);
}
}
If you saw correctly, it:
manager.getForm().alreadyUpdated();
Modifies my GUI :
public void alreadyUpdated() {
content.getFirstLine().setText("");
content.getSecondLine().setText("Your client is already up-to-date!");
content.getPlayButton().setEnabled(true);
}
but nothing.. My gui is not changed !
Thank you
There are two problems with attempting to update a Swing data structure from another thread:
Swing is not designed to be multi-thread safe. You could corrupt its data structures.
There is nothing forcing it to see changes made in another thread. It can be running with values in registers or caches that are not affected by changes to memory.
As already noted in a comment, you need to use invokeAndWait or invokeLater to make your changes in the Swing event handling thread.
I whould like to block a method execution from more thab 3 threads. The method can be executed recursively. I have following agly code. Can I achive this by using better way?
private static class MyHolder {
private static Semaphore limitThreadsSemaphore = new Semaphore(3);
private static Set<Thread> asquiredThreads = new HashSet<Thread>();
}
#Override
public void someMethod() {
if (!MyHolder.asquiredThreads.contains(Thread.currentThread())) {
synchronized (MyHolder.asquiredThreads) {
if (!MyHolder.asquiredThreads.contains(Thread.currentThread())) {
try {
MyHolder.limitThreadsSemaphore.acquire();
MyHolder.asquiredThreads.add(Thread.currentThread());
} finally {
MyHolder.limitThreadsSemaphore.release();
MyHolder.asquiredThreads.remove(Thread.currentThread());
}
}
}
}
return super.someMethod();
}
Thanks.
The simplest approach would be to refactor the recursive method to be private and then have the public method unconditionally acquire the semaphore, call the private method and then release the semaphore again. The recursive calls route straight to the private method so don't go through the semaphore guard code.
If that is not an option then the simplest approach I can think of would be to use a ThreadLocal flag
ThreadLocal<Object> alreadyIn = new ThreadLocal<>();
public void someMethod() {
boolean needSem = (alreadyIn.get() == null);
if(needSem) {
semaphore.acquire();
alreadyIn.set(new Object());
}
try {
// do stuff
} finally {
if(needSem) {
alreadyIn.remove();
semaphore.release();
}
}
}
I guess "someMethod" is the method you want to block execution,yeah?. Why don'y you do this? :
private static class MyHolder {
private static Semaphore limitThreadsSemaphore = new Semaphore(3);
public boolean semaphoreAdquired = false; //Make it private
public Semaphore getSemaphore()
{
return limitThreadsSemaphore;
}
}
#Override
public void someMethod() {
boolean ReleaseSemaphore = false;
if(!semaphoreAdquired)
{
MyHolder.getSemaphore().acquire();
semaphoreAdquired = true;
ReleaseSemaphore = true;
}
super.someMethod();
if(ReleaseSemaphore)
{
MyHolder.getSemaphore().release();
semaphoreAdquired = false;
}
}
Based on the documentation for Semaphor, this should be achievable using only acquire() and release() around the critical section. Also, you should be able to put the semaphor in the current class, no need for a separate class to contain the Semaphor.
private static Semaphore limitThreadsSemaphore = new Semaphore(3);
#Override
public void someMethod() {
limitThreadsSemaphore.acquire();
// do work.
limitThreadsSemaphore.release();
}
Update: If you need to call a method recursively within a thread, then the easiest way is to use a helper method to acquire the semaphor, and then invoke the recursive method from that helper method after acquiring the sempahor. You would call the helper instead of the original method in all the initial calls.
private static Semaphore limitThreadsSemaphore = new Semaphore(3);
public void someMethodHelper() {
limitThreadsSemaphore.acquire();
someMethod();
limitThreadsSemaphore.release();
}
#Override
public void someMethod() {
// do work, with recursive calls.
}
Ok, so, I have a java class in which every method must run in a thread.
only one thread is executed per time and at a specific time.
I've implemented like this, with a inner class that extends Thread.
private class MyOperation extends Thread {
public static final String M1 = "method1";
public static final String M2 = "method2";
public static final String M3 = "method3";
protected long scheduledStartTime = 0;
private String type;
public MyOperation(String type, long milliSecondsToWait) {
this.type = type;
scheduledStartTime = System.currentTimeMillis() + mlliSecondsToWait;
}
#Override
public void run() {
switch(type){
case M1:
doMethod1();
break;
case M2:
doMethod3();
break;
case M3:
doMethod3();
break;
}
setCurrentOperation(null);
}
}
private void setCurrentOperation(MyOperation task) {
synchronized (currentOperation) {
this.currentOperation = task;
}
}
then I have the Thread queue and the current running thread
private MyOperation currentOperation;
private Queue <MyOperation> operationList;
And I'm fetching tasks like this:
private void fetchTasks() {
new Thread() {
#Override
public void run() {
while(true) {
if(currentOperation == null && !operationList.isEmpty()) {
currentOperation = getOperation();
while(currentOperation.scheduledStartTime > System.currentTimeMillis()) {
// do nothing, wait for proper time;
}
currentOperation.start();
}
}
}
}.start();
}
private MyOperation getOperation() {
synchronized (operationList) {
return operationList.remove();
}
}
and I'm adding thread to the queue like this, for example:
addOperation(new MyOperation(M1, 5));
private void addOperation(MyOperation task) {
synchronized (operationList) {
operationList.add(task);
}
}
My questions are:
Is there a better way to run each method in a diffent thread?
Is this way of fetching the threads queue correct?
Thank you very much
Just a little thing: if your operationsList is empty or the currentOperation is not null your thread starts going in circles really fast.
You could use Thread.wait() and .notify() to avoid this.
Also you are using currentOperation with and without synchronized. This could get you into trouble.
Have you condsidered using a ScheduledExecutorService (java.util.concurrent) to schedule your tasks?