I am emulating a simple connection between a client and a server. The client petitions are sent and the server proccesses them in a concurrent way: the server class extends Thread and the task is run when the object is created.
The server is always open, listening to petitions, when there is one then a object is created using the socket as a parameter, and the task is then run as I said.
I am trying to measure the time it takes to process all the petitions one client sends at once, but I can't manage to do it. With threads, pools and such I would usually take the initial time and take the time when I know everything finished and voila (usually after a join or checking if the pool is terminated).
But now I can't manage to know when all the tasks are done, because the server is always running.
Any ideas?
I'm going to try to sum up the code in case someone didn't understand:
import java.net.*;
import java.io.*;
public class MyServer extends Thread
{
Socket socket;
public MyServer(Socket s) { socket=s; this.start(); }
public void run()
{
// proccessing of the data sent by the client (just printing values)
// data is read properly, don't worry
socket.close();
}
public static void main(String[] args)
{
int port = 2001; // the same one the client is using
try
{
ServerSocket chuff = new ServerSocket(port, 3000);
while (true)
{
Socket connection = chuff.accept();
new MyServer(connection);
}
} catch (Exception e) {}
}
}
It's not clear from your question whether a client will (a) send more work down a single connection later, or (b) open multiple connections at once.
If it won't ever do either, then the processing of one connection is the unit of work to time (and in fact I think all you need to time is how long the thread is alive for).
If a client might do one of those things, then if you can, change your protocol so that clients send work in one single packet; then measure how long it takes to process one of those packets. This gives you an unambiguous definition of what you are actually measuring, the lack of which might be what is causing you problems here.
For each incoming connection, I would do it as follows:
Handover the connection to a Runnable class that performs the work.
Measure the time taken by the run method and at the end of run method, prepare a Statistics object that contains the client details and the time taken to run and post it to a LinkedBlockingQueue.
Have another thread that would poll this queue, extracts the Statistics object and updates the database or data where per-client run times are tracked.
If you want to be notified when no more connections are incomming you must set a SO_TIMEOUT, otherwise accept() blocks forever. Timeouts are enabled by invoking ServerSocket.setSoTimeout(int).
To measure performance each thread could update a shared variable with the time when they completed the task. Mark this variable as volatile to keep the values synchronized and wait until all your threads have terminated and accept has raised a java.net.SocketTimeoutException.
Note that you're also measuring the network latency between the incoming requests, is this inteded?
I would highly recommended instead of creating new Thread every time on accepting the client task consider using ExecutorService instead.
If you want to check the timing for performing number of tasks by server may be you can send list of task in one go as mentioned above and use CompletionService to check total time to complete all tasks(Runnable). Below is a sample test class to show how to capture completion time:
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.*;
public class ServerPerformanceTest {
public static void main(String[] args) {
System.out.println("Total time taken : " + totalTimeTaken(1000, 16));
}
public static long totalTimeTaken(final int taskCount, final int threadCount) {
//Mocking Dummy task send by client
Runnable clientTask = new Runnable() {
#Override
public void run() {
System.out.println("task done");
}
};
long startTime = System.currentTimeMillis();
//Prepare list of tasks for performance test
List<Runnable> tasks = Collections.nCopies(taskCount, clientTask);
ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
ExecutorCompletionService<String> completionService = new ExecutorCompletionService<String>(executorService);
//Submit all tasks
for (Runnable _task : tasks) {
completionService.submit(_task, "Done");
}
//Get from all Future tasks till all tasks completed
for (int i = 0; i < tasks.size(); i++) {
try {
completionService.take().get();
} catch (InterruptedException e) {
e.printStackTrace(); //do something
} catch (ExecutionException e) {
e.printStackTrace(); //do something
}
}
long endTime = System.currentTimeMillis();
return (endTime - startTime);
}
}
Related
A service I am using starts blocking requests after 5 are made within 1 second.
Using Java in Spring I am looking for a way to queue threads in such a way that up to 5 threads can access the critical section within a second and any other threads are queued up and released once there is bandwidth for them to continue.
Currently I've attempted this with a lock but it causes the thread to wait 1/5th of a second always, even if we wouldn't be at the max calls per second without sleeping.
Lock l = new ReentrantLock();
try {
l.lock();
//critical section
} finally {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
l.unlock();
}
With this implementation I never exceed the 5 per second but I also cause the response to be delayed by 200 milli after everything is ready to be returned to the user.
I need a solution that only delays threads when a delay is needed. In this case the 6th+ call in a second should be delayed but the first 5 do not need to be delayed. Likewise calls 6-11 could all go through at the same time.
This sort of rate-limiting is quite a common problem in microservice architectures, as it is part of the broader issue of addressing cascading failures. There are many libraries around to deal with this issue, and one of the most widely-used modern ones is called Resilience4j, which provides a RateLimiter implementation. You probably want something pretty close to this:
Create the limiter:
RateLimiterConfig config = RateLimiterConfig.custom()
.limitRefreshPeriod(Duration.ofSeconds(1))
.limitForPeriod(5)
.timeoutDuration(Duration.ofSeconds(4)) //or however long you want to wait before failing
.build();
// Create registry
RateLimiterRegistry rateLimiterRegistry = RateLimiterRegistry.of(config);
// Use registry
RateLimiter rateLimiter = rateLimiterRegistry
.rateLimiter("someServiceLimiter", config);
Use it:
// Decorate your call to BackendService.doSomething()
CheckedRunnable restrictedCall = RateLimiter
.decorateCheckedRunnable(rateLimiter, backendService::doSomething);
//Or, you can use an annotation:
#RateLimiter(name = "someServiceLimiter")
public void doSomething() {
//backend call
}
I think solving it using semaphore API would be the best approach.
import java.util.LinkedList;
import java.util.Queue;
import java.util.concurrent.*;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class BulkheadSemaphore {
private Queue<Long> enterQueue = new LinkedList<>();
private ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor();
private Semaphore semaphore;
public BulkheadSemaphore(final Long timeLimit, final int concurrentThreadsLimit) {
this.semaphore = new Semaphore(concurrentThreadsLimit);
executor.scheduleAtFixedRate(() -> {
final Long now = now();
while (!enterQueue.isEmpty() && now - enterQueue.peek() >= timeLimit) {
enterQueue.poll();
semaphore.release();
}
}, timeLimit, 200, TimeUnit.MILLISECONDS);
}
private Long now() {
return System.currentTimeMillis();
}
public void acquire() {
try {
semaphore.acquire();
} catch (InterruptedException e) {
// todo: handle exception
}
}
public void release() {
semaphore.release();
}
}
The api is quite simple:
Each thread entering the critical section, call bulkheadSemaphore.acqure()
After an external call execution finishes, call bulkheadSemaphore.release()
Why does it solve the problem?
This semaphore releases permits for threads which entered the
critical section long time ago.
It releases it's permits at a certain rate (I set it to 200ms, it can be smaller though). It also guarantees that if a work unit has been done quickly, the next thread will be able to start a new work unit.
Some threads would still face redundant waiting, however it doesn't happen every time and they'd spend 200ms at most.
As requests take time, I'd set timeLimit to 1.5 seconds to match your 1 second limitation.
P.S. Don't forget to shutdown executor service
I am setting up a simulator (for testing) of a server (Radius) which sends queries to another server (LDAP) using threads.
The queries need to be executed on a x per second basis.
I am using a scheduled thread pool executor with callable for this purpose so that I can create callables and submit them to the thread pool for execution.
Each thread should open its own connection and use it to query.
The thing is that I want the connection to be re-used by the same thread every time it is used.
To clarify:
If I have lets say a thread pool of 20 I want 20 connections to be created and used. (so I can send lets say 10.000 queries which will be processed in turn by the 20 threads/connections).
Now the (LDAP) server information to connect to is sent to the constructor of the callable and the callable sets up the connection for execution. Thereafter I retrieve the result using the future system of callable.
The problem with this is each time I create a callable the connection is being opened (and later closed of course).
I am looking for the best practice to keep the connections alive and them being re-used for each thread.
I have thought of some ways to implement this but they dont seem very efficient:
Use a connection pool inside my threadpool to retrieve a free connection when needed (Creates deadlock and other thread safety issues)
Use a static (or so) array with connections and using the thread number to retrieve its connection (Not foul proof either, see link)
What is the most efficient way of implementing this? <- old question, see edit for new question.
EDIT:
I was thinking because I cannot safely get a thread number, but the threadId is always unique, I can just use a
map<String/threadId, connection>
And pass the whole map (reference) to the callable. This way I can use something like: (pseudo code)
Connection con = map.get(this.getThreadId());
If (con == null){
con = new Connection(...);
map.put(this.getThreadId(), con)
}
It would also be possible to make the map static and just access it statically. This way I don't have to pass the map to the Callable.
This would be at least safe and doesnt force me to restructure my code.
New question:
What would be closer aligned with best practices; The above solution or Zim-Zam's solution?
And if the above is best, would it be better to go for the static solution or not?
I would implement this using a BlockingQueue that is shared between Callables, with the ScheduledThreadPoolExecutor putting x queries into the BlockingQueue every second
public class Worker implements Runnable {
private final BlockingQueue<Query> inbox;
private final BlockingQueue<Result> outbox;
public Worker(BlockingQueue<Query> inbox, BlockingQueue<Result> outbox) {
// create LDAP connection
this.inbox = inbox;
this.outbox = outbox;
}
public void run() {
try {
while(true) {
// waits for a Query to be available
Query query = inbox.take();
// execute query
outbox.add(new Result(/* result */));
}
} catch(InterruptedException e) {
// log and restart? close LDAP connection and return?
}
}
}
public class Master {
private final int x; // number of queries per second
private final BlockingQueue<Query> outbox = new ArrayBlockingQueue<>(4 * x);
private final BlockingQueue<Result> inbox = new ArrayBlockingQueue<>(4 * x);
private final ScheduledThreadPoolExecutor executor;
private final List<Future<?>> workers = new ArrayList<>(20);
private final Future<?> receiver;
public Master() {
// initialize executor
for(int i = 0; i < 20; i++) {
Worker worker = new Worker(inbox, outbox);
workers.add(executor.submit(worker));
}
receiver = executor.submit(new Runnable() {
public void run() {
while(!Thread.interrupted()) {
try {
Result result = inbox.take();
// process result
} catch(InterruptedException e) {
return;
}
}
}
}
}
executor.scheduleWithFixedDelay(new Runnable() {
public void run() {
// add x queries to the queue
}
}, 0, 1, TimeUnit.SECONDS);
}
Use BlockingQueue#add to add new Queries to outbox, if this throws an exception then your queue is full and you'll need to reduce the rate of query creation and/or create more workers. To break out of a worker's infinite loop call cancel(true) on its Future, this will throw an InterruptedException inside of the Worker.
I am trying to test the performance (in terms of execution time) for my webcrawler but I am having trouble timing it due to multi-threading taking place.
My main class:
class WebCrawlerTest {
//methods and variables etc
WebCrawlerTest(List<String> websites){
//
}
if(!started){
startTime = System.currentTimeMillis();
executor = Executors.newFixedThreadPool(32); //this is the value I'm tweaking
started=true;
}
for(String site : websites){
executor.submit(webProcessor = new AllWebsiteProcessorTest(site, deepSearch));
}
executor.shutdown();
//tried grabbing end time here with no luck
AllWebsiteProcessorTest class:
class AllWebsiteProcessorTest implements Runnable{
//methods and var etc
AllWebsiteProcessorTest(String site, boolean deepSearch) {
}
public void run() {
scanSingleWebsite(websites);
for(String email:emails){
System.out.print(email + ", ");
}
private void scanSingleWebsite(String website){
try {
String url = website;
Document document = Jsoup.connect(url).get();
grabEmails(document.toString());
}catch (Exception e) {}
With another class (with a main method), I create an instance of WebCrawlerTest and then pass in an array of websites. The crawler works fine but I can't seem to figure out how to time it.
I can get the start time (System.getCurrentTime...();), but the problem is the end time. I've tried adding the end time like this:
//another class
public static void main(.....){
long start = getCurrent....();
WebCrawlerTest w = new WebCrawlerTest(listOfSites, true);
long end = getCurrent....();
}
Which doesn't work. I also tried adding the end after executor.shutdown(), which again doesn't work (instantly triggered). How do I grab the time for the final completed thread?
After shutting down your executors pool
executor.shutdown();
//tried grabbing end time here with no luck
You can simply
executor.awaitTermination(TimeUnit, value)
This call will block untill all tasks are completed. Take the time, subtract T0 from it and voila, we have execution time.
shutdown() method just assures that no new tasks will be accepted into excution queue. Tasks already in the queue will be performed (shutdownNow() drops pending tasks). To wait for all currently running tasks to complete, you have to awaitTermination().
I have a server program which accepts client connections. These client connections can belong to many streams. For example two or more clients can belong to the same stream. Out of these streams one message I have to pass but I have to wait until all the streams are established. For this I maintain the following data structure.
ConcurrentHashMap<Integer, AtomicLong> conhasmap = new ConcurrentHashMap<Integer, AtomicLong>();
Integer is the stream ID and Long is the client number. To make one thread for a given stream to wait till AtomicLong reach a specific value I used the following loop. Actually the first packet of the stream puts it stream ID and the number of connections to wait. With each connection I decrease the connections to wait.
while(conhasmap.get(conectionID) != new AtomicLong(0)){
// Do nothing
}
However this loop blocks the other threads. According to this
answer it does a volatile read. How can I modify the code to wait the correct thread for a given stream until it reaches a specific value?
If you're using Java 8, CompletableFuture could be a good fit. Here's a complete, contrived example which is waiting for 5 clients to connect and send a message to a server (simulated using a BlockingQueue with offer/poll).
In this example, when the expected client connected message count is reached, a CompletableFuture hook is completed, which then runs arbitrary code on any thread of your choice.
In this example, you don't have any complex thread wait/notify setups or busy wait loops.
package so.thread.state;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
public class Main {
public static String CONNETED_MSG = "CONNETED";
public static Long EXPECTED_CONN_COUNT = 5L;
public static ExecutorService executor = Executors.newCachedThreadPool();
public static BlockingQueue<String> queue = new LinkedBlockingQueue<>();
public static AtomicBoolean done = new AtomicBoolean(false);
public static void main(String[] args) throws Exception {
// add a "server" thread
executor.submit(() -> server());
// add 5 "client" threads
for (int i = 0; i < EXPECTED_CONN_COUNT; i++) {
executor.submit(() -> client());
}
// clean shut down
Thread.sleep(TimeUnit.SECONDS.toMillis(1));
done.set(true);
Thread.sleep(TimeUnit.SECONDS.toMillis(1));
executor.shutdown();
executor.awaitTermination(1, TimeUnit.SECONDS);
}
public static void server() {
print("Server started up");
// track # of client connections established
AtomicLong connectionCount = new AtomicLong(0L);
// at startup, create my "hook"
CompletableFuture<Long> hook = new CompletableFuture<>();
hook.thenAcceptAsync(Main::allClientsConnected, executor);
// consume messages
while (!done.get()) {
try {
String msg = queue.poll(5, TimeUnit.MILLISECONDS);
if (null != msg) {
print("Server received client message");
if (CONNETED_MSG.equals(msg)) {
long count = connectionCount.incrementAndGet();
if (count >= EXPECTED_CONN_COUNT) {
hook.complete(count);
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
print("Server shut down");
}
public static void client() {
queue.offer(CONNETED_MSG);
print("Client sent message");
}
public static void allClientsConnected(Long count) {
print("All clients connected, count: " + count);
}
public static void print(String msg) {
System.out.println(String.format("[%s] %s", Thread.currentThread().getName(), msg));
}
}
You get output like this
[pool-1-thread-1] Server started up
[pool-1-thread-5] Client sent message
[pool-1-thread-3] Client sent message
[pool-1-thread-2] Client sent message
[pool-1-thread-6] Client sent message
[pool-1-thread-4] Client sent message
[pool-1-thread-1] Server received client message
[pool-1-thread-1] Server received client message
[pool-1-thread-1] Server received client message
[pool-1-thread-1] Server received client message
[pool-1-thread-1] Server received client message
[pool-1-thread-4] All clients connected, count: 5
[pool-1-thread-1] Server shut down
Your expression:
conhasmap.get(conectionID) != new AtomicLong(0)
will always be true because you are comparing the object references, which will never be equal, instead of the values. The better expression would be:
conhasmap.get(conectionID).longValue() != 0L)
, but looping like this without wait/notify logic within the loop is not a good practice because it uses CPU time constantly. Instead, each thread should call .wait() on the AtomicLong instance, and when it is decremented or incremented, you should call .notifyAll() on the AtomicLong instance to wake up each waiting thread to check the expression. The AtomicLong class may already be calling the notifyAll() method whenever it is modified, but I don't know for sure.
AtomicLong al = conhasmap.get(conectionID);
synchronized(al) {
while(al.longValue() != 0L) {
al.wait(100); //wait up to 100 millis to be notified
}
}
In the code that increments/decrements, it will look like:
AtomicLong al = conhasmap.get(conectionID);
synchronized(al) {
if(al.decrementAndGet() == 0L) {
al.notifyAll();
}
}
I personally would not use an AtomicLong for this counter because you are not benefiting from the lock-less behavior of the AtomicLong. Just use a java.lang.Long instead because you will need to synchronize on the counter object for the wait()/notify() logic anyway.
I just want to implement the following in Java , Do anyone have some idea..?
public String method1(){
//statement1
.
.
.
//statement 5
}
I want to set a timer for the statemen1 ( which involves some network communication ) . If the statement1 is not getting finished even after 25seconds , the control should go to statement 5 . how can I implement this in java ..?
You can make use of the java.util.TimerTask.
extend TimerTask and over-ride the run() method.
What you put in the run method is what should be executed every 25 seconds.
To start the timer do the following:
Timer tmer = new Timer("Network Timer",false);
ExtendedTimerTask extdTT = new ExtendedTimerTask(<params_go_here>)
tmer.schedule(extdTT,25000,25000);
You can parse the object which does the networking part at <params_go_here> and assign to a local variable in your ExtendedTimerTask.
When the timer executes you can do the necassary calls on your <params_go_here> object to see if its finished.
Please note that the checker will run in a seperate thread as java.util.TimerTask implements java.util.Runnable
Cool
You could do something like this:
private volatile Object resultFromNetworkConnection;
public String method1(){
resultFromNetworkConnection = null;
new Thread(){
public void run(){
//statement1
.
.
.
// assign to result if the connection succeeds
}
}.start();
long start = System.currentMilis();
while (System.currentMilis() - start < 25 * 1000) {
if (resultFromNetworkConnection != null) break;
Thread.sleep(100);
}
// If result is not null, you can use it, otherwise, you can ignore it
//statement 5
}
If there is no time-out parameter for the blocking method at statement1, you would have to put statement1 in a separate thread, then wait(25000) for it to finish, if the wait times-out, you go ahead with statement 5 and ignore the result of the blocking call.
I/O operations (including network communication) are synchronous. So you can configure a timeout for the particular network communication, and you will have the desired behaviour. How exactly to configure the timeout - depends on what you are using.
You mention network communication, so I'll give a rough example with an InputStream from a Socket with a timeout set that may apply to other classes. While you could make timer threads, this is simpler.
socket.setSoTimeout(25 * 1000);
try
{
data = readMyData(socket.getInputStream());
doStuff(data);
}
catch(SocketTimeoutException e){ }
doStatement5();
Here's is a pattern that you can use. The idea is to start a separate thread to do the network stuff. The "main" thread will wait for the adequate time and check a shared variable that indicates if the networking stuff did his job on time.
public class TestConstrainNetworkOP {
private Object lock = new Object();
private Object dataAvailable;
private Object constrainedNetworkOp() throws InterruptedException {
Thread t = new Thread(new DoTask());
t.start();
Thread.sleep(25000);
synchronized (lock) {
if (dataAvailable != null) {
//the data arrived on time
}
else{
//data is not available and
//maybe throw a timeoutexception
}
}
}
public class DoTask implements Runnable {
#Override
public void run() {
// do the networking
synchronized (lock) {
// save your data here
dataAvailable = new Long(1);
}
}
}
}
This is a useful pattern if you don't too much control over the network layer (e.g. RMI, EJB). If you are writing the network communication by yourself, then you can set the timeout direct to the socket (as people previously said) or use Java NIO