In Java EE7, the JAX-RS Client API provides a high-level API for accessing any REST resources. According to the documentation, "Clients are heavy-weight objects that manage the client-side communication infrastructure. Initialization as well as disposal of a Client instance may be a rather expensive operation. It is therefore advised to construct only a small number of Client instances in the application. "
In order to avoid create client frequently, I am going to cache the client instance and reuse it. Is the client instance thread safe since it can be used by concurrent threads?
Is there any performance issue if I only create a instance of the client and reuse it for all the requests?
I am not sure but I think this is a implementation-specific decision.
I couldn't find in the JAX-RS 2.0 specification nor in the Javadoc anything granting that javax.ws.rs.client.Client is thread-safe. But in the Resteasy (an implementor of JAX-RS) documentation I found:
One default decision made by HttpClient and adopted by Resteasy is the
use of org.apache.http.impl.conn.SingleClientConnManager, which
manages a single socket at any given time and which supports the use
case in which one or more invocations are made serially from a single
thread. For multithreaded applications, SingleClientConnManager may be
replaced by
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager:
ClientConnectionManager cm = new ThreadSafeClientConnManager();
HttpClient httpClient = new DefaultHttpClient(cm);
ApacheHttpClient4Engine engine = new ApacheHttpClient4Engine(httpClient);
Source: http://docs.jboss.org/resteasy/docs/3.0.9.Final/userguide/html/RESTEasy_Client_Framework.html#transport_layer
Based in these information I guess that the answer for your question is likely to be "no".
PLEASE BE AWARE: Although this is the accepted answer, this is implementation specific and was correct for the Jersey 1 Client. For that you absolutely should share a single instance. Creating a client per request is a huge performance overhead
The JavaDoc is mostly answering your question already- yes it's thread-safe and you can and should reuse it. There can be a performance issue from not reusing it, i.e. if you create a Client for every HTTP request you make your performance will suck really bad.
Related
I am trying to integrate QFJ into a single-threaded application. At first I was trying to utilize QFJ with my own TCP layer, but I haven't been able to work that out. Now I am just trying to integrate an initiator. Based on my research into QFJ, I would think the overall design should be as follows:
The application will no longer be single-threaded, since the QFJ initiator will create threads, so some synchronization is needed.
Here I am using an SocketInitiator (I only handle a single FIX session), but I would expect a similar setup should I go for the threaded version later on.
There are 2 aspects to the integration of the initiator into my application:
Receiving side (fromApp callback): I believe this is straightforward, I simply push messages to a thread-safe queue consumed by my MainProcessThread.
Sending side: I'm struggling to find documentation on this front. How should I handle synchronization? Is it safe to call Session.sendToTarget() from the MainProcessThread? Or is there some synchronization I need to put in place?
As Michael already said, it is perfectly safe to call Session.sendToTarget() from multiple threads, even concurrently. But as far as I see it you only utilize one thread anyway (MainProcessThread).
The relevant part of the Session class is in method sendRaw():
private boolean sendRaw(Message message, int num) {
// sequence number must be locked until application
// callback returns since it may be effectively rolled
// back if the callback fails.
state.lockSenderMsgSeqNum();
try {
.... some logic here
} finally {
state.unlockSenderMsgSeqNum();
}
Other points:
Here I am using an SocketInitiator (I only handle a single FIX session), but I would expect a similar setup should I go for the threaded version later on.
Will you always use only one Session? If yes, then there is no use in utilizing the ThreadedSocketInitiator since all it does is creating a thread per Session.
The application will no longer be single threaded, since the QFJ initiator will create threads
As already stated here Use own TCP layer implementation with QuickFIX/J you could try passing an ExecutorFactory. But this might not be applicable to your specific use case.
I was wondering getPort call creates a new port or just serves the same one all the time, thus making it not thread-safe?
the thread safety nature of the JAX-WS seems to be depend on the underlying implementation. see this question. Are you trying to reduce the GC overhead or reusing the Service object?
My Servler spends quite some time in reading request.getInputStream() and writing to response.getOutputStream(). In the long run, this can be a problem as its blocking a thread for nothing but reading/writing literally a few bytes per second. (*)
I'm never interested in a partial request data, the processing should not start before the request is completely available. Similarly for the response.
I guess, asynchronous IO would solve it, but I wonder what's the proper way. Maybe a servlet Filter replacing the ServletInputStream by a wrapped ByteArrayInputStream, using request.startAsync and calling the chained servlet after having collected the whole input?
Is there already such a filter?
Should I write one or should I use a different approach?
Note that what I mean is to avoid wasting threads on slow servlet streams. This isn't the same as startAsync which avoids wasting threads just waiting for some event.
And yes, at the moment it'd be a premature optimization.
My read loop as requested
There's nothing interesting in my current input stream reading method, but here you are:
private byte[] getInputBytes() throws IOException {
ServletInputStream inputStream = request.getInputStream();
final int len = request.getContentLength();
if (len >= 0) {
final byte[] result = new byte[len];
ByteStreams.readFully(inputStream, result);
return result;
} else {
return ByteStreams.toByteArray(inputStream);
}
}
That's all and it blocks when data aren't available; ByteStreams come from Guava.
Summary of my understanding so far
As the answers clearly state, it's impossible to work with servlet streams without wasting a thread on them. Neither the servlet architecture nor the common implementation expose anything allowing to say "buffer the whole data and call me only when you collected everything", albeit they use NIO and could do it.
The reason may be that usually a reverse proxy like nginx gets used, which can do it. nginx does this buffering by default and it couldn't be even switched off until two years ago.
Actually a supported case???
Given that many negative answer, I'm not sure, but it looks like my goal
to avoid wasting threads on slow servlet streams
is actually fully supported: Since 3.1, there's ServletInputStream.html#setReadListener which seems to be meant exactly for this. The thread allocated for processing Servlet#Service initially calls request.startAsync(), attaches the listener and gets returned to the pool by simply returning from service. The listener implements onDataAvailable(), which gets called when it's possible to read without blocking, adds a piece of data and returns. In onAllDataRead(), I can do the whole processing of the collected data.
There's an example, how it can be done with Jetty. It seems to cover non-blocking output as well.
(*) In the logfiles, I can see requests taking up to eight seconds which get spend on reading the input (100 bytes header + 100 bytes data). Such cases are rare, but they do happen, although the server is mostly idle. So I guess, it's a mobile client on a very bad connection (some users of ours connect from places having such bad connectivity).
HttpServletRequest#startAsync() isn't useful for this. That's only useful for push things like web sockets and the good 'ol SSE. Moreover, JSR356 Web Socket API is built on top of it.
Your concrete problem is understood, but this definitely can't be solved from the servlet on. You'd only end up wasting yet more threads for the very simple reason because the container has already dedicated the current thread to the servlet request until the request body is read fully up to the last bit, even if it's ultimately read by a newly spawned async thread.
To save threads, you actually need a servletcontainer which supports NIO and if necessary turn on that feature. With NIO, a single thread can handle as many TCP connections as the available heap memory allows it, instead of that a single thread is allocated per TCP connection. Then, in your servlet you don't at all need to worry about this delicate I/O task.
Almost all modern servletcontainers support it: Undertow (WildFly), Grizzly (GlassFish/Payara), Tomcat, Jetty, etc. Some have it by default enabled, others require extra configuration. Just refer their documentation using the keyword "NIO".
If you'd actually also want to save the servlet request thread itself, then you'd basically need to go a step back, drop servlets and implement a custom NIO based service on top of an existing NIO connector (Undertow, Grizzly, Jetty, etc).
You can't. The Servlet container allocates the thread to the request, and that's the end of it, it's allocated. That's the model. If you don't like that, you will have to stop using Servlets.
Even if you could solve (1), you can't start async I/O on an input stream.
The way to handle slow requests is to time them out, by setting the appropriate setting for whatever container you're using ... if you actually have a problem, and it's far from clear that you really do, with a mostly idle server and this only happening rarely.
Your read loop makes a distinction without a difference. Just read the request input stream to its end. The servlet container already ensures that end of stream happens at the content-length if provided.
There's a class called org.apache.catalina.connector.CoyoteAdapter, which is the class that receives the marshaled request from TCP worker thread. It has a method called "service" which does the bulk of the heavy lifting. This method is called by another class: org.apache.coyote.http11.Http11Processor which also has a method of the same name.
I find it interesting that I see so many hooks in the code to handle async io, which makes me wonder if this is not a built in feature of the container already? Anyway, with my limited knowledge, the best way that I can think of to implement the feature you are talking about, would be to create a class:
public class MyAsyncReqHandlingAdapter extends CoyoteAdapter and #Override service() method and roll your own... I don't have the time to devote to doing this now, but I may revisit in the future.
In this method you would need a way to identify slow requests and handle them, by handing them off to a single threaded nio processor and "complete" the request at that level, which, given the source code:
https://github.com/apache/tomcat/blob/075920d486ca37e0286586a9f017b4159ac63d65/java/org/apache/coyote/http11/Http11Processor.java
https://github.com/apache/tomcat/blob/3361b1321201431e65d59d168254cff4f8f8dc55/java/org/apache/catalina/connector/CoyoteAdapter.java
You should be able to figure out how to do. Interesting question and yes it can be done. Nothing I see in the spec says that it cannot...
There is this note in the akka-stream docs stating as follows:
… a reusable flow description cannot be bound to “live” resources, any connection to or allocation of such resources must be deferred until materialization time. Examples of “live” resources are already existing TCP connections, a multicast Publisher, etc.; …
I have several questions concerning the note:
Apart from the these two examples, what other resource counts as a live?
Anything that cannot be safely (deep)copied? Like a Thread?
Should I also avoid sharing anything that's not thread-safe?
What about an ActorRef existing in the ActorSystem used by the ActorFlowMaterializer?
How to defer allocation until materialization time? Is it safe for example to allocate it in the constructor of a PushPullStage but not in the create function of a FlowGraph?
The problem here is a common problem if we consider webservices, RMI connections or any other communication protocol. It's always recommended sharing "primitive" values then references, because marshalling/unmarshalling or serializing/unserializing is always a headache. Also think of different types of environments communicating each other. Sharing solid values is a safe way to solve communication.
Akka by itself is a good example of "microservices" communicating actors each other. When I read the documentation of Akka, one good word defines Akka actors very well. Actors are like mailbox clients and you can think of each client has a mailbox. When you pass a variable, it's just like you got a new email.
Short result of long story, be avoid sharing "dependent" objects that can be invalidated before it's read from another actor. Additionally, if your system names actorRefs dynamically, avoid calling them by its reference.
"Materializing" is explained in docs of akka-streams.
The process of materialization may be parameterized, e.g. instantiating a blueprint for handling a TCP connection’s data with specific information about the connection’s address and port information. Additionally, materialization will often create specific objects that are useful to interact with the processing engine once it is running, for example for shutting it down or for extracting metrics. This means that the materialization function takes a set of parameters from the outside and it produces a set of results. Compositionality demands that these two sets cannot interact, because that would establish a covert channel by which different pieces could communicate, leading to problems of initialization order and inscrutable runtime failures.
So use parameters instead of passing "connection" itself.
Deferring a live resource is not a big think. That means if you use one connection for all system, you should keep it alive always. Or when you create a transaction in actor-1 and send it to actor-2, you shouldn't terminate the transaction in actor-1 until actor-2 finished its job with transaction.
Then how you can understand ? Then you use "Future" and "offer()".
Hope I understand your question and hope I can express myself.
I have a web-app where when the user submits a request, we send a JMS message to a remote service and then wait for the reply. (There are also async requests, and we have various niceties set up for message replay, etc, so we'd prefer to stick with JMS instead of, say, HTTP)
In How should I implement request response with JMS?, ActiveMQ seems to discourage the idea of either temporary queues per request or temporary consumers with selectors on the JMSCorrelationID, due to the overhead involved in spinning them up.
However, if I use pooled consumers for the replies, how do I dispatch from the reply consumer back to the original requesting thread?
I could certainly write my own thread-safe callback-registration/dispatch, but I hate writing code I suspect has has already been written by someone who knows better than I do.
That ActiveMQ page recommends Lingo, which hasn't been updated since 2006, and Camel Spring Remoting, which has been hellbanned by my team for its many gotcha bugs.
Is there a better solution, in the form of a library implementing this pattern, or in the form of a different pattern for simulating synchronous request-reply over JMS?
Related SO question:
Is it a good practice to use JMS Temporary Queue for synchronous use?, which suggests that spinning up a consumer with a selector on the JMSCorrelationID is actually low-overhead, which contradicts what the ActiveMQ documentation says. Who's right?
In a past project we had a similar situation, where a sync WS request was handled with a pair of Async req/res JMS Messages. We were using the Jboss JMS impl at that time and temporary destinations where a big overhead.
We ended up writing a thread-safe dispatcher, leaving the WS waiting until the JMS response came in. We used the CorrelationID to map the response back to the request.
That solution was all home grown, but I've come across a nice blocking map impl that solves the problem of matching a response to a request.
BlockingMap
If your solution is clustered, you need to take care that response messages are dispatched to the right node in the cluster. I don't know ActiveMQ, but I remember JBoss messaging to have some glitches under the hood for their clusterable destinations.
I would still think about using Camel and let it handle the threading, perhaps without spring-remoting but just raw ProducerTemplates.
Camel has some nice documentation about the topic and works very well with ActiveMQ.
http://camel.apache.org/jms#JMS-RequestreplyoverJMS
For your question about spinning up a selector based consumer and the overhead, what the ActiveMQ docs actually states is that it requires a roundtrip to the ActiveMQ broker, which might be on the other side of the globe or on a high delay network. The overhead in this case is the TCP/IP round trip time to the AMQ broker. I would consider this as an option. Have used it muliple times with success.
A colleague suggested a potential solution-- one response queue/consumer per webapp thread, and we can set the return-address to the response queue owned by that particular thread. Since these threads are typically long-lived (and are re-used for subsequent web requests), we only have to suffer the overhead at the time the thread is spawned by the pool.
That said, this whole exercise is making me rethink JMS vs HTTP... :)
I have always used CorrelationID for request / response and never suffered any performance issues. I can't imagine why that would be a performance issue at all, it should be super fast for any messaging system to implement and quite an important feature to implement well.
http://www.eaipatterns.com/RequestReplyJmsExample.html has the tow main stream solutions using replyToQueue or correlationID.
It's an old one, but I've landed here searching for something else and actually do have some insights (hopefully will be helpful to someone).
We have implemented very similar use-case with Hazelcast being our chassis for
cluster's internode comminication. The essense is 2 datasets: 1 distributed map for responses, 1 'local' list of response awaiters (on each node in cluster).
each request (receiving it's own thread from Jetty) creates an entry in the map of local awaiters; the entry has obviously the correlation UID and an object that will serve as a semaphore
then the request is being dispatched to the remote (REST/JMS) and the original thread starts waiting on the semaphore; UID must be part of the request
remote returns the response and writes it into the responses map with the correlated UID
responses map is being listened; if the UID of the newly coming response is found in the map of the local awaiters, it's semaphore is being notified, original request's thread is being released, picking up the response from the responses map and returning it to the client
This is a general description, I can update an answer with a few optimizations we have, in case there will be any interest.