Alright, I've already asked one question regarding this, but needed a bit more info. I'll try to be coherent with my question as much as I can. (Since I am not sure of the concepts).
Background
I have a java web project(dynamic). I am writing Restful webservices. Below is a snippet from my class
/services
class Services{
static DataSource ds;
static{
ds = createNewDataSource;
}
/serviceFirst
#Consumes(Something)
#produces(Something)
public List<Data> doFirst(){
Connection con = ds.getConnection();
ResultSet res = con.execute(preparedStatement);
//iterate over res, and create list of Data.
return list;
}
}
This is a very basic functionality that I have stated here.
I've got tomcat server where I've deployed this. I've heard that Tomcat has a threadpool, of size 200 (by default). Now my question is that, how exactly does the threadpool work here.
Say I have two requests coming in at the same time. That means that 2 of the threads from the threadpool will get to work. Does this mean that both the threads will have an instance of my class Services? Because below is how I understand the threads and concurrency.
public class myThread extends Thread(){
public void run(){
//do whatever you wan to do here;
}
}
In the above, when I call start on my Thread it will execute the code in run() method and all the objects that it creates in there, will belong to it.
now, coming back to the Tomcat, is there somewhere a run() method written that instantiates the Services class, and that is how the threadpool handles 200 concurrent requests. (Obviously, I understant they will require 200 cores for them to execute concurrently, so ignore that).
Because otherwise, if tomcat does not have 200 different threads having the same path of execution (i.e. my Services class), then how exactly will it handle the 200 concurrent requests.
Thanks
Tomcat's thread pool works, more or less, like what you would get from an ExecutorService (see Executors).
YMMV. Tomcat listens for requests. When it receives a request, it puts this request in a queue. In parallel, it maintains X threads which will continuously attempt to take from this queue. They will prepare the ServletRequest and ServletResponse objects, as well as the FilterChain and appropriate Servlet to invoke.
In pseudocode, this would look like
public void run() {
while (true) {
socket = queue.take();
ServletRequest request = getRequest(socket.getInputStream());
ServletResponse response = generateResponse(socket.getOutputStream());
Servlet servletInstance = determineServletInstance(request);
FilterChain chain = determineFilterChainWithServlet(request, servletInstance);
chain.doFilter(request,response); // down the line invokes the servlet#service method
// do some cleanup, close streams, etc.
}
}
Determining the appropriate Servlet and Filter instances depends on the URL path in the request and the url-mappings you've configured. Tomcat (and every Servlet container) will only ever manage a single instance of a Servlet or Filter for each declared <servlet> or <filter> declaration in your deployment descriptor.
As such, every thread is potentially executing the service(..) method on the same Servlet instance.
That's what the Servlet Specification and the Servlet API, and therefore Tomcat, guarantee.
As for your Restful webservices, read this. It describes how a resource is typically an application scoped singleton, similar to the Servlet instance managed by a Servlet container. That is, every thread is using the same instance to handle requests.
Related
I'm having an issue with a web service with users trying to guess application IDs by looping over random IDs.
The bad requests are coming from random IPs, so I cannot just ban their IP (unless I do it dynamically, but I'm not looking into that yet).
Currently when I detect a client that has made 10 bad app ID attempts I put them on a block list in my app, and reject further requests from that IP for the day.
I want to minimize the amount of work my server needs to do, as the bad client will continue to send 1000s of requests even though they get rejected. I know there are dynamic Firewall solutions, but want something easy to implement in my app for now. Currently I am sleeping for 5 seconds to reduce the calls, but what I want to do is just not send a response to the client, so it has to timeout.
Anyone know how to do this in Java, in JAX-RS?
My service is like,
#Path("/api")
public class MyServer {
#GET
#Consumes(MediaType.APPLICATION_XML)
#Produces(MediaType.APPLICATION_XML)
#Path("/my-request")
public String myRequest(String type,
#Context HttpServletRequest requestContext,
#Context HttpServletResponse response) {
...
}
See:
How to stop hack/DOS attack on web API
You are looking for asynchronous responses which are supported by JAX-RS. The tutorial for Jersey contains some examples of how to implement an asynchronous response to a request.
With asynchronous responses, the thread that is responsible for answering a request is freed for handling another request already before a request is completed. This feature is activated by adding a parameter with the #Suspended annotation. What you would need to do additionally, would be to register a dedicated scheduler that is responsible for waking up your requests after a given time-out like in this example:
#Path("/api")
public class MyServer {
private ScheduledExecutorService scheduler = ...;
#GET
#Consumes(MediaType.APPLICATION_XML)
#Produces(MediaType.APPLICATION_XML)
#Path("/my-request")
public String myRequest(String type,
#Context HttpServletRequest requestContext,
#Context HttpServletResponse response,
#Suspended AsyncResponse asyncResponse) {
scheduler.schedule(new Runnable() {
#Override
public void run() {
asyncResponse.resume(...)
}
}, 5, TimeUnit.SECOND);
}
}
This way, no thread is blocked for the waiting time of five seconds what gives an opportunity for handling other requests in the meantime.
JAX-RS does not offer a way of completely discarding a request without an answer. You will need to keep the connection open to produce a time out, if you terminate the connection, a user is notified of the termination. Best you could do would be to never answer an asynchronous request but this will still consume some ressources. If you wanted to avoid this, you would have to solve the problem outside of JAX-RS, for example by proxying the request by another server.
One way to do that would be to set up mod_proxy where you could answer the proxy with an error code for the mallicious requests and set up a very large retry limit for such requests.
I may suggest move IP deny logic from REST to plain HTTP Filter:
#WebFilter(urlPatterns = "/*", asyncSupported = true)
#WebListener
public class HttpFilter implements Filter {
#Override
public void init(FilterConfig filterConfig) throws ServletException { }
#Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
if(denyIP(servletRequest.getRemoteAddr())) {
AsyncContext asyncContext = servletRequest.startAsync();
asyncContext.setTimeout(100000);
}else{
filterChain.doFilter(servletRequest, servletResponse);
}
}
#Override
public void destroy() { }
private boolean denyIP(String address){
//todo
return true;
}
}
It is cheaper for application server: no XML/JSON deserialization, no call to REST classes. Also you may notice that I never call asyncContext.start. I check Wildfly 8.2 server. In this case Wildfly does not use thread per request. I sent a lot of requests, but amount of threads was constant.
PS
trying to guess application IDs by looping over random IDs
It is not DOS attack. It is brute force attack.
There are many possible solutions, with the restrictions you have given 2 possible solutions come to my mind:
1) Use a forward proxy that already has support for limiting requests. I personally have used Nginx and can recommend it partly because it is simple to set up. Relevant rate limiting config: Limit Req Module
2) Use Asynchronous JAX-RS to let timeout the malicious request that you detected. Sane request can be processed directly. But beware of the consequences, either way such an approach will consume resources on the server!
You can try asyncContext supported from Tomcat 3.0.
This feature decouples web request handler and processer. In your case, the thread which accepts the request has to wait/sleep more than timeout configured. Making the same thread sleep for such a long time would starve them and it would drastically affect the performance of servers. So, asynchronous processing is the right way to go.
I have used asyncContext with Java single thread executor http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
It worked well for me. I had similar business case, where I had to mock my application.
Refer this for implementation http://peter-braun.org/2013/04/asynchronous-processing-from-servlet-3-0-to-jax-rs-2-0/
Single thread executor would not eat resources and its ideal for this use case.
I have not tried this.... just a shot in the dark here, so take it with a grain of salt. So the problem is that once you detect something fishy and put the IP in block-mode you do not want to waste another iota of resources on this request and also cause them to waste time timingout. But your framework will respond if you throw an exception. How about interrupting your current thread? You can do this by Thread.currentThread().interrupt();. The hope is that the Java container processing the request will check the interrupt status. It might not be doing that. I know I have seen IO related classes not process requests because the interrupted flag was set.
EDIT: If interrupting your thread does not work, you can also try throwing an InterruptedException. It might accomplish the desired affect.
As far as I know, in the Servlet specification there is no such mechanism. As JAX-RS implementations use servlets (at least Jersey and Resteasy), you do not have a standard way to achieve that in Java.
The idea of using the aynchronous JAX-RS is better than Thread.sleep(5000), but it will still use some resources and is nothing but a way to process the request later, instead of ignoring the request for always.
I once solved a similar problem by creating a TCP/IP tunnel application.
It's a small application that listens to an external port (e.g. the http port 80). Under normal conditions, all received calls are accepted, creating dedicated socket objects. These individual sockets then call the real (hidden) webserver, which runs on the same physical server, or any server within your local network.
The tunnel itself is like a dispatcher, but can also act like a loadbalancer. It can be multifunctional.
The thing is that you're working low-level with sockets at this point. If you hand a list of banned ip-addresses to this application, then it can shut-out applications on a very low-level (not even accepting their socket calls at all).
You could integrate this in the very same java application as well. But I think it is more flexible to regard this as a seperate application.
(Let me know if you need some source code, I may have some code laying around to get you started)
I am analyzing some jersey 2.0 code and i have a question on how the following method works:
#Stateless
#Path("/mycoolstuff")
public class MyEjbResource {
…
#GET
#Asynchronous //does this mean the method executes on child thread ?
public void longRunningOperation(#Suspended AsyncResponse ar) {
final String result = executeLongRunningOperation();
ar.resume(result);
}
private String executeLongRunningOperation() { … }
}
Lets say im at a web browser and i type in www.mysite/mycoolstuff
this will execute the method but im not understanding what the asyncResponse is used for neither the #Asynchronous annotation. From the browser how would i notice its asychnronous ? what would be the difference in removing the annotation ? Also the suspended annotation after reading the documentation i'm not clear its purpose.
is the #Asynchronous annotation simply telling the program to execute this method on a new thread ? is it a convenience method for doing "new Thread(.....)" ?
Update: this annotation relieves the server of hanging onto the request processing thread. Throughput can be better. Anyway from the official docs:
Request processing on the server works by default in a synchronous processing mode, which means that a client connection of a request is processed in a single I/O container thread. Once the thread processing the request returns to the I/O container, the container can safely assume that the request processing is finished and that the client connection can be safely released including all the resources associated with the connection. This model is typically sufficient for processing of requests for which the processing resource method execution takes a relatively short time. However, in cases where a resource method execution is known to take a long time to compute the result, server-side asynchronous processing model should be used. In this model, the association between a request processing thread and client connection is broken. I/O container that handles incoming request may no longer assume that a client connection can be safely closed when a request processing thread returns. Instead a facility for explicitly suspending, resuming and closing client connections needs to be exposed. Note that the use of server-side asynchronous processing model will not improve the request processing time perceived by the client. It will however increase the throughput of the server, by releasing the initial request processing thread back to the I/O container while the request may still be waiting in a queue for processing or the processing may still be running on another dedicated thread. The released I/O container thread can be used to accept and process new incoming request connections.
#Suspended have more definite if you used it, else it will not make any difference of using it.
Let's talk about benefits of it:
#Suspended will pause/Suspend the current thread until it gets response,by default #NO_TIMEOUT no suspend timeout set. So it doesn't mean your request response (I/O)thread will get free and be available for other request.
Now Assume you want your service to be a response with some specific time, but the method you are calling from resource not guarantee the response time, then how will you manage your service response time? At that time, you can set suspend timeout for your service using #Suspended, and even provide a fall back response when time get exceed.
Below is some sample of code for setting suspend/pause timeout
public void longRunningOperation(#Suspended AsyncResponse ar) {
ar.setTimeoutHandler(customHandler);
ar.setTimeout(10, TimeUnit.SECONDS);
final String result = executeLongRunningOperation();
ar.resume(result);
}
for more details refer this
The #Suspended annotation is added before an AsyncResponse parameter on the resource method to tell the underlying web server not to expect this thread to return a response for the remote caller:
#POST
public void asyncPost(#Suspended final AsyncResponse ar, ... <args>) {
someAsyncMethodInYourServer(<args>, new AsyncMethodCallback() {
#Override
void completed(<results>) {
ar.complete(Response.ok(<results>).build());
}
#Override
void failed(Throwable t) {
ar.failed(t);
}
}
}
Rather, the AsyncResponse object is used by the thread that calls completed or failed on the callback object to return an 'ok' or throw an error to the client.
Consider using such asynchronous resources in conjunction with an async jersey client. If you're trying to implement a ReST service that exposes a fundamentally async api, these patterns allow you to project the async api through the ReST interface.
We don't create async interfaces because we have a process that takes a long time (minutes or hours) to run, but rather because we don't want our threads to ever sleep - we send the request and register a callback handler to be called later when the result is ready - from milliseconds to seconds later - in a synchronous interface, the calling thread would be sleeping during that time, rather than doing something useful. One of the fastest web servers ever written is single threaded and completely asynchronous. That thread never sleeps, and because there is only one thread, there's no context switching going on under the covers (at least within that process).
The #suspend annotation makes the caller actually wait until your done work. Lets say you have a lot of work to do on another thread. when you use jersey #suspend the caller just sits there and waits (so on a web browser they just see a spinner) until your AsyncResponse object returns data to it.
Imagine you had a really long operation you had to do and you want to do it on another thread (or multiple threads). Now we can have the user wait until we are done. Don't forget in jersey you'll need to add the " true" right in the jersey servlet definition in web.xml to get it to work.
Servlet 3.0 allows the 'request' thread (or 'main' thread) to delegate long-running processing to some other thread, so as to free itself to receive more requests. Agreed.
That is, we are achieving scalability (of requests) by utilizing multi-threading.
But this requires that my 'Servlet container JVM' is capable of such processing.
What if I have a multi-tiered architecture where the 'Servlet container JVM' is only the entry point, while the logic for servicing requests lies somewhere else in some other JVM (henceforth called as 'Service JVM' in this post).
What if I want to post the incoming 'request' (or atleast the relevant attributes of the request) to a JMS queue and let the 'request' be grabbed and processed by one out of a pool of 'Service JVMs' ? Wouldnt it be better to delegate the responsibility of sending the 'response' (say as JSON) also to this Service JVM ?
I dont think 'AsyncContext' can be passed meaningfully outside of the Servlet container JVM. So, how to really delegate request-processing and response-sending, to be done by distributed services (JVMs) ?
In terms of code/pseudo-code, my question is:
#WebServlet(urlPatterns = "/AsyncServlet", asyncSupported=true)
public class AsyncServlet extends HttpServlet {
protected void doGet(HttpServletRequest request,
HttpServletResponse response)
throws ServletException, IOException {
AsyncContext asyncCtx = request.startAsync();
// Put the asyncCtx in a JMS queue so as to be picked by another
// service JVM that can really service this request.
// return back to receiving requests and dont worry abt sending a response
// The service JVM will take care of sending the appropriate response
// as it has the data necessary for the response.
}
}
One option seems to be to have Worker threads (in the Servlet container JVM) wait for responses from the Service JVMs. After a Service JVM does the actual processing, it can communicate the results (thru messages or otherwise) to the respective Worker thread, and have the Worker thread send the GET response.
I want to know if there is (am sure there shd be !) a better alternative than this, as this seems so convoluted !
Set context as async
Store context inside singleton bean
Send a jms request
Process jms request
Send jms reply
Get context for reply from the singleton bean
Send reply to client
You might want to set a timer for cleanup and you can replace jms with async one-way ejb calls
On Google App Engine (GAE) it is possible for frontend instances to create up to 10 threads to maximize throughput. According to this page, such multi-threading can be accomplished as follows:
Runnable myTask = new Runnable({
#Override
public void run() {
// Do whatever
}
});
ThreadFactory threadFactory = ThreadManager.currentRequestThreadFactory();
// GAE caps frontend instances to 10 worker threads per instance.
threadFactory.newRequestThread(myTask);
To hit my GAE server-side, I'll expose many servlets mapped to certain URLs, such as the FizzServlet mapped to http://myapp.com/fizz:
public class FizzServlet extends HttpServlet {
#Override
public void doGet(HttpServletRequest request,
HttpServletResponse response) throws IOException {
// Handle the request here. Somehow send it to an available
// worker thread.
}
}
I guess I'm choking on how to connect these two ideas. As far as I see it, you have 3 different mechanisms/items here:
The App Engine instance itself, whose lifecycle I can "hook" by implementing a ServletContextListener and run custom code when GAE fires up the instance; and
This ThreadFactory/ThreadManager stuff (above)
The servlets/listeners
I guess I'm wondering how to implement code such that every time a new request comes into, say, FizzServlet#doGet, how to make sure that request gets sent to an available thread (if there is one available). That way, if FizzServlet was the only servlet I was exposing, it could get called up to 10 times before it would cause a new (11th) incoming request to hang while a previous request was processing.
I'm looking for the glue code between the servlet and this thread-creating code. Thanks in advance.
I guess I'm wondering how to implement code such that every time a new request comes into, say, FizzServlet#doGet, how to make sure that request gets sent to an available thread (if there is one available). That way, if FizzServlet was the only servlet I was exposing, it could get called up to 10 times before it would cause a new (11th) incoming request to hang while a previous request was processing.
That's what the GAE servlet engine does for you. You deploy an app containing a servlet, and when a request comes in, the servlet engine uses a thread to process the request and calls your servlet. You don't have anything to do.
If your servlet's doGet() or doPost() method, invoked by GAE, needs to perform several tasks in parallel (like contacting several other web sites for example), then you'll start threads by yourself as explained in the page you linked to.
In simple description, I have a servlet and response time is long so I decided to divide it into two parts, one just composes a response to client, and second let's say performs some business logic and stores result in DB. To decrease response time I execute business logic asynchronously using ThreadPoolExecutor in combination with ArrayBlockingQueue. Using ArrayBlockingQueue I can ensure original FIFO ordering if requests were sequential for the same client. This is important prerequisite.
Here is a snippet:
Servlet
public class HelloServlet extends HttpServlet {
AsyncExecutor exe = new AsyncExecutor();
protected void doGet(HttpServletRequest req,
HttpServletResponse resp) throws ServletException, IOException {
PrintWriter w = resp.getWriter();
exe.executeAsync(exe.new Task(this));
w.print("HELLO TO CLIENT");
}
protected void someBusinessMethod(){
// long time execution here
}
}
and Executor
public class AsyncExecutor {
static final BlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(10, true);
static final Executor executor = new ThreadPoolExecutor(3, 5, 20L, TimeUnit.SECONDS, queue);
public void executeAsync(Task t){
boolean isTaskAccepted = false;
while(!isTaskAccepted){
try {
executor.execute(t);
isTaskAccepted = true;
} catch (RejectedExecutionException e){
}
}
}
class Task implements Runnable{
private HelloServlet servlet;
Task(HelloServlet servlet){
this.servlet = servlet;
}
#Override
public void run() {
// just call back to servlet's business method
servlet.someBusinessMethod();
}
}
}
This implementation works fine if I deploy it only to one Tomcat node, since I have only one ArrayBlockingQueue in application. But if I have several nodes and load balancer in front then I can not guarantee FIFO ordering of requests for async execution for the same client since I have already several Queues.
My question is, how it is possible to guarantee the same order of requests to be executed asynchronously for the same client in clustered (multi node) deployment? I think ActiveMQ probably a solution (not preferable for me), or load balancer configuration, or can it be implemented in code?
Hope some of these ideas help.
Thanks Sam for you prompt suggestions.
In the first post I described a problem in very simplified way so to clarify it better let's say I have a legacy web app deployed to Tomcat and it serves some Licensing Model(old one). Then we got a new Licensing Model (this is a GlassFish app) and we need to use it alongside with old one to be in sync. For the end user such integration must be transparent and not intrusive. So user request is served like this.
caller send a request (create subscription for example)
execute business logic of the the new licensing model
execute business logic of the the old licensing model
despite the result of the p.3 return response of p.2 in format of old licensing model back to caller
(optional) handle failure of p.3 if any
This was implemented with Aspect which intercepts requests of p.1 and executes the rest of stuff sequentially. And as I said in previous post p.3 execution time can be long that's why I want to make it asynchronous. Let's have a look at snippet of Aspect (instead of Servlet from the first post).
#Aspect #Component public class MyAspect {
#Autowired
private ApplicationContext ctx;
#Autowired
private AsyncExecutor asyncExecutor;
#Around("#annotation(executeOpi)")
public Object around(ProceedingJoinPoint jp, ExecuteOpi executeOpi) throws Throwable {
LegacyMapper newModelExecutor = ctx.getBean(executeOpi.legacyMapper());
// executes a new model and then return result in the format of old model
Object result = newModelExecutor.executeNewModelLogic(joinPoint.getArgs());
// executes old model logic asynchronously
asyncExecutor.executeAsync(asyncExecutor.new Task(this, jp)
return object
}
public void executeOldModelLogic(ProceedingJoinPoint jp) throws Throwable{
// long time execution here
jp.proceed();
}
}
With this implementation as in the first post, I can guarantee a FIFO order of executeOldModelLogic methods, if requests come to the same Tomcat node. But with multi-node deployment and round robin LB in front I can end-up with such a case when for the same caller "update subscription in old model" can come first to ArrayBlockingQueue than "create subscription in old model", which of course a bad logical bug.
And as for points you suggested:
p1, p2 and p4: I probably can't use it as a solution since I don't have a state of object as such. You see that I pass to Runnable task a references of Aspect and JoinPoint to make a call back of executeOldModelLogic from Runnable to Aspect
p3 Don't know about this might be worthwhile to investigate
p5 This is a direction I want go for further investigation, I have a gut feeling it is only way to solve my problem in the given conditions.
There are some solutions that come to mind off hand.
Use the database: post the jobs to be run in a database table, have a secondary server process run the jobs, and place results in an output table. Then when users call back to the web page, it can pick up any results waiting for them from the output table.
Use a JMS service: this is a pretty lightweight messaging service, which would integrate with your Tomcat application reasonably well. The downside here is that you have to run another server side component, and build the integration layer with your app. But that's not a big disadvantage.
Switch to a full J2EE container (Java App Server): and use an EJB Singleton. I have to admit, I don't have any experience with running a Singleton across separate server instances, but I believe that some of them may be able to handle it.
Use EHCache or some other distributed cache: I built a Queue wrapper around EHCache to enable it to be used like a FIFO queue, and it also has RMI (or JMS) replication, so multiple nodes will see the same data.
Use the Load Balancer: If your load balancer supports session level balancing, then all requests for a single user session can be directed to the same node. In a big web environment where I worked we were unable to share user session state across multiple servers, so we set up load balancing to ensure that the user's session always was directed to same web server.
Hope some of these ideas help.