I'm having an issue with a web service with users trying to guess application IDs by looping over random IDs.
The bad requests are coming from random IPs, so I cannot just ban their IP (unless I do it dynamically, but I'm not looking into that yet).
Currently when I detect a client that has made 10 bad app ID attempts I put them on a block list in my app, and reject further requests from that IP for the day.
I want to minimize the amount of work my server needs to do, as the bad client will continue to send 1000s of requests even though they get rejected. I know there are dynamic Firewall solutions, but want something easy to implement in my app for now. Currently I am sleeping for 5 seconds to reduce the calls, but what I want to do is just not send a response to the client, so it has to timeout.
Anyone know how to do this in Java, in JAX-RS?
My service is like,
#Path("/api")
public class MyServer {
#GET
#Consumes(MediaType.APPLICATION_XML)
#Produces(MediaType.APPLICATION_XML)
#Path("/my-request")
public String myRequest(String type,
#Context HttpServletRequest requestContext,
#Context HttpServletResponse response) {
...
}
See:
How to stop hack/DOS attack on web API
You are looking for asynchronous responses which are supported by JAX-RS. The tutorial for Jersey contains some examples of how to implement an asynchronous response to a request.
With asynchronous responses, the thread that is responsible for answering a request is freed for handling another request already before a request is completed. This feature is activated by adding a parameter with the #Suspended annotation. What you would need to do additionally, would be to register a dedicated scheduler that is responsible for waking up your requests after a given time-out like in this example:
#Path("/api")
public class MyServer {
private ScheduledExecutorService scheduler = ...;
#GET
#Consumes(MediaType.APPLICATION_XML)
#Produces(MediaType.APPLICATION_XML)
#Path("/my-request")
public String myRequest(String type,
#Context HttpServletRequest requestContext,
#Context HttpServletResponse response,
#Suspended AsyncResponse asyncResponse) {
scheduler.schedule(new Runnable() {
#Override
public void run() {
asyncResponse.resume(...)
}
}, 5, TimeUnit.SECOND);
}
}
This way, no thread is blocked for the waiting time of five seconds what gives an opportunity for handling other requests in the meantime.
JAX-RS does not offer a way of completely discarding a request without an answer. You will need to keep the connection open to produce a time out, if you terminate the connection, a user is notified of the termination. Best you could do would be to never answer an asynchronous request but this will still consume some ressources. If you wanted to avoid this, you would have to solve the problem outside of JAX-RS, for example by proxying the request by another server.
One way to do that would be to set up mod_proxy where you could answer the proxy with an error code for the mallicious requests and set up a very large retry limit for such requests.
I may suggest move IP deny logic from REST to plain HTTP Filter:
#WebFilter(urlPatterns = "/*", asyncSupported = true)
#WebListener
public class HttpFilter implements Filter {
#Override
public void init(FilterConfig filterConfig) throws ServletException { }
#Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
if(denyIP(servletRequest.getRemoteAddr())) {
AsyncContext asyncContext = servletRequest.startAsync();
asyncContext.setTimeout(100000);
}else{
filterChain.doFilter(servletRequest, servletResponse);
}
}
#Override
public void destroy() { }
private boolean denyIP(String address){
//todo
return true;
}
}
It is cheaper for application server: no XML/JSON deserialization, no call to REST classes. Also you may notice that I never call asyncContext.start. I check Wildfly 8.2 server. In this case Wildfly does not use thread per request. I sent a lot of requests, but amount of threads was constant.
PS
trying to guess application IDs by looping over random IDs
It is not DOS attack. It is brute force attack.
There are many possible solutions, with the restrictions you have given 2 possible solutions come to my mind:
1) Use a forward proxy that already has support for limiting requests. I personally have used Nginx and can recommend it partly because it is simple to set up. Relevant rate limiting config: Limit Req Module
2) Use Asynchronous JAX-RS to let timeout the malicious request that you detected. Sane request can be processed directly. But beware of the consequences, either way such an approach will consume resources on the server!
You can try asyncContext supported from Tomcat 3.0.
This feature decouples web request handler and processer. In your case, the thread which accepts the request has to wait/sleep more than timeout configured. Making the same thread sleep for such a long time would starve them and it would drastically affect the performance of servers. So, asynchronous processing is the right way to go.
I have used asyncContext with Java single thread executor http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
It worked well for me. I had similar business case, where I had to mock my application.
Refer this for implementation http://peter-braun.org/2013/04/asynchronous-processing-from-servlet-3-0-to-jax-rs-2-0/
Single thread executor would not eat resources and its ideal for this use case.
I have not tried this.... just a shot in the dark here, so take it with a grain of salt. So the problem is that once you detect something fishy and put the IP in block-mode you do not want to waste another iota of resources on this request and also cause them to waste time timingout. But your framework will respond if you throw an exception. How about interrupting your current thread? You can do this by Thread.currentThread().interrupt();. The hope is that the Java container processing the request will check the interrupt status. It might not be doing that. I know I have seen IO related classes not process requests because the interrupted flag was set.
EDIT: If interrupting your thread does not work, you can also try throwing an InterruptedException. It might accomplish the desired affect.
As far as I know, in the Servlet specification there is no such mechanism. As JAX-RS implementations use servlets (at least Jersey and Resteasy), you do not have a standard way to achieve that in Java.
The idea of using the aynchronous JAX-RS is better than Thread.sleep(5000), but it will still use some resources and is nothing but a way to process the request later, instead of ignoring the request for always.
I once solved a similar problem by creating a TCP/IP tunnel application.
It's a small application that listens to an external port (e.g. the http port 80). Under normal conditions, all received calls are accepted, creating dedicated socket objects. These individual sockets then call the real (hidden) webserver, which runs on the same physical server, or any server within your local network.
The tunnel itself is like a dispatcher, but can also act like a loadbalancer. It can be multifunctional.
The thing is that you're working low-level with sockets at this point. If you hand a list of banned ip-addresses to this application, then it can shut-out applications on a very low-level (not even accepting their socket calls at all).
You could integrate this in the very same java application as well. But I think it is more flexible to regard this as a seperate application.
(Let me know if you need some source code, I may have some code laying around to get you started)
I have exposed some rest api using Jersey 2(Tomcat server) and successfully implemented Basic authentication(only needed authentication stuff not authorization) using ContainerRequestFilter filter as below
public class AuthFilter implements ContainerRequestFilter{
#Context
HttpServletRequest request;
#Override
public void filter(ContainerRequestContext context) {
............................
//getting username/password authorization header and validating
When I told the same to my Lead, he said don't use filters as every time your rest api is hit, this filter will get invoked.Therefore, implement basic authentication security at container level.I am using Tomcat server.
In web.xml, this is defined
<servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
Is the above he is referring to?
Can anyone please guide me how to implement the way my lead is saying?
The documentation gives you examples on how to configure this via web.xml. You'll need to configure this using a login-config that belongs to a realm. The web container then takes care of securing resources based on URL patterns.
Note that the data is sent in plain text (in encoded form) via a HTTP header, so you'll need to think of ways to ensure that is not snooped on (like HTTPS).
Whether you check this header on a filter or on the container does not relieve you of the overhead required for making the check (which is probably negligible, but I've never profiled this area of the code to quote numbers).
We have 2 different web applications that are running on the same tomcat.
webapp#1, webapp#2.
Webapp#1 is connecting to webapp#2 via this service method:
this.restTemplate.postForObject(url,
request,
responseType);
webapp#2 is receiving this request in the following controller:
#RequestMapping(value = "/bla", method = RequestMethod.POST)
#ResponseStatus(value = HttpStatus.OK)
#ResponseBody
public ResponseDTO requestSomething(#RequestBody RequestDTO requestDTO, HttpServletRequest request) {
return new ResponseDTO("Hello");
}
Now, we have a security requirement that this controller we are revealing in webapp#2 will only recieve requests from webapp#1.
What are our alternatives in achieving that? Do we have to create a new session in webapp#2 from webapp#1? if so where do the credentials comes from ? should we agree on something predefined?
Does spring security have any way to solve this?
Thanks!
I'm happy if someone can tell me if there is a special (and good) way for webapps in the same container, but AFAIK these are the options:
Option 1: Disregard that they are in the same Tomcat
In other words, do it like the two webapps were on two different locations. For example, you can use HTTP Basic authentication; it is rather easy do a ClientHttpRequestFactory implementation that adds the Authorization header for RestTemplate on the client side (webapp#1) and Spring Security has built-in support for handling it on the server side (webapp#2). With Basic authentication, communication can be stateless and no session is required. Only disadvantage is that both web apps need to know the credentials.
Option 2: Check for localhost
The idea is that in webapp#2 check where the request comes from. Compare ServletRequest.getRemoteAddr() with 127.0.0.1 (or any other loopback address). If you want to apply Spring Security, you need create a custom filter in the security chain. Advantage: webapp#1 does not need any credentials. Disadvantage: Depending on your server setup, this can be unsafe! If a user can open connections on the machine, it can pretend to be webapp#1. Be extra careful if there is a proxy of some kind on the machine.
I've an interceptor class where in postHandle method, I need to capture all the event request for my application for Audit. I want to implement the gateway pattern between my service and repository. I don't know how to implement the gateway layer which can convert my request object of HttpServletRequest to my domain object and pass it to repository layer. Below is my code:
#Override
public void postHandle(HttpServletRequest request,
HttpServletResponse response,
Object handler,
ModelAndView modelAndView) throws Exception {
String requestURI = request.getRequestURI();
auditService.save((Audit)reuest);
}
How can I send the request object to service and then to repository through gateway?
(I'm speaking from a Spring Integration stand-point, but this should apply).
The gateway pattern is used to put data into the system from a service. This "service" could be a simple java class or could be tied to a connector in same way to be input from an outside source. The gateway defines what "channel" is used and essentially what data is on that channel (though that need not be explicit).
To create a gateway, you would define what data exists on the gateway and how that will connect with other services. You could use spring integration, or some form of Listeners. Or another. The choice is yours.
Define the gateway where the data comes into your system. This looks to be your Http Request Handler. Here you would push data into the gateway, and don't have a care what happens on the other end. You would then create two endpoints, one for your service so the data is processed correctly and one for your repository so an archive is maintained. (You could even add another later if you wanted.)
I have found numerous academic answers to this question, but I would like one from practitioners in the field.
Background
I would like to create a Java-based RESTful API, using the Grails framework for a variety of mobile clients (iOS and Android) to access protected resources through my service. I require authentication on certain requests, and I already have SSL setup over the wire (so all requests occur over https). My web API will eventually be exposed as a service to other web applications.
Problem
What authentication method do people recommend for a web service that is to be consumed by mobile devices, and eventually other web applications?
These are what I see as my choices. Can you tell me when would be appropriate use cases for each one?
I can do HTTP Basic authentication
I can do HTTP Digest authentication
I can implement OAuth authentication (1.0 or 2.0)?
I can pass the credentials as parameters in my request
I can use an authentication method above, and then pass a delegate/token around for authentication
I can implement my own custom HTTP authentication headers
I can use cookies and pass those to the server on each request
Other...?
Need
If you have one leaning one way or another, I'd like to know why you would choose that method. Better yet, if you're doing this in Grails, I'm very interested.
I already know...
I've already read through the excellent answers here and Richardson and Ruby's book, Restful Web Services.
REST is stateless protocol, thus using "work sessions" - I mean login/work/(auto)logout concept is somewhat questionable. Sending credentials as a parameter with each request seems to be the most often used method due to its simplicity. Just keep in mind that
1) The api url must be SSL only - it makes sense to use a dedicated domain AND ip address, e.g. api.example.com and configure your web server to handle SSL only for this address and domain. Just to avoid accidental disclosure of credentials.
2) Avoid using login/password with the request if possible, use "API key" (a shared secret) instead. You always may use "API key" instead of login/password if authentication is all you need, that is you do not need authorization (all users share the same permissions) and no need for logging (who did what).
3) Rather than sending the API key with each request, it is better to "sign" the request using shared secret and supply the signature. With the key long enough you may use this technique over plain unencrypted http.
=== responding to a comment ===
If you need authorization, just use basic authentication:
HTTPBuilder builder = new HTTPBuilder("https://api.example.com/v1/foo/bar")
builder.auth.basic(login, password)
builder.headers.put('Accept', 'application/json')
def result = builder.request(POST, JSON) { req ->
body = [
....
]
response.'201' = { resp, json ->
....
}
response.success = { resp, json ->
....
}
response.failure = { resp ->
log.error "failure, ${resp.statusLine}"
}
}