I have a Spring Boot REST service that sometimes call third party services as a part of a request. I would like to set a timeout on all my resources (let's say 5 seconds), so that if any request handling (the whole chain, from incoming to response) takes longer than 5 seconds my controllers responds with HTTP 503 instead of the actual response. It would be awesome if this was just a Spring property, for example setting
spring.mvc.async.request-timeout=5000
but I haven't had any luck with that. I've also tried extending WebMvcConfigurationSupport and overriding configureAsyncSupport:
#Override
public void configureAsyncSupport(final AsyncSupportConfigurer configurer) {
configurer.setDefaultTimeout(5000);
configurer.registerCallableInterceptors(timeoutInterceptor());
}
#Bean
public TimeoutCallableProcessingInterceptor timeoutInterceptor() {
return new TimeoutCallableProcessingInterceptor();
}
without any luck.
I suspect I have to manually time all my third party calls, and if they take too long, throw a timeout exception. Is that right? Or is there any easier, holistic solution that covers all my request endpoints?
You need to return a Callable<> if you want spring.mvc.async.request-timeout=5000 to work.
#RequestMapping(method = RequestMethod.GET)
public Callable<String> getFoobar() throws InterruptedException {
return new Callable<String>() {
#Override
public String call() throws Exception {
Thread.sleep(8000); //this will cause a timeout
return "foobar";
}
};
}
The #Transactional annotation takes a timeout parameter where you can specify timeout in seconds for a specific method in the #RestController
#RequestMapping(value = "/method",
method = RequestMethod.POST,
produces = MediaType.APPLICATION_JSON_VALUE)
#Timed
#Transactional(timeout = 120)
A fresh answer for Spring Boot 2.2 is required as server.connection-timeout=5000 is deprecated. Each server behaves differently, so server specific properties are recommended instead.
SpringBoot embeds Tomcat by default, if you haven't reconfigured it with Jetty or something else. Use server specific application properties like server.tomcat.connection-timeout or server.jetty.idle-timeout.
As commented by Wilkinson:
Setting the connection timeout will only result in a timeout when the
client connects but is then too slow to send its request. If you want
the client to wait for a maximum of 30 seconds for a response you will
have to configure that on the client-side. If you want the server-side
to only spend a maximum of 30 seconds handling the request there is no
way to guarantee that as you cannot force the thread that is handling
the request to stop.
You can also try setting spring.mvc.async.request-timeout
I would suggest you have a look at the Spring Cloud Netflix Hystrix starter to handle potentially unreliable/slow remote calls. It implements the Circuit Breaker pattern, that is intended for precisely this sorta thing.
See offcial docs for more information.
You can try server.connection-timeout=5000 in your application.properties. From the official documentation:
server.connection-timeout= # Time in milliseconds that connectors will
wait for another HTTP request before closing the connection. When not
set, the connector's container-specific default will be used. Use a
value of -1 to indicate no (i.e. infinite) timeout.
On the other hand, you may want to handle timeouts on the client side using Circuit Breaker pattern as I have already described in my answer here: https://stackoverflow.com/a/44484579/2328781
if you are using RestTemplate than you should use following code to implement timeouts
#Bean
public RestTemplate restTemplate() {
return new RestTemplate(clientHttpRequestFactory());
}
private ClientHttpRequestFactory clientHttpRequestFactory() {
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setReadTimeout(2000);
factory.setConnectTimeout(2000);
return factory;
}}
The xml configuration
<bean class="org.springframework.web.client.RestTemplate">
<constructor-arg>
<bean class="org.springframework.http.client.HttpComponentsClientHttpRequestFactory"
p:readTimeout="2000"
p:connectTimeout="2000" />
</constructor-arg>
In Spring properties files, you can't just specify a number for this property. You also need to specify a unit. So you can say spring.mvc.async.request-timeout=5000ms or spring.mvc.async.request-timeout=5s, both of which will give you a 5-second timeout.
I feel like none of the answers really solve the issue. I think you need to tell the embedded server of Spring Boot what should be the maximum time to process a request. How exactly we do that is dependent on the type of the embedded server used.
In case of Undertow, one can do this:
#Component
class WebServerCustomizer : WebServerFactoryCustomizer<UndertowServletWebServerFactory> {
override fun customize(factory: UndertowServletWebServerFactory) {
factory.addBuilderCustomizers(UndertowBuilderCustomizer {
it.setSocketOption(Options.READ_TIMEOUT, 5000)
it.setSocketOption(Options.WRITE_TIMEOUT, 25000)
})
}
}
Spring Boot official doc: https://docs.spring.io/spring-boot/docs/2.2.0.RELEASE/reference/html/howto.html#howto-configure-webserver
You can configure the Async thread executor for your Springboot REST services. The setKeepAliveSeconds() should consider the execution time for the requests chain. Set the ThreadPoolExecutor's keep-alive seconds. Default is 60. This setting can be modified at runtime, for example through JMX.
#Bean(name="asyncExec")
public Executor asyncExecutor()
{
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(3);
executor.setMaxPoolSize(3);
executor.setQueueCapacity(10);
executor.setThreadNamePrefix("AsynchThread-");
executor.setAllowCoreThreadTimeOut(true);
executor.setKeepAliveSeconds(10);
executor.initialize();
return executor;
}
Then you can define your REST endpoint as follows
#Async("asyncExec")
#PostMapping("/delayedService")
public CompletableFuture<String> doDelay()
{
String response = service.callDelayedService();
return CompletableFuture.completedFuture(response);
}
Related
In my Spring Boot project, a couple of REST API endpoints require a class whose initialization takes several minutes. Because of this, it takes several minutes to start the REST API.
Is it possible (using Spring Boot) to make so that these few endpoints are initialized asynchronously i.e. all other endpoints are initialized right away and REST API starts working and these endpoints are initialized whenever the class that they need is initialized and are simply not available to the user before that?
I tried looking into #Async and other ways to make things asynchronous in Spring Boot, but that did not help.
I would really appreciate some help.
Try #Lazy annotation. When it's applied to the spring component, it will be initialized on the first call.
Some resources:
https://www.baeldung.com/spring-lazy-annotation
Java Doc
There's nothing built into Spring to do what you want, but you could implement it yourself by returning a 404 Not Found response while the service is initializing and a 200 OK once it's available. The following is one way to implement that:
#RestController
class ExampleController {
private final Future<SlowInitializationService> service;
ExampleController() {
this.service = ForkJoinPool.commonPool().submit(SlowInitializationService::new);
}
#GetMapping
ResponseEntity<Result> example() throws InterruptedException, ExecutionException {
if (this.service.isDone()) {
return new ResponseEntity<>(this.service.get().perform(), HttpStatus.OK);
}
return new ResponseEntity<>(HttpStatus.NOT_FOUND);
}
}
Rather than using the common pool, you may want to inject an Executor or similar. It'll depend on the rest of your app and how it handles threading.
In the latest versions of spring cloud Netflix's official documentation (for example 2.0.2.RELEASE, the last GA version) it states:
When Spring Retry is present, load-balanced RestTemplates, Feign, and Zuul automatically retry any failed requests (assuming your configuration allows doing so).
But that seems wrong concerning the use of standalone Ribbon (i.e. load-balanced RestTemplates). I couldn't make it work nor found any working example. Moreover, I found other sources stating quite the opposite was true, such as:
spring-cloud with RestTemplate//Ribbon/Eureka - retry when server not available
Spring Cloud - Getting Retry Working In RestTemplate?).
So, it's the documentation wrong or is the whole world missing something?
We found the same issue; Zuul will neither retry nor failover to alternative servers in the ribbon server list using the default (Apache Http Client) configuration. We tracked it down to this line of code in the RibbonLoadBalancingHttpClient class:
#Override
public RequestSpecificRetryHandler getRequestSpecificRetryHandler(
RibbonApacheHttpRequest request, IClientConfig requestConfig) {
return new RequestSpecificRetryHandler(false, false, RetryHandler.DEFAULT,
requestConfig);
}
The hardcoded false, false parameters effectively disable retry from ever happening. It is easy to fix this.
Add this to your Zuul main application class:
#RibbonClients(
defaultConfiguration = {EurekaRibbonClientConfiguration.class,
MyRibbonConfiguration.class})
Create a class, MyRibbonConfiguration and use it to emulate the ribbonLoadBalancingHttpClient method here except we override and fix the getRequestSpecificRetryHandler method that's causing problems.
public class MyRibbonConfiguration {
#RibbonClientName
private String name = "client";
#Bean
public RibbonLoadBalancingHttpClient ribbonLoadBalancingHttpClient(
IClientConfig config, ServerIntrospector serverIntrospector,
ILoadBalancer loadBalancer, RetryHandler retryHandler, CloseableHttpClient httpClient) {
RibbonLoadBalancingHttpClient client =
new RibbonLoadBalancingHttpClient(httpClient, config, serverIntrospector) {
#Override
public RequestSpecificRetryHandler getRequestSpecificRetryHandler(
RibbonApacheHttpRequest request, IClientConfig requestConfig) {
return new RequestSpecificRetryHandler(true, true, RetryHandler.DEFAULT,
requestConfig);
}
};
client.setLoadBalancer(loadBalancer);
client.setRetryHandler(retryHandler);
Monitors.registerObject("Client_" + this.name, client);
return client;
}
}
Adjust true, true to reference from properties if you want. With this in place Zuul will start respecting the ribbon.MaxAutoRetries and ribbon.MaxAutoRetriesNextServer properties.
Note that if you are using Ribbon with Feign for server->server calls then those Feign calls are not affected because OpenFeign hardcodes those two parameters to true, true.
I am using current Spring boot version (1.4.x) and wondering if it has any default timeout for api calls. I have tested it by putting breakpoints but it was keep waiting and didn't time-out.
I was also trying to configure default timeout for all my spring-boot apps by using some annotation or yml settings.
I found couple of alternatives (one of them here) but using callable actually adding extra non-business logic code where setting something in xml bean is out of fashion in latest spring boot applications.
You can try server.connection-timeout=5000 in your application.properties.From the official documentation:
server.connection-timeout= # Time in milliseconds that connectors will wait for another HTTP request before closing the connection. When not set, the connector's container-specific default will be used. Use a value of -1 to indicate no (i.e. infinite) timeout.
UPDATE:
Just noticed that you use microservice architecture, so in case you need to handle timeouts when communicating between microservices, I would suggest handling it on the client side instead of the server side. If the microservice you are trying to call is overloaded and its performance degrades to the point where it drastically affects the user experience sometimes it's better to return some fallback data than just drop the request.
Imagine we have an e-commerce web-site that has microservice architecture and one of its microservices that gives recommendations to the user becomes extremely slow. In this case, the preferred solution would be to return some fallback data which could be top 10 popular products this month rather than showing 5xx error page to the customer. Moreover, in case subsequent requests fail with a timeout, we can make a decision to avoid sending requests to the 'recommendation-service' and return fallback data immediately. After some time we can try sending a request to the 'recommendation-service' again, and if it became healthy - just use it instead of the fallback data.
This is called Circuit Breaker pattern and there is already an implementation of it in a framework called Hystrix. Here is a nice article explaining it in depth: http://www.baeldung.com/spring-cloud-netflix-hystrix. Spring Cloud Feign + Spring Cloud Hystrix looks really nice especially taking into account that they work with Discovery services out-of-the-box (e.g. Spring Cloud Eureka).
I agree all above options and tried below option in my spring boot application. It works perfectly fine now. Below is the code sample as a bean. Now just need to #Autowire RestTemplate wherever(java class) I need it.
#Bean
public RestTemplate restTemplate() {
RestTemplate restTemplate = new RestTemplate();
((SimpleClientHttpRequestFactory) restTemplate.getRequestFactory()).setConnectTimeout(15000);
((SimpleClientHttpRequestFactory) restTemplate.getRequestFactory()).setReadTimeout(15000);
return restTemplate;
}
There are a couple of ways to do this:
1) Using ClientHttpRequestFactory with RestTemplate:
public RestTemplate restTemplate() {
return new RestTemplate(clientHttpRequestFactory());
}
private ClientHttpRequestFactory clientHttpRequestFactory() {
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setReadTimeout(timeinMillis);
factory.setConnectTimeout(timeinMillis);
return factory;
}
2) Second way is to use callable but I guess you have already explored that solution.
The timeout can be set using the connectionTimeout property of Tomcat.
Please refer this answer how to set it for Tomcat.
Configuring maxKeepAliveRequests in Spring Boot embedded Tomcat
You can create a configuration file using annotation #Configuration
For using RestTemplate:
` #Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return new RestTemplateBuilder()
.setConnectTimeout(Duration.ofMillis(30000))
.setReadTimeout(Duration.ofMillis(30000))
.build();
}`
For WebClient Timeouts:
#Bean
public WebClient webClient() {
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(
HttpClient.create().responseTimeout(Duration.ofMillis(30000))
))
.build();
}
After the configuration is done, in a controller or any other file where you want to use this RestTemplate or WebClient just autowire it like:
#Autowired
private RestTemplate restTemplate;
or
#Autowired
private WebClient webClient;
I got a load-test setup of spring websocket server (based on Jetty and spring version 4.3.2.RELEASE) and client, that generates many connections (based on spring's sample java websocket client). The code below sends data to given websocket session: the snippet exploits the case where sessionId can be used instead of User ID (Spring WebSocket #SendToSession: send message to specific session). I may execute this code very often, every 2-3 milliseconds. I use SimpleMessageBroker.
public void publishToSessionUsingTopic(String sessionId, String subscriptionTopic, Map<String, CacheRowModel> payload) {
String subscriptionTopicWithoutUser = subscriptionTopic.replace(USER_ENDPOINT, "");
// necessary message headers for per-session send
SimpMessageHeaderAccessor headerAccessor = SimpMessageHeaderAccessor.create(SimpMessageType.MESSAGE);
headerAccessor.setSessionId(sessionId);
headerAccessor.setLeaveMutable(true);
simpMessagingTemplate.convertAndSendToUser(sessionId, subscriptionTopicWithoutUser, Collections.singletonList(payload), headerAccessor.getMessageHeaders());
}
When this code is executed very frequently (every 2-3 milliseconds) for ~100 sessions, while I see in my logs that it was run and called the convertAndSendToUser, some of the sessions won't receive the message. I appreciate any suggestions about how this could be cleared.
Well, I think your problem is with the:
#Bean
public ThreadPoolTaskExecutor clientOutboundChannelExecutor() {
TaskExecutorRegistration reg = getClientOutboundChannelRegistration().getOrCreateTaskExecRegistration();
ThreadPoolTaskExecutor executor = reg.getTaskExecutor();
executor.setThreadNamePrefix("clientOutboundChannel-");
return executor;
}
where it uses this config for the Executor:
protected ThreadPoolTaskExecutor getTaskExecutor() {
ThreadPoolTaskExecutor executor = (this.taskExecutor != null ? this.taskExecutor : new ThreadPoolTaskExecutor());
executor.setCorePoolSize(this.corePoolSize);
executor.setMaxPoolSize(this.maxPoolSize);
executor.setKeepAliveSeconds(this.keepAliveSeconds);
executor.setQueueCapacity(this.queueCapacity);
executor.setAllowCoreThreadTimeOut(true);
return executor;
}
See, there is no RejectedExecutionHandler configured. And by default it is like:
private RejectedExecutionHandler rejectedExecutionHandler = new ThreadPoolExecutor.AbortPolicy();
So, when you have enough many messages and tasks for them exceed the ThreadPool, any extra are just aborted.
To fix the issue you should implement WebSocketMessageBrokerConfigurer and override its configureClientOutboundChannel() to provide some custom taskExecutor(ThreadPoolTaskExecutor taskExecutor) for example with the new ThreadPoolExecutor.CallerRunsPolicy().
I am currently using Spring Cloud and Feign to consume a Microservice in my application. Since it can happen, that a database connection or the like fails in a single service instance, making it return 500 HTTP status code, I want to make sure, that the next server is retried by the service's clients. Currently, Ribbon's retry mechanism works like a charm when the service is not running at all, however it still returns instantly an error when it receives a 500 status code, without any retry.
Is it possible to configure the Feign clients or their underlying Ribbon load balancers to retry the next server, if an instance returns a 500 response?
The configuration is pretty much the same as in this thread: Does Feign retry require some sort of configuration?
I would love to use an implementation like Ribbons' HttpResponseValidator (https://github.com/Netflix/ribbon/blob/master/ribbon/src/main/java/com/netflix/ribbon/http/HttpResponseValidator.java), but I couldn't find anything usable for Spring Cloud and its Feign/Ribbon integration
This question is very old and the solution was probably already found or wasn't possible at the time. Anyway, I think that answer might still help someone 8 ).
Please use this as a reference, this code is not intended for production use.
Feign allows you to configure errorDecoder - this is the place where magic happens.
Feign.Builder builder = Feign.builder()
.errorDecoder(new RetryOnScaleErrorDecoder())
Here is the implementation, I use that class to retry request on HTTP error 429 I get from AWS when service is scaling
public static class RetryOnScaleErrorDecoder implements ErrorDecoder {
#Override
public Exception decode(String methodKey, Response response) {
FeignException exception = errorStatus(methodKey, response);
// This is a terrible part please check how feign.codec.ErrorDecoder.RetryAfterDecoder is implemented for proper parsing of retry-after header
Collection<String> headers = response.headers().get("Retry-After");
String repeatAfterString = "0";
if (Objects.nonNull(headers)) {
repeatAfterString = Iterables.getFirst(headers, "0");
}
assert repeatAfterString != null;
Date repeatAfter = new Date(currentTimeMillis());
if (repeatAfterString.matches("^[0-9]+$")) {
try {
long deltaMillis = SECONDS.toMillis(Long.parseLong(repeatAfterString));
repeatAfter = new Date(currentTimeMillis() + deltaMillis);
} catch (NumberFormatException ignored) {
// TODO: logging
}
}
// That's the part where we decide to retry based on status code value
if (exception.status() == 429) {
return new RetryableException(
response.status(),
exception.getMessage(),
response.request().httpMethod(),
exception,
repeatAfter
);
}
return exception;
}
}
I think that in conjunction with Ribbon it will produce desired result.
Try to this config:
MY-SPRING-API.ribbon.retryableStatusCodes=404,500
This is the same question:
Feign client and Spring retry
document is :
https://docs.spring.io/spring-cloud-netflix/docs/2.2.10.RELEASE/reference/html/#retrying-failed-requests