I am using hystrix javanica collapser in spring boot, but I found it did not work, my code just like this below:
service class:
public class TestService {
#HystrixCollapser(batchMethod = "getStrList")
public Future<String> getStr(String id) {
System.out.println("single");
return null;
}
#HystrixCommand
public List<String> getStrList(List<String> ids) {
System.out.println("batch,size=" + ids.size());
List<String> strList = Lists.newArrayList();
ids.forEach(id -> strList.add("test"));
return strList;
}
}
where I use:
public static void main(String[] args) {
TestService testService = new TestService();
HystrixRequestContext context = HystrixRequestContext.initializeContext();
Future<String> f1= testService.getStr("111");
Future<String> f2= testService.getStr("222");
try {
Thread.sleep(3000);
System.out.println(f1.get()); // nothing printed
System.out.println(f2.get()); // nothing printed
} catch (Exception e) {
}
context.shutdown();
}
It printed 3 single instead of 1 batch.
I want to know what's wrong with my code, a valid example is better.
I can't find a hystrix javanica sample on the internet, So I have to read the source code to solve this problem, now it's solved, and this is my summary:
when you use hystrix(javanica) collapser in spring-boot, you have to:
Defined a hystrixAspect spring bean and import hystrix-strategy.xml;
Annotate single method with #Hystrix Collapser annotate batch method with #HystrixCommand;
Make your single method need 1 parameter(ArgType) return Future , batch method need List return List and make sure size of args be equal to size of return.
Set hystrix properties batchMethod, scope, if you want to collapse requests from multiple user threads, you must set the scope to GLOBAL;
Before you submit a single request, you must init the hystrix context with HystrixRequestContext.initializeContext(), and shutdown the context when your request finish;
Given a class SchedulerResource which has the following createSchedules method and a bunch of constants used in the method, how can I use mockito to write a unit-test for the createSchedules method?
#PostMapping
public ResponseEntity<CustomResponse> createScheduler(#Valid #RequestBody SchedulerDTO schedulerDTO) {
if(schedulerDTO != null)
{
schedulerService.saveScheduler(schedulerDTO);
customResponse.setMessage("Schedule has been created!");
return new ResponseEntity<>(customResponse ,HttpStatus.OK);
} else {
customResponse.setMessage("Not Create!");
return new ResponseEntity<>(customResponse,HttpStatus.NOT_FOUND);
}
}
Test class:
#Test
public void createScheduler_Success() throws Exception {
SchedulerDTO scheduler = new SchedulerDTO();
Long sId = new Long(2);
scheduler.setSchedulerId(sId);
scheduler.setLinearChannelId((long)1);
scheduler.setDurationMs((long) 5000);
scheduler.setStatus(StatusEnum.NEW);
scheduler.setStartTime("2018-03-01T05:55:25");
scheduler.setEndTime("2018-03-01T05:57:25");
when(schedulerService.saveScheduler(scheduler)).thenReturn(scheduler);
mockMvc.perform(post("/linear/api/1.0/schedules")
.contentType(MediaType.APPLICATION_JSON)
.content(asJsonString(scheduler)))
.andExpect(status().isOk())
.andExpect(jsonPath("$.message", is("Schedule has been created!")));
}
So is ok with :
if(schedulerDTO != null)
{
schedulerService.saveScheduler(schedulerDTO);
customResponse.setMessage("Schedule has been created!");
return new ResponseEntity<>(customResponse ,HttpStatus.OK);
}
But what about:
else{
customResponse.setMessage("Not Create!");
return new ResponseEntity<>(customResponse,HttpStatus.NOT_FOUND);
}
So, - how can I write for the case where schedulerDTO == null?
Simple: you pass in null, and then you put down different specs for your mockMvc object, such as andExpect(status().isNotFound() (or something alike).
Beyond that, you can use methods like verifyZeroInteractions() to ensure no calls went to that mocked service object for example.
In that sense, it really isn't much different from testing the other case: you step back, and look at all the things that happen in the else branch, and then you think of ways how to observe/verify them.
I have used spring declarative retry in my project like
#Service
class Service {
#Async #Retryable(maxAttempts=12, backoff=#Backoff(delay=100, maxDelay=500))
public service() {
// ... do something
}
}
Now, I have two questions.
Is it fine to use retry with async, I don't have any issue, just
want to be sure.
The second requirement is, if the process fails I want to log it to log file including the number of remaining retries. So, is there a way to pass, or obtain the number of remaining retries from inside the method?
There is no way around using the annotations, #Recover annotated method executes only after the last failed retry, not after each one failing.
Refer to this github documentation
An excerpt from the link above- "Call the "service" method and if it fails with a RemoteAccessException then it will retry (up to three times by default), and then execute the "recover" method if unsuccessful."
Even with using RetryTemplate the Retry callback is called only after all retries are exhausted.
Another excerpt form the same link- "When a retry is exhausted the RetryOperations can pass control to a different callback, the RecoveryCallback. To use this feature clients just pass in the callbacks together to the same method"
You should use the #Recover annotation to perform an action on each fail and keep a count inside your object outside of the methods. Make sure no other methods interact with this counter. Here is the basic premise:
#Service
class Service {
private int attemptsLeft=12;
#Retryable(maxAttempts=12, backoff=#Backoff(delay=100, maxDelay=500))
public service() {
// ... do something that throws a KnownException you create to catch later on.
}
#Recover
public void connectionException(KnownException e) {
this.attemptsLeft = this.attemptsLeft-1; //decrease your failure counter
Logger.warn("Retry attempts left:{}",attemptsLeft);
}
}
If you don't want a member variable tracking count, you might need to ditch the annotations and declare the RetryTemplate to get access to the context, with the getRetryCount() method.
public String serviceWithRetry() {
RetryTemplate retryTemplate = new RetryTemplate();
final SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(12);
retryTemplate.setRetryPolicy(retryPolicy);
FixedBackOffPolicy backOffPolicy = new FixedBackOffPolicy();
backOffPolicy.setInterval(100L);
retryTemplate.setBackOffPolicy(backOffPolicy);
retryTemplate.execute(new RetryCallback<Void, RuntimeException>()
return retryTemplate.execute(new RetryCallback<Void, RuntimeException>() {
#Override
public void doWithRetry(RetryContext context) {
LOG.info("Retry of connection count: {}", context.getRetryCount());
return //something with your connection logic
}
});
}
I am using spring-cloud-starter (ie.. spring boot with all the microservices features). When I create hystrix method in a component annotated using the javanica #HystrixCommand, follow the directions on the javanica github site (https://github.com/Netflix/Hystrix/tree/master/hystrix-contrib/hystrix-javanica) to make that method run async, regardless of whether I use their 'Future<>' or Reactive execution 'Observable<>', nothing runs/executes and I get
java.lang.ClassCastException: springbootdemo.EricComponent$1 cannot be cast to springbootdemo.Eric whenever I attempt to pull the result (in the case of Future<>) or get a callback (in case of Reactive Execution .. and println's dont trigger so it really didnt run).
public class Application { ...
}
#RestController
#RequestMapping(value = "/makebunchofcalls/{num}")
class EricController { ..
#RequestMapping(method={RequestMethod.POST})
ArrayList<Eric> doCalls(#PathVariable Integer num) throws IOException {
ArrayList<Eric> ale = new ArrayList<Eric>(num);
for (int i =0; i<num; i++) {
rx.Observable<Eric> oe = this.ericComponent.doRestTemplateCallAsync(i);
oe.subscribe(new Action1<Eric>() {
#Override
public void call(Eric e) { // AT RUNTIME, ClassCastException
ale.add(e);
}
});
}
return ale;
}
#Component
class EricComponent { ...
// async version =========== using reactive execution via rx library from netflix ==============
#HystrixCommand(fallbackMethod = "defaultRestTemplateCallAsync", commandKey = "dogeAsync")
public rx.Observable<Eric> doRestTemplateCallAsync(int callNum) {
return new ObservableResult<Eric>() {
#Override
public Eric invoke() { // NEVER CALLED
try {
ResponseEntity<String> result = restTemplate.getForEntity("http://doges/doges/24232/photos", String.class); // actually make a call
System.out.println("*************** call successfull: " + new Integer(callNum).toString() + " *************");
} catch (Exception ex) {
System.out.println("=============== call " + new Integer(callNum).toString() + " not successfull: " + ex.getMessage() + " =============");
}
return new Eric(new Integer(callNum).toString(), "ok");
}
};
}
public rx.Observable<Eric> defaultRestTemplateCallAsync(int callNum) {
return new ObservableResult<Eric>() {
#Override
public Eric invoke() {
System.out.println("!!!!!!!!!!!!! call bombed " + new Integer(callNum).toString() + "!!!!!!!!!!!!!");
return new Eric(new Integer(callNum).toString(), "bomb");
}
};
}
}
Why would I be getting back an EricComponent$1 instead of a Eric? btw, Eric is just a simple class with 2 strings... its ommitted.
I am figuring that I must have to explicitly execute, but that alludes me because: 1) Doing it with Future<> the queue() method is not available as the documentation claims and 2) doing it with Observable<> there really isn't a way to execute it that I get.
Do you have the #EnableHystrix annotation on you application class?
The subscribe method is asynchronous and you are trying to populate a list in a synchronous controller method so there may be a problem there. Can you change the subscribe to toBlockingObservable().forEach() and see if that helps?
Update #1
I was able to duplicate. Your default method should not return an Observable<Eric>, just an Eric.
public Eric defaultRestTemplateCallAsync(final int callNum) {
System.out.println("!!!!!!!!!!!!! call bombed " + new Integer(callNum) + "!!!!!!!!!!!!!");
return new Eric(new Integer(callNum).toString(), "bomb");
}
Update #2
See my code here https://github.com/spencergibb/communityanswers/tree/so26372319
Update #3
When I commented out the fallbackMethod attribute, it complained that it couldn't find a public version of EricComponent for AOP. I made EricComponent public static and it worked. A top level class in its own file would work to. My code, linked above, works (assuming the restTemplate call works) and returns n OK.
I've been thinking around the Java feature that evaluates annotation values in compile-time and it seems to really make difficult externalizing annotation values.
However, I am unsure whether it is actually impossible, so I'd appreciate any suggestions or definitive answers on this.
More to the point, I am trying to externalize an annotation value which controls delays between scheduled method calls in Spring, e.g.:
public class SomeClass {
private Properties props;
private static final long delay = 0;
#PostConstruct
public void initializeBean() {
Resource resource = new ClassPathResource("scheduling.properties");
props = PropertiesLoaderUtils.loadProperties(resource);
delay = props.getProperties("delayValue");
}
#Scheduled(fixedDelay = delay)
public void someMethod(){
// perform something
}
}
Suppose that scheduling.properties is on classpath and contains property key delayValue along with its corresponding long value.
Now, this code has obvious compilation errors since we're trying to assign a value to final variable, but that is mandatory, since we can't assign the variable to annotation value, unless it is static final.
Is there any way of getting around this? I've been thinking about Spring's custom annotations, but the root issue remains - how to assign the externalized value to annotation?
Any idea is welcome.
EDIT: A small update - Quartz integration is overkill for this example. We just need a periodic execution with sub-minute resolution and that's all.
The #Scheduled annotation in Spring v3.2.2 has added String parameters to the original 3 long parameters to handle this. fixedDelayString, fixedRateString and initialDelayString are now available too:
#Scheduled(fixedDelayString = "${my.delay.property}")
public void someMethod(){
// perform something
}
Thank you both for your answers, you have provided valuable info which led me to this solution, so I upvoted both answers.
I've opted to make a custom bean post processor and custom #Scheduled annotation.
The code is simple (essentially it is a trivial adaptation of existing Spring code) and I really wonder why they didn't do it like this from the get go. BeanPostProcessor's code count is effectively doubled since I chose to handle the old annotation and the new one.
If you have any suggestion on how to improve this code, I'll be glad to hear it out.
CustomScheduled class (annotation)
#Target({ElementType.METHOD, ElementType.ANNOTATION_TYPE})
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface CustomScheduled {
String cron() default "";
String fixedDelay() default "";
String fixedRate() default "";
}
CustomScheduledAnnotationBeanPostProcessor class
public class CustomScheduledAnnotationBeanPostProcessor implements BeanPostProcessor, Ordered, EmbeddedValueResolverAware, ApplicationContextAware, ApplicationListener<ContextRefreshedEvent>, DisposableBean
{
private static final Logger LOG = LoggerFactory.getLogger(CustomScheduledAnnotationBeanPostProcessor.class);
// omitted code is the same as in ScheduledAnnotationBeanPostProcessor......
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
// processes both #Scheduled and #CustomScheduled annotations
public Object postProcessAfterInitialization(final Object bean, String beanName) throws BeansException {
final Class<?> targetClass = AopUtils.getTargetClass(bean);
ReflectionUtils.doWithMethods(targetClass, new MethodCallback() {
public void doWith(Method method) throws IllegalArgumentException, IllegalAccessException {
Scheduled oldScheduledAnnotation = AnnotationUtils.getAnnotation(method, Scheduled.class);
if (oldScheduledAnnotation != null) {
LOG.info("#Scheduled found at method {}", method.getName());
Assert.isTrue(void.class.equals(method.getReturnType()), "Only void-returning methods may be annotated with #Scheduled.");
Assert.isTrue(method.getParameterTypes().length == 0, "Only no-arg methods may be annotated with #Scheduled.");
if (AopUtils.isJdkDynamicProxy(bean)) {
try {
// found a #Scheduled method on the target class for this JDK proxy -> is it
// also present on the proxy itself?
method = bean.getClass().getMethod(method.getName(), method.getParameterTypes());
} catch (SecurityException ex) {
ReflectionUtils.handleReflectionException(ex);
} catch (NoSuchMethodException ex) {
throw new IllegalStateException(String.format(
"#Scheduled method '%s' found on bean target class '%s', " +
"but not found in any interface(s) for bean JDK proxy. Either " +
"pull the method up to an interface or switch to subclass (CGLIB) " +
"proxies by setting proxy-target-class/proxyTargetClass " +
"attribute to 'true'", method.getName(), targetClass.getSimpleName()));
}
}
Runnable runnable = new ScheduledMethodRunnable(bean, method);
boolean processedSchedule = false;
String errorMessage = "Exactly one of 'cron', 'fixedDelay', or 'fixedRate' is required.";
String cron = oldScheduledAnnotation.cron();
if (!"".equals(cron)) {
processedSchedule = true;
if (embeddedValueResolver != null) {
cron = embeddedValueResolver.resolveStringValue(cron);
}
cronTasks.put(runnable, cron);
}
long fixedDelay = oldScheduledAnnotation.fixedDelay();
if (fixedDelay >= 0) {
Assert.isTrue(!processedSchedule, errorMessage);
processedSchedule = true;
fixedDelayTasks.put(runnable, fixedDelay);
}
long fixedRate = oldScheduledAnnotation.fixedRate();
if (fixedRate >= 0) {
Assert.isTrue(!processedSchedule, errorMessage);
processedSchedule = true;
fixedRateTasks.put(runnable, fixedRate);
}
Assert.isTrue(processedSchedule, errorMessage);
}
CustomScheduled newScheduledAnnotation = AnnotationUtils.getAnnotation(method, CustomScheduled.class);
if (newScheduledAnnotation != null) {
LOG.info("#CustomScheduled found at method {}", method.getName());
Assert.isTrue(void.class.equals(method.getReturnType()), "Only void-returning methods may be annotated with #CustomScheduled.");
Assert.isTrue(method.getParameterTypes().length == 0, "Only no-arg methods may be annotated with #CustomScheduled.");
if (AopUtils.isJdkDynamicProxy(bean)) {
try {
// found a #CustomScheduled method on the target class for this JDK proxy -> is it
// also present on the proxy itself?
method = bean.getClass().getMethod(method.getName(), method.getParameterTypes());
} catch (SecurityException ex) {
ReflectionUtils.handleReflectionException(ex);
} catch (NoSuchMethodException ex) {
throw new IllegalStateException(String.format("#CustomScheduled method '%s' found on bean target class '%s', "
+ "but not found in any interface(s) for bean JDK proxy. Either "
+ "pull the method up to an interface or switch to subclass (CGLIB) "
+ "proxies by setting proxy-target-class/proxyTargetClass " + "attribute to 'true'", method.getName(),
targetClass.getSimpleName()));
}
}
Runnable runnable = new ScheduledMethodRunnable(bean, method);
boolean processedSchedule = false;
String errorMessage = "Exactly one of 'cron', 'fixedDelay', or 'fixedRate' is required.";
boolean numberFormatException = false;
String numberFormatErrorMessage = "Delay value is not a number!";
String cron = newScheduledAnnotation.cron();
if (!"".equals(cron)) {
processedSchedule = true;
if (embeddedValueResolver != null) {
cron = embeddedValueResolver.resolveStringValue(cron);
}
cronTasks.put(runnable, cron);
LOG.info("Put cron in tasks map with value {}", cron);
}
// fixedDelay value resolving
Long fixedDelay = null;
String resolverDelayCandidate = newScheduledAnnotation.fixedDelay();
if (!"".equals(resolverDelayCandidate)) {
try {
if (embeddedValueResolver != null) {
resolverDelayCandidate = embeddedValueResolver.resolveStringValue(resolverDelayCandidate);
fixedDelay = Long.valueOf(resolverDelayCandidate);
} else {
fixedDelay = Long.valueOf(newScheduledAnnotation.fixedDelay());
}
} catch (NumberFormatException e) {
numberFormatException = true;
}
}
Assert.isTrue(!numberFormatException, numberFormatErrorMessage);
if (fixedDelay != null && fixedDelay >= 0) {
Assert.isTrue(!processedSchedule, errorMessage);
processedSchedule = true;
fixedDelayTasks.put(runnable, fixedDelay);
LOG.info("Put fixedDelay in tasks map with value {}", fixedDelay);
}
// fixedRate value resolving
Long fixedRate = null;
String resolverRateCandidate = newScheduledAnnotation.fixedRate();
if (!"".equals(resolverRateCandidate)) {
try {
if (embeddedValueResolver != null) {
fixedRate = Long.valueOf(embeddedValueResolver.resolveStringValue(resolverRateCandidate));
} else {
fixedRate = Long.valueOf(newScheduledAnnotation.fixedRate());
}
} catch (NumberFormatException e) {
numberFormatException = true;
}
}
Assert.isTrue(!numberFormatException, numberFormatErrorMessage);
if (fixedRate != null && fixedRate >= 0) {
Assert.isTrue(!processedSchedule, errorMessage);
processedSchedule = true;
fixedRateTasks.put(runnable, fixedRate);
LOG.info("Put fixedRate in tasks map with value {}", fixedRate);
}
Assert.isTrue(processedSchedule, errorMessage);
}
}
});
return bean;
}
}
spring-context.xml config file
<beans...>
<!-- Enables the use of a #CustomScheduled annotation-->
<bean class="org.package.CustomScheduledAnnotationBeanPostProcessor" />
</beans>
Some spring annotations support SpEL.
First:
<context:property-placeholder
location="file:${external.config.location}/application.properties" />
And then, for example:
#Value("${delayValue}")
private int delayValue;
I'm not sure if #Scheduled supports SPeL, though, but in general, that's the approach.
In regard to scheduling, check this post of mine and this related question
A better way to do this is to define the scheduling in xml using the task name space
<context:property-placeholder location="scheduling.properties"/>
<task:scheduled ref="someBean" method="someMethod" fixed-delay="${delayValue}"/>
If you for some reason want to do it with annotation, you need to create an annotation that has another optional attribute where you can specify the property name or better still a property placeholder expression or Spel expression.
#MyScheduled(fixedDelayString="${delay}")
If you want to make this work with annotation rather than bean configuration xml, you can use the following annotations: #Component, #PropertySource with PropertySourcesPlaceholderConfigurer Bean itself, like this:
#Component
#PropertySource({ "classpath:scheduling.properties" })
public class SomeClass {
#Scheduled(fixedDelay = "${delay}")
public void someMethod(){
// perform something
}
#Bean
public static PropertySourcesPlaceholderConfigurer placeHolderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
We can use a field value from other beans. Suppose we have a bean named someBean with a field someValue equal to 10. Then, 10 will be assigned to the field:
#Value("#{someBean.someValue}")
private Integer someBeanValue;
Reference: A Quick Guide to Spring #Value - Baeldung