I've configured a route to extract some data from exchanges and aggregate them; here is simple summary:
#Component
#RequiredArgsConstructor
public class FingerprintHistoryRouteBuilder extends RouteBuilder {
private final FingerprintHistoryService fingerprintHistoryService;
#Override
public void configure() throws Exception {
from("seda:httpFingerprint")
.aggregate( (AggregationStrategy) (oldExchange, newExchange) -> {
final FingerprintHistory newFingerprint = extract(newExchange);
if (oldExchange == null) {
List<FingerprintHistory> fingerprintHistories = new ArrayList<>();
fingerprintHistories.add(newFingerprint);
newExchange.getMessage().setBody(fingerprintHistories);
return newExchange;
}
final Message oldMessage = oldExchange.getMessage();
final List<FingerprintHistory> fingerprintHistories = (List<FingerprintHistory>) oldMessage.getBody(List.class);
fingerprintHistories.add(newFingerprint);
return oldExchange;
})
.constant(true)
.completionSize(aggregateCount)
.completionInterval(aggregateDuration.toMillis())
.to("direct:processFingerprint")
.end();
from("direct:processFingerprint")
.process(exchange -> {
List<FingerprintHistory> fingerprintHistories = exchange.getMessage().getBody(List.class);
fingerprintHistoryService.saveAll(fingerprintHistories);
});
strong text
}
}
The problem is aggregation completion never works for example this is a sample of my test:
#SpringBootTest
class FingerprintHistoryRouteBuilderTest {
#Autowired
ProducerTemplate producerTemplate;
#Autowired
FingerprintHistoryRouteBuilder fingerprintHistoryRouteBuilder;
#Autowired
CamelContext camelContext;
#MockBean
FingerprintHistoryService historyService;
#Test
void api_whenAggregate() {
UserSearchActivity activity = ActivityFactory.buildSampleSearchActivity("127.0.0.1", "salam", "finger");
Exchange exchange = buildExchange();
exchange.getMessage().setBody(activity);
ReflelctionTestUtils.setField(fingerprintHistoryRouteBuilder, "aggregateCount", 1);
ReflectionTestUtils.setFiled(fingerprintHistoryRouteBuilder, "aggregateDuration", Duration.ofNanos(1));
producerTemplate.send(FingerprintHistoryRouteBuilder.FINGERPRINT_HISTORY_ENDPOINT, exchange);
Mockito.verify(historyService).saveAll(Mockito.any());
}
Exchange buildExchange() {
DefaultExchange defaultExchange = new DefaultExchange(camelContext);
defaultExchange.setMessage(new DefaultMessage(camelContext));
return defaultExchange;
}
}
with the following result:
Wanted but not invoked: fingerprintHistoryService bean.saveAll(
);
I build this simplified example, and the test passes, so it looks like your usage of aggregate is probably correct.
Have you considered that your Mockito.verify() call is happening before the exchange finishes routing? You could test this by removing the verify call and adding a .log() statement to the FINGERPRINT_PROCESS_AGGREGATION route. If you see the log output during execution, you know the exchange is being routed as you expect. If this is the case, then your verify() call needs to be able to wait for the exchange to finish routing. I don't use mockito much, but it looks like you can do this:
Mockito.verify(historyService, timeout(10000)).saveAll(Mockito.any());
Related
I want to call a Rest API using springboot till a field in the response (hasMoreEntries) is 'Y'. Currently, am simply using a while loop and checking the response for the flag and calling the API again. PFB pseudocode. Is there any other efficient way to do this OR what is the best way.
String hasMoreEntries="Y";
while(!hasMoreEntries.equals("N"))
{
response = \\PERFORM REST SERVICE CALL HERE
hasMoreEntries=respone.body().getHasMoreEntries();
}
You can use Spring Retry mechanism.
For example you can create RetryTemplate bean with custom exception ResponseValidateException that is thrown when the response is invalid:
#Bean
public RetryTemplate retryTemplate() {
return RetryTemplate.builder()
.retryOn(ResponseValidateException.class)
.infiniteRetry()
.fixedBackoff(2000L)
.build();
}
And your service:
#Service
public class YourService {
#Autowired
private RetryTemplate retryTemplate;
public void bar() {
final ResponseEntity<SomeObject> entity = retryTemplate.execute(retryContext -> {
final ResponseEntity<SomeObject> response = null;
if ("Y".equals(response.getBody().getHasMoreEntries())) {
return response;
} else {
throw new ResponseValidateException();
}
});
}
}
You can also look at custom RetryPolicy (for example CircuitBreakerRetryPolicy) classes to add them in your RetryTemplate bean for your cases.
so, i have this #Component class for listening topic from kafka
#Component
#Data
#Slf4j
public class KafkaConsumer {
public List<String> saveReserveStock = new ArrayList<>();
#KafkaListener(topics = "topic")
public void listenReserveStock(ConsumerRecord<?, ?> consumerRecord) {
System.out.println("==================================================================");
System.out.println("consuming records at: " + DateTime.now().toLocalDateTime());
System.out.println("consuming topic: " + consumerRecord.topic());
saveReserveStock.add(consumerRecord.value().toString());
saveReserveStock.add("dummy data");
saveReserveStock.forEach(System.out::println);
System.out.println("consumed at: " + DateTime.now().toLocalDateTime());
System.out.println("==================================================================");
System.out.println("end at: " + DateTime.now().toLocalDateTime());
}
public void emptyConsumer(){
saveReserveStock = new ArrayList<>();
}
}
and this is embedded kafka configuration
#Slf4j
#EnableKafka
public abstract class EmbeddedKafkaIntegrationTest {
#Autowired
protected static EmbeddedKafkaBroker embeddedKafkaBroker = new EmbeddedKafkaBroker(1, false);
#Autowired
protected KafkaConsumer kafkaConsumer;
#Autowired
private ReactorKafkaProducer reactorKafkaProducer;
protected abstract void setUp();
private static boolean started;
#BeforeClass
public static void createBroker(){
log.info("start test class");
Map<String, String> propertiesMap = new HashMap<>();
propertiesMap.put("listeners", "PLAINTEXT://localhost:9092");
embeddedKafkaBroker.brokerProperties(propertiesMap);
if (!started) {
try {
embeddedKafkaBroker.afterPropertiesSet();
log.info("before class - kafka connected to: "+embeddedKafkaBroker.getBrokersAsString());
}
catch (Exception e) {
log.error("Embedded broker failed to start", e);
}
started = true;
}
}
#Before
public void doSetUp() {
log.info("before - kafka connected to: "+embeddedKafkaBroker.getBrokersAsString());
kafkaConsumer.emptyConsumer();
this.setUp();
}
#After
public void tearDown() {
kafkaConsumer.emptyConsumer();
embeddedKafkaBroker.getZookeeper().getLogDir().deleteOnExit();
}
#AfterClass
public static void destroy(){
log.info("end test class");
}
}
then in my test class, using #Autowired for that KafkaConsumer class
and in the test class i have this to get message from the listener that already consumend
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {ImsStockApplication.class},
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#Slf4j
public class IntegrationTest extends EmbeddedKafkaIntegrationTest {
#Value("${local.server.port}")
private int port;
#Autowired
private KafkaConsumer kafkaConsumer;
#Autowired
private ReactorKafkaProducer reactorKafkaProducer;
#Before
public void setUp() {
RestAssured.port = port;
}
#Test
public void success_SubDetail() {
reactorKafkaProducer.send("topic", event).block();
Awaitility.await().atMost(10, TimeUnit.SECONDS).untilAsserted(() -> {
log.info("AWAITILITY AT: " + DateTime.now().toLocalDateTime());
Assert.assertTrue(kafkaConsumer.getFailDecreaseGoodsReceipt().size() > 0);
Assert.assertTrue(kafkaConsumer.getSaveReserveStock().size() > 0);
Assert.assertTrue(kafkaConsumer.getSaveBindStock().size() > 0);
});
}
}
but the result sometimes got failure (list empty)...
it's like the list variable is empty, while it should be not empty
below is the log where the listener receive the message and store it to the list
==================================================================
consuming records at: 2022-07-10T14:16:46.748
consuming topic: topic
{"id":9721,"eventId":"eventId","organizationCode":"ORG","createdDate":1657437282742,"lastModifiedDate":1657437282742,"routingId":"routingId"}
dummy data
consumed at: 2022-07-10T14:16:46.748
==================================================================
end at: 2022-07-10T14:16:46.748
and in my test class when i tried to access the variable, it got empty. it keep waiting for the list to be filled
AWAITILITY AT: 2022-07-10T14:16:46.829
AWAITILITY AT: 2022-07-10T14:16:46.945
AWAITILITY AT: 2022-07-10T14:16:47.056
AWAITILITY AT: 2022-07-10T14:16:47.164
AWAITILITY AT: 2022-07-10T14:16:47.273
AWAITILITY AT: 2022-07-10T14:16:47.384
AWAITILITY AT: 2022-07-10T14:16:47.490
AWAITILITY AT: 2022-07-10T14:16:47.598
if we looked at the timestamp, the list shouldn't be empty right? but why my test got failed?
Where did i go wrong?
Thanks
IMHO the question lacks information... So providing not a real answer but rather a series of possible answers that can help to find the solution.
There are many possible reasons for the test to fail and its not necessarily because of your test code.
One possible reason is that when you connect to kafka you start to listen to the "latest" messages (offset = latest) In this case you won't be able to consume the messages that are already in the topic.
While this can be the answer to the question really, but Maybe you can post the code that actually sends the message to the topic. And this is the real question here.
Another possible reason is the number of partitions. If the listener is configured to use the same consumer group as other listeners that might exist in the running application - maybe it doesn't get the partition that eventually receives the message
Its also possible that the reason is in the code itself, but again you don't show all the configuration at least the configuration of the test here.
An example of such a possible issue is that the #KafkaListener is not handled properly, so spring makes a spring bean from the component (after all it can be autowired from within the test) but doesn't plug in the whole kafka infrastructure under the hood.
I'm pretty new to webflux and I am struggling to understand how to test this Controller function.
public Mono<ResponseEntity<Void>> functionName(final Request request) {
RequestDto Dto = RequestMapper.
INSTANCE.toDto(request);
service.functionName(Dto);
return Mono.just(new ResponseEntity<Void>(HttpStatus.OK));
}
You could use WebTestClient that provides fluent API for verifying responses. Here is a short example
#WebFluxTest
class ControllerTest {
#Autowired
private WebTestClient client;
#BeforeEach
void setUp(ApplicationContext context) {
client = WebTestClient.bindToApplicationContext(context).build();
}
#Test
void test() {
client.get()
.uri("/test")
.exchange()
.expectStatus().isOk()
.expectBody().isEmpty();
}
}
In addition, pay attention to endpoint implementation. In reactive you need to build the flow and Webflux will subscribe to it for every request. In case, service.functionName is blocking (non-reactive), consider running it on a separate Scheduler using .subscribeOn(Schedulers.boundedElastic()). For details, check How Do I Wrap a Synchronous, Blocking Call?.
If the Callable resolves to null, the resulting Mono completes empty and we could return 404 applying switchIfEmpty operator.
#GetMapping(path = "/test")
public Mono<ResponseEntity<Void>> functionName(Request request) {
return Mono.fromCallable(() -> {
RequestDto Dto = RequestMapper.INSTANCE.toDto(request);
return service.functionName(Dto);
}).subscribeOn(Schedulers.boundedElastic())
.map(res -> new ResponseEntity<>(HttpStatus.OK))
.switchIfEmpty(Mono.just(new ResponseEntity<>(HttpStatus.NOT_FOUND)));
}
I'm new to Camel and I need to understand how to unit test my route that has two endpoints. The first endpoints gets a user ID and uses that for the second endpoint.
public RouteBuilder routeBuilder() {
return new RouteBuilder() {
#Override
public void configure() throws HttpOperationFailedException {
this.from(MyServiceConstant.ROUTE)
.setHeader(...)
.setHeader(...)
.to(MyConstants.THE_FIRST_ROUTE)
.setHeader(...)
.setHeader(...)
.process(...)
.setProperty(...)
.to(MyConstants.THE_SECOND_ROUTE)
}
};
}
So I have to mock both the MyConstants.THE_FIRST_ROUTE and MyConstants.THE_SECOND_ROUTE in my Test class. I did that but am not sure how to write the test. All I'm doing is hitting the second endpoint but don't know how to trigger the first.
#Produce(uri = MyServiceConstant.ROUTE)
private MyService myService;
#EndpointInject(uri = "mock:" + MyConstants.THE_FIRST_ROUTE)
private MockEndpoint mockFirstService;
#EndpointInject(uri = ""mock:" + MyConstants.THE_SECOND_ROUTE)
private MockEndpoint mockSecondService;
#Test
#DirtiesContext
public void getDetails()throws Exception {
// **The missing part**: Is this the right way to call my first service?
this.mockFirstService.setUserId("123456");
// this returns a JSON that I'll compare the service response to
this.mockSecondService.returnReplyBody(...PATH to JSON file);
UserDetail userDetailsInfo = this.myService.getUserDetails(...args)
// all of my assertions
assertEquals("First name", userDetailsInfo.getFirstName());
MockEndpoint.assertIsSatisfied();
}
I got some time today to quickly hack around some demo code, with Camel Spring boot archetype. Here we go. My route produces messages from a timer component. Explicit delivery to an endpoint is not used.
//Route Definition - myBean::saySomething() always returns String "Hello World"
#Component
public class MySpringBootRouter extends RouteBuilder {
#Override
public void configure() {
from("timer:hello?period={{timer.period}}").routeId("hello_route")
.transform().method("myBean", "saySomething")
.to("log:foo")
.setHeader("test_header",constant("test"))
.to("log:bar");
}
}
#RunWith(CamelSpringBootRunner.class)
#SpringBootTest
public class MySpringBootRouterTest {
#Autowired
SpringCamelContext defaultContext;
#EndpointInject("mock:foo")
private MockEndpoint mockFoo;
#EndpointInject("mock:bar")
private MockEndpoint mockBar;
#Test
#DirtiesContext
public void getDetails() throws Exception {
assertNotNull(defaultContext);
mockBar.expectedHeaderReceived("test_header", "test");
mockBar.expectedMinimumMessageCount(5);
MockEndpoint.setAssertPeriod(defaultContext, 5_000L);
MockEndpoint.assertIsSatisfied(mockFoo, mockBar);
mockFoo.getExchanges().stream().forEach( exchange -> assertEquals(exchange.getIn().getBody(),"Hello World"));
//This works too
//mockBar.assertIsSatisfied();
//mockFoo.assertIsSatisfied();
}
#Before
public void attachTestProbes() throws Exception {
//This is Camel 3.0 API with RouteReifier
RouteReifier.adviceWith(defaultContext.getRouteDefinition("hello_route"), defaultContext, new AdviceWithRouteBuilder() {
#Override
public void configure() throws Exception {
//Hook into the current route, intercept log endpoints and reroute them to mock
interceptSendToEndpoint("log:foo").to("mock:foo");
interceptSendToEndpoint("log:bar").to("mock:bar");
}
});
}
}
Warning to visitors from future: The test case here demonstrates how to intercept log: endpoints with mock: and set expectations on them. The test case may not be testing anything worthwhile.
Here is a link to the Unit Test cases for the Mock component. It shows how to implement tests with mock: endpoints and CamelTestSupport. #Roman Vottner is completely right in his comment.
This test case may be of specific interest to you since it shows how to swap an smtp: endpoint with a mock: endpoint. Additionally, here is official documentation on how to mock existing endpoints (To use them like test probes).
Caveat: Please bear in mind that Camel 3.0 API is quite different from Camel 2.x API, in this region. Good luck!
We are using Project Reactor to run a particular operation asynchronously as per the code in ServiceTest below. To test this code, as per ServiceTest below, when setting up the Mono for the async operation we make the Mono pass it's result to a DirectProcessor with doOnNext that the test has access to, and then carry out our test call and assertions with StepVerifier.
The JavaDoc of StepVerifier#assertNext reads
Any AssertionErrors thrown by the consumer will be rethrown during verification.
We have found that is true only when the immediate scheduler (Schedulers.immediate()) is used and is not true when the single scheduler (Schedulers.single()) is used. When the single scheduler is used, AssertionErrors are not re-thrown, i.e. the test always passes.
Is it possible, and if so, how, to use the single scheduler and have AssertionErrors rethrown during verification as per the JavaDoc?
#Service
#RequiredArgsConstructor
public class Service implements WithReactive, WithTestProcessor<Response> {
#Getter
#Setter
private DirectProcessor<Response> processor = DirectProcessor.create();
#Setter
private Scheduler scheduler = Schedulers.single();
public void doAction() {
Mono.fromSupplier(this::doActionAsync)
.doOnNext(processor::onNext)
.subscribeOn(scheduler)
.subscribe();
}
private Response doActionAsync() {
...
}
...
}
public interface WithReactive {
void setScheduler(Scheduler scheduler);
}
public interface WithTestProcessor<T> {
void setProcessor(DirectProcessor<T> processor);
DirectProcessor<T> getProcessor();
}
#RunWith(SpringRunner.class)
#SpringBootTest
public class ServiceTest {
#Inject
private Collection<WithTestProcessor> withTestProcessors;
#Before
public void setTestProcessors() {
withTestProcessors.forEach(withTestProcessor -> withTestProcessor.setProcessor(DirectProcessor.create()));
}
#Inject
private Collection<WithReactive> withReactives;
#Before
public void makeReactiveSynchronous() {
withReactives.forEach(withReactive -> withReactive.setScheduler(Schedulers.immediate()));
}
#Test
private void test() {
StepVerifier.create(service.getProcessor())
.then(service::doAction)
.assertNext(response -> assertThat(logExtractor.getInsertsByTable("assets")).hasSize(1))
.thenCancel()
.verify();
}
}
This is a combination of three factors: the initial then, the fact that subscription happens in parallel of the verification due to subscribeOn and the thenCancel.
One workaround is to give enough time to the onNext to happen before the StepVerifier executes thenCancel, by putting a thenAwait(Duration.ofMillis(10)) before the thenCancel.