How to Mock a Kafka Consumer endpoint on a Camel Route? - java

I have a Camel endpoint which is basically a Kafka Consumer reading from a topic and sending the information to a database. It is working fine, however, I am struggling to unit test it as I haven't been able to mock the Kafka endpoint. Can anyone help me in mocking a Kafka Consumer in a Camel Route?
#Override
public void configure() {
from(kafka:eph?brokers=localhost:9092...).routeId("KafkaConsumer")
.to(direct:updateDatabase)
}

To unit test your route, you may do that with a standard camel spring boot test.
During the test, the Kafka producer(in Camel's view) can be swapped in with a direct component and mock messages can be delivered there. To see if your routes are processing those messages properly, Mock endpoints can be used.
//Route definition
#Component
public class KafkaRoute extends RouteBuilder {
public static final String KAFKA_ROUTE_NAME = "kafka-route";
#Override
public void configure() throws Exception {
from("kafka:eph?brokers=localhost:9092").routeId(KAFKA_ROUTE_NAME)
.log(LoggingLevel.INFO, "Message: ${body} received on the topic: ${headers[kafka.TOPIC]} ")
.to("direct:updateDatabase");
from("direct:updateDatabase").log(LoggingLevel.INFO, "DB Updated.");
}
}
import java.util.HashMap;
import java.util.Map;
import org.apache.camel.CamelContext;
import org.apache.camel.EndpointInject;
import org.apache.camel.Produce;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.builder.AdviceWithRouteBuilder;
import org.apache.camel.component.kafka.KafkaConstants;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.test.spring.CamelSpringBootRunner;
import org.apache.camel.test.spring.MockEndpoints;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.annotation.DirtiesContext;
#RunWith(CamelSpringBootRunner.class)
#SpringBootTest
#DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)
#MockEndpoints("direct:*")
public class KafkaRouteTest {
#Autowired
CamelContext camelContext;
#Produce
ProducerTemplate mockKafkaProducer;
#EndpointInject("mock:direct:updateDatabase")
MockEndpoint finalSink;
#Test
public void testKafkaRoute() throws Exception {
//Here we swap the FROM component in the KafkaRoute.KAFKA_ROUTE_NAME with a direct component, direct:kafka-from
AdviceWithRouteBuilder.adviceWith(camelContext, KafkaRoute.KAFKA_ROUTE_NAME, routeBuilder -> {
routeBuilder.replaceFromWith("direct:kafka-from");
});
Map<String, Object> headers = new HashMap<>();
headers.put(KafkaConstants.TOPIC, "testTopic");
//Send mock message to the route
mockKafkaProducer.sendBodyAndHeaders("direct:kafka-from", "test-body", headers);
//Assertions. You may do additional assertions with the likes of Mockito
finalSink.expectedBodiesReceived("test-body");
finalSink.expectedHeaderReceived(KafkaConstants.TOPIC, "testTopic");
finalSink.assertIsSatisfied();
}
}
Camel Kafka component is already unit tested, there is no point in replicating all those tests in your code base. However, if you really want to do testing against a real Kafka instance, you may use test containers. Here is a full blown example, from the Camel repository itself, using test containers.

Simply externalize the endpoint URI in a property (for example with Spring Property facility)
from(consumerEndpoint).routeId("KafkaConsumer")
Then in your production configuration, you use the real endpoint
consumerEndpoint=kafka:eph?brokers=localhost:9092...
Whereas in your test configuration, you use a direct endpoint
consumerEndpoint=direct:consumer
This one is easy to trigger from a Camel route test
producer.sendBody("direct:consumer", myMessageBody);

Related

How to clear/ignore ehcache's data and test via MockRestServiceServer?

We are running our component test-cases in which we load some data using cache.. Now the problem is when we try other test-cases, we want to reset the cache because it then doesn't test with the other data. How can we achieve this. We are using spring boot with Java and using Ehcache.
You can inject the org.springframework.cache.CacheManager bean into your tests and use it to clear the cache before or after each test. Assuming there is a cache named testCache, a test class that clears the cache would look something like below:
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.cache.CacheManager;
#SpringBootTest
public class IntegrationTest {
#Autowired
private CacheManager cacheManager;
#BeforeEach
public void setup() {
cacheManager.get("testCache").clear();
}
#Test
public void testSomething() {
}
}
You can find a reference spock-based test that does the same thing on github

Using GCP Secret Manager in integration tests in Java

I have a pretty big code base written in Java. I have a lot of integration tests with both Kafka and Bigtable using JUnits ExternalResource. I have introduced fetching of secrets from GCP Secret Manager in my code. I now want to write integration tests for that as well.
So my scenario is that I want to create a mock username/password, create a secret of that username/password in my mock GCP Secret Manager, access the secrets and then use it to connect to my mock-service that requires that username/password. So, in reality, I'm connecting to a Kafka broker in my test with SSL and I want to simulate the entire flow with fetching of secrets.
The problem is, I can't find any documentation on how to do it. Google has great other documentation on how to emulate Bigtable, but I can't find any documentation on how to emulate/mock a Secret Manager. Has anyone ran into something similar?
In case you are using Spring Boot you may mock SecretManagerOperations interface using #TestConfiguration as follows:
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Primary;
import org.springframework.test.web.servlet.MockMvc;
import com.google.cloud.spring.secretmanager.SecretManagerOperations;
#AutoConfigureMockMvc
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class MySecretIntegrationTest {
#Autowired
private MockMvc mockMvc;
#Test
public void testGettingSecretResource() throws Exception {
mockMvc.perform(get("/api/resource")).andExpect(status().isOk());
}
#TestConfiguration
public static class SecretManagerConfiguration {
#Bean
#Primary
public SecretManagerOperations secretManagerProducer() throws Exception{
SecretManagerOperations secretManager = mock(SecretManagerOperations.class);
when(secretManager.getSecretString("secret_id")).thenReturn("top secret");
return secretManager;
}
}
}
Annotate the producer method with #Primary so that the mocked bean gets autowired during the test instead of the default implementation SecretManagerTemplate.
"GCP doesn't have any Emulator for Secret Manager" - #Guillaume Blaquiere.

Spring Integration MessageDeliveryException

I am trying to test my Spring Integration setup, but am receiving
"MessageDeliveryException: Dispatcher has no subscribers for channel".
I'm using a QueueChannel and I don't think it needs a handler (from what I can see in the docs).
I'm using Spring Integration Java DSL to define the integration flow programmatically, rather than using a context.
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.integration.channel.QueueChannel;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.dsl.channel.MessageChannels;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
#RunWith(SpringJUnit4ClassRunner.class)
public class SimplifiedIntegrationTest {
#Test
public void simpleTest() {
MessageChannel inputChannel = MessageChannels.direct().get();
QueueChannel outputChannel = MessageChannels.queue().get();
IntegrationFlows.from(inputChannel).channel(outputChannel).get();
inputChannel.send(MessageBuilder.withPayload("payload").build());
Message<?> outMessage = outputChannel.receive(0);
Assert.notNull(outMessage);
}
}
The IntegrationFlow must always be registered in the context as a bean. I’m not sure what is the source making you think differently, but since you don’t register it as a bean in context, there is no that flow configuration magic done by the particular BeanPostProcessor. And that’s why you get that "Dispatcher has no subscribers for channel” error.
See IntegrationFlowContext if you register your Integration Flows manually: https://docs.spring.io/spring-integration/docs/5.0.0.BUILD-SNAPSHOT/reference/html/java-dsl.html#java-dsl-runtime-flows.
That doc is for Spring Integration 5.0, but IntegrationFlowContext behaves the same way in Java DSL 1.2.x.

Spring Boot - infinite loop service

I want to build a headless application which will query the DB in infinite loop and perform some operations in certain conditions (e.g. fetch records with specific values and when found launch e-mail sending procedure for each message).
I want to use Spring Boot as a base (especially because of Actuator to allow expose health-checks), but for now I used Spring Boot for building REST web-services.
Is there any best practices or patterns to follow when building infinite loop applications ? Does anyone tried to build it based on Spring Boot and can share with me his architecture for this case ?
Best regards.
Do not implement an infinite loop yourself. Let the framework handle it using its task execution capabilities:
#Service
public class RecordChecker{
//Executes each 500 ms
#Scheduled(fixedRate=500)
public void checkRecords() {
//Check states and send mails
}
}
Don't forget to enable scheduling for your application:
#SpringBootApplication
#EnableScheduling
public class Application {
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class);
}
}
See also:
Scheduling Tasks
What I'm using is a message broker and a consumer put at the spring boot application to do the job.
There are several options. My approach is to start a loop on an ApplicationReadyEvent, and abstract away the loop logic into an injectable service. In my case it was a game loop, but this pattern should work for you as well.
package com.ryanp102694.gameserver;
import com.ryanp102694.gameserver.service.GameProcessor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;
#Component
public class GameLauncher implements ApplicationListener<ApplicationReadyEvent> {
private static Logger logger = LoggerFactory.getLogger(GameLauncher.class);
private GameProcessor gameProcessor;
#Autowired
public GameLauncher(GameProcessor gameProcessor){
this.gameProcessor = gameProcessor;
}
#Override
public void onApplicationEvent(ApplicationReadyEvent event) {
logger.info("Starting game process.");
gameProcessor.start();
while(gameProcessor.isRunning()){
logger.debug("Collecting user input.");
gameProcessor.collectInput();
logger.debug("Calculating next game state.");
gameProcessor.nextGameState();
logger.debug("Updating clients.");
gameProcessor.updateClients();
}
logger.info("Stopping game process.");
gameProcessor.stop();
}
}

Mocking expected result from service using spring testing framework

I want to write a test case for Restful web service using Spring testing framework. I mocked the service and able to run the testcase successfully.
But, as service is mocked, it is returning the empty response. So, I want to set the expected output from the service.
I can achieve it using different mocking frameworks like Mockito or Jmockit (In below code it is with Mockito).
But, is it possible without any addition/external testing frameworks apart from internal Spring testing framework.
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mockito;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.web.WebAppConfiguration;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import org.springframework.web.context.WebApplicationContext;
import java.util.Arrays;
import static org.hamcrest.Matchers.*;
import static org.hamcrest.Matchers.is;
import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.model;
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = {TestContext.class, WebAppContext.class})
#WebAppConfiguration
public class TodoControllerTest {
private MockMvc mockMvc;
#Autowired
private TodoService todoServiceMock;
#Test
public void findAll_ShouldAddTodoEntriesToModelAndRenderTodoListView() throws Exception {
Todo first = new TodoBuilder()
.id(2L)
.description("Lorem ipsum")
.title("Bar")
.build();
/**
Need mocking technique from Spring Testing Framework
*/
when(todoServiceMock.findAll()).thenReturn(Arrays.asList(first));
mockMvc.perform(get("/"))
.andExpect(status().isOk())
.andExpect(view().name("todo/list"))
.andExpect(forwardedUrl("/WEB-INF/jsp/todo/list.jsp"))
.andExpect(model().attribute("todos", hasSize(2)))
.andExpect(model().attribute("todos", hasItem(
allOf(
hasProperty("id", is(1L)),
hasProperty("description", is("Lorem ipsum")),
hasProperty("title", is("Foo"))
)
)))
.andExpect(model().attribute("todos", hasItem(
allOf(
hasProperty("id", is(2L)),
hasProperty("description", is("Lorem ipsum")),
hasProperty("title", is("Bar"))
)
)));
}
}
One possible solution is to extend a class that you want to test and override methods that you want to mock up. Then in a separate configuration file you define a bean which would be wired in place of a real object. Next use this configuration in your test class.
This is actually Mockito spy's behaviour. If I were you I would stick to it as it provides more flexibility and saves a lot of boilerplate code.
You aren't showing the relevant part of your configuration, but it should be fully possible. In your TodoService you should have injected your data layer dependencies using their interfaces as per Spring best practices, and those dependencies can be replaced (using Spring config) with dummy/stub classes implementing those interfaces that provide the required test data.

Categories

Resources