I am trying to get the Non-Blocking response from the Micronaut kafka implementation, however the return value in not working.
public class ProductManager implements IProductManager{
private final ApplicationContext applicationContext;
public ProductManager(ApplicationContext applicationContext) {
this.applicationContext = applicationContext;
}
#Override
public ProductViewModel findFreeText(String text) {
final ProductViewModel model = new ProductViewModel();
IProductProducer client = applicationContext.getBean(IProductProducer.class);
client.findFreeText(text).subscribe(item -> {
System.out.println(item);
});
return model;
}
}
The subscribe method is not working, the debugger never comes to this point. I want to get the value back from the kafka listener
kafka producer
#KafkaClient
public interface IProductProducer {
#Topic(ProductTopicConstants.GET_FREE_TEXT_SEARCH)
Flowable<ProductViewModel> findFreeText(String text);
}
Kafka Listener
#KafkaListener(offsetReset = OffsetReset.EARLIEST)
public class ProductListener {
private static final Logger LOG = LoggerFactory.getLogger(ProductListener.class);
#Topic(ProductTopicConstants.GET_FREE_TEXT_SEARCH)
public Flowable<Product>> findByFreeText(String text) {
LOG.info("Listening value = ", text);
return Flowable.just(new Product("This is the test", 0,"This is test description"));
}
}
Micronaut documentation for non-blocking method
https://docs.micronaut.io/1.0.0.M3/guide/index.html#_reactive_and_non_blocking_method_definitions
Related
I am trying to move away from now deprecated annotations like #EnableBinding and #Output but could not find a simple example to do it in a functional way. These are the files currently:
KafkaConfig.java
#Configuration
#EnableBinding({
CcRegistrationFailureChannel.class
})
public class KafkaConfig {
}
CcRegistrationFailureChannel.java
public interface CcRegistrationFailureChannel {
String CC_REGISTRATION = "cc-registration";
#Output(CC_REGISTRATION)
MessageChannel ccFailureChannel();
}
CcRegistrationFailurePublisher.java
#Log4j2
#Component
public class CcRegistrationFailurePublisher {
public void publish(MessageChannel outputChannel, EventPayload payLoad) {
boolean success = outputChannel.send(MessageBuilder
.withPayload(payLoad)
.setHeader(MessageHeaders.CONTENT_TYPE, MimeTypeUtils.APPLICATION_JSON)
.build()
);
if (!success) {
log.error("CC Registration Failure: publish message failed");
}
}
}
and message publish is invoked from following code:
#Autowired
private final CcRegistrationFailurePublisher ccRegistrationFailurePublisher;
public void sendCCRegistrationFailure(String internalUserId) {
Long unixEpochTime = Instant.now().getEpochSecond();
CcRegistrationFailureEventPayload ccRegistrationFailureEventPayload =
new CcRegistrationFailureEventPayload(internalUserId, unixEpochTime, CcFailureEventType.ADD_CC_FAILURE);
ccRegistrationFailurePublisher.publish(ccRegistrationFailureChannel.ccFailureChannel(), ccRegistrationFailureEventPayload);
}
How can I migrate from the current state to using functional way recommended by Spring?
Use a StreamBridge https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream.html#_sending_arbitrary_data_to_an_output_e_g_foreign_event_driven_sources
bridge.send(bindingName, message);
I am building a graphql application with spring-boot-starter-webflux 2.5.6 and com.graphql-java-kickstart:graphql-spring-boot-starter:12.0.0.
At this point the application is running fine since com.graphql-java-kickstart is easy to start with.
With http-Requests I can call Queries and run Mutations and I am even able to create and get updates via Subscriptions over websockets.
But for my application Queries and Mutations also have to run via websocket.
It seems that in com.graphql-java-kickstart:graphql-spring-boot-starter you can only configure a subscription endpoint as websocket.
Adding an additional websocket via 'extends Endpoint' and '#ServerEndpoint' did nothing at all.
I also tried to add my own HandlerMapping:
#PostConstruct
public void init()
{
Map<String, Object> map = new HashMap<String, Object>(
((SimpleUrlHandlerMapping) webSocketHandlerMapping).getUrlMap());
map.put("/mysocket", myWebSocketHandler);
//map.put("/graphql", myWebSocketHandler);
((SimpleUrlHandlerMapping) webSocketHandlerMapping).setUrlMap(map);
((SimpleUrlHandlerMapping) webSocketHandlerMapping).initApplicationContext();
}
This seems to work with the /mysocket Topic but how do I enable it for /graphql, it seems like there is already a handler listening on:
WARN 12168 --- [ctor-http-nio-2] notprivacysafe.graphql.GraphQL : Query failed to parse : ''
And how to connect the websocket with my GraphQLMutationResolvers?
My entry point to a solution for this problem was to create a RestController and connect the ServerWebExchange to a WebSocketHandler in the WebSocketService like this:
#RestController
#RequestMapping("/")
public class WebSocketController
{
private static final Logger logger = LoggerFactory.getLogger(WebSocketController.class);
private final GraphQLObjectMapper objectMapper;
private final GraphQLInvoker graphQLInvoker;
private final GraphQLSpringInvocationInputFactory invocationInputFactory;
private final WebSocketService service;
#Autowired
public WebSocketController(GraphQLObjectMapper objectMapper, GraphQLInvoker graphQLInvoker,
GraphQLSpringInvocationInputFactory invocationInputFactory, WebSocketService service)
{
this.objectMapper = objectMapper;
this.graphQLInvoker = graphQLInvoker;
this.invocationInputFactory = invocationInputFactory;
this.service = service;
}
#GetMapping("${graphql.websocket.path:graphql-ws}")
public Mono<Void> getMono(ServerWebExchange exchange)
{
logger.debug("New connection via GET");
return service.handleRequest(exchange,
new GraphQLWebsocketMessageConsumer(exchange, objectMapper, graphQLInvoker, invocationInputFactory));
}
#PostMapping("${graphql.websocket.path:graphql-ws}")
public Mono<Void> postMono(ServerWebExchange exchange)
{
...
}
}
In this prototype state the WebSocketHandler is also implementing the Consumer which is called to handle each WebSocketMessage:
public class GraphQLWebsocketMessageConsumer implements Consumer<String>, WebSocketHandler
{
private static final Logger logger = LoggerFactory.getLogger(GraphQLWebsocketMessageConsumer.class);
private final ServerWebExchange swe;
private final GraphQLObjectMapper objectMapper;
private final GraphQLInvoker graphQLInvoker;
private final GraphQLSpringInvocationInputFactory invocationInputFactory;
private final Sinks.Many<String> publisher;
public GraphQLWebsocketMessageConsumer(ServerWebExchange swe, GraphQLObjectMapper objectMapper,
GraphQLInvoker graphQLInvoker, GraphQLSpringInvocationInputFactory invocationInputFactory)
{
...
publisher = Sinks.many().multicast().directBestEffort();
}
#Override
public Mono<Void> handle(WebSocketSession webSocketSession)
{
Mono<Void> input = webSocketSession.receive().map(WebSocketMessage::getPayloadAsText).doOnNext(this).then();
Mono<Void> sender = webSocketSession.send(publisher.asFlux().map(webSocketSession::textMessage));
return Mono.zip(input, sender).then();
}
#Override
public void accept(String body)
{
try
{
String query = extractQuery(body);
if(query == null)
{
return;
}
GraphQLRequest request = objectMapper.readGraphQLRequest(query);
GraphQLSingleInvocationInput invocationInput = invocationInputFactory.create(request, swe);
Mono<ExecutionResult> executionResult = Mono.fromCompletionStage(graphQLInvoker.executeAsync(invocationInput));
Mono<String> jsonResult = executionResult.map(objectMapper::serializeResultAsJson);
jsonResult.subscribe(publisher::tryEmitNext);
} catch (Exception e)
{
...
}
}
#SuppressWarnings("unchecked")
private String extractQuery(final String query) throws Exception
{
Map<String, Object> map = (Map<String, Object>) objectMapper.getJacksonMapper().readValue(query, Map.class);
...
return queryPart;
}
#Override
public List<String> getSubProtocols()
{
logger.debug("getSubProtocols called");
return Collections.singletonList("graphql-ws");
}
}
This solution does not yet touch security aspects like authentication or session handling.
The following class in included into several consumer applications:
#Component
#Configuration
public class HealthListener {
public static final String HEALTH_CHECK_QUEUE_NAME = "healthCheckQueue";
public static final String HEALTH_CHECK_FANOUT_EXCHANGE_NAME = "health-check-fanout";
#Bean
public Binding healthListenerBinding(
#Qualifier("healthCheckQueue") Queue queue,
#Qualifier("instanceFanoutExchange") FanoutExchange exchange) {
return BindingBuilder.bind(queue).to(exchange);
}
#Bean
public FanoutExchange instanceFanoutExchange() {
return new FanoutExchange(HEALTH_CHECK_FANOUT_EXCHANGE_NAME, true, false);
}
#Bean
public Queue healthCheckQueue() {
return new Queue(HEALTH_CHECK_QUEUE_NAME);
}
#RabbitListener(queues = HEALTH_CHECK_QUEUE_NAME)
public String healthCheck() {
return "some result";
}
}
I'm trying to send a message to fanout exchange, and receive all replies, to know which consumers are running.
I can send a message and get the first reply like this:
#Autowired
RabbitTemplate template;
// ...
String firstReply = template.convertSendAndReceiveAsType("health-check-fanout", "", "", ParameterizedTypeReference.forType(String.class));
However I need to get all repliest to this message, not just the first one. I need to set up a reply listener, but I'm not sure how.
The (convertS|s)endAndReceive.*() methods are not designed to handle multiple replies; they are strictly one request/one reply methods.
You would need to use a (convertAndS|s)end() method to send the request, and implement your own reply mechanism, perhaps using a listener container for the replies, together with some component to aggregate the replies.
You could use something like a Spring Integration Aggregator for that, but you would need some mechanism (ReleaseStrategy) that would know when all expected replies are received.
Or you can simply receive the discrete replies and handle them individually.
EDIT
#SpringBootApplication
public class So54207780Application {
public static void main(String[] args) {
SpringApplication.run(So54207780Application.class, args);
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> template.convertAndSend("fanout", "", "foo", m -> {
m.getMessageProperties().setReplyTo("replies");
return m;
});
}
#RabbitListener(queues = "queue1")
public String listen1(String in) {
return in.toUpperCase();
}
#RabbitListener(queues = "queue2")
public String listen2(String in) {
return in + in;
}
#RabbitListener(queues = "replies")
public void replyHandler(String reply) {
System.out.println(reply);
}
#Bean
public FanoutExchange fanout() {
return new FanoutExchange("fanout");
}
#Bean
public Queue queue1() {
return new Queue("queue1");
}
#Bean
public Binding binding1() {
return BindingBuilder.bind(queue1()).to(fanout());
}
#Bean
public Queue queue2() {
return new Queue("queue2");
}
#Bean
public Binding binding2() {
return BindingBuilder.bind(queue2()).to(fanout());
}
#Bean
public Queue replies() {
return new Queue("replies");
}
}
and
FOO
foofoo
I am currently working on Kafka module where I am using spring-kafka abstraction of Kafka communication. I am able to integrate the producer & consumer from real implementation standpoint however, I am not sure how to test (specifically integration test) the business logic surrounds at consumer with #KafkaListener. I tried to follow spring-kafk documentation and various blogs on the topic but none of those answer my intended question.
Spring Boot test class
//imports not mentioned due to brevity
#RunWith(SpringRunner.class)
#SpringBootTest(classes = PaymentAccountUpdaterApplication.class,
webEnvironment = SpringBootTest.WebEnvironment.NONE)
public class CardUpdaterMessagingIntegrationTest {
private final static String cardUpdateTopic = "TP.PRF.CARDEVENTS";
#Autowired
private ObjectMapper objectMapper;
#ClassRule
public static KafkaEmbedded kafkaEmbedded =
new KafkaEmbedded(1, false, cardUpdateTopic);
#Test
public void sampleTest() throws Exception {
Map<String, Object> consumerConfig =
KafkaTestUtils.consumerProps("test", "false", kafkaEmbedded);
consumerConfig.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
consumerConfig.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
ConsumerFactory<String, String> cf = new DefaultKafkaConsumerFactory<>(consumerConfig);
ContainerProperties containerProperties = new ContainerProperties(cardUpdateTopic);
containerProperties.setMessageListener(new SafeStringJsonMessageConverter());
KafkaMessageListenerContainer<String, String>
container = new KafkaMessageListenerContainer<>(cf, containerProperties);
BlockingQueue<ConsumerRecord<String, String>> records = new LinkedBlockingQueue<>();
container.setupMessageListener((MessageListener<String, String>) data -> {
System.out.println("Added to Queue: "+ data);
records.add(data);
});
container.setBeanName("templateTests");
container.start();
ContainerTestUtils.waitForAssignment(container, kafkaEmbedded.getPartitionsPerTopic());
Map<String, Object> producerConfig = KafkaTestUtils.senderProps(kafkaEmbedded.getBrokersAsString());
producerConfig.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
producerConfig.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
ProducerFactory<String, Object> pf =
new DefaultKafkaProducerFactory<>(producerConfig);
KafkaTemplate<String, Object> kafkaTemplate = new KafkaTemplate<>(pf);
String payload = objectMapper.writeValueAsString(accountWrapper());
kafkaTemplate.send(cardUpdateTopic, 0, payload);
ConsumerRecord<String, String> received = records.poll(10, TimeUnit.SECONDS);
assertThat(received).has(partition(0));
}
#After
public void after() {
kafkaEmbedded.after();
}
private AccountWrapper accountWrapper() {
return AccountWrapper.builder()
.eventSource("PROFILE")
.eventName("INITIAL_LOAD_CARD")
.eventTime(LocalDateTime.now().toString())
.eventID("8730c547-02bd-45c0-857b-d90f859e886c")
.details(AccountDetail.builder()
.customerId("idArZ_K2IgE86DcPhv-uZw")
.vaultId("912A60928AD04F69F3877D5B422327EE")
.expiryDate("122019")
.build())
.build();
}
}
Listener Class
#Service
public class ConsumerMessageListener {
private static final Logger LOGGER = LoggerFactory.getLogger(ConsumerMessageListener.class);
private ConsumerMessageProcessorService consumerMessageProcessorService;
public ConsumerMessageListener(ConsumerMessageProcessorService consumerMessageProcessorService) {
this.consumerMessageProcessorService = consumerMessageProcessorService;
}
#KafkaListener(id = "cardUpdateEventListener",
topics = "${kafka.consumer.cardupdates.topic}",
containerFactory = "kafkaJsonListenerContainerFactory")
public void processIncomingMessage(Payload<AccountWrapper,Object> payloadContainer,
Acknowledgment acknowledgment,
#Header(KafkaHeaders.RECEIVED_TOPIC) String topic,
#Header(KafkaHeaders.RECEIVED_PARTITION_ID) String partitionId,
#Header(KafkaHeaders.OFFSET) String offset) {
try {
// business logic to process the message
consumerMessageProcessorService.processIncomingMessage(payloadContainer);
} catch (Exception e) {
LOGGER.error("Unhandled exception in card event message consumer. Discarding offset commit." +
"message:: {}, details:: {}", e.getMessage(), messageMetadataInfo);
throw e;
}
acknowledgment.acknowledge();
}
}
My question is: In the test class I am asserting the partition, payload etc which is polling from BlockingQueue, however, my question is how can I verify that my business logic in the class annotated with #KafkaListener is getting executed properly and routing the messages to different topic based on error handling and other business scenarios. In some of the examples, I saw CountDownLatch to assert which I don't want to put in my business logic to assert in a production grade code. Also the message processor is Async so, how to assert the execution, not sure.
Any help, appreciated.
is getting executed properly and routing the messages to different topic based on error handling and other business scenarios.
An integration test can consume from that "different" topic to assert that the listener processed it as expected.
You could also add a BeanPostProcessor to your test case and wrap the ConsumerMessageListener bean in a proxy to verify the input arguments are as expected.
EDIT
Here is an example of wrapping the listener in a proxy...
#SpringBootApplication
public class So53678801Application {
public static void main(String[] args) {
SpringApplication.run(So53678801Application.class, args);
}
#Bean
public MessageConverter converter() {
return new StringJsonMessageConverter();
}
public static class Foo {
private String bar;
public Foo() {
super();
}
public Foo(String bar) {
this.bar = bar;
}
public String getBar() {
return this.bar;
}
public void setBar(String bar) {
this.bar = bar;
}
#Override
public String toString() {
return "Foo [bar=" + this.bar + "]";
}
}
}
#Component
class Listener {
#KafkaListener(id = "so53678801", topics = "so53678801")
public void processIncomingMessage(Foo payload,
Acknowledgment acknowledgment,
#Header(KafkaHeaders.RECEIVED_TOPIC) String topic,
#Header(KafkaHeaders.RECEIVED_PARTITION_ID) String partitionId,
#Header(KafkaHeaders.OFFSET) String offset) {
System.out.println(payload);
// ...
acknowledgment.acknowledge();
}
}
and
spring.kafka.consumer.enable-auto-commit=false
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.listener.ack-mode=manual
and
#RunWith(SpringRunner.class)
#SpringBootTest(classes = { So53678801Application.class,
So53678801ApplicationTests.TestConfig.class})
public class So53678801ApplicationTests {
#ClassRule
public static EmbeddedKafkaRule embededKafka = new EmbeddedKafkaRule(1, false, "so53678801");
#BeforeClass
public static void setup() {
System.setProperty("spring.kafka.bootstrap-servers",
embededKafka.getEmbeddedKafka().getBrokersAsString());
}
#Autowired
private KafkaTemplate<String, String> template;
#Autowired
private ListenerWrapper wrapper;
#Test
public void test() throws Exception {
this.template.send("so53678801", "{\"bar\":\"baz\"}");
assertThat(this.wrapper.latch.await(10, TimeUnit.SECONDS)).isTrue();
assertThat(this.wrapper.argsReceived[0]).isInstanceOf(Foo.class);
assertThat(((Foo) this.wrapper.argsReceived[0]).getBar()).isEqualTo("baz");
assertThat(this.wrapper.ackCalled).isTrue();
}
#Configuration
public static class TestConfig {
#Bean
public static ListenerWrapper bpp() { // BPPs have to be static
return new ListenerWrapper();
}
}
public static class ListenerWrapper implements BeanPostProcessor, Ordered {
private final CountDownLatch latch = new CountDownLatch(1);
private Object[] argsReceived;
private boolean ackCalled;
#Override
public int getOrder() {
return Ordered.HIGHEST_PRECEDENCE;
}
#Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
if (bean instanceof Listener) {
ProxyFactory pf = new ProxyFactory(bean);
pf.setProxyTargetClass(true); // unless the listener is on an interface
pf.addAdvice(interceptor());
return pf.getProxy();
}
return bean;
}
private MethodInterceptor interceptor() {
return invocation -> {
if (invocation.getMethod().getName().equals("processIncomingMessage")) {
Object[] args = invocation.getArguments();
this.argsReceived = Arrays.copyOf(args, args.length);
Acknowledgment ack = (Acknowledgment) args[1];
args[1] = (Acknowledgment) () -> {
this.ackCalled = true;
ack.acknowledge();
};
try {
return invocation.proceed();
}
finally {
this.latch.countDown();
}
}
else {
return invocation.proceed();
}
};
}
}
}
I am beginner with spring - boot and webservice, I have two exercises to do, they would know How to implement the client and the run () method; to send and receive strings trough In this webservice?
PROJECT CONSUMER
public class Service implements Runnable {
public static void main(String args[]) {
System.out.println("Send the messages....");
Thread thread = new Thread(new Service());
thread.start();
}
public void run() {
// Loop receiving messages
}
}
PROJECT PRODUCER
#Path("/greet")
#Component
public class GreetResource {
static Logger logger = LoggerFactory.getLogger(GreetResource.class);
#Autowired
Client client;
public GreetResource() {
logger.info("Resource Initialized");
}
#GET
#Path("/echo/{msg}")
#Produces({ MediaType.TEXT_PLAIN_VALUE })
public Response echo(#PathParam("msg") String message) {
return Response.ok().entity(message).build();
}
#POST
#Path("/send")
#Consumes({ MediaType.TEXT_PLAIN_VALUE })
public boolean sendMessage(String greeting) {
client.sendAMessage(greeting);
return true;
}
}
PROJECT PRODUCER
#EnableAutoConfiguration
public class Application {
private static final Logger logger = LoggerFactory.getLogger(Application.class);
#Autowired
private Environment env;
#PostConstruct
public void initApplication() throws IOException {
if (env.getActiveProfiles().length == 0) {
logger.warn("No Spring profile configured, running with default configuration");
} else {
logger.info("Running with Spring profile(s) : {}", Arrays.toString(env.getActiveProfiles()));
Collection<String> activeProfiles = Arrays.asList(env.getActiveProfiles());
if (activeProfiles.contains("dev") && activeProfiles.contains("prod")) {
logger.error("You have misconfigured your application! "
+ "It should not run with both the 'dev' and 'prod' profiles at the same time.");
}
}
}
public static void main(String[] args) {
logger.info("weather application service starting...");
SpringApplication.run(Application.class, args);
}
}
PROJECT PRODUCER
import org.springframework.stereotype.Component;
#Component
public class Client {
public boolean sendAMessage(String greeting) {
// Send the message
return false;
}
}
Any tips to implement method run () and sendAMessage()?
If you are using Spring Boot and want to implement web-service , then easiest way is to implement web service using RESTapi.
You can use #ResponseBody or #RestController annotations to expose service.