I have 2 Spring Boot microservices. Microservice (B) calls a reactive api exposed by Microservice (A).
Microservice (A) RestController code :
#RestController
#RequestMapping(value = "/documents")
public class ElasticDocumentController {
private static final Logger LOG = LoggerFactory.getLogger(ElasticDocumentController.class);
private final ElasticQueryService elasticQueryService;
public ElasticDocumentController(ElasticQueryService queryService) {
this.elasticQueryService = queryService;
}
#GetMapping(value = "/", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ElasticQueryServiceResponseModel> getAllDocuments() {
Flux<ElasticQueryServiceResponseModel> response = elasticQueryService.getAllDocuments();
response = response.log();
LOG.info("Returning from query reactive service for all documents");
return response;
}
}
When I call the getAllDocuments() api from Postman, I can see the documents scrolling in the output cosole. So Microservice (A) is correct.
But when I call the api from Microservice (B), I cannot retrieve any documents. Microservice (B) cannot communicate with Microservice (A).
Microservice (B) Service code :
#Service
public class TwitterElasticQueryWebClient implements ElasticQueryWebClient {
private static final Logger LOG = LoggerFactory.getLogger(TwitterElasticQueryWebClient.class);
private final WebClient.Builder webClientBuilder;
private final ElasticQueryWebClientConfigData elasticQueryWebClientConfigData;
public TwitterElasticQueryWebClient(
#Qualifier("webClientBuilder") WebClient.Builder clientBuilder,
ElasticQueryWebClientConfigData configData
) {
this.webClientBuilder = clientBuilder;
this.elasticQueryWebClientConfigData = configData;
}
#Override
public Flux<ElasticQueryWebClientResponseModel> getAllData() {
LOG.info("Querying all data");
return webClientBuilder
.build()
.get()
.uri("/")
.accept(MediaType.valueOf(elasticQueryWebClientConfigData.getQuery().getAccept()))
.retrieve()
.bodyToFlux(ElasticQueryWebClientResponseModel.class);
}
}
Microservice (B) config code :
#Configuration
public class WebClientConfig {
private final ElasticQueryWebClientConfigData.WebClient webClientConfig;
public WebClientConfig(ElasticQueryWebClientConfigData webClientConfigData) {
this.webClientConfig = webClientConfigData.getWebClient();
}
#Bean("webClientBuilder")
WebClient.Builder webClientBuilder() {
return WebClient.builder()
.baseUrl(webClientConfig.getBaseUrl())
.defaultHeader(HttpHeaders.CONTENT_TYPE, webClientConfig.getContentType())
.defaultHeader(HttpHeaders.ACCEPT, webClientConfig.getAcceptType())
.clientConnector(new ReactorClientHttpConnector(HttpClient.from(getTcpClient())))
.codecs(configurer -> configurer.defaultCodecs()
.maxInMemorySize(webClientConfig.getMaxInMemorySize()));
}
private TcpClient getTcpClient() {
return TcpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, webClientConfig.getConnectTimeoutMs())
.doOnConnected(connection -> {
connection.addHandlerLast(new ReadTimeoutHandler(webClientConfig.getReadTimeoutMs(), TimeUnit.MILLISECONDS));
connection.addHandlerLast(new WriteTimeoutHandler(webClientConfig.getWriteTimeoutMs(), TimeUnit.MILLISECONDS));
});
}
}
Microservice (B) application.yml :
elastic-query-web-client:
webclient:
connect-timeout-ms: 10000
read-timeout-ms: 10000
write-timeout-ms: 10000
max-in-memory-size: 10485760 # 10MB
content-type: 'application/json'
accept-type: 'text/event-stream'
base-url: 'http://localhost:8183/reactive-elastic-query-service/documents'
query:
method: POST
uri: "/get-doc-by-text"
accept: ${elastic-query-web-client.webclient.accept-type}
server:
port: 8184
spring:
webflux:
base-path: /reactive-elastic-query-web-client
thymeleaf:
cache: false
reactive:
max-chunk-size: 8192
codec:
max-in-memory-size: 25MB
Microservice (B) controller :
#Controller
public class QueryController {
private static final Logger LOG = LoggerFactory.getLogger(QueryController.class);
private final ElasticQueryWebClient elasticQueryWebClient;
public QueryController(ElasticQueryWebClient webClient) {
this.elasticQueryWebClient = webClient;
}
#GetMapping("/all")
public String queryAll(Model model) {
Flux<ElasticQueryWebClientResponseModel> responseModels = elasticQueryWebClient.getAllData();
responseModels = responseModels.log();
IReactiveDataDriverContextVariable reactiveData = new ReactiveDataDriverContextVariable(responseModels, 1);
model.addAttribute("elasticQueryWebClientResponseModels", reactiveData);
model.addAttribute("searchText", "");
model.addAttribute("elasticQueryWebClientRequestModel", ElasticQueryWebClientRequestModel.builder().build());
LOG.info("Returning from reactive client controller for all data");
return "home";
}
}
There are no exceptions in the output consoles.
I don't see what I am missing here.
Related
I am using Spring boot to implement reactive micro-services. However, the reactive code is never executed in lambda function. My implementation as below. publishEventScheduler is created during application start-up. I am using this code together with Kafka to send an event to user micro-service to create user.
MainServiceApplication.java
public class MainServiceApplication {
private final Integer threadPoolSize;
private final Integer taskQueueSize;
public MainServiceApplication(
#Value("${app.threadPoolSize:10}") Integer threadPoolSize,
#Value("${app.taskQueueSize:100}") Integer taskQueueSize) {
this.threadPoolSize = threadPoolSize;
this.taskQueueSize = taskQueueSize;
}
#Bean
public Scheduler publishEventScheduler() {
LOG.info("Creating a message scheduler with connectionPoolSize = {}", threadPoolSize);
return Schedulers.newBoundedElastic(threadPoolSize, taskQueueSize, "publish-pool");
}
public static void main(String[] args) {
SpringApplication.run(MainServiceApplication.class, args);
}
}
MainIntegration.java
the function createUser() is called with a POST request from Postman (break point stop at subscribeOn(publishEventScheduler)) but sendMessageUser() is never executed (break point in the function not working)
#Component
public class MainIntegration implements UserService, TodoService {
private final String todoServiceUrl;
private final String userServiceUrl;
private final WebClient webClient;
private final StreamBridge streamBridge;
private final Scheduler publishEventScheduler;
public MainIntegration(
#Qualifier("publishEventScheduler") Scheduler publishEventScheduler,
WebClient.Builder webClient,
StreamBridge streamBridge,
#Value("${app.user-service.host}") String userServiceHost,
#Value("${app.user-service.port}") int userServicePort
) {
this.publishEventScheduler = publishEventScheduler;
this.webClient = webClient.build();
this.streamBridge = streamBridge;
userServiceUrl = "http://" + userServiceHost + ":" + userServicePort + "/user";
}
#Override
public Mono<User> createUser(User body) {
return Mono.fromCallable(() -> {
sendMessageUser("user-out-0", new Event<Event.Type, String, User >(Event.Type.CREATE, body.getUserName(), body));
return body;
}).subscribeOn(publishEventScheduler);
}
private void sendMessageUser(String bindingName, Event<Type, String, User> event) {
LOG.debug("Sending a {} message to {}", bindingName, event.getEventType());
Message<Event<Type, String, User>> message = MessageBuilder.withPayload(event)
.setHeader("partitionKey", event.getKey())
.build();
streamBridge.send(bindingName, message);
}
application.yaml
server.port: 7000
server.error.include-message: always
app:
user-service:
host: localhost
port: 7002
spring:
cloud:
stream:
default-binder: kafka
default-contentType: application/json
bindings:
user-out-0:
destination: user-service
producer:
required-groups: auditGroup
kafka:
binder:
brokers: 127.0.0.1
defaultBrokerPort: 2181
rabbitmq:
host: 127.0.0.1
port: 5672
username: guest
password: guest
Circuit breaker works, fallback is called, but circuit breaker doesn't change it's state and every time send request to failed service.
Tried the same YAML config with rest template - works correctly.
Feign client
#FeignClient(
name = MyFeignClient.SERVICE_NAME,
url = "https://httpbin.org/",
configuration = {FeignClientConfiguration.class})
public interface MyFeignClient {
String SERVICE_NAME = "producer-service";
#GetMapping(value = "/status/502")
ResponseEntity<String> gerRequest();
}
Fallback class
public class MyFallback implements MyFeignClient {
private final Exception cause;
public MyFallback(Exception cause) {
this.cause = cause;
}
public ResponseEntity<String> gerRequest() {
if (cause instanceof HttpServerErrorException){
return ResponseEntity.of(Optional.of(cause.getMessage()));
} else {
return ResponseEntity.of(Optional.of(cause.getMessage()));
}
}
}
Feign client configuration
#RequiredArgsConstructor
public class FeignClientConfiguration {
private final CircuitBreakerRegistry registry;
#Bean
#Scope("prototype")
public Feign.Builder feignBuilder() {
CircuitBreaker circuitBreaker = registry.circuitBreaker("producer-service");
FeignDecorators decorators = FeignDecorators.builder()
.withCircuitBreaker(circuitBreaker)
.withFallbackFactory(MyFallback::new)
.build();
return Resilience4jFeign.builder(decorators);
}
}
Circuit breaker YAML config
resilience4j.circuitbreaker:
configs:
default:
registerHealthIndicator: true
slidingWindowType: COUNT_BASED
slidingWindowSize: 5
minimumNumberOfCalls: 3
permittedNumberOfCallsInHalfOpenState: 1
automaticTransitionFromOpenToHalfOpenEnabled: true
waitDurationInOpenState: 5s
failureRateThreshold: 50
eventConsumerBufferSize: 10
writableStackTraceEnabled: true
recordExceptions:
- org.springframework.web.client.HttpServerErrorException
- java.util.concurrent.TimeoutException
- java.io.IOException
shared:
slidingWindowSize: 100
permittedNumberOfCallsInHalfOpenState: 30
waitDurationInOpenState: 1s
failureRateThreshold: 50
eventConsumerBufferSize: 10
instances:
producer-service:
baseConfig: default
I have a WebClient implementantion with some logs ands "Trace" implementation, in this code I add some header in my req and wait for end.
The implementation works without error, but after next.exchange(), my filter is executed again:
My client builder:
private final WebClient.Builder builder;
builder
.baseUrl(url)
.filter(new GenericAwsXrayHandler(url))
.build();
and my filter:
public class GenericAwsXrayHandler implements ExchangeFilterFunction {
private final String url;
public GenericAwsXrayHandler(final String url) {
this.url = url;
}
#Override
public Mono<ClientResponse> filter(final ClientRequest request, final ExchangeFunction next) {
log.info("executing filter");
final Subsegment subsegment = AWSXRay.getGlobalRecorder().beginSubsegment(url);
final TraceHeader header = getTraceHeader(request, subsegment);
final ClientRequest clientRequest =
ClientRequest.from(request)
.header(TraceHeader.HEADER_KEY, header.toString())
.build();
logRequest(clientRequest);
return next
.exchange(clientRequest)
.doOnSuccess(clientResponse -> {
logResponse(clientResponse);
addResponseInformation(subsegment, clientResponse);
})
.doOnError(ex -> {
subsegment.setFault(true);
subsegment.addException(ex);
})
.doFinally(x -> {
AWSXRay.getGlobalRecorder().endSubsegment(subsegment);
});
}
My output log:
2021/07/21 08:33:21.101 [main] [INFO ] b.c.i.o.r.a.o.p.h.h.GenericAwsXrayHandler - executing filter
2021/07/21 08:33:21.108 [main] [INFO ] b.c.i.o.r.a.o.p.h.h.GenericAwsXrayHandler - executing filter
Debugging, I see the filter running again just after return next.exchange(clientRequest).
Did I missed something?
I'm trying to write contract test to this service:
#RestController
#RequestMapping(path = "/api/form")
public class FormController {
private RestOperations restOperations;
#Autowired
public FormController(RestOperations restOperations) {
this.restOperations = restOperations;
}
#PostMapping(path = "/submit", consumes = MediaType.APPLICATION_JSON_UTF8_VALUE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
public ResponseEntity<SubmitFormResponse> submitForm(#RequestBody #Valid SubmitFormCommand submitFormCommand) {
return restOperations.postForEntity("http://127.0.0.1:9000/api/form/submit", submitFormCommand, SubmitFormResponse.class);
}
}
SubmitFormCommand contains only String "message" and SubmitFormResponse contains Boolean "success"
My RestClient for this service:
#Component
public class FormControllerClient {
#Autowired
private RestOperations restOperations;
public ResponseEntity<SubmitFormResponse> submitForm(SubmitFormCommand submitFormCommand) {
HttpEntity<SubmitFormCommand> request = new HttpEntity<>(submitFormCommand);
return restOperations.exchange("http://localhost:1234/api/form/submit", HttpMethod.POST, request, SubmitFormResponse.class);
}
And Contract test class of consumer looks like this:
#RunWith(SpringRunner.class)
#SpringBootTest
public class ContactFormClientTest {
#Rule
public PactProviderRuleMk2 pactProviderRuleMk2 = new PactProviderRuleMk2("formservice", "localhost", 1234, this);
#Autowired
private FormControllerClient formControllerClient;
#Pact(state = "provider accets submit contact form", provider = "formservice", consumer = "formclient")
public RequestResponsePact submitFormPact(PactDslWithProvider builder) {
return builder
.given("provider accetps form submit")
.uponReceiving("a request to POST form")
.path("/api/form/submit")
.method("POST")
.willRespondWith()
.status(200)
.matchHeader("Content-Type", "application/json;charset=UTF-8")
.body(new PactDslJsonBody()
.stringType("message", "TestMessage"))
.toPact();
}
#Test
#PactVerification(fragment = "submitFormPact")
public void verifySubmitFormPact() {
SubmitFormCommand submitFormCommand = new SubmitFormCommand("TestMessage");
ResponseEntity<SubmitFormResponse> response = formControllerClient.submitForm(submitFormCommand);
assertNotNull(response);
}
}
Every time I run the test it says "Connection refused" and I don't understand what I did wrong with a setup, my FormController would be a consumer in this case since it calls another service to submit the form.
Plugin in pom.xml for building Pact file looks like this :
<plugin>
<!-- mvn pact:publish -->
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-maven_2.11</artifactId>
<version>3.5.10</version>
<configuration>
<pactDirectory>../pacts</pactDirectory>
<pactBrokerUrl>http://localhost:1234</pactBrokerUrl>
<projectVersion>${project.version}</projectVersion>
</configuration>
</plugin>
The problem is you are placing your request body in the response. Your pact should look like:
#Pact(state = "provider accets submit contact form", provider = "formservice", consumer = "formclient")
public RequestResponsePact submitFormPact(PactDslWithProvider builder) {
return builder
.given("provider accetps form submit")
.uponReceiving("a request to POST form")
.path("/api/form/submit")
.method("POST")
.body(new PactDslJsonBody()
.stringType("message", "TestMessage"))
.willRespondWith()
.status(200)
.matchHeader("Content-Type", "application/json;charset=UTF-8")
.body(new PactDslJsonBody()
.booleanType("sucess", true))
.toPact();
}
I'm working on simple chat module for my application using Spring WebFlux with ReactiveMongoRepository on backend and Angular 4 on front. I'm able to receive data through WebSocketSession but after streaming all messages from db i want to keep the connection so i could update message list. Can anyone give me clues how to achieve that, or maybe i'm following wrong assumptions ?
Java Backend responsible for WebSocket, my subscriber only logs current state, nothing relevant there:
WebFluxConfiguration:
#Configuration
#EnableWebFlux
public class WebSocketConfig {
private final WebSocketHandler webSocketHandler;
#Autowired
public WebSocketConfig(WebSocketHandler webSocketHandler) {
this.webSocketHandler = webSocketHandler;
}
#Bean
#Primary
public HandlerMapping webSocketMapping() {
Map<String, Object> map = new HashMap<>();
map.put("/websocket-messages", webSocketHandler);
SimpleUrlHandlerMapping mapping = new SimpleUrlHandlerMapping();
mapping.setOrder(10);
mapping.setUrlMap(map);
return mapping;
}
#Bean
public WebSocketHandlerAdapter handlerAdapter() {
return new WebSocketHandlerAdapter();
}
}
WebSocketHandler Implementation
#Component
public class MessageWebSocketHandler implements WebSocketHandler {
private final MessageRepository messageRepository;
private ObjectMapper mapper = new ObjectMapper();
private MessageSubscriber subscriber = new MessageSubscriber();
#Autowired
public MessageWebSocketHandler(MessageRepository messageRepository) {
this.messageRepository = messageRepository;
}
#Override
public Mono<Void> handle(WebSocketSession session) {
session.receive()
.map(WebSocketMessage::getPayloadAsText)
.map(this::toMessage)
.subscribe(subscriber::onNext, subscriber:: onError, subscriber::onComplete);
return session.send(
messageRepository.findAll()
.map(this::toJSON)
.map(session::textMessage));
}
private String toJSON(Message message) {
try {
return mapper.writeValueAsString(message);
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
}
private Message toMessage(String json) {
try {
return mapper.readValue(json, Message.class);
} catch (IOException e) {
throw new RuntimeException("Invalid JSON:" + json, e);
}
}
}
and MongoRepo
#Repository
public interface MessageRepository extends
ReactiveMongoRepository<Message,String> {
}
FrontEnd Handling:
#Injectable()
export class WebSocketService {
private subject: Rx.Subject<MessageEvent>;
constructor() {
}
public connect(url): Rx.Subject<MessageEvent> {
if (!this.subject) {
this.subject = this.create(url);
console.log('Successfully connected: ' + url);
}
return this.subject;
}
private create(url): Rx.Subject<MessageEvent> {
const ws = new WebSocket(url);
const observable = Rx.Observable.create(
(obs: Rx.Observer<MessageEvent>) => {
ws.onmessage = obs.next.bind(obs);
ws.onerror = obs.error.bind(obs);
ws.onclose = obs.complete.bind(obs);
return ws.close.bind(ws);
});
const observer = {
next: (data: Object) => {
if (ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify(data));
}
}
};
return Rx.Subject.create(observer, observable);
}
}
in other service i'm mapping observable from response to my type
constructor(private wsService: WebSocketService) {
this.messages = <Subject<MessageEntity>>this.wsService
.connect('ws://localhost:8081/websocket-messages')
.map((response: MessageEvent): MessageEntity => {
const data = JSON.parse(response.data);
return new MessageEntity(data.id, data.user_id, data.username, data.message, data.links);
});
}
and finally subscribtion with send function which i can't use because of closed connection:
ngOnInit() {
this.messages = [];
this._ws_subscription = this.chatService.messages.subscribe(
(message: MessageEntity) => {
this.messages.push(message);
},
error2 => {
console.log(error2.json());
},
() => {
console.log('Closed');
}
);
}
sendTestMessage() {
this.chatService.messages.next(new MessageEntity(null, '59ca30ac87e77d0f38237739', 'mickl', 'test message angular', null));
}
Assuming your chat messages are being persisted to the datastore as they're being received, you could use the tailable cursors feature in Spring Data MongoDB Reactive (see reference documentation).
So you could create a new method on your repository like:
public interface MessageRepository extends ReactiveSortingRepository< Message, String> {
#Tailable
Flux<Message> findWithTailableCursor();
}
Note that tailable cursors have some limitations: you mongo collection needs to be capped and entries are streamed in their order of insertion.
Spring WebFlux websocket support does not yet support STOMP nor message brokers, but this might be a better choice for such a use case.