Develop Reference

How to stop and start a PubSub programatically through the MessageReceiver in Java

Is it possible to pause and start a GCP PubSub Subscriber(pull) programatically using Java?
I have the following code for the MessageReceiver:
MessageReceiver receiver = (message, consumer) -> {
System.out.println("Received message: " + message.getData().toStringUtf8() + " at " + LocalDateTime.now());
Mono<Void> mono = Mono.just(message.getData().toStringUtf8())
.flatMap(deserializePayload())
.doOnSuccess(creditNoteRequestDTO -> configureMDC(message.getAttributesMap(), creditNoteRequestDTO.getOrderNumber()))
.doOnError(error -> log.error("Problem while parsing CreditNoteRequest DTO {}", message.getData().toStringUtf8(), error))
.map(creditNoteRequestMapper::mapToCreditNoteRequest)
.flatMap(creditNoteRequestProcess::process)
.doOnError(throwable -> {
if (throwable instanceof CallNotPermittedException) ConstantUtils.ENABLED_SUBSCRIBER = false;
consumer.nack();
})
.doOnSuccess(unused -> {
log.info("Doing ACK on payload {}", message.getData().toStringUtf8());
consumer.ack();
});
if (Boolean.TRUE.equals(ConstantUtils.ENABLED_SUBSCRIBER)) {
mono.subscribe();
} else {
System.err.println("Subscription has been disabled.");
//mono.subscribe().dispose();
try {
System.err.println("PRE: " + LocalDateTime.now() + " " + ConstantUtils.ENABLED_SUBSCRIBER);
TimeUnit.SECONDS.sleep(15);
ConstantUtils.ENABLED_SUBSCRIBER = true;
System.err.println("POST: " + LocalDateTime.now() + " " + ConstantUtils.ENABLED_SUBSCRIBER);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
And the following for the subscriber:
Subscriber subscriber = null;
FlowControlSettings controlSettings = FlowControlSettings
.newBuilder()
.setMaxOutstandingElementCount(2000L)
.build();
subscriber = Subscriber
.newBuilder(subscription, receiver)
.setFlowControlSettings(controlSettings)
.build();
// Start the subscriber.
subscriber.startAsync().awaitRunning();
System.out.printf("Listening for messages on %s:\n", subscription);
System.out.printf("Max elements count: %s\n", subscriber.getFlowControlSettings().getMaxOutstandingElementCount());
// Allow the subscriber to run for 45s unless an unrecoverable error occurs.
Subscriber finalSubscriber = subscriber;
new java.util.Timer().schedule(
new java.util.TimerTask() {
#Override
public void run() {
System.err.println("Subscriber has been shut down.");
finalSubscriber.stopAsync();
}
},
120000
);
It is somehow possible to use finalSubscriber.stopAsync(); on the MessageReceiver or pause the messageReceiver?
UPDATE: Thanks, I managed to stop it, but how do I restart it? Now, when I try to restart, nothing happens.
private void createSubscriber() {
ProjectSubscriptionName subscription = ProjectSubscriptionName.of("txd-boss-dev", "circuit-breaker-test-sub");
this.subscriber = Subscriber.newBuilder(subscription, getMessageReceiver()).build();
}
private void runSubscriber(boolean start) {
if(start) {
try {
this.subscriber.startAsync().awaitRunning();
System.out.printf("Listening for messages on %s:\n", this.subscriber.getSubscriptionNameString());
subscriber.awaitTerminated(240, TimeUnit.SECONDS);
} catch (TimeoutException e) {
System.err.println("Shutting down subscription.");
runSubscriber(false);
}
} else {
subscriber.stopAsync();
}
}
#EventListener(ContextRefreshedEvent.class)
public void test() {
createSubscriber();
runSubscriber(true);
}

You need to return the same subscriber object to start and stop it:
check some google examples here.
here is a sketch (adapt for your class):
private Subscriber StartSubscriber() {
Subscriber subscriber = null;
try {
subscriber = Subscriber.newBuilder(subscriptionName, receiver).build();
subscriber.startAsync().awaitRunning();
System.out.printf("Listening for messages on %s:\n", subscriptionName.toString());
subscriber.awaitTerminated(30, TimeUnit.SECONDS);
return subscriber;
} catch (TimeoutException timeoutException) {
// Shut down the subscriber after 30s. Stop receiving messages.
System.out.printf("Error \n");
subscriber.stopAsync();
}
}
private void StopSubscriber(Subscriber subscriber) {
System.out.printf("Stoping subscriber\n");
subscriber.stopAsync();
}
public void test() {
Subscriber subscriber = StartSubscriber();
StopSubscriber(subscriber);
}

Redisson async procesing messages

I'm trying to aply Redisson features for my project as message broker and I have a question. Is it possible to push Redisson to precceding recieved messages asynchronously? I have created a small example, sent 4 messages from different URL's. I expected, that Redisson proceeded them asynchronously, but it did it one by one.
Here the implementation:
public class RedisListenerServiceImpl implements MessageListener<String> {
private static final Logger log = LoggerFactory.getLogger(RedisListenerServiceImpl.class);
private final ObjectMapper objectMapper = new ObjectMapper();
#Override
public void onMessage(CharSequence channel, String stringMsg) {
log.info("Message received: {}", stringMsg);
MessageDto msg;
try {
msg = objectMapper.readValue(stringMsg, MessageDto.class);
} catch (final IOException e) {
log.error("Unable to deserialize message: {}", e.getMessage(), e);
return;
}
try {
//Do my stuff
} catch (Exception e) {
log.error("Unable to get service from factory: {}", e.getMessage(), e);
}
}
}
And the config:
#Configuration
public class RedisListenerConfig {
#Autowired
public RedisListenerConfig(RedissonClient redisClient,
MessageListener redisListenerService,
#Value("${redis.sub.key}") String redisSubKey) {
RTopic subscribeTopic = redisClient.getTopic(redisSubKey);
subscribeTopic.addListenerAsync(String.class, redisListenerService);
}
}

It's expected behavior. If you want your messages to be processed concurrently when the Listener onMessage() method is triggered, just use a thread pool to process it.
Since Redisson doesn't know how many threads you want to consume the triggered events, it leaves the implementation detail to you.
public class RedisListenerServiceImpl implements MessageListener<String> {
private static final Logger log = LoggerFactory.getLogger(RedisListenerServiceImpl.class);
private final ObjectMapper objectMapper = new ObjectMapper();
private final ExecutorService executorService = Executors.newFixedThreadPool(10);
#Override
public void onMessage(CharSequence channel, String stringMsg) {
log.info("Message received: {}", stringMsg);
MessageDto msg;
try {
msg = objectMapper.readValue(stringMsg, MessageDto.class);
executorService.submit(()->{
System.out.println("do something with message: "+msg);
});
} catch (final IOException e) {
log.error("Unable to deserialize message: {}", e.getMessage(), e);
return;
}
try {
//Do my stuff
} catch (Exception e) {
log.error("Unable to get service from factory: {}", e.getMessage(), e);
}
}

Spring integration - Publisher Confirms with timeout?

This is my current setup:
queue1 and queue2 are marged together with integration flow to channel1:
#Bean
public IntegrationFlow q1f() {
return IntegrationFlows
.from(queue1InboundAdapter())
...
.channel(amqpInputChannel())
.get();
}
#Bean
public IntegrationFlow q2f() {
return IntegrationFlows
.from(queue2InboundAdapter())
...
.channel(amqpInputChannel())
.get();
}
then, everything is aggregated and then confirmed after aggregated message is confirmed by rabbitmq:
#Bean
public IntegrationFlow aggregatingFlow() {
return IntegrationFlows
.from(amqpInputChannel())
.aggregate(...
.expireGroupsUponCompletion(true)
.sendPartialResultOnExpiry(true)
.groupTimeout(TimeUnit.SECONDS.toMillis(10))
.releaseStrategy(new TimeoutCountSequenceSizeReleaseStrategy(200, TimeUnit.SECONDS.toMillis(10)))
)
.handle(amqpOutboundEndpoint())
.get();
}
#Bean
public AmqpOutboundEndpoint amqpOutboundEndpoint() {
AmqpOutboundEndpoint outboundEndpoint = new AmqpOutboundEndpoint(ackTemplate());
outboundEndpoint.setConfirmAckChannel(manualAckChannel());
outboundEndpoint.setConfirmCorrelationExpressionString("#root");
outboundEndpoint.setExchangeName(RABBIT_PREFIX + "ix.archiveupdate");
outboundEndpoint.setRoutingKeyExpression(routingKeyExpression()); //forward using patition id as routing key
return outboundEndpoint;
}
ackTemplate() is set with cf that has springFactory.setPublisherConfirms(true);.
The problem I see is that once in 10 days, there are some messages that are stuck in unacknowledged state in rabbitmq.
My guess is that somehow publish of message is waiting for rabbit to do PUBLISHER CONFIRMS but it never gets it and times out? In this case, I never ACK message in queue1. Is this possible?
So just one more time complete workflow:
[two queues -> direct channel -> aggregator (keeps channel and tag values) -> publish to rabbit -> rabbit returns ACK via publisher confirms -> spring confirms all messages on channel+values that it kept in memory for aggregated message]
I also have my implementation of aggregator (since I need to manually ack messages from both q1 and q2):
public abstract class AbstractManualAckAggregatingMessageGroupProcessor extends AbstractAggregatingMessageGroupProcessor {
public static final String MANUAL_ACK_PAIRS = PREFIX + "manualAckPairs";
private AckingState ackingState;
public AbstractManualAckAggregatingMessageGroupProcessor(AckingState ackingState){
this.ackingState = ackingState;
}
#Override
protected Map<String, Object> aggregateHeaders(MessageGroup group) {
Map<String, Object> aggregatedHeaders = super.aggregateHeaders(group);
List<ManualAckPair> manualAckPairs = new ArrayList<>();
group.getMessages().forEach(m -> {
Channel channel = (Channel)m.getHeaders().get(AmqpHeaders.CHANNEL);
Long deliveryTag = (Long)m.getHeaders().get(AmqpHeaders.DELIVERY_TAG);
manualAckPairs.add(new ManualAckPair(channel, deliveryTag, ackingState));
});
aggregatedHeaders.put(MANUAL_ACK_PAIRS, manualAckPairs);
return aggregatedHeaders;
}
}
UPDATE
This is how rabbit admin looks (2 unacked messages for a long time, and it will not be ACKED untill restart - when it is redelivered):

In Spring AMQP version 2.1 (Spring Integration 5.1), We added a Future<?> and returned message to the CorrelationData to assist with this kind of thing. If you are using an older version, you can subclass CorrelationData (and you'd have to handle setting the future and returned message in your code).
This, together with a scheduled task, can detect missing acks...
#SpringBootApplication
#EnableScheduling
public class Igh2755Application {
public static void main(String[] args) {
SpringApplication.run(Igh2755Application.class, args);
}
private final BlockingQueue<CorrelationData> futures = new LinkedBlockingQueue<>();
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> {
SuccessCallback<? super Confirm> successCallback = confirm -> {
System.out.println((confirm.isAck() ? "A" : "Na") + "ck received");
};
FailureCallback failureCallback = throwable -> {
System.out.println(throwable.getMessage());
};
// Good - ack
CorrelationData correlationData = new CorrelationData("good");
correlationData.getFuture().addCallback(successCallback, failureCallback);
this.futures.put(correlationData);
template.convertAndSend("", "foo", "data", correlationData);
// Missing exchange nack, no return
correlationData = new CorrelationData("missing exchange");
correlationData.getFuture().addCallback(successCallback, failureCallback);
this.futures.put(correlationData);
template.convertAndSend("missing exchange", "foo", "data", correlationData);
// Missing queue ack, with return
correlationData = new CorrelationData("missing queue");
correlationData.getFuture().addCallback(successCallback, failureCallback);
this.futures.put(correlationData);
template.convertAndSend("", "missing queue", "data", correlationData);
};
}
#Scheduled(fixedDelay = 5_000)
public void checkForMissingAcks() {
System.out.println("Checking pending acks");
CorrelationData correlationData = this.futures.poll();
while (correlationData != null) {
try {
if (correlationData.getFuture().get(10, TimeUnit.SECONDS).isAck()) {
if (correlationData.getReturnedMessage() == null) {
System.out.println("Ack received OK for " + correlationData.getId());
}
else {
System.out.println("Message returned for " + correlationData.getId());
}
}
else {
System.out.println("Nack received for " + correlationData.getId());
}
}
catch (InterruptedException e) {
Thread.currentThread().interrupt();
System.out.println("Interrupted");
}
catch (ExecutionException e) {
System.out.println("Failed to get an ack " + e.getCause().getMessage());
}
catch (TimeoutException e) {
System.out.println("Timed out waiting for ack for " + correlationData.getId());
}
correlationData = this.futures.poll();
}
System.out.println("No pending acks, exiting");
}
}
.
Checking pending acks
Ack received OK for good
Nack received for missing exchange
Message returned for missing queue
No pending acks, exiting
With Spring Integration there is a confirmCorrelationExpression which can be used to create the CorrelationData instance.
EDIT
With Spring Integration...
#SpringBootApplication
#EnableScheduling
public class Igh2755Application {
public static void main(String[] args) {
SpringApplication.run(Igh2755Application.class, args);
}
private final BlockingQueue<CorrelationData> futures = new LinkedBlockingQueue<>();
public interface Gate {
void send(#Header("exchange") String exchange, #Header("rk") String rk, String payload);
}
#Bean
#DependsOn("flow")
public ApplicationRunner runner(Gate gate) {
return args -> {
gate.send("", "foo", "good");
gate.send("junque", "rk", "missing exchange");
gate.send("", "junque", "missing queue");
};
}
#Bean
public IntegrationFlow flow(RabbitTemplate template) {
return IntegrationFlows.from(Gate.class)
.handle(Amqp.outboundAdapter(template)
.confirmCorrelationExpression("#correlationCreator.create(#root)")
.exchangeNameExpression("headers.exchange")
.routingKeyExpression("headers.rk")
.returnChannel(returns())
.confirmAckChannel(acks())
.confirmNackChannel(acks()))
.get();
}
#Bean
public MessageChannel acks() {
return new DirectChannel();
}
#Bean
public MessageChannel returns() {
return new DirectChannel();
}
#Bean
public IntegrationFlow ackFlow() {
return IntegrationFlows.from("acks")
/*
* Work around a bug because the correlation data is wrapped and so the
* wrong future is completed.
*/
.handle(m -> {
System.out.println(m);
if (m instanceof ErrorMessage) { // NACK
NackedAmqpMessageException nme = (NackedAmqpMessageException) m.getPayload();
CorrelationData correlationData = (CorrelationData) nme.getCorrelationData();
correlationData.getFuture().set(new Confirm(false, "Message was returned"));
}
else {
((CorrelationData) m.getPayload()).getFuture().set(new Confirm(true, null));
}
})
.get();
}
#Bean
public IntegrationFlow retFlow() {
return IntegrationFlows.from("returns")
.handle(System.out::println)
.get();
}
#Bean
public CorrelationCreator correlationCreator() {
return new CorrelationCreator(this.futures);
}
public static class CorrelationCreator {
private final BlockingQueue<CorrelationData> futures;
public CorrelationCreator(BlockingQueue<CorrelationData> futures) {
this.futures = futures;
}
public CorrelationData create(Message<String> message) {
CorrelationData data = new CorrelationData(message.getPayload());
this.futures.add(data);
return data;
}
}
#Scheduled(fixedDelay = 5_000)
public void checkForMissingAcks() {
System.out.println("Checking pending acks");
CorrelationData correlationData = this.futures.poll();
while (correlationData != null) {
try {
if (correlationData.getFuture().get(10, TimeUnit.SECONDS).isAck()) {
if (correlationData.getReturnedMessage() == null
&& !correlationData.getId().equals("Message was returned")) {
System.out.println("Ack received OK for " + correlationData.getId());
}
else {
System.out.println("Message returned for " + correlationData.getId());
}
}
else {
System.out.println("Nack received for " + correlationData.getId());
}
}
catch (InterruptedException e) {
Thread.currentThread().interrupt();
System.out.println("Interrupted");
}
catch (ExecutionException e) {
System.out.println("Failed to get an ack " + e.getCause().getMessage());
}
catch (TimeoutException e) {
System.out.println("Timed out waiting for ack for " + correlationData.getId());
}
correlationData = this.futures.poll();
}
System.out.println("No pending acks, exiting");
}
}

you can declare connection as bean
#Bean
public ConnectionFactory createConnectionFactory(){
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("127.0.0.1", 5672);
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
connectionFactory.setVirtualHost("/");
connectionFactory.setPublisherReturns(true);
connectionFactory.setPublisherConfirmType(ConfirmType.SIMPLE);
return connectionFactory;
}
Then RabbitTemplate as
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMandatory(true);
rabbitTemplate.setConfirmCallback(callback);
return rabbitTemplate;
}
Where callback is implementation of ConfirmCallback interface
and while sending you can just wait for confirmation
System.out.println("Sending message...");
rabbitTemplate.convertAndSend(rabbitMQProperties.getEXCHANGENAME(),
rabbitMQProperties.getQUEUENAME(), "hello from rabbit");
rabbitTemplate.waitForConfirms(1);
waitforconfirms will take time in milisecond. I put it as 1 for testing purpose.

Send messages from server to websocket clients

I'm building a websocket application using glassfish I need in a given event my server to send message to all connected clients. I can send and receive messages from both, but I am not able to use the class server to send messages.
My server class has the following body:
#ApplicationScoped
#ServerEndpoint("/actions")
public class DeviceWebSocketServer {
#Inject
private DeviceSessionHandler sessionHandler;
#OnOpen
public void open(Session session) {
sessionHandler.addSession(session);
}
#OnClose
public void close(Session session) {
sessionHandler.removeSession(session);
}
#OnError
public void onError(Throwable error) {
Logger.getLogger(DeviceWebSocketServer.class.getName()).log(Level.SEVERE, null, error);
}
#OnMessage
public void handleMessage(String message, Session session) {
System.out.println("Chegou uma mensagem: " + message);
System.out.println("Na sessao: " + session.getId());
try (JsonReader reader = Json.createReader(new StringReader(message))) {
JsonObject jsonMessage = reader.readObject();
if ("add".equals(jsonMessage.getString("action"))) {
Device device = new Device();
device.setName(jsonMessage.getString("name"));
device.setDescription(jsonMessage.getString("description"));
device.setType(jsonMessage.getString("type"));
device.setStatus("Off");
sessionHandler.addDevice(device);
}
if ("remove".equals(jsonMessage.getString("action"))) {
int id = (int) jsonMessage.getInt("id");
sessionHandler.removeDevice(id);
}
if ("toggle".equals(jsonMessage.getString("action"))) {
int id = (int) jsonMessage.getInt("id");
sessionHandler.toggleDevice(id);
}
}
}
How do I send messages to customers after receiving an event? Should I instantiate my class server?

Theres a example on howto send messages to all connected clients in this whiteboard app:
https://netbeans.org/kb/docs/javaee/maven-websocketapi.html
#ServerEndpoint(value="/whiteboardendpoint", encoders = {FigureEncoder.class}, decoders = {FigureDecoder.class})
public class MyWhiteboard {
private static Set<Session> peers = Collections.synchronizedSet(new HashSet<Session>());
#OnMessage
public void broadcastFigure(Figure figure, Session session) throws IOException, EncodeException {
System.out.println("broadcastFigure: " + figure);
for (Session peer : peers) {
if (!peer.equals(session)) {
peer.getBasicRemote().sendObject(figure);
}
}
}

Where I may wrong while connected to StompClient?

I need help with connect on android to my WebSocket server based on Spring boot. Source code of this server I have taken https://spring.io/guides/gs/messaging-stomp-websocket/
Everything works fine on server and browser client on this sample,
but if I use StompClient (https://github.com/NaikSoftware/StompProtocolAndroid) to connect on my socket I am getting mStompClient.isConnected() == false and conand mStompClient.send(...) doesn't send anything (?).
After few minutes socketweb server closes the connection and I get in my log: '~~ Stomp connection closed'.
Web server locates on Heroku cloud system.
There is my connecting code from android activity:
private StompClient mStompClient;
private void connectStomp(){
mStompClient = Stomp.over(WebSocket.class, "wss://myserver/gs-guide-websocket");
mStompClient.topic("/topic/greetings").subscribe(new Action1<StompMessage>() {
#Override
public void call(StompMessage stompMessage) {
Log.w(TAG, "== "+stompMessage.getPayload());
}
});
mStompClient.connect();
mStompClient.lifecycle().subscribe(new Action1<LifecycleEvent>() {
#Override
public void call(LifecycleEvent lifecycleEvent) {
switch (lifecycleEvent.getType()) {
case OPENED:
Log.w(TAG, "~~ Stomp connection opened");
break;
case ERROR:
Log.e(TAG, "~~ Error", lifecycleEvent.getException());
break;
case CLOSED:
Log.w(TAG, "~~ Stomp connection closed "+lifecycleEvent.getMessage());
break;
}
}
});
}
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_maps);
connectStomp();
}
// Send test request to server
public void onSend(View view){
Log.w(TAG,"onSend: click");
mStompClient.send("/app/hello","Test").subscribe(new Observer<Void>() {
#Override
public void onCompleted() {
Log.w(TAG, "~~~~ onCompleted");
}
#Override
public void onError(Throwable e) {
Log.w(TAG, "~~~~ onCompleted "+e.getMessage());
}
#Override
public void onNext(Void aVoid) {
Log.w(TAG, "~~~~ onNext ");
}
});
if (mStompClient.isConnected()){
mStompClient.send("/app/hello","test msg").subscribe();
Log.w("aaaa : ","onCreate: connected");
}
}
It would be my mistake but if I connect to my server socket with spring boot WebSocketStompClient everithing works fine:
private SockJsClient sockJsClient;
private WebSocketStompClient stompClient;
private final WebSocketHttpHeaders headers = new WebSocketHttpHeaders();
#Before
public void setup() {
List<Transport> transports = new ArrayList<>();
transports.add(new WebSocketTransport(new StandardWebSocketClient()));
this.sockJsClient = new SockJsClient(transports);
this.stompClient = new WebSocketStompClient(sockJsClient);
this.stompClient.setMessageConverter(new MappingJackson2MessageConverter());
}
#Test
public void getGreeting() throws Exception {
final CountDownLatch latch = new CountDownLatch(1);
final AtomicReference<Throwable> failure = new AtomicReference<>();
StompSessionHandler handler = new TestSessionHandler(failure) {
#Override
public void afterConnected(final StompSession session, StompHeaders connectedHeaders) {
session.subscribe("/topic/greetings", new StompFrameHandler() {
#Override
public Type getPayloadType(StompHeaders headers) {
return Greeting.class;
}
#Override
public void handleFrame(StompHeaders headers, Object payload) {
Greeting greeting = (Greeting) payload;
try {
System.out.println(greeting.getContent());
assertEquals("Hello, Spring!", greeting.getContent());
} catch (Throwable t) {
System.out.println(t.getMessage());
failure.set(t);
} finally {
session.disconnect();
latch.countDown();
}
}
});
try {
session.send("/app/hello", "Test");
} catch (Throwable t) {
failure.set(t);
latch.countDown();
}
}
};
this.stompClient.connect("wss://myserver/gs-guide-websocket", this.headers, handler, 443);
if (latch.await(10, TimeUnit.SECONDS)) {
if (failure.get() != null) {
throw new AssertionError("", failure.get());
}
}
else {
fail("Greeting not received");
}
}
private class TestSessionHandler extends StompSessionHandlerAdapter {
private final AtomicReference<Throwable> failure;
public TestSessionHandler(AtomicReference<Throwable> failure) {
this.failure = failure;
}
#Override
public void handleFrame(StompHeaders headers, Object payload) {
this.failure.set(new Exception(headers.toString()));
}
#Override
public void handleException(StompSession s, StompCommand c, StompHeaders h, byte[] p, Throwable ex) {
this.failure.set(ex);
}
Any ideas? Thanks a lot!

I used the same Library in order to connect with Stomp based web socket. There were some configurations on the server side. On the android side, I was using URL as starting with "ws://" and ending with "websocket" like "ws://" + SERVER_URL + "/websocket".
See this answer for server side https://stackoverflow.com/a/41751897/5392825