Develop Reference

Camel NotifyBuilder always returns false

I've got a fairly simple route:
PerfStubRouteBuilder.java
public class PerfStubRouteBuilder extends SpringRouteBuilder {
/* (non-Javadoc)
* #see org.apache.camel.builder.RouteBuilder#configure()
*/
#Override
public void configure() throws Exception {
from("direct:test-aces-input")
.log("Creating test Accident Number header")
.setHeader("AccidentNumber", simple("AB999999999"))
.log("Test Accident Number header created : ${header.AccidentNumber}")
.end();
}
}
and I'm testing it as follows:
PerfStubRouteBuilderTest.java
public class PerfStubRouteBuilderTest extends CamelTestSupport {
#Produce(uri = "direct:test-aces-input")
ProducerTemplate template;
PerfStubRouteBuilder route = new PerfStubRouteBuilder();
Exchange exch = new DefaultExchange(context);
#Override
protected RouteBuilder createRouteBuilder() {
return route;
}
#Test
public void test_PerfStubRouteBuilder_happyPath_addsAccidentNumberHeaderToExchange() throws Exception {
startCamelContext();
NotifyBuilder notify = new NotifyBuilder(context).from("direct:test-aces-input").whenDone(1).filter(header("AccidentNumber").isEqualTo("AB999999999")).create();
template.send(exch);
assertTrue(notify.matches(10, TimeUnit.SECONDS));
stopCamelContext();
}
}
assertTrue always come back as false, so I suspect I'm not using the NotifyBuilder correctly although at this point I can't be certain. Essentially I want to check that an exchange makes it through the route and the header specified in the actual route is added to the exchange. What I want to happen is for a match to occur if an exchange with that header value combo makes it to the end of the route, hence the filter step. I want to avoid adding an endpoint at the end of the route, for example, via AdviceWith given how small and simple the route itself is , it seems a bit heavyweight to start adding in mock endpoints for such a minor test
UPDATE:
Tried removing the filter portion from the expression, leaving the NotifyBuilder as NotifyBuilder notify = new NotifyBuilder(context).from("direct:test-aces-input").whenDone(1).create();
, the test still fails

Create the exchange from the endpoint in the test like
Endpoint endpoint = context.getEndpoint("direct:test-aces-input");
Exchange exchange = endpoint.createExchange();
template.send(exchange);
or just directly use sendBody
tempalte.sendBody("")

Junit and Spring 5 : All my classes should be multithreading tests expect one class, how can I achieve this? SSL handshake with concurrent

I'm working on a Spring 5 project and have some very special expectations with junit. Spring 5 now support junit multithreading and that definitely works very well, I'm now running my hundreds of tests into method parrallel multithreading. But I just setup recently my whole automatic mailing system which works like a charm but that's where it start to be problematic : I run a class that send all my mails to test them, and so they are being sent concurently. But as I just tried right now to test it with not only one email at a time but several, I get a strange SSL handshake error which I related to the fact that concurrent mail sending is not supported by most mail clients.
That's where goes my interrogation: how can I run all my test classes with parallel methods execution except for that email batch sending class?
Maybe I should think about a mail queue to avoid this kind of problem in live? Anyone has an idea?
By the way, in case you wonder I'm yet using gmail client to send mail as I didn't configured it yet for our live mail sending but it will be achieved using dedicated 1and1.fr smtp email client.
Thanks for your patience!

For those who feels interested about the solution, here is how I solved it:
I created a new Singleton class which would handle the queue :
public class EmailQueueHandler {
/** private Constructor */
private EmailQueueHandler() {}
/** Holder */
private static class EmailQueueHandlerHolder
{
/** unique instance non preinitialized */
private final static EmailQueueHandler INSTANCE = new EmailQueueHandler();
}
/** access point for unique instanciation of the singleton **/
public static EmailQueueHandler getInstance()
{
return EmailQueueHandlerHolder.INSTANCE;
}
private List<EmailPreparator> queue = new ArrayList<>();
public void queue(EmailPreparator email) {
waitForQueueHandlerToBeAvailable();
queue.add(email);
}
public List<EmailPreparator> getQueue()
{
waitForQueueHandlerToBeAvailable();
List<EmailPreparator> preparators = queue;
queue = new ArrayList<>();
return preparators;
}
// This method is used to make this handler thread safe
private synchronized void waitForQueueHandlerToBeAvailable(){}
}
I then created a CRON task using #Schedule annotation in my Scheduler bean in which I would correctly handle any mail sending fail.
#Scheduled(fixedRate = 30 * SECOND)
public void sendMailsInQueue()
{
List<EmailPreparator> queue = emailQueueHandler.getQueue();
int mailsSent = queue.size();
int mailsFailed = 0;
for(EmailPreparator preparator : queue)
{
try {
// And finally send the mail
emailSenderService.sendMail(preparator);
}
// If mail sending is not activated, mail sending function will throw an exception,
// Therefore we have to catch it and only throw it back if the email was really supposed to be sent
catch(Exception e)
{
mailsSent --;
// If we are not in test Env
if(!SpringConfiguration.isTestEnv())
{
mailsFailed ++;
preparator.getEmail().setTriggeredExceptionName(e.getMessage()).update();
// This will log the error into the database and eventually
// print it to the console if in LOCAL env
new Error()
.setTriggeredException(e)
.setErrorMessage(e.getClass().getName());
}
else if(SpringConfiguration.SEND_MAIL_ANYWAY_IN_TEST_ENV || preparator.isForceSend())
{
mailsFailed ++;
throw new EmailException(e);
}
}
}
log.info("CRON Task - " + mailsSent + " were successfuly sent ");
if(mailsFailed > 0)
log.warn("CRON Task - But " + mailsFailed + " could not be sent");
}
And then I called this mail queue emptyer methods at the end of each unit test in my #After annotated method to make sure it's called before I unit test the mail resulted. This way I'm aware of any mail sending fail even if it appear in PROD env and I'm also aware of any mail content creation failure when testing.
#After
public void downUp() throws Exception
{
proceedMailQueueManuallyIfNotAlreadySent();
logger.debug("After Test");
RequestHolder requestHolder = securityContextBuilder.getSecurityContextHolder().getRequestHolder();
// We check mails sending if some were sent
if(requestHolder.isExtResultsSent())
{
for(ExtResults results : requestHolder.getExtResults())
{
ExtSenderVerificationResolver resolver =
new ExtSenderVerificationResolver(
results,
notificationParserService
);
resolver.assertExtSending();
}
}
// Some code
}
protected void proceedMailQueueManuallyIfNotAlreadySent()
{
if(!mailQueueProceeded)
{
mailQueueProceeded = true;
scheduler.sendMailsInQueue();
}
}

Spring Boot rabbit mq spring.rabbitmq.listener.simple.concurrency never works

In SpringBoot rabbit mq project,I have the configuration like that.
#EnableRabbit
#Configuration
public class MQConfig {
private final String primaryQueueName;
private final String deadLetterQueueName;
#Autowired
private ConnectionFactory cachingConnectionFactory;
public MQConfig(
#Value("${primaryQueue.name}")String primaryQueueName,
#Value("${deadLetterQueue.name}")String deadLetterQueueName){
this.primaryQueueName=primaryQueueName;
this.deadLetterQueueName=deadLetterQueueName;
}
#Bean
public Queue primaryQueue() {
Map<String, Object> args = new HashMap<String, Object>();
// The default exchange
args.put("x-dead-letter-exchange", "");
// Route to the incoming queue when the TTL occurs
args.put("x-dead-letter-routing-key", deadLetterQueueName);
// TTL 500 seconds
args.put("x-message-ttl", 300000);
return new Queue(primaryQueueName, false, false, false, args);
}
#Bean
public Queue deadLetterQueue() {
return new Queue(deadLetterQueueName);
}}
And my listener looks like,
#Component
public class Receiver {
private static final Logger LOGGER = LoggerFactory.getLogger(Receiver.class);
#Autowired
private JavaMailSender mailSender;
#RabbitListener(queues = "${primaryQueue.name}")
public void receiveMessage(SimpleMailMessage message){
LOGGER.info("Sending email message : "+message);
try {
mailSender.send(message);
}catch(Exception e) {
LOGGER.error("Failed to deliver email",e);
throw e;
}
}}
So far so good, the messages come to primary queue, the consumer consumes them and they are gone to dead letter queue if for some reason the consumer is not able to consume them in given TTL 300000.
Now, I want to configure the listener even further with spring application.properties, but they have no impact on it at all.
E.g. I set spring.rabbitmq.listener.simple.concurrency=3, still it creates a single consumer, explicitly setting it in code works however. Nether works other settings,
spring.rabbitmq.listener.simple.acknowledge-mode= # Acknowledge mode of container.
spring.rabbitmq.listener.simple.auto-startup=true # Start the container automatically on startup.
spring.rabbitmq.listener.simple.concurrency= # Minimum number of consumers.
spring.rabbitmq.listener.simple.default-requeue-rejected= # Whether or not to requeue delivery failures; default `true`.
spring.rabbitmq.listener.simple.idle-event-interval= # How often idle container events should be published in milliseconds.
spring.rabbitmq.listener.simple.max-concurrency= # Maximum number of consumers.
spring.rabbitmq.listener.simple.prefetch= # Number of messages to be handled in a single request. It should be greater than or equal to the transaction size (if used).
spring.rabbitmq.listener.simple.retry.enabled=false # Whether or not publishing retries are enabled.
spring.rabbitmq.listener.simple.retry.initial-interval=1000 # Interval between the first and second attempt to deliver a message.
spring.rabbitmq.listener.simple.retry.max-attempts=3 # Maximum number of attempts to deliver a message.
spring.rabbitmq.listener.simple.retry.max-interval=10000 # Maximum interval between attempts.
spring.rabbitmq.listener.simple.retry.multiplier=1.0 # A multiplier to apply to the previous delivery retry interval.
spring.rabbitmq.listener.simple.retry.stateless=true # Whether or not retry is stateless or stateful.
spring.rabbitmq.listener.simple.transaction-size=
https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html
Any ideas? Am I configuring something incorrectly ?

You reference the docs for the current version of Spring Boot. Is this what you are using, or something older? This property used to be spring.rabbitmq.listener.concurrency without the "simple" in there.
Double check the version, for example: https://docs.spring.io/spring-boot/docs/1.4.3.RELEASE/reference/html/common-application-properties.html

Spring AMQP #RabbitListener custom retry on business error best practice

Following scenario: I have a #RabbitListener picking up messages from RabbitMQ. Sometimes the message runs into an error because I can not find the related business object. In this case I have no possibility to reply to the sender so I just want to ignore this message after certain amount of retry.
My solution: In my #RabbitListener whenever it is not possible to find the business object I throw a custom runtime exception. In my configuration I have a RetryOperationsInterceptor with max attempts and a custom recoverer.
What is the best practice for handling such cases? Can I configure diffrent recoverer class when having more than one #RabbitListener?
See my configuraion:
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setMessageConverter(new CustomMessageConverter());
factory.setConnectionFactory(connectionFactory());
factory.setAcknowledgeMode(AcknowledgeMode.AUTO);
factory.setConcurrentConsumers(1);
factory.setMaxConcurrentConsumers(20);
Advice[] adviceChain = new Advice[] { interceptor() };
factory.setAdviceChain(adviceChain);
return factory;
}
#Bean
RetryOperationsInterceptor interceptor() {
return RetryInterceptorBuilder.stateless()
.maxAttempts(5)
.recoverer(new CustomRejectAndRecoverer())
.build();
}
And this is my CustomRejectAndRecoverer:
public class CustomRejectAndRecoverer implements MessageRecoverer {
#Override
public void recover(Message message, Throwable cause) {
if (ExceptionUtils.getRootCause(cause) instanceof BusinessObjectNotFoundRuntimeException) {
throw new ListenerExecutionFailedException("Retry Policy Exhausted",
new AmqpRejectAndDontRequeueException(cause), message);
}
}
}

You currently need a different container factory for each different retry configuration.
In 2.0, we have added a new errorHandler attribute to the annotation so each listener can have a customized error handler, regardless of the container factory it was created by.
This was in the first milestone release; the current milestone is M2 and M3 will be out shortly. The GA release is expected in June.

If common logic from the shared SimpleRabbitListenerContainerFactory doesn't fit your requirements for particular #RabbitListener, you have to declare a new one, with those custom options, e.g. a new MessageRecoverer on the matter.
For that purpose the #RabbitListener has attribute:
/**
* The bean name of the {#link org.springframework.amqp.rabbit.listener.RabbitListenerContainerFactory}
* to use to create the message listener container responsible to serve this endpoint.
* <p>If not specified, the default container factory is used, if any.
* #return the {#link org.springframework.amqp.rabbit.listener.RabbitListenerContainerFactory}
* bean name.
*/
String containerFactory() default "";

How to create Processor with Transaction and DLQ with Rabbit binding?

I'm just starting to learn Spring Cloud Streams and Dataflow and I want to know one of important use cases for me. I created example processor Multiplier which takes message and resends it 5 times to output.
#EnableBinding(Processor.class)
public class MultiplierProcessor {
#Autowired
private Source source;
private int repeats = 5;
#Transactional
#StreamListener(Processor.INPUT)
public void handle(String payload) {
for (int i = 0; i < repeats; i++) {
if(i == 4) {
throw new RuntimeException("EXCEPTION");
}
source.output().send(new GenericMessage<>(payload));
}
}
}
What you can see is that before 5th sending this processor crashes. Why? Because it can (programs throw exceptions). In this case I wanted to practice fault prevention on Spring Cloud Stream.
What I would like to achieve is to have input message backed in DLQ and 4 messages that were send before to be reverted and not consumed by next operand (just like in normal JMS transaction). I tried already to define following properties in my processor project but without success.
spring.cloud.stream.bindings.output.producer.autoBindDlq=true
spring.cloud.stream.bindings.output.producer.republishToDlq=true
spring.cloud.stream.bindings.output.producer.transacted=true
spring.cloud.stream.bindings.input.consumer.autoBindDlq=true
Could you tell me if it possible and also what am I doing wrong? I would be overwhelmingly thankful for some examples.

You have several issues with your configuration:
missing .rabbit in the rabbit-specific properties)
you need a group name and durable subscription to use autoBindDlq
autoBindDlq doesn't apply on the output side
The consumer has to be transacted so that the producer sends are performed in the same transaction.
I just tested this with 1.0.2.RELEASE:
spring.cloud.stream.bindings.output.destination=so8400out
spring.cloud.stream.rabbit.bindings.output.producer.transacted=true
spring.cloud.stream.bindings.input.destination=so8400in
spring.cloud.stream.bindings.input.group=so8400
spring.cloud.stream.rabbit.bindings.input.consumer.durableSubscription=true
spring.cloud.stream.rabbit.bindings.input.consumer.autoBindDlq=true
spring.cloud.stream.rabbit.bindings.input.consumer.transacted=true
and it worked as expected.
EDIT
Actually, no, the published messages were not rolled back. Investigating...
EDIT2
OK; it does work, but you can't use republishToDlq - because when that is enabled, the binder publishes the failed message to the DLQ and the transaction is committed.
When that is false, the exception is thrown to the container, the transaction is rolled back, and RabbitMQ moves the failed message to the DLQ.
Note, however, that retry is enabled by default (3 attempts) so, if your processor succeeds during retry, you will get duplicates in your output.
For this to work as you want, you need to disable retry by setting the max attempts to 1 (and don't use republishToDlq).
EDIT3
OK, if you want more control over the publishing of the errors, this will work, when the fix for this JIRA is applied to Spring AMQP...
#SpringBootApplication
#EnableBinding({ Processor.class, So39018400Application.Errors.class })
public class So39018400Application {
public static void main(String[] args) {
SpringApplication.run(So39018400Application.class, args);
}
#Bean
public Foo foo() {
return new Foo();
}
public interface Errors {
#Output("errors")
MessageChannel errorChannel();
}
private static class Foo {
#Autowired
Source source;
#Autowired
Errors errors;
#StreamListener(Processor.INPUT)
public void handle (Message<byte[]> in) {
try {
source.output().send(new GenericMessage<>("foo"));
source.output().send(new GenericMessage<>("foo"));
throw new RuntimeException("foo");
}
catch (RuntimeException e) {
errors.errorChannel().send(MessageBuilder.fromMessage(in)
.setHeader("foo", "bar") // add whatever you want, stack trace etc.
.build());
throw e;
}
}
}
}
with properties:
spring.cloud.stream.bindings.output.destination=so8400out
spring.cloud.stream.bindings.errors.destination=so8400errors
spring.cloud.stream.rabbit.bindings.errors.producer.transacted=false
spring.cloud.stream.rabbit.bindings.output.producer.transacted=true
spring.cloud.stream.bindings.input.destination=so8400in
spring.cloud.stream.bindings.input.group=so8400
spring.cloud.stream.rabbit.bindings.input.consumer.transacted=true
spring.cloud.stream.rabbit.bindings.input.consumer.requeue-rejected=false
spring.cloud.stream.bindings.input.consumer.max-attempts=1