In my little microservice, I created a Producer Kafka to send the messages with errors (messages having errors in the JSON format) inside the DeadLetter in this way :
#Component
public class KafkaProducer {
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
public void sendDeadLetter(String message) {
kafkaTemplate.send("DeadLetter", message);
}
}
I would like to create a JUnitTest for the completeness of the project, but I have no idea how to create the eventuality of a possible JSON error in order to create the test. I thank everyone for any possible help and advice
To create a JUnitTest consistent with your code. I should recreate the case where you pass it a warped or invalid JSON. In your case, I would opt to configure a MockConsumer from which to read any message that the logic of your code will be invited to the dead letter.
To have a usable test structure, I recommend something like this:
#KafkaListener(topics = "yourTopic")
public void listen(String message) {
messages.add(message);
}
For testing a basic structure could be
#Test
public void testDeadLetter(){
//Set up a mockConsumer
MockConsumer<String,String> yourMockConsumer = new MockConsumer<String,String> (OffsetResetStrategy.EARLIEST);
yourMockConsumer.subscribe(Collections.singletonList("yourTopic"));
//Sending message on embedded Kafka broker
String error = "ERRORE";
kafkaTemplate.send("yourTopic", error);
//Reading the message may take a second
Thread.sleep(1000);
//Create an Assert that checks you that the message is equal to the error specified above
}
I hope it will be useful to you!
You can create Kafka topic using testcontainers and write your tests on top of that.
Sharing an example on how to use testcontainers https://github.com/0001vrn/testcontainers-example
Related
I am trying to send message on kafka when there is an insertion in a particular table Record.
I consider that this part of the application will be considered a Supplier/Producer.
I have following code.
#Bean
public Supplier<RecordAlteredEvent> affectedRecordEventEmitter() {
return (/*how do I pass the data?*/) -> {
log.info("SENDING_MESSAGE TO RECORD_EVENT_TOPIC");
return new RecordAlteredEvent();
};
}
I actually want to send the data. So, I am looking for something more like following:
#Bean
public Function<RecordAlteredEvent, RecordAlteredEvent> alteredRecordEventEmitter() {
return (RecordAlteredEvent recordAlteredEvent) -> {
log.info(SENDING_MESSAGE, VDP_USERS_EVENT_TOPIC, recordAlteredEvent.toString());
return recordAlteredEvent;
};
}
Is this possible? How do I configure that ? So, for example if
I declare a 'Supplier' then how do I pass data to it ?
And if I declare a 'Function' it seems that the input will be received from the topic and forwarded. And I am not receiving the data from topic. I am receiving data from REST API/DB.
I am not sure if the following can somehow help: But with a basic try I could not succeed. hence, an example will help a lot.
#Autowired
private StreamBridge streamBridge;
// Can following work somehow in-side a function that is, for example, in spring-boot-service
streamBridge.send("myDestination", body);
For reference:
https://cloud.spring.io/spring-cloud-stream/reference/html/spring-cloud-stream.html#spring_cloud_function
https://tanzu.vmware.com/developer/guides/scs-gs/
Section : Generating Loan Applications
How can create a producer using Spring Cloud Kafka Stream 3.1
Following worked
declare field
private final StreamBridge streamBridge;
Declared functions
public void delegateRecordsEventSupplier(#NotNull final RecordsEvent alteredRecordsEvent) {
streamBridge.send("affectedRecordsEvent-out-0", alteredRecordsEvent);
}
These functions when called will place the message on the topic
I am new to Akka and trying to write some code in Play Framework 2 in Java and use Akka.
To create an actor and send a test message to it, I have:
public class Global extends GlobalSettings {
#Override
public void onStart(Application app) {
final ActorRef testActor = Akka.system().actorOf(Props.create(TestActor.class), "testActor");
testActor.tell("hi", ActorRef.noSender());
}
}
This work perfectly fine and I see that my actor received my message, here is the code for my actor:
public class TestActor extends UntypedActor {
#Override
public void onReceive(Object message) throws Exception {
if(message.toString().equals("hi")){
System.out.println("I received a HI");
}else{
unhandled(message);
}
}
}
Very simple.
However, If I try to send a message from a controller:
public static Result index() {
final ActorRef testActor = Akka.system().actorFor("testActor");
testActor.tell("hi", ActorRef.noSender());
return ok(index.render("Your new application is ready."));
}
I get this message on terminal:
[INFO] [09/20/2014 11:40:30.850] [application-akka.actor.default-dispatcher-4] [akka://application/testActor] Message [java.lang.String] from Actor[akka://application/deadLetters] to Actor[akka://application/testActor] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
Can someone help me with this? why the first usage works and the second one fails? Thanks
The actorFor method requires the entire path, and your actor lives in the user space, so you have to use actorFor("/user/testActor"). Currently you are sending it to application/testActor, which would be a top-level actor in the ActorSystem itself.
By the way, actorFor is deprecated (at least in the Scala API) and replaced by actorSelection.
For more information, refer to the excellent documentation.
actorFor should get the path to the actor which is probably "akka://System/user/testActor". It also does not create the actor, meaning it should be exist.
Anyway, is there a reason that inside the controller you use the actorFor and not the actorOf? It had been deprecated and shouldn't be use.
I am coming from Sidekiq and am now moving to a Java solution for distributed jobs. I came across RabbitMQ and ActiveMQ, but it seems those brokers use plaintext or raw byte[] messages. I was wondering if it's possible to send custom messages using these frameworks?
Ideally I would just define a Java class for each specific message type and use that in both worker and producer. Is such a thing possible? Or should I look at other types of middleware?
MyOwnMessageFormat message = new MyOwnMessageFormat(content)
channel.send(message)
Message message = channel.receive()
if (message.class == MyOwnMessageFormat)
{
doSomething();
}
Exchanging messages through message brokers in Java is much easier when done through a service bus like Camel. You dont loose the flexibility of configuring your broker endpoint and still your code is isolated from the particular transport used or the message format. E.g. you can deploy ActiveMQ and later switch to RabbitMQ without having to update your code - just the service bus configuration. Or you can switch from plain Java serialization to JSON when sending messages to the broker by adding a message transformer. Again your business layer does not have to be modified.
Here's a sample that uses POJO producing/consuming where the producer calls a regular Java interface and the consumer implements the interface. The sample assumes that the sender/receiver are instantiated with Spring in order for the Camel endpoints to be injected
Message sender:
interface MyService {
MyResult addTask(MyTask task);
}
class Sender {
#Produce(uri="activemq:queue:myservice")
MyService service;
public void run() {
MyTask task = new MyTask();
MyResult result = service.addTask(task);
}
Message receiver:
class Receiver {
#Consume(uri="activemq:queue:myservice")
public MyResult addTask(MyTask task) {
return new MyResult();
}
}
MyTask & MyResult need to be serializable.
I think learning the Camel framework is not very hard while it can be very rewarding.
In JMS,as long as your custom message object is Serializable, you can use an ObjectMessage,Stream Message or Map Message according to your requirement like below to send(Object Message):
MessageProducer producer = session.createProducer( destination );
ObjectMessage message = session.createObjectMessage( getMyObject() );
producer.send( message );
For receiving:
Message message = consumer.receive();
if (message instanceof ObjectMessage) {
Object object = ((ObjectMessage) message).getObject();
Hope this helps you!
Yes, Before publishing the message to RabbitMQ, convert your message to a Json String.
Now on the receiving end, when the message is received, Parse it to convert into the same format.
In ruby this can be done with
while sending => Message.to_json
on Receiving => message = JSON.parse(received_msg)
To get the class of data, you can send a variable specifying the class of the data along with the data.
This is what worked for me with the latest version of ActiveMQ
...
MessageConsumer consumer=session.createConsumer(destination);
while(true) {
javax.jms.Message message=consumer.receive();
ActiveMQObjectMessage queueMessage=(ActiveMQObjectMessage)message;
Object payload=queueMessage.getObject();
if(payload instanceof NotificationMessage) {
this.sendMessage((NotificationMessage)payload);
}
}
The NotificationMessage object extends Serializable and looks like this
public static class NotificationMessage implements Serializable {
/**
*
*/
private static final long serialVersionUID = 1631373969001850200L;
public String to;
public String data;
}
My application sends message to Amazon Simple Notification Service (SNS) topic but sometime (6/10) I get java.net.UnknownHostException:sqs.ap-southeast-1.amazonaws.com. The reason of exception is described in the amazon web services discussion forums, please look: https://forums.aws.amazon.com/thread.jspa?messageID=499290񹹚.
My problem is similar to what described in forums of amazon but my rate of publishing messages to topic is very dynamic. It can be 1 message/second or 1 message/minute or no message in an hour. I am looking for a cleaner, better and safe approach, which guaranties sending of message to SNS topic.
Description of problem in detail:
Topic_Arn= arn of SNS topic where application wants to publish message
msg = Message to send in topic
// Just a sample example which publish message to Amazon SNS topic
class SimpleNotificationService {
AmazonSNSClient mSnsClient = null;
static {
createSnsClient()
}
private void static createSnsClient() {
Region region = Region.getRegion(Regions.AP_SOUTHEAST_1);
AWSCredentials credentials = new
BasicAWSCredentials(AwsPropertyLoader.getInstance().getAccessKey(),
AwsPropertyLoader.getInstance().getSecretKey());
mSqsClient = new AmazonSQSClient(credentials);
mSqsClient.setRegion(region);
}
public void static publishMessage(String Topic_Arn, String msg) {
PublishRequest req = new PublishRequest(Topic_Arn, msg);
mSnsClient.publish(req);
}
}
class which calls SimpleNotificationService
class MessagingManager {
public void sendMessage(String message) {
String topic_arn = "arn:of:amazon:sns:topic";
SimpleNotificationService.publishMessage(topic_arn, message);
}
}
Please note that this is a sample code, not my actual code. Here can be class design issue but please ignore those if they are not related to problem.
My thought process says to have try-catch block inside sendMessage, so when we catch UnknownHostException then again retry but I am not sure how to write this in safer, cleaner and better way.
So MessagingManager class will look something like this:
class MessagingManager {
public void sendMessage(String message) {
String topic_arn = "arn:of:amazon:sns:topic";
try {
SimpleNotificationService.publishMessage(topic_arn, message);
} catch (UnknownHostException uhe) {
// I need to catch AmazonClientException as aws throws
//AmazonClientException when sees UnknownHostException.
// I am mentioning UnknownHostException for non-aws user to understand
// my problem in better way.
sendMessage(message); // Isn't unsafe? - may falls into infinite loop
}
}
}
I am open for answers like this: java.net.UnknownHostException: Invalid hostname for server: local but my concern is to dependent on solution at application code-level and less dependent on changes to machine. As my server application is going to run in many boxes (developer boxes, testing boxes or production boxes). If changes in machine host-files or etc is only guaranted solution then I prefer that to include with code level changes.
Each AWS SDK implements automatic retry logic. The AWS SDK for Java automatically retries requests, and you can configure the retry settings using the ClientConfiguration class.
Below is the sample example to create SNS client. It retries for 25 times if encounters UnKnownHostException. It uses default BackOff and retry strategy. If you want to have your own then you need to implement these two interfaces: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/retry/RetryPolicy.html
private void static createSnsClient() {
Region region = Region.getRegion(Regions.AP_SOUTHEAST_1);
AWSCredentials credentials = new
BasicAWSCredentials(AwsPropertyLoader.getInstance().getAccessKey(),
AwsPropertyLoader.getInstance().getSecretKey());
ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setMaxErrorRetry(25);
clientConfiguration.setRetryPolicy(new RetryPolicy(null, null, 25, true));
mSnsClient = new AmazonSNSClient(credentials, clientConfiguration);
mSnsClient.setRegion(region);
}
Have you considering looking into the JVM TTL for the DNS Cache?
http://docs.aws.amazon.com/AWSSdkDocsJava/latest//DeveloperGuide/java-dg-jvm-ttl.html
i need to mock a javax.mail.Session object in my unit tests.
The class javax.mail.Session is marked final so Mockito is not able to create a mock. Does anyone have an idea how to fix this?
Edit:
My test is an Arquillian test and has already an annotation #RunWith(Arquillian.class). Therefore powermock is not an option.
You may refactor your code a little bit. In the "Working with Legacy Code" book by Martin Fowler, he describes a technique of separating the external API (think Java Mail API) from your own application code. The technique is called "Wrap and Skin" and is pretty simple.
What you should do is simply:
Create an interface MySession (which is your own stuff) by extracting methods from the javax.mail.Session class.
Implement that interface by creating a concrete class (looking a bit like the original javax.mail.Session)
In each method DELEGATE the call to equivalent javax.mail.Session method
Create your mock class which implements MySession :-)
Update your production code to use MySession instead of javax.mail.Session
Happy testing!
EDIT: Also take a look at this blog post: http://www.mhaller.de/archives/18-How-to-mock-a-thirdparty-final-class.html
Use PowerMockito to mock it.
#RunWith(PowerMockRunner.class)
// We prepare PartialMockClass for test because it's final or we need to mock private or static methods
#PrepareForTest(javax.mail.Session.class)
public class YourTestCase {
#Test
public void test() throws Exception {
PowerMockito.doReturn(value).when(classUnderTest, "methodToMock", "parameter1");
}
}
You can use the Mock JavaMail project. I first found it from Kohsuke Kawaguchi. Alan Franzoni also has a primer for it on his blog.
When you put this jar file in your classpath, it substitutes any sending of mail to in memory mailboxes that can be checked immediately. It's super easy to use.
Adding this to your classpath is admittedly a pretty heavy handed way to mock something, but you rarely want to send real emails in your automated tests anyway.
If you want to mock a final classes you can use the JDave unfinalizer which can be found here : http://jdave.org/documentation.html#mocking
It uses CGLib to alter the bytecode dynamically when the JVM is loaded to transform the class as a non final class.
This library can then be used with JMock2 ( http://www.jmock.org/mocking-classes.html ) to make your tests because as far as I know, Mockito is not compatible with JDave.
Use Java 8 Functions!
public class SendEmailGood {
private final Supplier<Message> messageSupplier;
private final Consumer<Message> messageSender;
public SendEmailGood(Supplier<Message> messageSupplier,
Consumer<Message> messageSender) {
this.messageSupplier = messageSupplier;
this.messageSender = messageSender;
}
public void send(String[] addresses, String from,
String subject, String body)
throws MessagingException {
Message message = messageSupplier.get();
for (String address : addresses) {
message.addRecipient
(Message.RecipientType.TO, new InternetAddress(address));
}
message.addFrom(new InternetAddress[]{new InternetAddress(from)});
message.setSubject(subject);
message.setText(body);
messageSender.accept(message);
}
}
Then your test code will look something like the following:
#Test
public void sendBasicEmail() throws MessagingException {
final boolean[] messageCalled = {false};
Consumer<Message> consumer = message -> {
messageCalled[0] = true;
};
Message message = mock(Message.class);
Supplier<Message> supplier = () -> message;
SendEmailGood sendEmailGood = new SendEmailGood(supplier, consumer);
String[] addresses = new String[2];
addresses[0] = "foo#foo.com";
addresses[1] = "boo#boo.com";
String from = "baz#baz.com";
String subject = "Test Email";
String body = "This is a sample email from us!";
sendEmailGood.send(addresses, from, subject, body);
verify(message).addRecipient(Message.RecipientType.TO, new InternetAddress("foo#foo.com"));
verify(message).addRecipient(Message.RecipientType.TO, new InternetAddress("boo#boo.com"));
verify(message).addFrom(new InternetAddress[]{new InternetAddress("baz#baz.com")});
verify(message).setSubject(subject);
verify(message).setText(body);
assertThat(messageCalled[0]).isTrue();
}
To create an integration test, plugin the real Session, and Transport.
Consumer<Message> consumer = message -> {
try {
Transport.send(message);
} catch (MessagingException e) {
e.printStackTrace();
}
};
Supplier<Message> supplier = () -> {
Properties properties = new Properties();
return new MimeMessage(Session.getDefaultInstance(properties));
};
See the PowerMock docs for running under JUnit 3, since it did not have runners, or use a byte-code manipulation tool.
If you can introduce Spring into your project you can use the JavaMailSender and mock that. I don't know how complicated your requirements are.
import org.springframework.mail.SimpleMailMessage;
import org.springframework.mail.javamail.JavaMailSender;
#Test
public void createAndSendBookChangesMail() {
// Your custom service layer object to test
MailServiceImpl service = new MailServiceImpl();
// MOCK BEHAVIOUR
JavaMailSender mailSender = mock(JavaMailSender.class);
service.setMailSender(mailSender);
// PERFORM TEST
service.createAndSendMyMail("some mail message content");
// ASSERT
verify(mailSender).send(any(SimpleMailMessage.class));
}
This is a pretty old question, but you could always implement your own Transport using the JavaMail API. With your own transport, you could just configure it according to this documentation. One benefit of this approach is you could do anything you want with these messages. Perhaps you would store them in a hash/set that you could then ensure they actually got sent in your unit tests. This way, you don't have to mock the final object, you just implement your own.
I use the mock-javamail library. It just replace the origin javamail implementation in the classpath. You can send mails normally, it just sends to in-memory MailBox, not a real mailbox.
Finally, you can use the MailBox object to assert anything you want to check.