Develop Reference

NPE Testing Kafka Producer Using Embedded Kafka

I've written a basic spring boot service that consumes some data via rest API and publishes it to rabbitmq and kafka.
To test the service class handling kafka producing, I followed this guide: https://www.baeldung.com/spring-boot-kafka-testing
In isolation, the test (KafkaMessagingServiceImplTest) works perfectly both in intellij idea and via mvn on the command line. Running all project tests in idea works fine. However, when I run all project tests via maven on the command line, this test fails with an NPE when trying to make the assertion on the payload String.
I've narrowed down the location of the root problem to another test class (AppPropertiesTest) which is solely testing my AppProperties component (which is a component I use to pull config from application.properties in a tidy way). When, and only when, the tests within that test class are run alongside the failing test using 'mvn clean install' in project root, does the NPE show up. Commenting out the tests in this class or annotating it with #DirtiesContext fixes the problem. Apparently something loaded into the spring context by this test class causes an issue with the timing/order of events/countdownlatch in the other test. Of course, I don't want to use #DirtiesContext as it can lead to a much slower build as the project increases in complexity. It also does not explain the problem.. and I can't handle that :)
AppPropertiesTest uses constructor injection to inject the AppProperties component. It also extends a abstract class 'GenericServiceTest' which is annotated by:
#SpringBootTest
#TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
and contains nothing else. As you probably know, the SpringBootTest annotation builds a test spring context and wires in boilerplate to allow effective testing of a spring app's dependency injection etc. and the TestConstructor annotation allows constructor injection in some of my tests. FWIW, I have tried removing the TestConstructor annotation and using plain old Autowiring in the AppProperties class to see if it makes a difference but it does not.
The failing test class also extends GenericServiceTest, as it requires the spring context to inject some of the dependencies such as the consumer and the messaging service being tested and AppProperties instance within etc.
So I know where the problem lies but I don't know what the problem is. Even when the test fails with the NPE, I can see in the logs that the consumer has successfully consumed the message before the failure, as per the Baeldung guide :
TestKafkaConsumer : received payload='ConsumerRecord(topic = test-kafka-topic, partition = 0, leaderEpoch = 0, offset = 0, CreateTime = 1618997289238, serialized key size = -1, serialized value size = 43, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = This is a test message to be sent to Kafka.)'
However, the payLoad is null when we get back to the assertion. I've tried all kinds of things like Thread.sleep() in the failing test to give it more time and I've increased the await() timeout but no joy.
I find it bizarre that the tests are fine in IDEA and in isolation. Now it's starting to drive me a little crazy and I can't debug it because the problem doesn't occur in my IDE.
If anyone has any ideas, it would be greatly appreciated!
Thanks.
EDIT: Someone very reasonably suggested that I add some code so here goes :)
The Failing Test (fails at assertTrue(payload.contains(testMessage)) because payLoad is null). The autowired kafkaMessagingService simply has the dependencies of AppProperties and KakfaTemplate injected and calls kafkaTemplate.send():
#EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:9092", "port=9092" })
class KafkaMessagingServiceImplTest extends GenericServiceTest {
#Autowired
#Qualifier("kafkaMessagingServiceImpl")
private IMessagingService messagingService;
#Autowired
private TestKafkaConsumer kafkaConsumer;
#Value("${app.topicName}")
private String testTopic;
#Test
public void testSendAndConsumeKafkaMessage() throws InterruptedException {
String testMessage = "This is a test message to be sent to Kafka.";
messagingService.sendMessage(testMessage);
kafkaConsumer.getLatch().await(2000, TimeUnit.MILLISECONDS);
String payload = kafkaConsumer.getPayload();
assertTrue(payload.contains(testMessage));
}
The TestConsumer (used to consume in the test above)
#Component
public class TestKafkaConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(TestKafkaConsumer.class);
private CountDownLatch latch = new CountDownLatch(1);
private String payload = null;
#KafkaListener(topics = "${app.topicName}")
public void receive(ConsumerRecord<?, ?> consumerRecord) {
LOGGER.info("received payload='{}'", consumerRecord.toString());
setPayload(consumerRecord.toString());
latch.countDown();
}
public CountDownLatch getLatch() {
return latch;
}
public String getPayload() {
return payload;
}
public void setPayload(String payload) {
this.payload = payload;
}
Project dependencies:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
<version>2.5.8.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
<!-- https://mvnrepository.com/artifact/org.mockito/mockito-all -->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.10.19</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.springframework.kafka/spring-kafka-test -->
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<version>2.5.6.RELEASE</version>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
AppPropertiesTest class (the context of which seems to cause the problem)
class AppPropertiesTest extends GenericServiceTest {
private final AppProperties appProperties;
public AppPropertiesTest(AppProperties appProperties) {
this.appProperties = appProperties;
}
#Test
public void testAppPropertiesGetQueueName() {
String expected = "test-queue";
String result = appProperties.getRabbitMQQueueName();
assertEquals(expected, result);
}
#Test
public void testAppPropertiesGetDurableQueue() {
boolean isDurableQueue = appProperties.isDurableQueue();
assertTrue(isDurableQueue);
}
}
The AppProperties class that the AppPropertiesTest class is testing:
#Component
#ConfigurationProperties("app")
public class AppProperties {
// a whole bunch of properties by name that are prefixed by app. in the application.properties file. Nothing else
}
The Generic service test class which both tests extend.
#SpringBootTest
#TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
public abstract class GenericServiceTest {
}
The failure (you can see on the line above the payload has been received and printed out).
2021-04-21 14:15:07.113 INFO 493384 --- [ntainer#0-0-C-1] service.TestKafkaConsumer : received payload='ConsumerRecord(topic = test-kafka-topic, partition = 0, leaderEpoch = 0, offset = 0, CreateTime = 1619010907076, serialized key size = -1, serialized value size = 43, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = This is a test message to be sent to Kafka.)'
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.791 s <<< FAILURE! - in
service.KafkaMessagingServiceImplTest
[ERROR] testSendAndConsumeKafkaMessage Time elapsed: 2.044 s <<< ERROR!
java.lang.NullPointerException
at service.KafkaMessagingServiceImplTest.testSendAndConsumeKafkaMessage(KafkaMessagingServiceImplTest.java:42)

The problem is that TestListener is a #Component so it is being added twice - the record is going to the other instance.
I added more debugging to verify the getter is called on a different instance.
#Component
public class TestKafkaConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(TestKafkaConsumer.class);
private final CountDownLatch latch = new CountDownLatch(1);
private String payload = null;
#KafkaListener(id = "myListener", topics = "${app.kafkaTopicName}")
public void receive(ConsumerRecord<?, ?> consumerRecord) {
LOGGER.info("received payload='{}'", consumerRecord.toString());
setPayload(consumerRecord.toString());
if (payload != null) {
LOGGER.info(this + ": payload is not null still");
}
latch.countDown();
if (payload != null) {
LOGGER.info(this + ": payload is not null after latch countdown");
}
}
public CountDownLatch getLatch() {
return latch;
}
public String getPayload() {
LOGGER.info(this + ": getting Payload");
return payload;
}
public void setPayload(String payload) {
this.payload = payload;
}
}
If you don't want to use #DirtiesContext, you can at least stop the listener containers after the tests complete:
#SpringBootTest
#TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
public abstract class GenericDataServiceTest {
#AfterAll
static void stopContainers(#Autowired KafkaListenerEndpointRegistry registry) {
registry.stop();
}
}
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Unable to use full text search with oak lucene

I am trying out jackrabbit oak with lucene in a file node store. The index definition record is created successfully but it seems the index record is not created.
The full project I am working on is here
What I am doing in the code is initializa a repository, create the lucene index definition, add a test data with a single property "name" with value "foo".
Sleep for 5 second for async indexing to complete, then perform the following query in a retry loop
select * from [nt:base] where contains(., 'foo').
No result is returned.
I have tried oak-run console to retrieve lc info on oak:index/lucene directory, it displays no result as well.
This is the main part of the code
public static void main( String[] args ) throws Exception
{
init();
createLuceneIndex();
createTestData();
performQuery();
}
private static void init() throws InvalidFileStoreVersionException, IOException {
LuceneIndexProvider provider = new LuceneIndexProvider();
FileStore fs = FileStoreBuilder.fileStoreBuilder(new File("repository")).build();
nodestore = SegmentNodeStoreBuilders.builder(fs).build();
repository = new Jcr(new Oak(nodestore))
.withAsyncIndexing("async", 3)
.with(new LuceneIndexEditorProvider())
.with((QueryIndexProvider) provider)
.with((Observer)provider)
.withAsyncIndexing()
.createRepository();
}
private static void createLuceneIndex() throws RepositoryException {
Session session = createAdminSession();
Node indexesNode = session.getRootNode().getNode("oak:index");
IndexDefinitionBuilder idxBuilder = new IndexDefinitionBuilder();
IndexRule indexRules = idxBuilder.indexRule("nt:unstructured");
indexRules.sync();
indexRules.property("name").analyzed().nodeScopeIndex();
idxBuilder.async("async");
idxBuilder.includedPaths("/");
Node documentIndex = indexesNode.addNode("lucene", "oak:QueryIndexDefinition");
idxBuilder.build(documentIndex);
session.save();
session.logout();
}
private static void createTestData() throws LoginException, RepositoryException {
Session session = createAdminSession();
Node test = session.getRootNode().addNode("test");
test.setProperty("name", "foo");
session.save();
session.logout();
}
private static void performQuery() throws Exception {
final Session session = createAdminSession();
TimeUnit.MICROSECONDS.sleep(5);
QueryManager qm = session.getWorkspace().getQueryManager();
final Query q = qm.createQuery("select * from [nt:base] where contains(., 'foo')", Query.JCR_SQL2);
new RetryLoop(new RetryLoop.Condition() {
public String getDescription() {
return "Full text query";
}
public boolean isTrue() throws Exception {
QueryResult r = q.execute();
return r.getNodes().hasNext();
}
}, 20, 500);
}
the pom file dependencies
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.jcr</groupId>
<artifactId>jcr</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.apache.jackrabbit</groupId>
<artifactId>oak-jcr</artifactId>
<version>1.21-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.jackrabbit</groupId>
<artifactId>oak-core</artifactId>
<version>1.21-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.jackrabbit</groupId>
<artifactId>oak-jcr</artifactId>
<version>1.21-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.jackrabbit</groupId>
<artifactId>oak-segment-tar</artifactId>
<version>1.21-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.jackrabbit</groupId>
<artifactId>oak-lucene</artifactId>
<version>1.21-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.30</version>
</dependency>
<dependency>
<groupId>com.codahale.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>3.0.2</version>
</dependency>

Springcloud bus custom messages cannot be sent through rabbitmq

When using the springcloud bus, a custom message is created and sent through rabbitmq, but after the message is sent, it does not go to rabbitmq. When you try to call /actuator/bus-refresh, you can see the bus messages emitted from the console page of rabbitmq.
I tried to start a micro service to register a custom event listener but failed to receive it. However, if the sender registers a listener himself, he can receive it but does not send it from rabbitmq.
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.3.RELEASE</version>
<relativePath/>
</parent>
<dependencies>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bus-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Greenwich.SR1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
About #RemoteApplicationEventScan annotations, I code all under the same package, so I should be able to scan to TestEvent. I also tried to specify basepackage.
#SpringBootApplication
#RemoteApplicationEventScan
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
#RestController
#RequestMapping("test")
public class TestController {
#Autowired
private ApplicationContext context;
#RequestMapping("test")
public String test() {
final TestEvent testEvent = new TestEvent(this, context.getId(), null,"test");
context.publishEvent(testEvent);
return "success";
}
}
#Data
public class TestEvent extends RemoteApplicationEvent {
private String action;
public TestEvent(Object source, String originService, String destinationService, String action) {
super(source, originService, destinationService);
this.action = action;
}
}
When I called http://localhost:8080/actuator/bus-refresh I can see the information in the rabbitmq.
{" type ":" AckRemoteApplicationEvent ", "timestamp" : 1554350325406, "originService" : "application: 0: b3461fbec3536203a7020ff9d24bb11b", "destinationService" : "* *", "id" : "e6b875bd - 2402-494 - f - a870 - 4917324 d2c5c ackId ", "" :" af93075e - 55 d2-41 f8 - ba27 - e3c80cf19eea ", "ackDestinationService" : "* *", "the event" is: "org. Springframework. Cloud. Bus. Event. RefreshRemoteApplicationEvent"}.
But when I call to http://localhost:8080/test/test, I don't.

I came into the same issue a few days ago and it turned out that it was because the originService was incorrect. passing in context.getId() as originService doesn't work.
Short answer: use org.springframework.cloud.bus.BusProperties#id. You can inject BusProperties to your component. Or you can configure your own spring cloud bus id as stated in the document.
I am not 100% sure this is the proper way. Maybe I missed something in the document. It is just based on what I read from the source code of org.springframework.cloud.bus.BusAutoConfiguration, method acceptLocal.
Hope it works for you.

PUT and POST not working from J2SE platform (RESTLET Version 3.0-M1)

I'm working on a project with GWT and J2SE clients. The GWT part is working great, but now there are problems with the J2SE client;
"The server understands the content type of the request entity and the
syntax of the request entity is correct but was unable to process the
contained instructions"
"The serialized representation must have this media type:
application/x-java-serialized-object or this one:
application/x-java-serialized-object+xml"
This code was working some months/versions ago... Both PUT and POST produce this error while GET is working. Whats wrong here?
Here's a really simple test case
// Shared Interface
public interface J2SeClientServerResourceInt
{
#Post("json")
public J2seStatusDto postJ2seStatus(J2seStatusDto pJ2seStatusDto);
}
// Java Bean
public class J2seStatusDto implements Serializable
{
private static final long serialVersionUID = 6901448809350740172L;
private String mTest;
public J2seStatusDto()
{
}
public J2seStatusDto(String pTest)
{
setTest(pTest);
}
public String getTest()
{
return mTest;
}
public void setTest(String pTest)
{
mTest = pTest;
}
}
// Server
public class J2seServerResource extends ClaireServerResource implements J2SeServerResourceInt
{
#Override
public J2seStatusDto postJ2seStatusDto(J2seStatusDto pJ2seStatusDto)
{
return pJ2seStatusDto;
}
}
// J2SE Client
public class ClaireJsSeTestClient
{
public static void main(String[] args)
{
Reference lReference = new Reference("http://localhost:8888//rest/j2se");
ClientResource lClientResource = new ClientResource(lReference);
lClientResource.accept(MediaType.APPLICATION_JSON);
J2SeServerResourceInt lJ2SeServerResource = lClientResource.wrap(J2SeServerResourceInt.class);
J2seStatusDto lJ2seStatusDto = new J2seStatusDto("TEST");
J2seStatusDto lJ2seResultDto = lJ2SeServerResource.postJ2seStatusDto(lJ2seStatusDto);
}
}
// Maven J2Se Client
<dependencies>
<dependency>
<groupId>org.restlet.jse</groupId>
<artifactId>org.restlet</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.jse</groupId>
<artifactId>org.restlet.ext.jackson</artifactId>
<version>3.0-M1</version>
</dependency>
</dependencies>
// Maven GAE Server
<dependency>
<groupId>org.restlet.gae</groupId>
<artifactId>org.restlet</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.gae</groupId>
<artifactId>org.restlet.ext.servlet</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.gae</groupId>
<artifactId>org.restlet.ext.jackson</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.gae</groupId>
<artifactId>org.restlet.ext.gwt</artifactId>
<version>3.0-M1</version>
</dependency>
<dependency>
<groupId>org.restlet.gwt</groupId>
<artifactId>org.restlet</artifactId>
<version>3.0-M1</version>
<scope>compile</scope>
</dependency>

Thierry Boileau fixed our problem (mistake);
https://github.com/restlet/restlet-framework-java/issues/1029#issuecomment-76212062
Due to the constraints of the GAE platform (there is no support of chunked encoding) you have to specify that the request entity is buffered first;
cr.setRequestEntityBuffering(true);
Thanks for the great support at restlet.com

How to test static method invokation in Android?

I have an Android activity and I want to write a unit test, which verifies that in onResume the activity checks whether the Internet is available.
public class MyActivity {
#Override
protected void onResume() {
super.onResume();
setContentView(R.layout.connect_to_server);
// Internet availability check
final IInternetAvailabilityChecker checker = InternetAvailabilityChecker.create(this);
if (!checker.isInternetAvailable())
{
Utils.showMessageBox(this, R.string.app_name,
R.string.internet_not_available);
return;
}
In the test, I want to verify that MyActiviy.onResume calls the InternetAvailabilityChecker.create method.
How can I do it (with any free mocking framework compatible with Android) ?
I tried to use PowerMock for this (see example below), but when I try to run the test, I get errors like MockTest.java:7: package org.powermock.core.classloader.annotations does not exist.
Maven:
<properties>
<powermock.version>1.5.1</powermock.version>
</properties>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito</artifactId>
<version>${powermock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
</dependency>
Unit test:
#RunWith(PowerMockRunner.class)
#PrepareForTest( { InternetAvailabilityChecker.class })
public class MyActivityPowerMockTest {
#Test
#Ignore
public void test()
{
final IInternetAvailabilityChecker checker = mock(IInternetAvailabilityChecker.class);
when(checker.isInternetAvailable()).thenReturn(false);
mockStatic(InternetAvailabilityChecker.class);
expect(InternetAvailabilityChecker.create(any(Activity.class))).andReturn(checker);
replay(InternetAvailabilityChecker.class);
final MyActivity objectUnderTest = new MyActivity();
objectUnderTest.onResume();
// Verify that the method InternetAvailabilityChecker.create was called
verify(InternetAvailabilityChecker.class);
// TODO: Verify that Utils.showMessageBox has been invoked
}
}

You appear to be missing a Maven dependency. According to this query the annotations are supplied by:
org.powermock / powermock-core
org.powermock / powermock-easymock-single-jar-release-full
org.powermock / powermock-mockito-single-jar-release-full