Develop Reference

how to prevent jdbc from trying to connect to a mysql database during unit testing

I'm making an application for a school project, but I'm running into the issue that when I try to run the unit tests that it tries to connect to the database while starting up the application, which isn't required for the tests (because it will be mocked), and is not available in the CI/CD pipeline.
jdbc connection error
I'm building my project in Java Maven Springboot and would like to know how I can prevent it from trying to connect to the database when running my test.
here is a link to my repository: https://gitlab.com/kwetter_jack/Kwetter_posts/-/tree/ci_cd_setup

Your test classes have #SpringBootTest annotation which will start a Spring application context - as your application uses a database the tests will also try to setup and use a database connection.
The simplest solution is to remove the annotation so the tests no longer try to connect to a database. You'll probably need to mock some more dependencies as a result as Spring is no longer creating these for you. You could also have a look at https://www.baeldung.com/spring-boot-testing for some other ideas how you could alter your tests.
Alternatively if you do want / need the application context to run you can add a application.yaml for the tests that defines and uses a in memory DB so the tests have something to connect to - see https://www.baeldung.com/spring-boot-h2-database for details how to do this.

Just change value under spring.datasource to H2 database to prevent
The application connect the real database.
Test application.yml
FYI, You no need to copy all config from original application.yml, just only some config that you need to override.

while I was investigating the spring boot H2 in-memory database (as suggested by Chris Olive and Paranaaan) I also came across the option of using a test container. after looking into this I saw that this enables the project to create a temp docker container with a MySQL image that I can use during the testing of my project, considering I was planning on using docker anyway for the integration testing of my microservices project I attempted this and it worked as I had hoped it would.
if anyone is interested in the test container solution that I used, the information can be found here:
https://www.testcontainers.org/modules/databases/mysql/

How to package a spring boot project using spring data elastcsearch with maven without skiping test?

I'm putting together a spring boot project that uses spring boot elasticsearch, the plan is to package the project on my local machine as a .jar file, send it to my offsite server, and run it.
On my server, elasticsearch is set to accept connections via local network's ip addresses only (i.e. curl localhost:9200 won't work but curl 172.168.0.7:9200 does), which i'm not able to access on my local machine.
When i run the project, i set the elasticsearch uris in my yml files to localhost, and it works fine. But when i try to package the project with maven, it runs test, which then connects to 172.168.0.7:9200 and of course fails, leaving me with a failed message like can't connect to that address.
The workaround is to set maven to skip tests altogehter, but that does't seem to be the right way to do it.
What should i do so that i can package the project with test?
Did a little research and most of them were about unit test, but in my case i didn't write any test classes, don't have surefire, i'm just stuck in the process of packaging.

Run integration tests in-memory(embedded) MongoDB in Spring application

I am trying to make the integration tests run in memory in MongoDB. I would appreciate any idea or suggestion.
What I did:
A dependency on de.flapdoodle.embed:de.flapdoodle.embed.mongo is added to provide embedded MongoDB for integration tests.
The embedmongo-maven-plugin is added to start or stop an instance of MongoDB during a Maven build.
My idea here is when I run 'mvn clean test' command the integration tests connect to the embedded mongo instance and not the "live" one(mongo-java-driver). After running test, I can see in the consolo log embedded mongo is started:
o.s.b.a.mongo.embedded.EmbeddedMongo : 2021-08-04T15:23:14.303+0300 I CONTROL [initandlisten] MongoDB starting : pid=26673 port=54871 dbpath=/var/folders/qv/......
However, I think it doesn't work as expected. Because, in the console log, I can also see:
org.mongodb.driver.cluster : Monitor thread successfully connected to server with description ServerDescription{address=...
If I shut down the live instance of Mongo, tests failed. :(
Is there anyone to help me to overcome this issue? I want to thank in advance for your time.

Writing Java/Maven integration tests using webservice database and jms

I'm maintaining a Java Enterprise Application that exposes a few webservices and interacts with a few JMS queues and a database.
I've got most of the code unit tested and I thought all was good until in a refactoring I moved a class in another package and that changed a namespace in my webservice without me noticing and breaking all clients.
An Integration Test would have caught that, so I'm trying to write one.
The application is deployed on JBoss EAP 6.4, how can I make a webservice call in my mvn verify step?
What about JMS Queues? They are configured on the Application Server.
Is the application supposed to be already deployed?
Am I supposed to deploy the application with maven to a JBoss installation before the verify step or start an embedded webserver?
Most of the docs around are confusing to me and often I see suggestion to mock stuff, which is not integration testing (and I already do in Unit tests).
Someone told me "just use Jenkins" and I read the docs, installed it and still don't understand how that is supposed to help me write integration tests since all it does is run mvn verify anyway.

This topic and is too broad, there might be many different correct answers for this question and will depend a lot on the technologies you're using, so I'll focus first in this part only:
that changed a namespace in my webservice without me noticing and
breaking all clients
You can create unit tests for endpoints too, I do that all the time with spring-boot. For example, the code below starts the application, runs the testEndpoint() test and shuts down the application right after.
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = {MyApplication.class})
#WebAppConfiguration
#IntegrationTest("server.port:0")
public class MyControllerTest {
#Value("${local.server.port}")
private int port;
#Test
public void testEndpoint() {
String endpointUrl = "http://localhost:" + port;
// make a HTTP request here to test the endpoint
}
}
I believe this can be done with any spring-mvc application, but the code would be a bit different and not as easy as with spring-boot.
Although this would usually catch most bugs in endpoints, it doesn't eliminate the need of integration tests against a deployed project.
So focusing on the bigger picture now, if you want to create end-to-end tests, including WebServices, JMS queues and databases, I suggest creating a separate project with tests only, probably using Cucumber or something similar. This project should be triggered in Jenkins (or any other CI tool) whenever needed (e.g. before a deployment, every hour, every commit and/or every night) and it will require that all applications are already deployed.
Alternatively, you could have Jenkins deploy an application and run integration tests only against this one application. In this case the tests will depend on all other applications to be already deployed.

Amazon Kinesis + Integration Tests

I'm currently working on a series of web-services which we need to integrate with Kinesis - the implementation has been done, however we have a series of integration tests (our web-services are all using Spring Boot so we use the #WebIntegrationTest annotation on our test classes to start up a local instance of the server and then call our resources with a TestRestTemplate) which are currently trying and failing to connect to the real Kinesis.
Although in ordinary unit tests it's not a problem to mock out calls to the methods within the Kinesis library, we can't really do this in the integration tests as the whole application stack is wired up with Spring. For a few other things (such as OAuth2 and calls to our other web-services) we've been able to use WireMock to mock out the actual endpoints - what I'd really like to do is use WireMock in this fashion to mock out the call to the AmazonKinesisClient but I can't find any advice on how to do this.
Alternatively I have seen that some AWS components have test libraries written by third parties which allow you to run a local version of it (e.g.: DynamoDbLocal) but can't find such a solution for Kinesis.
Is anyone able to give me some advice on how to run integration tests with Kinesis?

It might already be too late to give the solution but I will add what my team has done to replicate AWS resources locally as we use a lot of Kinesis, DynamoDb, S3 and cloudWatch.
We have created wrappers around Localstack -> https://github.com/localstack/localstack that allow us to spin up local instances of the necessary services as docker containers using docker-compose.
A typical docker-compose.yml file for us looks like:
version: '2'
services:
localstack:
image: "localstack/localstack"
environment:
- SERVICES=kinesis,dynamodb,cloudwatch
ports:
- "4568"
- "4569"
- "4582"
Then during the setup phase for the integration-tests, our wrapper fires up docker-compose up and runs the tests against the local infrastructure.
Later during tear-down, the wrapper kills the containers.

I ran into the same issue and the only mock implementation I found so far was a nodejs one: https://github.com/mhart/kinesalite
It does the trick - I managed to run my Java Kinesis client against it, just had to set the endpoint on the kinesis.properties:
kinesisEndpoint=http://localhost:4567
The downside is that it is not trivial to use it during build time tests - need to figure a way to start the mock kinesis before the test (using a maven plugin or something), didn't get to it yet..

Just a small addition to the existing answers. BTW, they are great, you should really use tools like localstack to start fake AWS services before the test during the test phase.
If you're using JUnit 5 in your tests, your life could be even simpler with JUnit 5 extensions for AWS, a few JUnit 5 extensions that could be useful for testing AWS-related code. These extensions can be used to inject clients for AWS service mocks provided by tools like localstack. Both AWS Java SDK v 2.x and v 1.x are supported:
#ExtendWith(DynamoDB.class)
class AmazonDynamoDBInjectionTest {
#AWSClient(
endpoint = Endpoint.class
)
private AmazonDynamoDB client;
#Test
void test() throws Exception {
Assertions.assertNotNull(client);
Assertions.assertEquals(
Collections.singletonList("table"),
client.listTables().getTableNames().stream().sorted().collect(Collectors.toList())
);
}
}
Here, client will be just injected in your test class and configured according to the Endpoint configuration class.