Develop Reference

Load neo4j dump in test container during integration test run

I am trying to write integration test for neo4j using spring boot. I am using test container. Can anyone help me how to load database dump file to testcontainer?

here's one way to do this (using current Neo4j 5.3). I have a dump file called neo4j.dump created from a local instance and I use withCopyFileToContainer to copy it to the container before it starts.
As I use the community edition in this example, there is no online backup/restore, but only dump/load. So therefor I have to change the startup command. The load needs to happen before Neo4j starts. I can't stop Neo4j in the container because it would stop the container.
Therefor I create a small shell script that executes the desired other command and than delegates to the original entry point.
This script is transferred with file mode 0100555, corresponding to r-xr-xr-x so that it is executable.
Finally, the container is started with the above script as command.
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.neo4j.driver.AuthTokens;
import org.neo4j.driver.Driver;
import org.neo4j.driver.GraphDatabase;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.images.builder.Transferable;
import org.testcontainers.utility.MountableFile;
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
public class LoadDumpTest {
Neo4jContainer<?> neo4j;
Driver driver;
#BeforeAll
void initNeo4j() {
neo4j = new Neo4jContainer<>("neo4j:5.3.0")
.withCopyFileToContainer(MountableFile.forClasspathResource("neo4j.dump"),
"/var/lib/neo4j/data/dumps/neo4j.dump")
.withCopyToContainer(Transferable.of("""
#!/bin/bash -eu
/var/lib/neo4j/bin/neo4j-admin database load neo4j
/startup/docker-entrypoint.sh neo4j
""", 0100555), "/startup/load-dump-and-start.sh")
.withCommand("/startup/load-dump-and-start.sh")
.withLogConsumer(f -> System.out.print(f.getUtf8String()));
neo4j.start();
driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", neo4j.getAdminPassword()));
}
#Test
void dataShouldHaveBeenLoaded() {
try (var session = driver.session()) {
var numNodes = session.run("MATCH (n) RETURN count(n)").single().get(0).asLong();
Assertions.assertTrue(numNodes > 0);
}
}
#AfterAll
void stopNeo4j() {
neo4j.stop();
}
}
Edit:
If you are on a recent enterprise edition, Christophe suggested the following solution on Twitter, which I do personally think it's superior, as it is less hacky.
https://github.com/ikwattro/neo4j-5-testcontainers-restore-backup/blob/main/src/test/java/dev/ikwattro/Neo4j5RestoreBackupExampleTest.java
It makes use of seed URIs for databases. Copying or binding the resource in his example works the same as in mine.

If you're using Neo4j 5, I suggest you make backups instead of dumps. Using backups you can take advantage of the seedUri option when creating a database, meaning you can create a database from an URI pointing to a backup on disk ( or in the neo4j container ). https://neo4j.com/docs/operations-manual/current/clustering/databases/#cluster-seed-uri
Here is an example using Testcontainers
package dev.ikwattro;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.neo4j.driver.AuthTokens;
import org.neo4j.driver.Driver;
import org.neo4j.driver.GraphDatabase;
import org.neo4j.driver.Session;
import org.neo4j.driver.SessionConfig;
import org.testcontainers.containers.BindMode;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import static org.assertj.core.api.Assertions.assertThat;
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
#Testcontainers(disabledWithoutDocker = true)
public class Neo4j5RestoreBackupExampleTest {
#Container
private Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:5.3.0-enterprise")
.withAdminPassword("password")
.withEnv("NEO4J_dbms_memory_heap_max__size", "256M")
.withEnv("NEO4J_dbms_databases_seed__from__uri__providers", "URLConnectionSeedProvider")
.withClasspathResourceMapping("backups", "/backups", BindMode.READ_ONLY)
.withEnv("NEO4J_ACCEPT_LICENSE_AGREEMENT", "yes");
#BeforeAll
void beforeAll() {
neo4j.start();
createDbFromBackup();
}
#Test
void testCreatingDbFromBackup() {
try (Driver driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", "password"))) {
try (Session session = driver.session(SessionConfig.forDatabase("worldcup22"))) {
var result = session.run("MATCH (n) RETURN count(n) AS c").single().get("c").asLong();
assertThat(result).isPositive();
}
}
}
private void createDbFromBackup() {
try (Driver driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", "password"))) {
try (Session session = driver.session(SessionConfig.forDatabase("system"))) {
session.run("""
CREATE DATABASE worldcup22 OPTIONS { existingData: "use", seedUri: "file:///backups/world-cup-2022-neo4j.backup"}
""");
}
}
}
}
You can find a working maven project here https://github.com/ikwattro/neo4j-5-testcontainers-restore-backup

JUnit sampler not working properly with CSV data set Config for multiple Threads Jmeter

I tried to use Junit Request Sampler for login to my test application with multiple users by using CSV data set config. Ex : I set number of thread count as 2 and set two user login details in .csv file and i run the test. The result was open the two firefox browsers and one browser logged successfully and other one is not get username and password into username and password fields in login page. This is my selenium script code. Please anyone can suggest the reason for this issue ?
import org.apache.jmeter.protocol.java.sampler.JUnitSampler;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
public class testClass {
static WebDriver driver;
JUnitSampler sampler = new JUnitSampler();
String userName = sampler.getThreadContext().getVariables().get("username");
String password = sampler.getThreadContext().getVariables().get("password");
String Empnamecsv = sampler.getThreadContext().getVariables().get("Empname");
#BeforeClass
public static void setUpBeforeClass() throws Exception
{
System.setProperty("webdriver.gecko.driver", "D:\\Automation\\Geckodriver\\V0.19.0\\geckodriver.exe");
driver = new FirefoxDriver();
}
#Test
public void loadHomePage() throws InterruptedException
{
driver.get("http://localhost/testWeb");
Thread.sleep(1000);
}
#Test
public void login() throws InterruptedException
{
driver.findElement(By.id("txtusername")).sendKeys(userName);
driver.findElement(By.id("txtpassword")).sendKeys(password);
driver.findElement(By.id("btnsubmit")).click();
Thread.sleep(1000);
String name = driver.findElement(By.xpath("/html/body/div[1]/div[2]/div[3]/span[1]/span[1]")).getText();
Assert.assertEquals(name,namecsv);
}
}

Try updating your "login" method to look like:
#Test
public void login() throws InterruptedException {
org.apache.jmeter.threads.JMeterVariables vars = org.apache.jmeter.threads.JMeterContextService.getContext().getVariables();
driver.findElement(By.id("txtusername")).sendKeys(vars.get(userName));
driver.findElement(By.id("txtpassword")).sendKeys(vars.get(password));
driver.findElement(By.id("btnsubmit")).click();
Thread.sleep(1000);
String name = driver.findElement(By.xpath("/html/body/div[1]/div[2]/div[3]/span[1]/span[1]")).getText();
Assert.assertEquals(name, vars.get("Empnamecsv"));
}
References:
JMeterContextService
JMeterContext
JMeterVariables
Also be aware of alternative ways of JMeter and Selenium integration:
WebDriver Sampler - comes with Selenium client libraries and Configuration Elements to set up the browser instance. Automatically handles multi-threaded execution
JSR223 Sampler - you can use it instead of JUnit, this way you won't have to recompile the code to bring changes. Assuming you choose Groovy language the performance will be nearly the same as for Java

Your issue is that you're initializing userName and password once:
String userName = sampler.getThreadContext().getVariables().get("username");
String password = sampler.getThreadContext().getVariables().get("password");
String Empnamecsv = sampler.getThreadContext().getVariables().get("Empname");
In JMeter you can test JUnit classes but here it seems you're creating it in JUnit test class:
JUnitSampler sampler = new JUnitSampler();
So there is a conception problem.
If you want to use a CSV, then create a Test plan that uses a Webdriver Sampler and use JMeter assertions for your checks.
See an example here:
http://www.ubik-ingenierie.com/blog/leverage-your-load-testing-using-jmeter-and-selenium-webdriver/

Unit testing a verticle deployment

I have a simple Verticle that reads configuration from a properties file and loads in into vertx config. I have written a unit test to test the deployment of this verticle and possible cause of test failure is non availability of the properties file at the location.
When I run the test, unit test passes irrespective of whether I change the properties file name or path and the handler says the verticle was deployed successfully.
Am I doing something wrong here? Below is my code
import io.vertx.config.ConfigRetrieverOptions;
import io.vertx.config.ConfigStoreOptions;
import io.vertx.core.DeploymentOptions;
import io.vertx.core.json.JsonObject;
import io.vertx.rxjava.config.ConfigRetriever;
import io.vertx.rxjava.core.AbstractVerticle;
/**
* This is the main launcher verticle, the following operations will be executed in start() method of this verticle:
* 1. Read configurations from application.properties file
* 2. Deploy all other verticles in the application
*/
public class LauncherVerticle extends AbstractVerticle {
#Override
public void start() throws Exception {
//set up configuration from the properties file
ConfigStoreOptions fileStore = new ConfigStoreOptions()
.setType("file")
.setFormat("properties")
.setConfig(new JsonObject().put("path", System.getProperty("vertex.config.path"));
//create config retriever options add properties to filestore
ConfigRetrieverOptions options = new ConfigRetrieverOptions().addStore(fileStore);
ConfigRetriever configRetriever = ConfigRetriever.create(vertx, options);
DeploymentOptions deploymentOptions = new DeploymentOptions();
//Deploy verticles after the config has been loaded
//The configurations are loaded into JsonConfig object
//This JsonConfig object can be accessed in other verticles using the config() method.
configRetriever.rxGetConfig().subscribe(s -> {
//pass on the JsonConfig object to other verticles through deployment options
deploymentOptions.setConfig(s);
vertx.deployVerticle(AnotherVerticle.class.getName(), deploymentOptions);
}, e -> {
log.error("Failed to start application : " + e.getMessage(), e);
try {
stop();
} catch (Exception e1) {
log.error("Unable to stop vertx, terminate the process manually : "+e1.getMessage(), e1);
}
});
}
}
This is my unit test
import io.vertx.ext.unit.TestContext;
import io.vertx.ext.unit.junit.VertxUnitRunner;
import io.vertx.rxjava.core.Vertx;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import rx.Single;
#RunWith(VertxUnitRunner.class)
public class LoadConfigurationTest {
/**
* Config should be loaded successfully
*
* #param context
*/
#Test
public void loadConfigTest(TestContext context) {
/*
* Set the system property "vertx.config.path" with value "application.properties"
* This system property will be used in the Launcher verticle to read the config file
*/
System.setProperty("vertx.config.path", "/opt/vertx/config/application.properties");
//create vertx instance
Vertx vertx = Vertx.vertx();
Single<String> single = vertx.rxDeployVerticle(LauncherVerticle.class.getName());
single.subscribe(s -> {
vertx.rxUndeploy(s);
}, e -> {
Assert.fail(e.getMessage());
});
}
/**
* Test for negative use case - file not available in the specified location
*
* #param context
*/
#Test
public void loadConfigFailTest(TestContext context) {
//set path = non existing path
System.setProperty("vertx.config.path", "/non/existing/path/application.properties");
//create vertx instance
Vertx vertx = Vertx.vertx();
Single single = vertx.rxDeployVerticle(LauncherVerticle.class.getName());
single.subscribe(s -> {
//not executing this statement
Assert.fail("Was expecting error but Verticle deployed successfully");
}, e -> {
//not executing this statement either
System.out.println("pass");
});
}
}

Can you try the below code inside your LauncherVerticle the changes only include using AbstractVerticles start with Future which is a neat way for handling the config loading and everything around the same during your starup.
public class LauncherVerticle extends AbstractVerticle {
#Override
public void start(Future<Void> startFuture) throws Exception {
ConfigStoreOptions fileStore = new ConfigStoreOptions()
.setType("file")
.setFormat("properties")
.setConfig(new JsonObject().put("path", System.getProperty("vertex.config.path")));
ConfigRetrieverOptions options = new ConfigRetrieverOptions().addStore(fileStore);
ConfigRetriever configRetriever = ConfigRetriever.create(vertx, options);
DeploymentOptions deploymentOptions = new DeploymentOptions();
configRetriever.rxGetConfig().subscribe(s -> {
deploymentOptions.setConfig(s);
vertx.deployVerticle(AnotherVerticle.class.getName(),
deploymentOptions,
result -> startFuture.complete()
);
},
startFuture::fail
);
}
}
startFuture there, would help you to control the state of your verticle loading.
Also remember that #Constantine way for handing the test is best way, use of Async to prevent your tests passing without actually asserting anything.

Seems like there is nothing wrong with your verticle. However, there is something in tests - the asynchronous nature of verticle deployment is not taken into account. These test methods finish immediately instead of waiting for verticle deployment, and JUnit test that does not result in AssertionError is a passed test. You have to signal completion explicitly using Async.
Please see an example for your negative scenario below:
import io.vertx.ext.unit.Async;
import io.vertx.ext.unit.TestContext;
import io.vertx.ext.unit.junit.RunTestOnContext;
import io.vertx.ext.unit.junit.VertxUnitRunner;
import io.vertx.rxjava.core.Vertx;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
#RunWith(VertxUnitRunner.class)
public class LoadConfigurationTest {
#Rule
public RunTestOnContext runTestOnContextRule = new RunTestOnContext();
#Test
public void testConfigLoading_shouldFail_whenConfigDoesNotExist(TestContext context) {
// create an Async instance that controls the completion of the test
Async async = context.async();
// set non existing path
System.setProperty("vertx.config.path", "/non/existing/path/application.properties");
// take vertx instance and wrap it with rx-ified version
Vertx vertx = Vertx.newInstance(runTestOnContextRule.vertx());
vertx.rxDeployVerticle(LauncherVerticle.class.getName()).subscribe(s -> {
context.fail("Was expecting error but Verticle deployed successfully"); // failure
}, e -> {
async.complete(); // success
});
}
}
Also please note that you can take a Vertx instance from RunTestOnContext rule (as in the snippet above).

Adding REST route to an existing Jetty endpoint in Camel at runtime

I have been inventing a way how to work around the problem of adding consumers to a jetty endpoint (it does not allow multiple consumers). The way we do it in our company is to build our own router and a broadcasting endpoint which consumes from jetty and routes requests to underlying "subscriptions". Only one of them will eventually process the request. It kind of works but it's not completely ok, since recently when updating to latest Camel we have found our custom built component to leak memory and in general I consider using built-in functionality over custom hacks.
I started investigating the Camel REST API and found it very nice and pretty much replacing our home-grown component apart from one thing - you cannot re-configure it at runtime - you have to stop the context basically for this to work. Below I include my unit test with a happy path and the path that fails. Frankly I think is a bug, but if there is a legitimate way to achieve what I want, I'd like to hear sound advice:
package com.anydoby.camel;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.fail;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.commons.io.IOUtils;
import org.junit.Before;
import org.junit.Test;
/**
* Test tries to add/remove routes at runtime.
*/
public class RoutesTest {
private DefaultCamelContext ctx;
#Before
public void pre() throws Exception {
ctx = new DefaultCamelContext();
new RouteBuilder(ctx) {
#Override
public void configure() throws Exception {
restConfiguration("jetty").host("localhost").port(8080);
rest("/")
.get("/issues/{isin}").route().id("issues")
.process(e -> e.getOut().setBody("Here's your issue " + e.getIn().getHeader("isin"))).endRest()
.get("/listings").route().id("listings").process(e -> e.getOut().setBody("some listings"));
}
}.addRoutesToCamelContext(ctx);
ctx.start();
}
#Test
public void test() throws IOException {
{
InputStream stream = new URL("http://localhost:8080/issues/35").openStream();
assertEquals("Here's your issue 35", IOUtils.toString(stream));
}
{
InputStream stream = new URL("http://localhost:8080/listings").openStream();
assertEquals("some listings", IOUtils.toString(stream));
}
}
#Test
public void disableRoute() throws Exception {
ctx.stopRoute("issues");
ctx.removeRoute("issues");
try (InputStream stream = new URL("http://localhost:8080/issues/35").openStream()) {
fail();
} catch (Exception e) {
}
new RouteBuilder(ctx) {
#Override
public void configure() throws Exception {
rest().get("/issues/{isin}/{sedol}").route().id("issues")
.process(e -> e.getOut()
.setBody("Here's your issue " + e.getIn().getHeader("isin") + ":" + e.getIn().getHeader("sedol")))
.endRest();
}
}.addRoutesToCamelContext(ctx);
{
InputStream stream = new URL("http://localhost:8080/issues/35/65").openStream();
assertEquals("Here's your issue 35:65", IOUtils.toString(stream));
}
}
}
The disableRoute() test fails since I cannot add another consumer to an existing endpoint.
So my question is - "is there a way to add a new URL mapping to a restful camel-jetty endpoint"? If you do it during first configuration it works fine, but when later you want to reconfigure one of the routes the error is:
org.apache.camel.FailedToStartRouteException: Failed to start route because of Multiple consumers for the same endpoint is not allowed: jetty:http://localhost:8080/issues/%7Bisin%7D/%7Bsedol%7D?httpMethodRestrict=GET

How to run Spring Shell scripts in a JUnit test

I have a Spring Shell-based application and a couple of scripts. Is there an easy way to run the scripts in a JUnit test such that a test fails, if some exception/error occurs during the execution of the script?
The purpose of the tests is to make sure that all correct scripts run without errors.
Update 1:
Here's a little helper class for running scripts in JUnit:
import org.apache.commons.io.FileUtils;
import org.springframework.shell.Bootstrap;
import org.springframework.shell.core.CommandResult;
import org.springframework.shell.core.JLineShellComponent;
import java.io.File;
import java.io.IOException;
import java.util.List;
import static org.fest.assertions.api.Assertions.*;
public class ScriptRunner {
public void runScript(final File file) throws IOException
{
final Bootstrap bootstrap = new Bootstrap();
final JLineShellComponent shell = bootstrap.getJLineShellComponent();
final List<String> lines = FileUtils.readLines(file);
for (final String line : lines) {
execVerify(line, shell);
}
}
private void execVerify(final String command, final JLineShellComponent shell) {
final CommandResult result = shell.executeCommand(command);
assertThat(result.isSuccess()).isTrue();
}
}

You can create an instance of Bootstrap, get the shell out of it and then executeCommand() (including the shell command) on it.
You may be interested in what is done in Spring XD for this: https://github.com/spring-projects/spring-xd/blob/master/spring-xd-shell/src/test/java/org/springframework/xd/shell/AbstractShellIntegrationTest.java (although there are a lot of XD specific details)