I'm trying to run Couchbase v.5.1.1 docker container for test purposes via Test container with fixed exposed ports, like:
trait CouchbaseTestEnvironment extends ForAllTestContainer {
this: Suite =>
def couchbaseContainer: FixedHostPortGenericContainer = {
val consumer = new Slf4jLogConsumer(LoggerFactory.getLogger(getClass))
/*
* Couchbase should know which ports are exposed for client, because this is how it exposes services.
* E.g. client ask only for on port - say 8091. And query service port is 8093. So client, won't ask for every port,
* instead CB will tell client on which port query service exposed, that's why CB should be aware about port mapping.
* That's why we need to give CB port mappings
*
* See for more details:
* https://stackoverflow.com/questions/59277436/couchbase-in-docker-for-integration-tests-make-the-ports-8092-8093-8094-and-8
*/
def randomPort: Int = {
val (from, to) = (32768, 35000) //linux private port range
from + Random.nextInt(to - from)
}
val random8091Port = randomPort
val random8092Port = randomPort
val random8093Port = randomPort
val random8094Port = randomPort
val random11210Port = randomPort
val container = FixedHostPortGenericContainer(
imageName = "couchbase:community-5.0.1",
exposedHostPort = random8091Port,
exposedContainerPort = random8091Port,
env = Map(
"COUCHBASE_RANDOM_PORT_8091" -> random8091Port.toString,
"COUCHBASE_RANDOM_PORT_8092" -> random8092Port.toString,
"COUCHBASE_RANDOM_PORT_8093" -> random8093Port.toString,
"COUCHBASE_RANDOM_PORT_8094" -> random8094Port.toString,
"COUCHBASE_RANDOM_PORT_11210" -> random11210Port.toString
)
)
container.container.withFixedExposedPort(random8092Port, random8092Port)
container.container.withFixedExposedPort(random8093Port, random8093Port)
container.container.withFixedExposedPort(random8094Port, random8094Port)
container.container.withFixedExposedPort(random11210Port, random11210Port)
container.container.withLogConsumer(consumer)
container
}
}
So as you can see 5 FIXED ports should be exposed.
But, when I'm running tests I actually can see, that instead other ports exposed with random ports:
docker ps
f4fc1ce06544 couchbase:community-5.0.1 "/entrypoint.sh /opt…" 59 seconds ago Up 1 second 0.0.0.0:55264->8091/tcp, 0.0.0.0:55263->8092/tcp, 0.0.0.0:55262->8093/tcp, 0.0.0.0:55261->8094/tcp, 0.0.0.0:55260->11207/tcp, 0.0.0.0:55259->11210/tcp, 0.0.0.0:55258->11211/tcp, 0.0.0.0:55257->18091/tcp, 0.0.0.0:55256->18092/tcp, 0.0.0.0:55255->18093/tcp, 0.0.0.0:55254->18094/tcp unruffled_mendel
03b491ac2ea8 testcontainersofficial/ryuk:0.3.0
So as you can see another ports was exposed, and mapped to random ports instead fixed.
As far as I understand, test containers, ignores ports I gave, and instead exposes ports from Couchbase Dockerfile: https://github.com/couchbase/docker/blob/master/community/couchbase-server/5.1.1/Dockerfile#L74
EXPOSE 8091 8092 8093 8094 8095 8096 11207 11210 11211 18091 18092 18093 18094 18095 18096
Can I somehow force Test containers to ignore EXPOSE instruction?
Partially helped question: Couchbase in docker for integration tests: Make the ports 8092, 8093, 8094 and 8095 configurable to be able to use docker’s random ports
Can I somehow force Test containers to ignore EXPOSE instruction?
I don't know if there is a simple configuration option for this, but a workaround solution I found is to use an advanced feature of the docker-java create container command customization. I'm providing an example in Java, translate it to Scala yourself, please. Apply it as the last command before returning a container object from your function:
container.withCreateContainerCmdModifier(
cmd -> cmd.getHostConfig().withPublishAllPorts(false)
);
The main point here is the usage of .withPublishAllPorts(false). From my understanding, this is the same as --publish-all (or -P) arguments of the docker run command. Testcontainers library sets this value to true by default. This modification overrides it to false.
With this configuration no ports are published at all for your example, not the 5 fixed as expected:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2ee4fb91b97c couchbase:community-5.0.1 "/entrypoint.sh couc…" 33 seconds ago Up 32 seconds 8091-8094/tcp, 11207/tcp, 11210-11211/tcp, 18091-18094/tcp trusting_keldysh
This is because in the answer you provided, the author created a special custom docker image of couchbase, which "understands" environment variables like COUCHBASE_RANDOM_PORT_8091. Your code uses the standard couchbase image couchbase:community-5.0.1, which basically just ignores these environment variables. So in order to run counchbase on not standard internal ports, you need to build a custom image with the "magic" configure-node.sh script, which tunes couchbase config using values provided in environment variables.
I hope it helps anyhow :)
Related
I am trying to POC accessing DynamoDB via an Apache Camel application. Obviously Dynamo DB will run in AWS but for development purposes we have it running locally as a docker container.
It was very easy to create a Dynamo BD table locally and put some items in there manually. I used for this my intelij Dynamo DB console and all I had to provide was a custom end point http://localhost:8000 and the Default credential provider chain.
Now at some certain times of the day I would like to trigger a job that will scan the Dynamo DB items and perform some actions on the returned data.
from("cron:myCron?schedule=0 */5 * * * *")
.log("Running myCron scheduler")
.setHeader(Ddb2Constants.OPERATION, () -> Ddb2Operations.Scan)
.to("aws2-ddb:myTable")
.log("Performing some work on items");
When I am trying to run my application it fails to start complaining that the security token is expired which makes me think it is trying to go to AWS rather than accessing the local. I was unable to find anything about how would I set this. The camel dynamo db component (https://camel.apache.org/components/3.15.x/aws2-ddb-component.html) is talking about being able to configure the component with a DynamoDbClient but this is an interface and its implementation called DefaultDynamoDbClient is not public and so is the DefaultDynamoDbClientBuilder.
Assuming that you use Spring Boot as Camel runtime, the simplest way in your case is to configure the DynamoDbClient used by Camel thanks to options set in the application.properties as next:
# The value of the access key used by the component aws2-ddb
camel.component.aws2-ddb.accessKey=test
# The value of the secret key used by the component aws2-ddb
camel.component.aws2-ddb.secretKey=test
# The value of the region used by the component aws2-ddb
camel.component.aws2-ddb.region=us-east-1
# Indicates that the component aws2-ddb should use the new URI endpoint
camel.component.aws2-ddb.override-endpoint=true
# The value of the URI endpoint used by the component aws2-ddb
camel.component.aws2-ddb.uri-endpoint-override=http://localhost:8000
For more details please refer to the documentation of those options:
camel.component.aws2-ddb.accessKey
camel.component.aws2-ddb.secretKey
camel.component.aws2-ddb.region
camel.component.aws2-ddb.override-endpoint
camel.component.aws2-ddb.uri-endpoint-override
For other runtimes, it can be configured programatically as next:
Ddb2Component ddb2Component = new Ddb2Component(context);
String accessKey = "test";
String secretKey = "test";
String region = "us-east-1";
String endpoint = "http://localhost:8000";
ddb2Component.getConfiguration().setAmazonDDBClient(
DynamoDbClient.builder()
.endpointOverride(URI.create(endpoint))
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(accessKey, secretKey)
)
)
.region(Region.of(region))
.build()
);
I am running a process every 5 minutes and checking to make sure every container. If the container does not respond I can flag it as down. I have the containers IP address and I loop through each ip and check if it responds to a ping. If not I flag it as down. Is there a better way to do this? My code:
#Transactional
#Scheduled(fixedRate = 1000 * 60) //5 min
public void monitorHosts(){
Iterable<Ncl> ncls = nclRepository.findAll();
for(Ncl ncl: ncls){
for(String host: ncl.getHosts()){
Boolean isHostAlive = isHostAlive(host);
if(!isHostAlive){
Ncl nclWorking = nclRepository.findOne(ncl.getUuid());
if(nclWorking != null){
Set<String> hosts = nclWorking.getHosts().stream().filter(x -> x.equals(host)).collect(Collectors.toSet());
nclWorking.getHosts().clear();
nclWorking = nclRepository.save(nclWorking);
nclWorking.setHosts(hosts);
nclRepository.save(nclWorking);
}
}
}
}
}
private Boolean isHostAlive(String host){
try{
InetAddress address = InetAddress.getByName(host);
boolean reachable = address.isReachable(10000);
return reachable;
} catch (Exception e){
e.printStackTrace();
return false;
}
}
It mostly depends on what you need to do with the information about your containers.
There is a number of monitoring solutions available, which can monitor your containers and notify some one if there are some troubles.
If you have to use this info in some application, then you can use some solutions like Consul.io and let them check your services statuses, not containers (in most cases man aware of the service availability in the container, not container itself). Or you can use docker-api for Java, because ICMP-protocol is not always a good solution, especially in distributed networks.
I would use docker events
extract from the doc
https://docs.docker.com/engine/reference/commandline/events/#examples
use something like
docker events --filter 'event=stop'
This is more obtrusive but is a nice one:
Using HEALTHCHECK that docker provides
You can customize it for each application. Something like:
HEALTHCHECK --interval=5m --timeout=3s \
CMD curl -f http://localhost/ || exit 1
Then you can check the health status as this:
docker inspect --format='{{json .State.Health}}' <container_id>
Or you can use docker ps and see the STATUS column:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
225426fc7c93 ubuntu "tail -f /dev/null" 5 seconds ago Up 4 seconds (healthy)
Or you can query the docker API:
curl --unix-socket /var/run/docker.sock http:/v1.24/containers/json
PS: You can set the healthcheck at docker run time (without modifying the Dockerfile). Docs
In the API documentation for Java Spark (not Apache spark), you can specify a port of 0 to have it automatically select a port. Great!
However, I cannot figure out how to get that port after the server is started. I can see it in the logs:
15:41:12.459 [Thread-2] INFO spark.webserver.JettySparkServer - >> Listening on 0.0.0.0:63134
But I need to be able to get to it programmatically, so that my integration tests are able to run reliably every time.
So how do I get that port?
I could find no way to get this information in the API, and so I filed an issue on their github.
I was able to get at it via an ugly pile of reflection:
/**
* Meant to be called from a different thread, once the spark app is running
* This is probably only going to be used during the integration testing process, not ever in prod!
*
* #return the port it's running on
*/
public static int awaitRunningPort() throws Exception {
awaitInitialization();
//I have to get the port via reflection, which is fugly, but the API doesn't exist :(
//Since we'll only use this in testing, it's not going to kill us
Object instance = getInstance();
Class theClass = instance.getClass();
Field serverField = theClass.getDeclaredField("server");
serverField.setAccessible(true);
Object oneLevelDeepServer = serverField.get(instance);
Class jettyServerClass = oneLevelDeepServer.getClass();
Field jettyServerField = jettyServerClass.getDeclaredField("server");
jettyServerField.setAccessible(true);
//Have to pull in the jetty server stuff to do this mess
Server jettyServer = (Server)jettyServerField.get(oneLevelDeepServer);
int acquiredPort = ((ServerConnector)jettyServer.getConnectors()[0]).getLocalPort();
log.debug("Acquired port: {}", acquiredPort);
return acquiredPort;
}
This works well for me in our integration tests, but I'm not using https, and it does reach about two levels deep into the API via reflection grabbing protected fields. I could not find any other way to do it. Would be quite happy to be proven wrong.
This will work on Spark 2.6.0:
public static int start (String keystoreFile, String keystorePw)
{
secure(keystoreFile, keystorePw, null, null);
port(0);
staticFiles.location("/public");
get(Path.CLOCK, ClockController.time);
get(Path.CALENDAR, CalendarController.date);
// This is the important line. It must be *after* creating the routes and *before* the call to port()
awaitInitialization();
return port();
}
Without the call to awaitInitialization() port() would return 0.
I am trying to retrieve all the instances running in my AWS account (say instance id, etc). I use the following code. I am not able to print the instance ids. When I debug, I am just getting null values. But I have three instances running on AWS. Can someone point out what I am doing wrong here?
DescribeInstancesResult result = ec2.describeInstances();
List<Reservation> reservations = result.getReservations();
for (Reservation reservation : reservations) {
List<Instance> instances = reservation.getInstances();
for (Instance instance : instances) {
System.out.println(instance.getInstanceId());
}
}
The most common cause for issues like this is a missing region specification when initializing the client, see section To create and initialize an Amazon EC2 client within Create an Amazon EC2 Client for details:
Specifically, step 2 only creates an EC2 client without specifying the region explicitly:
2) Use the AWSCredentials object to create a new AmazonEC2Client instance, as follows:
amazonEC2Client = new AmazonEC2Client(credentials);
This yields a client talking to us-east-1 - surprisingly, the AWS SDKs and the AWS Management Console use different defaults even as outlined in step 3, which also shows how to specify a different endpoint:
3) By default, the service endpoint is ec2.us-east-1.amazonaws.com. To specify a different endpoint, use the setEndpoint method. For example:
amazonEC2Client.setEndpoint("ec2.us-west-2.amazonaws.com");
The AWS SDK for Java uses US East (N. Virginia) as the default region
if you do not specify a region in your code. However, the AWS
Management Console uses US West (Oregon) as its default. Therefore,
when using the AWS Management Console in conjunction with your
development, be sure to specify the same region in both your code and
the console. [emphasis mine]
The differing defaults are easy to trip over, and the respective default in the AWS Management Console has in fact changed over time - as so often in software development, I recommend to always be explicit about this in your code to avoid such subtle error sources.
I am using the Docker-Java API found here https://github.com/docker-java/docker-java.
I have a dockerfile that I have to build 3 containers for. I am trying to automate the process of building 3 images and the commands to go with them when running the shell inside. The 3 containers must be in the same local network in order for them to communicate. I am able to do this manually just fine...
So first, using the docker-java API, I am building a custom network using the following function:
private void createNetwork() {
CreateNetworkResponse networkResponse = dockerClient.createNetworkCmd()
.withName("ETH")
.withDriver("bridge")
.withAttachable(true)
.exec();
System.out.printf("Network %s created...\n", networkResponse.getId());
}
This works great, and if I run docker network ls, I can see the ETH network listed.
The next step is building the image. I am running the following function:
public String buildImage(String tag) {
String imageID = dockerClient.buildImageCmd()
.withDockerfile(new File("/Dockerfile"))
.withPull(true)
.withNoCache(false)
.withTags(new HashSet<>(Collections.singletonList(tag)))
.withNetworkMode("ETH")
.exec(new BuildImageResultCallback())
.awaitImageId();
System.out.println("Built image: " + imageID);
return imageID;
}
So the image builds fine and I can see the image when I run the docker images command in terminal. I do expect that the image to be connected to the ETH network, but I do not see that.
I thought that maybe I have to connect to the network when creating the container instead then, so I pass the same commands I would if I were to manually do this when building the container through the following function:
private String createContainer(String name, String imageID, int port) {
CreateContainerResponse container = dockerClient
.createContainerCmd(name)
.withImage(imageID)
.withCmd("docker", "run", "--rm", "-i", "-p", port + ":" + port, "--net=ETH", name)
.withExposedPorts(new ExposedPort(port))
.exec();
dockerClient.startContainerCmd(container.getId()).exec();
return container.getId();
}
Unfortunately, when passing in the arguments like this, the built container does not show up in the ETH network when running the command docker network inspect ETH.
I'm not sure what I am doing wrong. If I build the image using the API, and then run the following command manually, docker run --rm -it -p 8545:8545 --net=ETH miner_one everything works fine. Any help would be greatly appreciated. Thank you!
The docker-java client supports a subset of the Docker Remote API. To connect to a network when you create the container set the NetworkMode field
(see HostConfig -> NetworkMode in Container Create section)
Network mode to use for this container. Supported standard values are:
bridge, host, none, and container:. Any other value is taken
as a custom network's name to which this container should connect to.
Therefore, in order for the container to connect to the custom network set the value of the network mode to ETH.
In Java, for older versions of the Docker-Java client, use the withNetworkMode() method:
CreateContainerResponse container = dockerClient
.createContainerCmd(name)
.withImage(imageID)
.withNetworkMode("ETH")
...
In the latest version, the methods in CreateContainerCmd used to set the fields in HostConfig are deprecated. Use withHostConfig() instead:
CreateContainerResponse container = dockerClient.createContainerCmd(name)
.withImage(imageID)
.withHostConfig(HostConfig.newHostConfig().withNetworkMode("ETH"))
...
Here is a basic example:
List<Network> networks = dockerClient.listNetworksCmd().withNameFilter("ETH").exec();
if (networks.isEmpty()) {
CreateNetworkResponse networkResponse = dockerClient
.createNetworkCmd()
.withName("ETH")
.withAttachable(true)
.withDriver("bridge").exec();
System.out.printf("Network %s created...\n", networkResponse.getId());
}
CreateContainerResponse container = dockerClient
.createContainerCmd("ubuntu")
.withName("my-ubuntu")
.withCmd("sleep", "10")
.withHostConfig(HostConfig
.newHostConfig()
.withNetworkMode("ETH")
.withAutoRemove(true))
.exec();
String containerId = container.getId();
dockerClient.startContainerCmd(containerId).exec();
Network ethNetwork = dockerClient.inspectNetworkCmd()
.withNetworkId("ETH")
.exec();
Set<String> containerIds = ethNetwork.getContainers().keySet();
if(containerIds.contains(containerId)) {
System.out.printf("Container with id:%s is connected to network %s%n", containerId, ethNetwork.getName());
}
It creates a network named ETH and a container my-ubuntu from an ubuntu image. The container is connected to the ETH network.
Hope this helps.