Develop Reference

Executable file not found in $PATH unkown

I am currently trying to set-up an instance of Open Trip Planner 2 in docker. I have created two compose files which execute the same docker image, but with different arguments.
However, currently I am having quite a lot of trouble getting the OTP JAR file to execute correctly.
I am getting the following error:
ERROR: for otp Cannot start service otp: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "otp": executable file not found in $PATH: unknown.
After a quick google search I found the
following question, but after changing the docker-compose command argument, the error still occurs.
My Dockerfile:
FROM openjdk:11
LABEL maintainer="Tristan van Triest <tristantriest#gmail.com>"
ENV OTP_VERSION=2.1.0
ADD https://repo1.maven.org/maven2/org/opentripplanner/otp/$OTP_VERSION/otp-$OTP_VERSION-shaded.jar /usr/local/share/java/
RUN ln -s otp-$OTP_VERSION-shaded.jar ./otp.jar
COPY . .
EXPOSE 8080
ENTRYPOINT [ "otp" ]
My docker-compose file:
version: '3.7'
services:
otp:
build:
context: .
dockerfile: dockerfile
restart: unless-stopped
ports:
- 9494:8080
volumes:
- ./graphs:/var/otp/graphs
environment:
- JAVA_OPTIONS=-Xmx8G
command:
[
'--maxThreads 4',
'--insecure',
'--verbose',
]
The "entrypoint" OTP file:
#!/bin/sh
exec java $JAVA_OPTIONS -jar otp.jar $#
I just simply want to be able to run the OTP Jar with different arguments. Is there an easier way to do this? Or if my current way is "correct", how can I solve the error?
Thanks,
Tristan

Docker compose gradle plugin exits with "docker inspect" requires at least 1 argument

I have the following docker-compose file:
version: '3.1'
services:
localstack:
image: localstack/localstack:0.9.6
ports:
- "4576:4576"
environment:
- SERVICES=sqs:4576
- HOSTNAME_EXTERNAL=localhost
- DEBUG=0
- START_WEB=0
primarydb:
image: mysql:5.6
environment:
MYSQL_USER: test-user
MYSQL_PASSWORD: test-password
MYSQL_DATABASE: testdb
MYSQL_ROOT_PASSWORD: test-password
ports:
- "3318:3306"
And I have the following gradle configuration:
apply plugin: 'docker-compose'
dockerCompose.isRequiredBy(project.tasks.getByName('test'))
dockerCompose {
useComposeFiles = ['docker-compose.yml']
startedServices = ['localstack', 'primarydb']
projectName = "${rootProject.name}"
}
project.tasks.getByName('test').doFirst {
systemProperty 'localstack.sqs.endpoint', "http://localhost:4576"
systemProperty 'mysql.url', "jdbc:mysql://localhost:3318"
}
And I am using the latest docker-compose plugin: com.avast.gradle:gradle-docker-compose-plugin:0.15.1
For some reason, the build works on my friend's Linux machine, but it doesn't work on MacOS 11.4.
When I run ./gradlew composeUp task it just fails with the following error message:
./gradlew composeUp
> Configure project :
2.5.25-SNAPSHOT
> Task :composeUp
Container primarydb-1 Running
Container localstack-1 Recreate
Container localstack-1 Recreated
Container localstack-1 Starting
Container localstack-1 Started
"docker inspect" requires at least 1 argument.
See 'docker inspect --help'.
Usage: docker inspect [OPTIONS] NAME|ID [NAME|ID...]
Return low-level information on Docker objects
> Task :composeUp FAILED
...
Any idea why this plugin doesn't work on a mac but works on my friend's machine?

I faced the same error. Running ./gradle composeUp --stacktrace gave a clue what com.avast.gradle:gradle-docker-compose-plugin:0.15.1 is doing. In essence to does:
docker-compose -p <project name> up
docker-compose -p <project name> config --services and finally
docker-compose -p <project name> ps --services
The third a last step is the one which was failing on me (and is the one currently failing on you.) Repeating the compose-commands manually and having a look into the Docker Desktop UI I could see, that docker-compose did alter my project-name by removing any dots from it. After respecifying
dockerCompose {
[...]
projectName = 'name-without-dots'
}
thus ensuring that docker-compose wont alter the project name, did solve the issue for me.
Your error cause migth still be a different one. Try running ./gradlew composeUp with --stacktrace, --info or even --debug to get a better understanding of what is going wrong.

How do I get Java testcontainers to work in Docker Multistage builds?

I have a problem similar to Run (Docker) Test Container in gitlab with Maven. The difference is that rather than my script running mvn directly it runs a docker multistage build that runs the test inside of the docker image. Unfortunately this doesn't appear to work for the PostgreSQL Test Container.
Dockerfile
#############
### build ###
#############
# base image
FROM maven:3-jdk-11 as build
# set working directory
WORKDIR /app
# add app
COPY . .
RUN export MAVEN_OPTS="-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true" && export MAVEN_CLI_OPTS="-B -U --batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
RUN mvn $MAVEN_CLI_OPTS clean install
############
### prod ###
############
# Yea this isn't right, but it crashes before it gets to this point. This is for example purposes only.
FROM openjdk:15-jdk-alpine
COPY --from=build /app/reproducer-testcontainer/target/reproducer-testcontainer.jar /reproducer-testcontainer.jar
CMD java -jar reproducer-testcontainer.jar
When I run mvn clean install it works properly and runs my test using the PostgreSQL Test Container. However, when I run docker build . it fails at the mvn clean install step with the below stack trace.
Stack trace:
13:05:01.250 [main] ERROR org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy - ping failed with configuration Environment variables, system properties and defaults. Resolved:
dockerHost=unix:///var/run/docker.sock
apiVersion='{UNKNOWN_VERSION}'
registryUrl='https://index.docker.io/v1/'
registryUsername='root'
registryPassword='null'
registryEmail='null'
dockerConfig='DefaultDockerClientConfig[dockerHost=unix:///var/run/docker.sock,registryUsername=root,registryPassword=<null>,registryEmail=<null>,registryUrl=https://index.docker.io/v1/,dockerConfigPath=/root/.docker,sslConfig=<null>,apiVersion={UNKNOWN_VERSION},dockerConfig=<null>]'
due to org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:51)
<snip>
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: com.sun.jna.LastErrorException: [2] No such file or directory
at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocket.<init>(UnixDomainSocket.java:62)
<snip>
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.sun.jna.LastErrorException: [2] No such file or directory
at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocketLibrary.connect(Native Method)
at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocket.<init>(UnixDomainSocket.java:57)
... 35 common frames omitted
In my CI pipeline I'd like to only run docker build . and not worry about having another stage that does the mvn clean install.
How do I fix the configuration to get the java PostgreSQL Testcontainers to work inside of a Docker build so that I can use it in a multi-stage build?
Full Code example: https://gitlab.com/raymondcg/reproducer-testcontainer

Not really Testcontainers related.
Testcontainers requires a valid Docker daemon. When you build images, there is no daemon mounted into the image build context.
You can easily verify that by doing:
RUN curl --unix-socket /var/run/docker.sock http:/_/_ping
Make this command return "OK" (no need to run the Testcontainers code), and your tests will pass as well.

You can overwrite testcontainers default docker host by adding:
ENV DOCKER_HOST=tcp://host.docker.internal:2375
to your build stage.

Apache druid No known server

I am trying to setup the Apache Druid on a single machine following quickstart guide here. When I start historical server, it shows io.druid.java.util.common.IOE: No known server exception on screen.
Command:
java `cat conf-quickstart/druid/historical/jvm.config xargs` \
-cp "conf-quickstart/druid/_common:conf-quickstart/druid/historical:lib/*" \
io.druid.cli.Main server historical
Full stack-trace-
2018-04-07T18:23:40,234 WARN [main]
io.druid.java.util.common.RetryUtils - Failed on try 1, retrying in
1,246ms. io.druid.java.util.common.IOE: No known server at
io.druid.discovery.DruidLeaderClient.getCurrentKnownLeader(DruidLeaderClient.java:276)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.discovery.DruidLeaderClient.makeRequest(DruidLeaderClient.java:128)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.query.lookup.LookupReferencesManager.fetchLookupsForTier(LookupReferencesManager.java:569)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.query.lookup.LookupReferencesManager.tryGetLookupListFromCoordinator(LookupReferencesManager.java:420)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.query.lookup.LookupReferencesManager.lambda$getLookupListFromCoordinator$4(LookupReferencesManager.java:398)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:63)
[java-util-0.12.0.jar:0.12.0] at
io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81)
[java-util-0.12.0.jar:0.12.0] at
io.druid.query.lookup.LookupReferencesManager.getLookupListFromCoordinator(LookupReferencesManager.java:388)
[druid-server-0.12.0.jar:0.12.0]
I have tried to setup from scratch many times with exactly the same steps mentioned on quick-start guide, but I am unable to resolve this error. How to resolve this error?

If you already tried to start druid, then delete the druid-X.Y.Z/log and druid-X.Y.Z/var folders.
Start zookeeper ./zookeeper-X.Y.Z/bin/zkServer.sh start
Recreate those folders you erased with druid-X.Y.Z/bin/init
Run each command in a new tab in this order
java `cat conf-quickstart/druid/coordinator/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/coordinator:lib/*" io.druid.cli.Main server coordinator
java `cat conf-quickstart/druid/overlord/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/overlord:lib/*" io.druid.cli.Main server overlord
java `cat conf-quickstart/druid/broker/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/broker:lib/*" io.druid.cli.Main server broker
java `cat conf-quickstart/druid/historical/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/historical:lib/*" io.druid.cli.Main server historical
java `cat conf-quickstart/druid/middleManager/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/middleManager:lib/*" io.druid.cli.Main server middleManager
You should now have 1 tab open for each of those commands (so 5).
Insert the data curl -X 'POST' -H 'Content-Type:application/json' -d #quickstart/wikiticker-index.json localhost:8090/druid/indexer/v1/task
You will then see {"task":"index_hadoop_wikiticker_2018-06-06T19:17:51.900Z"}

How to read docker env variables into Java code

I am trying to dockerize my Java application which tries to connect to a Rabbitmq server. I have passed the Rabbitmq Url via docker env variable and readig the same url using
System.getenv("RABBITMQ_URL")
, but it came out to be null. Is there anything wrong with the way I am reading the docker env variable ? Here is my Docker create command :
docker service create --name xxx --env
RABBITMQ_URL=amqp://rabbitmq:xxxx --network msgq --with-registry-auth
${imageName}

This is weird but, restarting the docker container just worked fine for me. Turns out , i have to restart the container when ever I update the network connection using "--network". Thanks

Seems to work just fine for me. Please see below:
>> cat Dockerfile
FROM java
COPY Test.java /Test.java
RUN javac Test.java
CMD java Test
>> cat Test.java
class Test {
public static void main(String[] j) {
System.out.println(System.getenv("RABBITMQ_URL"));
while (true) {}
}
}
>> docker build -t testj .
Sending build context to Docker daemon 6.656kB
Step 1/4 : FROM java
---> d23bdf5b1b1b
Step 2/4 : COPY Test.java /Test.java
---> Using cache
---> 2333685c6488
Step 3/4 : RUN javac Test.java
---> Using cache
---> 8d1e98d604b9
Step 4/4 : CMD java Test
---> Using cache
---> 6f9625f04966
Successfully built 6f9625f04966
Successfully tagged testj:latest
>> docker service create --name xxx --env RABBITMQ_URL=amqp://rabbitmq:xxxx --detach testj
937rbfctrds0z1mhpk1e7dlja
>> docker service logs xxx
xxx.1.acv6mqqy38pf#moby | amqp://rabbitmq:xxxx
>>