I am trying to run DynamoDB local for testing purposes. I followed the steps amazon provides for setting it up and running the jar by itself works fine (link to amazon's tutorial Here). However, the tutorial doesn't go over running the jar within your own project. I don't want all the other developers to have to grab a jar and run it locally every time they test their code.
That is where my question comes in. I've had a real hard time finding any examples online of how to configure a Gradle project to run the DynamoDB local server as part of my tests. I found the following maven example https://github.com/awslabs/aws-dynamodb-examples/blob/master/src/test/java/com/amazonaws/services/dynamodbv2/DynamoDBLocalFixture.java#L32 and am trying to convert it to a Gradle, but am getting errors for all of com.amazonaws.services.dynamodbv2.local import statements they are using. The errors are that the resource cannot be found.
I went into their project's pom and put the following into my build.gradle file to emulate it.
//dynamodb local dependencies
testCompile('com.amazonaws:aws-java-sdk-dynamodb:1.10.42')
testCompile('com.amazonaws:aws-java-sdk-cloudwatch:1.10.42')
testCompile('com.amazonaws:aws-java-sdk:1.3.0')
testCompile('com.amazonaws:amazon-kinesis-client:1.6.1')
testCompile('com.amazonaws:amazon-kinesis-connectors:1.1.1')
testCompile('com.amazonaws:dynamodb-streams-kinesis-adapter:1.0.2')
testCompile('com.amazonaws:DynamoDBLocal:1.10.5.1')
The import statements still fail. Here is an example of one that fails.
import com.amazonaws.services.dynamodbv2.local.embedded.DynamoDBEmbedded;
TL;DR
Has anyone managed to get the DynamoDB local JAR to execute as part of a Gradle project or have a link to a good tutorial (it doesn't have to be the tutorial I linked to).
We have DynamoDB local working with gradle. Here's what you need to add to your gradle.build file:
For gradle 4.x and below versions
1) Add to the repositories section:
maven {
url 'http://dynamodb-local.s3-website-us-west-2.amazonaws.com/release'
}
2) Add to the dependencies section (assuming you're using this for your tests):
testCompile group: 'com.amazonaws', name: 'DynamoDBLocal', version: 1.11.0
3) These next two steps are the tricky part. First copy the native files to a directory:
task copyNativeDeps(type: Copy) {
from (configurations.testCompile) {
include "*.dylib"
include "*.so"
include "*.dll"
}
into 'build/libs'
}
4) Then make sure you include this directory (build/libs in our case) in the java library path like so:
test.dependsOn copyNativeDeps
test.doFirst {
systemProperty "java.library.path", 'build/libs'
}
Now you should be able to run ./gradlew test and have your tests hit your local DynamoDB.
For Gradle 5.x the below solution works
maven {
url 'http://dynamodb-local.s3-website-us-west-2.amazonaws.com/release'
}
configurations {
dynamodb
}
dependencies {
testImplementation 'com.amazonaws:DynamoDBLocal:1.11.477'
dynamodb fileTree (dir: 'lib', include: ["*.dylib", "*.so", "*.dll"])
dynamodb 'com.amazonaws:DynamoDBLocal:1.11.477'
}
task copyNativeDeps(type: Copy) {
from configurations.dynamodb
into "$project.buildDir/libs/"
}
test.dependsOn copyNativeDeps
test.doFirst {
systemProperty "java.library.path", 'build/libs'
}
I run into the same problem and first I tried to add sqlite4java.library.path to the Gradle script as it has been mentioned in the other comments.
This worked for command line, but were not working when I was running the tests from IDE (IntelliJ IDEA), so finally I come up with a simple init method, that is called at the beginning of each of integration tests:
AwsDynamoDbLocalTestUtils.initSqLite();
AmazonDynamoDBLocal amazonDynamoDBLocal = DynamoDBEmbedded.create();
Implementation can be found here: https://github.com/redskap/aws-dynamodb-java-example-local-testing/blob/master/src/test/java/io/redskap/java/aws/dynamodb/example/local/testing/AwsDynamoDbLocalTestUtils.java
I put a whole example to GitHub, it might be helpful: https://github.com/redskap/aws-dynamodb-java-example-local-testing
In August 2018 Amazon announced new Docker image with Amazon DynamoDB Local onboard. It does not require downloading and running any JARs as well as adding using third-party OS-specific binaries like sqlite4java.
It is as simple as starting a Docker container before the tests:
docker run -p 8000:8000 amazon/dynamodb-local
You can do that manually for local development, as described above, or use it in your CI pipeline. Many CI services provide an ability to start additional containers during the pipeline that can provide dependencies for your tests. Here is an example for Gitlab CI/CD:
test:
stage: test
image: openjdk:8-alpine
services:
- name: amazon/dynamodb-local
alias: dynamodb-local
script:
- ./gradlew clean test
So, during the test task DynamoDB will be available on http://dynamodb-local:8000.
Another, more powerful tool is localstack. It supports two dozen of AWS services, DynamoDB is one of them. The isage is very similar, you have to start it before running the tests and it will expose AWS-compatible APIs on given ports:
test:
stage: test
image: openjdk:8-alpine
services:
- name: localstack/localstack
alias: localstack
script:
- ./gradlew clean test
The idea is to move all the configuration out of your build tool and tests and provide the dependency externally. Think of it as of dependency injection / IoC but for the whole service, not just a single bean. This way, you code is more clean and maintainable. You can see that even in the examples above: you can switch mock implementation from DynamoDB Local to localstack by simply changing the image part!
The easiest way, in my opinion, is to:
Download the JAR from here:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html#DynamoDBLocal.DownloadingAndRunning
Then unzip the downloaded folder and add its content to the /libs folder in the project (create the /libs folder before that)
Finally, add to the build.gradle:
dependencies {
runtime files('libs/DynamoDBLocal.jar')
}
I didn't want to create a specific configuration for dynamo for gradle 6+ so I tweaked the original answer instructions. Also, this is in kotlin gradle DSL rather than groovy.
val copyNativeDeps by tasks.creating(Copy::class) {
from(configurations.testRuntimeClasspath) {
include("*.dylib")
include("*.so")
include("*.dll")
}
into("$buildDir/libs")
}
tasks.withType<Test> {
dependsOn.add(copyNativeDeps)
doFirst { systemProperty("java.library.path", "$buildDir/libs") }
}
By leveraging the testRuntimeClasspath configuration, gradle is able to locate the relevant files for you without needing to create a custom configuration. Obviously this has the side effect that if your test runtime has many native deps, they will also be copied which would make the custom configuration approach more ideal.
Related
What I would like to achieve
I would like to have a manual gradle task that I can generate Java classes based on Json schema. However, I don't want this task to run when I run other fx. gradle build.
What I did
Firstly I've create simple gradle java project with
gradle init
Then I have added jsonschema2dataclass plugin and configure it as follows (my current build.gradle):
/*
* This file was generated by the Gradle 'init' task.
* (...)
*/
plugins {
// Apply the application plugin to add support for building a CLI application in Java.
id 'application'
id "org.jsonschema2dataclass" version "4.5.0"
}
repositories {
// Use Maven Central for resolving dependencies.
mavenCentral()
}
dependencies {
// Use JUnit Jupiter API for testing.
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.6.2'
// Use JUnit Jupiter Engine for testing.
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine'
// This dependency is used by the application.
implementation 'com.google.guava:guava:29.0-jre'
}
application {
// Define the main class for the application.
mainClass = 'GradlePlayground.App'
}
jsonSchema2Pojo {
includeGeneratedAnnotation = true
generateBuilders = true
targetPackage = 'org.example.api' // specify package for your needs
targetDirectoryPrefix = file("${project.rootDir}/app/src/main/java")
source.setFrom files("${project.rootDir}/app/src/main/resources/json")
}
What I've tried
Add task.enabled = false
Put plugin configuration into another task
Check source code of plugin to find a way to disconnect this task from build task
But all above trails have failed. When I run gradle tasks I can always see generateJsonSchema2DataClass and generateJsonSchema2DataClass0 as part of build tasks.
I'm using java 8 and gradle 6.9.3
I'm the author of the gradle plugin.
Short answer for your question is "no, it's not an intended flow for a normal project". However, it's always possible to create a new project, build it once and extract the sources.
Could you please explain, what the reason you want to exclude run from the build chain?
My only guess is to build and publish models. And if this is a case, it's possible to do this using some gradle magic, which described in discussions in the GitHub project
UPD: Based on your gradle script, I have a lot of "why" questions I'd like to ask, it'll be easier if you
I am trying to update a gradle 6.x version multi project application to gradle 7.3 as it is the first version to support Java 17. However, I am unable to progress past an issue arising from a task which is not declared in any of my build.gradle files. The error is below: ([] pieces are redacted)
Execution failed for task ':[root module]:[module]:processIntTestResources'.
> Entry [filename] is a duplicate but no duplicate handling strategy has been set. Please refer to https://docs.gradle.org/7.3/dsl/org.gradle.api.tasks.Copy.html#org.gradle.api.tasks.Copy:duplicatesStrategy for details.
I think is associated with a sourceSet for integration tests, however, I never explicitly copy and files for those tests. I have also set all copy instructions to have this configuration with regard to duplicate handling:
duplicatesStrategy = DuplicatesStrategy.INCLUDE
I also tried adding the above to projects which failed because that setting is only valid for Copy type tasks or blocks. I am by no means an expert in gradle and can add any relevant information needed, but I believe I have included what might be most relevant. I am really just looking for a direction I can head in to further debug this issue.
You didn't show much about how you have integration tests configured, but I ran into a similar problem. What saved me was this blog post by Tom Gregory:
Running integration tests in Gradle
Since links can disappear, let me copy and paste the most important part that I found relevant, which is regarding the new (as of Gradle 7.3) JVM Test Suite Plugin that adds support for integration tests. For me, this replaced my old integration test configuration:
testing {
suites {
integrationTest(JvmTestSuite) {
dependencies {
implementation project
}
}
}
}
tasks.named('check') {
dependsOn testing.suites.integrationTest
}
I also found the following useful, which is not in the above blog post, but is a leftover from my previous Gradle 6 configuration. This for me duplicates the 'test' dependencies for 'integrationTest'. This is not the recommended way of handling the test depedencies now (see the JVM Test Suite Plugin documentation), but I still found it useful to get me back running quickly:
configurations {
integrationTestImplementation.extendsFrom testImplementation
integrationTestRuntime.extendsFrom testRuntime
}
On a new environment gradle build takes quite a while because all dependencies have to be downloaded.
Is there a way to only download dependencies in order to speed up the following build?
That way we could for example already prefill a CI build environment.
Edit: Updated for Gradle 6+.
Some notes:
This new approach downloads jars into a folder, and then deletes the folder. So the result of having the jars in the Gradle cache is a side-effect.
It currently uses jars configured for the main source-set but could be generalized.
Even though it is neither efficient nor elegant, it can be useful if you actually want the jars (and transitive dependencies): simply comment-out the deletion of the runtime folder.
This solution can be handy when you want the jars (and transitive dependencies), as you simply have to comment-out deleting the folder.
Consider this build.gradle (as an arbitrary, concrete example):
apply plugin: 'java'
dependencies {
implementation 'org.apache.commons:commons-io:1.3.2'
implementation 'org.kie.modules:org-apache-commons-lang3:6.2.0.Beta2'
}
repositories {
jcenter()
}
task getDeps(type: Copy) {
from sourceSets.main.runtimeClasspath
into 'runtime/'
doFirst {
ant.delete(dir: 'runtime')
ant.mkdir(dir: 'runtime')
}
doLast {
ant.delete(dir: 'runtime')
}
}
Example run:
$ find /Users/measter/.gradle/caches -name "commons-io*1.3.2.jar"
$ gradle getDeps
$ find /Users/measter/.gradle/caches -name "commons-io*1.3.2.jar"
/Users/measter/.gradle/caches/modules-2/files-2.1/commons-io/commons-io/1.3.2/[snip]/commons-io-1.3.2.jar
I've found ./gradlew dependencies (as suggested by this user) to be very handy for Docker builds.
You can create a custom task that resolves all the configurations( in doing so, it will also download the dependencies without building the project)
task downloadDependencies {
doLast {
configurations.findAll{it.canBeResolved}.each{it.resolve()}
}
}
Run command ./gradlew downloadDependencies
My answer will favor the gradle plugins and built-in tasks.
I would use "gradle assemble" in the command-line.
It is a minor version of "gradle build".
This way, you may reduce the time of your preparations before running or building anything.
Check the link bellow for the documentation:
https://docs.gradle.org/current/userguide/java_plugin.html#lifecycle_tasks
In general, what is my recipe when I clone a new repository:
-gradle assemble
-do some coding
-gradle run (and basically test until done)
-gradle build (to make distributable files)
note: this last step may have adicional configurations for .jar files as outputs (depends on you).
I'm using gradle to build my android project and am not able to run single local unit test. I have several test classes and one of them is MockServerTest and I only want to run test methods in this class.
I tried using gradle -Dtest.single=MockServerTest test but it turned out running all my tests, including these in other test classes.
I also tried gradle test --tests MockServerTest but an error occurred said
Test filtering is not supported for given version of junit. Please upgrade junit version to at least 4.6.
But I'm using junit 4.12 in my gradle file
testCompile 'junit:junit:4.12'
I'm using gradle 2.4 with com.android.tools.build:gradle:1.2.3.
Also, how can I run a single test method inside a single test class?
BTW, I'm able to run single test method inside Android Studio, by right clicking on the test method and select run targetTestMethod() from the menu. But how can I achieve this in the terminal? I guess Android Studio also trigger a certain command to do this. How can I see what that command is?
Figured it out myself. I have to run
gradle testDebug --tests com.my.package.TestClassName
There are two things to note here.
1. You have to use gradle testDebug or gradle testRelease instead of just gradle test. If you have build variant, you have to use gradle testVariantNameDebug or gradle testVariantNameRelease
2. You have to specify the whole qualified class name, means including the package name.
You can use Android Gradle plugin DSL to set up test tasks filters like this:
android {
testOptions {
unitTests.all {
it.testNameIncludePattern = "*.SomeTest"
}
}
}
You can find more information on testOptions here and filters here.
Have you tried running gradle test -Dtest.single=MockServerTest? More information can be found here.
I am building an application using Gradle, JDK 8, Java FX, and Test FX. I need to be on JDK 8 for our application to work on all platforms with our tech stack. My problem is that I am unable to get code coverage into our build. I found this link...
https://github.com/jacoco/jacoco/issues/74
...and using the Jacoco preview build at the top, I was able to replace my intellij JARs and successfully run my tests and get the coverage. However, I am having trouble putting this into my build.gradle. From what I can tell, I need to add a local repository in my build script...
...
apply plugin: "jacoco"
...
buildscript {
repositories {
// Local Repo
// MVN Repo(s)
}
dependencies {
// Classpaths
}
}
jacoco {
toolVersion = "0.6.4.201311160552" // I need this specific version, which isn't on a MVN repo
}
...I tried to add my local repo several ways including...
flatDir(dirs: "lib")
flatDir dirs: "${projectDir}/lib"
maven { url uri("lib") }
one or two other ways I forget
...my lib folder contains the exact contents, unchanged, from the preview build zip's lib folder in the link above. It doesn't seem like gradle is having a problem locating the local repo, but it is having trouble finding the JAR. I assume there is something wrong with the way I am naming it or the way that it is "packaged". I have tried modifying the JAR names but I keep getting the error...
What went wrong:
A problem occurred configuring root project 'myProject'.
Could not resolve all dependencies for configuration ':classpath'.
Could not find :org.jacoco.agent:.
Required by:
:myProject:unspecified
...any ideas why my JAR is not being found? Thanks!
"Answering" my own question, despite the fact that I still haven't quite figured it out. Anyways, here are two links I found that seem to solve my problem...
http://forums.gradle.org/gradle/topics/jacocotestreport_is_skipping
...following some of these instructions allow my tests to run, but I am still not able to run "gradle jacocoTestReport" without it failing.
UPDATE
OKAY! I figured it out, the link above did help me figure it out. My problem was with the asm-all JAR, since there were several, I did not know which one to use. In order to get jacoco working with Java 1.8, you do not need to specify the toolVersion property. All you need to do is add the following to your dependencies block (not the buildscript block, the code block)...
jacocoAgent files(
"$projectDir/lib/org.jacoco.agent-0.6.4.201311160552.jar")
jacocoAnt files(
"$projectDir/lib/org.jacoco.ant-0.6.4.201311160552.jar",
"$projectDir/lib/org.jacoco.core-0.6.4.201311160552.jar",
"$projectDir/lib/org.jacoco.report-0.6.4.201311160552.jar",
"$projectDir/lib/asm-all-5.0_BETA.jar")
...where the asm-all-5.0_BETA.jar is taken from the org.ow2.asm group found at...
http://mvnrepository.com/artifact/org.ow2.asm/asm-all/5.0_BETA
...hope this helps!
for reference, latest jacoco libs are changed so i'm sharing the following snippet:
dependencies{
jacocoAgent files("$rootProject.projectDir/lib/org.jacoco.agent-0.8.3.201904130250.jar")
jacocoAnt files("$rootProject.projectDir/lib/org.jacoco.ant-0.8.3.201904130250.jar",
"$rootProject.projectDir/lib/org.jacoco.core-0.8.3.201904130250.jar",
"$rootProject.projectDir/lib/org.jacoco.report-0.8.3.201904130250.jar",
"$rootProject.projectDir/lib/asm-7.0.jar",
"$rootProject.projectDir/lib/asm-tree-7.0.jar",
"$rootProject.projectDir/lib/asm-commons-7.0.jar"
)
}