Deploy additional files in Gradle Application Plugin - java

I have a small Java/Gradle project. I'm using the Application plugin to create a zip distribution (using the distZip task). Using the standard configuration I get the following directories in my zip file:
/bin - The scripts to start the application go in here
/lib - Contains my project code in a JAR file and all dependency JAR files.
The trouble is that I would like a third directory: /conf where I can put my configuration files (instead of having them packaged inside my application JAR file.
I imagine that this is a pretty common requirement because things like log4j.xml and hibernate.properties would be better placed outside the JAR file. I just can't figure out how I can customise the behavior of the Application plugin to do this however.


I revisited this problem several months later and I finally have an elegant solution. The following code should be added to the gradle file:
distZip {
into(project.name) {
from '.'
include 'conf/*'
}
}
This adds an additional include to the distZip task. This copies the "conf" directory (including contents) into the Zip distribution.
The generated zip file contains a single directory which is the same as the project name. This is why the "into" part is required.


Actually, create a dist dir under the src dir in your project. Anything in this dir is copied by the application plugin (under applicationDistribution) when installApp or distZip is run.
Or edit applicationDistribution to do other things, if a simple copy is not enough.


For me, a simple
applicationDistribution.from("src/main/config/") {
into "config"
}
did the job. Of course you need to have your properties loaded correctly from within code. Especially if you move them from src/main/resources where they have been usable via classpath, into the new location. I circumvented this by adding a command line parameter which points to the configuration file.


I am not sure whether you can customize the application plugin, I have never used it. There is however other ways to achieve what you want to achieve.
You may create a /conf directory like this:
confDir = new File("$buildDir/conf")
You can then copy the files you need into this directory like this:
task copyConfFiles(type: Copy) {
from _wherever your files reside_
into confDir
include('**/*.properties') // your configuration files
}
You may then hook this copy task into the process like this:
distZip.dependsOn copyConfFiles
And last if you do not want your configurations in the final zip, you can do this:
distZip {
exclude('**/*.properties') // your configuration files
}
Again, there might be a better way. This is a way.


OP's self-answer may be good for his use case, but there are a few things I'd like to improve on:
His answer suggests that he has a directory conf parallel to the build.gradle. There is no such thing in the Maven Standard Directory Layout. The general consensus is to have a src/main/conf as had been hinted to in the docs:
If there are other contributing sources to the artifact build, they
would be under other subdirectories: for example src/main/antlr would
contain Antlr grammar definition files.
The target directory name is NOT project.name as had been pointed out in a comment.
If resource filtering is required, and it often is, then having a separate task is desirable. During local development, this task can be run to generate the filtered files. The distribution would merely use the output of this task (and unlike OP's answer, this also makes conf available to the tar distribution).
def props = new Properties()
file("src/main/filters/application.properties")
.withInputStream { props.load(it) }
import org.apache.tools.ant.filters.ReplaceTokens
task copyConf(type: Copy) {
from("src/main/conf/")
into("$buildDir/conf")
filesMatching("**/*.y*ml") {
filter(tokens: props, ReplaceTokens)
}
}
distributions {
main {
contents {
from(copyConf) {
into("conf")
}
}
}
}

Related

How to access static files from a test class in IntelliJ IDEA? [duplicate]

I know I can load a file from src/test/resources with:
getClass().getResource("somefile").getFile()
But how can I get the full path to the src/test/resources directory, i.e. I don't want to load a file, I just want to know the path of the directory?

You don't need to mess with class loaders. In fact it's a bad habit to get into because class loader resources are not java.io.File objects when they are in a jar archive.
Maven automatically sets the current working directory before running tests, so you can just use:
File resourcesDirectory = new File("src/test/resources");
resourcesDirectory.getAbsolutePath() will return the correct value if that is what you really need.
I recommend creating a src/test/data directory if you want your tests to access data via the file system. This makes it clear what you're doing.

Try working with the ClassLoader class:
ClassLoader classLoader = getClass().getClassLoader();
File file = new File(classLoader.getResource("somefile").getFile());
System.out.println(file.getAbsolutePath());
A ClassLoader is responsible for loading in classes. Every class has a reference to a ClassLoader. This code returns a File from the resource directory. Calling getAbsolutePath() on it returns its absolute Path.
Javadoc for ClassLoader: http://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoader.html

I would simply use Path from Java 7
Path resourceDirectory = Paths.get("src","test","resources");
Neat and clean!

If it's a spring project, we can use the below code to get files from src/test/resource folder.
File file = ResourceUtils.getFile(this.getClass().getResource("/some_file.txt"));

I have a Maven3 project using JUnit 4.12 and Java8.
In order to get the path of a file called myxml.xml under src/test/resources, I do this from within the test case:
#Test
public void testApp()
{
File inputXmlFile = new File(this.getClass().getResource("/myxml.xml").getFile());
System.out.println(inputXmlFile.getAbsolutePath());
...
}
Tested on Ubuntu 14.04 with IntelliJ IDE.
Reference here.
Note
The prepending / symbol is necessary, because Class.getResource(String) won't necessarily reveal entire file path (missing) along with FileNotFoundException.

All content in src/test/resources is copied into target/test-classes folder. So to get file from test resources during maven build you have to load it from test-classes folder, like that:
Paths.get(
getClass().getProtectionDomain().getCodeSource().getLocation().toURI()
).resolve(
Paths.get("somefile")
).toFile()
Break down:
getClass().getProtectionDomain().getCodeSource().getLocation().toURI() - give you URI to target/test-classes.
resolve(Paths.get("somefile")) - resolves someFile to target/test-classes folder.
Original anwser is taken from this

There are differences and constraints in options offered by #Steve C and #ashosborne1. They must be specified, I believe.
When can we can use: File resourcesDirectory = new File("src/test/resources");?
1 When tests are going to be run via maven only but not via IDE.
2.1 When tests are going to be run via maven or
2.2 via IDE and only one project is imported into IDE. (I use “imported” term, cause it is used in IntelliJ IDEA. I think users of eclipse also import their maven project). This will work, cause working directory when you run tests via IDE is the same as your project.
3.1 When tests are going to be run via maven or
3.2 via IDE, and more than one projects are imported into IDE (when you are not a student, you usually import several projects), AND before you run tests via IDE, you manually configure working directory for your tests. That working directory should refer to your imported project that contains the tests. By default, working directory of all projects imported into IDE is only one. Probably it is a restriction of IntelliJ IDEA only, but I think all IDEs work like this. And this configuration that must be done manually, is not good at all. Working with several tests existing in different maven projects, but imported into one big “IDE” project, force us to remember this and don’t allow to relax and get pleasure from your work.
Solution offered by #ashosborne1 (personally I prefer this one) requires 2 additional requirements that must be done before you run tests. Here is a list of steps to use this solution:
Create a test folder (“teva”) and file (“readme”) inside of “src/test/resources/”:
src/test/resources/teva/readme
File must be created in the test folder, otherwise, it will not work. Maven ignores empty folders.
At least once build project via mvn clean install. It will run tests also. It may be enough to run only your test class/method via maven without building a whole project. As a result your test resources will be copied into test-classes, here is a path: target/test-classes/teva/readme
After that, you can access the folder using code, already offered by #ashosborne1 (I'm sorry, that I could not edit this code inside of this list of items correctly):
public static final String TEVA_FOLDER = "teva"; ...
URL tevaUrl = YourTest.class.getClassLoader().getResource(TEVA_FOLDER);
String tevaTestFolder = new File(tevaUrl.toURI()).getAbsolutePath();
Now you can run your test via IDE as many times as you want. Until you run mvn clean. It will drop the target folder.
Creating file inside a test folder and running maven first time, before you run tests via IDE are needed steps. Without these steps, if you just in your IDE create test resources, then write test and run it via IDE only, you'll get an error. Running tests via mvn copies test resources into target/test-classes/teva/readme and they become accessible for a classloader.
You may ask, why do I need import more than one maven project in IDE and why so many complicated things? For me, one of the main motivation: keeping IDA-related files far from code. I first create a new project in my IDE. It is a fake project, that is just a holder of IDE-related files. Then, I import already existing maven projects. I force these imported projects to keep IDEA files in my original fake project only. As a result I don't see IDE-related files among the code. SVN should not see them (don't offer to configure svn/git to ignore such files, please). Also it is just very convenient.

The simplest and clean solution I uses, suppose the name of the test class is TestQuery1 and there is a resources directory in your test folder as follows:
├── java
│   └── TestQuery1.java
└── resources
└── TestQuery1
├── query.json
└── query.rq
To get the URI of TestQuery1 do:
URL currentTestResourceFolder = getClass().getResource("/"+getClass().getSimpleName());
To get the URI of one of the file TestQuery1, do:
File exampleDir = new File(currentTestResourceFolder.toURI());
URI queryJSONFileURI = exampleDir.toURI().resolve("query.json");

You can get where you are through
new File(".").getAbsolutePath()
then you can derive path to src/test/resources
usually it is just
new File("src/test/resources")

You can't use a file from a resource folder for tests in a common case. The reason is that resource files in the resource folder are stored inside a jar. So they don't have a real path in the file system.
The most simple solution can be:
Copy a file from resources to the temporary folder and get a path to that temporary file.
Do tests using a temporary path.
Delete the temporary file.
TemporaryFolder from JUnit can be used to create temporary files and delete it after test is complited. Classes from guava library are used to copy a file form resource folder.
Please, notice that if we use a subfolder in the resources folder, like good one, we don't have to add leading / to the resource path.
public class SomeTest {
#Rule
public TemporaryFolder tmpFolder = new TemporaryFolder();
#Test
public void doSomethinge() throws IOException {
File file = createTmpFileFromResource(tmpFolder, "file.txt");
File goodFile = createTmpFileFromResource(tmpFolder, "good/file.txt");
// do testing here
}
private static File createTmpFileFromResource(TemporaryFolder folder,
String classLoaderResource) throws IOException {
URL resource = Resources.getResource(classLoaderResource);
File tmpFile = folder.newFile();
Resources.asByteSource(resource).copyTo(Files.asByteSink(tmpFile));
return tmpFile;
}
}

With Spring you could easily read it from the resources folder (either main/resources or test/resources):
For example create a file: test/resources/subfolder/sample.json
#Test
public void testReadFile() {
String json = this.readFile("classpath:subfolder/sample.json");
System.out.println(json);
}
public String readFile(String path) {
try {
File file = ResourceUtils.getFile(path);
return new String(Files.readAllBytes(file.toPath()));
} catch (IOException e) {
e.printStackTrace();
}
return null;
}

List<String> lines = Files.readAllLines(Paths.get("src/test/resources/foo.txt"));
lines.forEach(System.out::println);

Wow the correct answer is not here yet!
MyClass.class.getResource("/somefile");
MyClass.class.getResourceAsStream("/somefile");
https://javachannel.org/posts/how-to-access-static-resources/

Use the following to inject Hibernate with Spring in your unit tests:
#Bean
public LocalSessionFactoryBean getLocalSessionFactoryBean() {
LocalSessionFactoryBean localSessionFactoryBean = new LocalSessionFactoryBean();
localSessionFactoryBean.setConfigLocation(new ClassPathResource("hibernate.cfg.xml"));
localSessionFactoryBean.setPackagesToScan("com.example.yourpackage.model");
return localSessionFactoryBean;
}
If you don't have the hibernate.cfg.xml present in your src/test/resources folder it will automatically fall back to the one in your src/main/resources folder.

Use .getAbsolutePath() on your File object.
getClass().getResource("somefile").getFile().getAbsolutePath()

With Spring, you can use this:
import org.springframework.core.io.ClassPathResource;
// Don't worry when use a not existed directory or a empty directory
// It can be used in #before
String dir = new ClassPathResource(".").getFile().getAbsolutePath()+"/"+"Your Path";

Change how Gradle's build directory is organized

So, I've recently (partially) completed a Java project with Gradle. Importantly, the project uses absolute pathing to access files in my resources folder, since those files will change after the JAR is made. When I use Eclipse's "export as runnable JAR" functionality, I get something that works perfectly - putting the .jar file in my main directory lets it find everything. However, using Gradle's build function doesn't, because Gradle adds extra layers between the .jar and the resources. To demonstrate, here's my "normal" directory:
./program root directory
|_program.jar
|_resources
|_[actual resources]
And here's the directory Gradle makes:
./build folder
|_libs
| |_program.jar
|_resources
|_main
|_[actual resources]
What I want from Gradle is:
./build folder
|_program.jar
|_resources
|_[actual resources]
Yes, I could manually move the resources and program.jar around in the directory to achieve this, but that feels wrong - this is exactly what Gradle is supposed to do for me, right? I know there has to be SOME way to do it. I just don't know how. So that's why I'm asking for help - how do I do this?

To change the output of resources:
sourceSets.main.output.resourcesDir = "$buildDir/resources"
To change where the JAR file is put:
jar {
// use destinationDir for Gradle < 5.1
destinationDirectory = buildDir
}
If all your resources are meant to be external you may want to exclude them from the JAR file:
jar {
include '**/*.class'
destinationDirectory = buildDir
}
That will only include .class files from the jar task's input. You can customize this using the include and exclude options.

Gradle copy command deprecated, breaks build

In Gradle 3.x I was able to get some xml mapping files to copy into the classes directory prior to build/jar via the following block:
copy{
from 'src/main/java/com/company/mapping'
into 'build/classes/main/java/com/company/mapping'
include '**/*.xml'
}
In Gradle 4.9 this has been deprecated in favor of:
task copyMappings(type: Copy){
from 'src/main/java/com/company/mapping'
into 'build/classes/main/java/com/company/mapping'
include '**/*.xml'
}
The copyMappings task succeeds, but build/jar does not wait for copyMappings to finish. I have tried variations on build.dependsOn and doFirst{ copyMappings } doLast{ build } but nothing seems to get me the desired effect of having the copied files in place in the 'into' path prior to jar.
This is for Windows 10.

This works for me with Gradle 4.9 on Mac OS:
apply plugin: 'java'
task copyMappings(type: Copy) {
from 'src/main/java/com/company/mapping'
into 'build/classes/main/java/com/company/mapping'
include '**/*.xml'
}
jar.dependsOn copyMappings
jar.doFirst {
assert new File("${projectDir}/build/classes/main/java/com/company/mapping/abc.xml").exists()
assert new File("${projectDir}/build/classes/main/java/com/company/mapping/def.xml").exists()
}
command line is gradle clean jar

I like to model things around source sets where appropriate as doing so let's the build work more reliably with a wide range of plugins and use cases. For example, imagine you want to run an application direct from its class files and resources rather than packaging it as a JAR first. You could make sure that the "run" task depends on the copy as well, but you'd have to do that for every instance where this is a requirement.
Source sets are the ideal solution because they have the concept of a runtime classpath, which will work for packaging, instrumentation, running, testing and so on.
With that in mind, I would go for this simple declaration and get rid of the copy task:
sourceSets {
main {
resources {
srcDir "src/main/java"
include "**/*.xml"
}
}
}
The XML files will end up in a different directory from your current approach, but that shouldn't matter unless you have tasks that assume the location rather than using the source set model to get the necessary information.
Note The above include directive applies to all the resources in src/main/resources as well. So if you have properties files or text files or anything else in there, they will be excluded. The simplest solution is to add all required resource file patterns to the include directive.

How do I specify an extra folder to be on the classpath for gradle's application plugin 'run' task?

I've successfully configured my gradle build script to create a zip distribution of my application with an extra 'config' folder at the root. This folder contains (at least right now) only one properties file in use by the application, and is on the classpath for the application.
What I'm looking for now, however, is a way to do the same with the 'run' task in the application plugin. When I try to run my application this way, (for testing), my program fails to run because of a class trying to access this properties file on the root of the classpath.
A bonus would be if I could get IntelliJ or Eclipse to also add this folder to its classpath just like the other folders (src/main/java, src/main/resources, ...) so I can run and debug my code from within the IDE without invoking a gradle task. I want to try to avoid as much as possible tying this code to any one IDE, so that when anybody needs to work on the project, they just need to import the build.gradle file and have the IDE make the appropriate config files it needs.
Here is my build.gradle file:
apply plugin: 'application'
mainClassName = "MainClass"
startScripts {
// Add config folder to classpath. Using workaround at
// https://discuss.gradle.org/t/classpath-in-application-plugin-is-building-always-relative-to-app-home-lib-directory/2012
classpath += files('src/dist/config')
doLast {
def windowsScriptFile = file getWindowsScript()
def unixScriptFile = file getUnixScript()
windowsScriptFile.text = windowsScriptFile.text.replace('%APP_HOME%\\lib\\config', '%APP_HOME%\\config')
unixScriptFile.text = unixScriptFile.text.replace('$APP_HOME/lib/config', '$APP_HOME/config')
}
}
repositories {
...
}
dependencies {
...
}
Likely what needs to happen is that I need to have the /src/dist/config folder to be copied into the build directory and added to the classpath, or have its contents be copied into a folder that is already on the classpath.

I ended up taking Opal's suggestion as a hint, and came up with the following solution. I added the following to my build.gradle file:
task processConfig(type: Copy) {
from('src/main/config') {
include '**/*'
}
into 'build/config/main'
}
classes {
classes.dependsOn processConfig
}
run {
classpath += files('build/config/main')
}
Alternatively, a simpler approach would be to add a runtime dependency to my project as such:
dependencies {
...
runtime files('src/main/config')
}
I didn't end up doing it this way, however, because my distribution package ended up having .properties files in the lib folder... and I'm just picky that way.

As you can see in the docs run is a task of type JavaExec. So classpath for it can be modified. Try to add config folder to the classpath. See here.

How to override files when creating a jar in gradle

We have a java project where we have some default configurations under src/main/resources and there are overwrites under project_root/configDeploy
For our mapred jars we want to copy both configs but allow config/deploy files to overwrite defaults in resources. So we can have myconf.xml in resource and myconf.xml in deploy, but the mapred fat jar generated only has myconf.xml
I have tried two different methods, try to have deploy copy and overwrite the resources conf:
from 'src/main/resources'
from 'conf/deploy'
but this will add two files inside the jar, so it didn't work
Then I tried to add only files from src/main/resources that are not in conf/deploy, something like:
into('conf'){
from{
'src/main/resources'
}
exclude{file('deploy/conf/')}
}
into('conf'){
from{
'deploy/conf'
}
}
but this didn't work, as a result none of the confs from resources were copies.
So question is if I have a two folders with files which some of them have same name how can I include their files in jar so I get files from both folders but for files that are in both I get only the version in second folder.
Thanks for your help!

To avoid the duplicate files in the JAR you can set the duplicates strategy to EXCLUDE on the task.
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
This will cause subsequent attempts to add the file to be ignored. Therefore, if you want files in 'deploy/conf' to take precedence you should define that copy spec first.

Categories

Resources