I have a question about Tycho packaging types and project structure. I currently have a project that has both source and test source directories following typical Maven convention, which is built using package type 'eclipse-plugin'. This results in tests not being run in the maven build. The most common solution I can find is to separate the test source to a project/bundle of its own and build that via 'eclipse-test-plugin'. My question: What are the drawbacks of keeping the Maven convention and just building the whole project as eclipse-test-plugin instead?
According to http://wiki.eclipse.org/Tycho/Packaging_Types:
"Tycho introduces new eclipse-test-plugin packaging type to represent such projects. Build behavior is like regular Eclipse plugins, but these are treated specially at test-time."
This makes me believe that there are actually no drawbacks apart from the resulting bundle possibly depending on bundles that are only required for the test source. However, this is also the case for building via 'eclipse-plugin' (without splitting). Is it safe to say that changing packaging type to 'eclipse-test-plugin' in this case only results in tests being run in the maven build with no other side-effects?
(...) there are actually no drawbacks apart from the resulting bundle possibly depending on bundles that are only required for the test source.
This is the main reason why there are separate eclipse-test-plugin modules for tests in Tycho.
There are no scopes in the OSGi manifest, so the test dependencies would pollute the dependencies of your productive bundle. When you then install your bundle into Eclipse or an RCP, it may pull in test bundles that your productive code doesn't actually need. For most users, this is enough reason to split their code into a productive and a test bundle.
Another potential reason for separating the tests is when you want to explicitly test via the OSGi API of your productive bundle. In this case, you would not use a fragment for the test, which ensures that your tests can only load classes from your productive bundle according to the OSGi package visibility rules.
Related
I have a project 'java11-core' that generates a test jar artifact to share with project 'java11-app'. These projects build fine with command line Maven, but within Eclipse, the classes shared in the test jar cannot be found.
Version Info:
Apache Maven 3.6.0 (command line and Eclipse)
Java version: 11.0.1, vendor: Oracle Corporation
Eclipse IDE: Version: 2018-09 (4.9.0)
M2E Plugin: 1.9.1.20180912-1601
I originally created these to projects as tradition non-JPMS projects. These projects compiled and ran tests normally as expected. After I added module-info.java to both java11-core and java11-app, the Eclipse compiler could not recognize the shared test files from the core project.
Here is a snapshot of the package explorer for an overview of the project structure.
The added java11-app and java11-core module-info contents respectively:
module com.java11.app {
exports com.java11.app;
requires com.java11.core;
}
module com.java11.core {
exports com.java11.core;
}
As you can see, I do not export the test utilities package from com.java11.core. I do not want to export the test packages because this would make the test classes publicly available. I also do not wish to introduce a new test project, because in real-world scenarios, this is very likely to require cyclic dependencies between test utilities and the projects they assist in testing.
Build errors for in AppTest.java. The failure reported by Eclipse is interesting is that it does not claim it cannot find the CoreTestUtil class, but rather:
The type com.java11.test.core.util.CoreTestUtil is not accessible AppTest.java /java11-app/src/test/java/com/java11/app line 8 Java Problem
CoreTestUtil cannot be resolved AppTest.java /java11-app/src/test/java/com/java11/app line 21 Java Problem
My assumption is that the lack of an export for this package from java11-core and/or the lack of a requires for this package in java11-app make eclipse believe the access is restricted, even though the classes exist in a separate test-jar.
The module path for java11-app shows it includes java11-core as a module, and the Without test code is set to No.
I know I am working with newly release features and suspect that sharing test classes across Eclipse JPMS project is not yet supported. But, I am not sure where to look (Eclipse? M2E plugin) for an update on it being supported. I am also not aware of a work-around that would allow me to be productive while adopting JPMS for my software projects.
For those that believe test utilities should not be shared this way...
This subject has been characterized as a best-practice issue that should be resolved by refactoring test utilities into a separate module. I respect this perspective, but in attempting to follow that guidance, I found myself being forced to violate other best-practices, including DRY (Don't Repeat Yourself), and cyclic dependencies between packages.
It is common for a test utility to emerge while developing a module that both assists in effective testing of that module, as well as depends on that module. This creates a cycle if those utilities are pulled out to separate module. Furthermore, some of these utilities can be equally useful when testing other modules that depend upon that module. This creates duplicate code if those utilities are copied to a new test module for dependents. This reasoning may have been why Maven 'test-jar' support was originally added.
Eclipse does not support multiple module-info per project: in whatever source folder (main or test), you must only have one module-info.
From Eclipse point of view, your only luck is to create a new Java project referencing the other and with its proper module-info/exports:
module mod.a {
exports com.example.a;
// com.example.a.Main
}
module mod.a.tests { // (1)
exports com.example.a.tests;
// com.example.a.tests.MainUtils calling com.example.a.Main
requires mod.a;
}
In case (1), you will have problems if you don't use mod.a.tests: Java will never find com.example.a.Main, probably because the second project shadows the first project.
I am not an OSGI expert, but I think that's one of those reason for why most Eclipse plugin do have a main and test projects: org.eclipse.m2e.core is patched by org.eclipse.m2e.core.tests
However module-info does not have any knowledge of "patches": you may patch a module on command line (java --patch-module), but not in module-info itself: perhaps Eclipse could do that on your behalf, but it don't.
As you can see, two project in Eclipse = two Maven module.
In the case of Maven, you can certainly create other artefacts with the same build (and I do think it tends to pollute the dependencies, because every time your secondary artefacts will requires a dependency, it would have to go to the common scope).
This can be done using maven-compiler-plugin, maven-shade-plugin and maven-jar-plugin:
I think you should not rely test-jar because you want to emulate the --patch-module of Java by merging the classes and test-classes directories.
You don't want to import this project in Eclipse due to multiple module-info; or you must ensure that its module-info is only visible to Maven (you can use a profile + m2e.version do detect m2e and disable it).
I fully agree with you. Why should I only use the src-main code from some core module when I also could inherit some src-test functionalities?
But how to handle the scope problem? When I use the "test"-scope I loose the relation to the src-main code. When I dont use the test scope I loose the relation to the src-test code.
My core test code does not change very often, so to get the stuff working in Eclipse
I install the test-jar to my local repository and everything works fine.
We are developing code in the context of a legacy Java application that heavily uses static members and system properties, expecting files in various locations on the disk. The builds are run in Maven.
We are trying to allow unit testing of our code without having to deploy, configure and start the whole application. I have managed to do this by patching a small number of classes in the framework, providing my own variants of the relevant source files in Maven's test sources in src/test/java.
As a next step I would like to make this patch re-usable by providing a JAR file that can be pulled in as a test dependency on any project that develops a part of the larger application. I would like to deploy this via our normal binary repository.
Surefire offers an option to set <additionalClasspathElements>, but according to the documentation this works only with absolute paths and will add the dependency at the end of the class path.
In theory ordering the project dependencies correctly could work, but I cannot find any documentation on how that order works across multiple scopes. I would need Maven to guarantee that my test dependency is loaded before the runtime ones.
What is a reliable way of patching classes for a Surefire run by using a JAR pulled via Maven's dependency resolution mechanisms?
I already found this post: Separating tests from src with Maven Project?.
I've just started working on a Java project (as I usually use .net), and one of the first > things that strikes me as odd is that in the Maven project there is a /src and a /test
directory where obviously the source code and the tests should go.
In .net I preferred to have the tests in a separate assembly/project, so for example I
would have:
MyProject
MyProject.Tests
That way I dont have to bloat my deployed code with any tests and it makes it easier to
test my code in true isolation and in alot of cases I didnt bother writing tests per
project, I would just have solution wide unit/integration/acceptance tests i.e
MySolution.UnitTests, MySolution.IntegrationTests.
However in Java it just seems to be bundled together, and I would rather separate it out, > however I hear that Maven is a cruel mistress when you want to do things differently to
the default structures.
So to reign this post back in, my main [question is]:
Is there a way to separate out the tests from the project [How?]
Although this question exactly describes what I try to achieve, the thread has not provided the solution for how to do it.
I'd like to know whether there is a way to have a separate project just for (unit) testing with JUnit. I want to have the actual source code in a "core" project and the according tests in a separate "test" project instead of having one single project with src/main and src/test paths.
However, I don't know how to configure the (parent) pom.xml files to achieve that.
So far, I defined a parent pom that declares the two projects as modules. In addition, for each of the two projects, I have a separate pom file declaring the required dependencies etc. Of course, the pom file of the test project defines the core project as a dependency.
I guess I have to configure the pom file of the core project to tell the testing plugin to look in the other project for the tests. But how should such behaviour be configured?
If you follow the Maven conventions (having both the src and test folders) you will have an easier time. Your tests will not be deployed along with your compiled source so I wouldn't worry about bloat. Maven will compile both a jar and a test jar file (assuming you're using jars). If you really want separate src/test modules then yes, the multi-module approach with a common parent is the way to go. The test module would have a dependency on the source module but not the other way around. Really this just amounts to reinventing what Maven is already doing for you though.
In the long run, I think you'd be happier using the conventional approach though as things will go a lot smoother.
Do not pay attention to the naysayers who will try to convince you that you have to do it in one of the established ways or else you will run into trouble. This is cargo cult engineering, and it reflects the cowardice of your average enterprise employee out there, who will rather die than try something different or think outside the box for a moment.
It is perfectly doable to have a huge multi-module maven project, with loads of tests, where not a single module contains both production and test subfolders, and instead every single module is either production, or test. That's the only way they do it in the DotNet world, and I never heard anyone complaining.
There exist situations where you absolutely have to split your modules this way, so maven has no option but to support this. Such situations arise when the dependencies are such that the tests of module A depend on module B which in turn depends on the production code of model A. If both the tests and production code of module A are in the same actual module, this causes a circular dependency, so the project is unbuildable. Such an arrangement is not commonplace, but it does happen some times. When it happens, you have to move the tests of A into a separate module C, which depends on both A and B, and leave only production code on A.
In maven, there is nothing special to it: in production modules you only specify <sourceDirectory>, while in test modules you only specify <testSourceDirectory>. Everything else is done as expected: Both modules have the same parent pom, and the parent pom references them both. JUnit and other test-related dependencies are only included by the test modules. It is so straightforward that it is trivial. (I am not sure what kind of trouble the OP was facing that made him ask the question.)
As a matter of fact, if it was not for the particular maven plugins that people use for running tests during continuous deployment, you would not even need <testSourceDirectory>, you could be using in all modules nothing but <sourceDirectory>. IntelliJ IDEA does not have a problem detecting and running tests even if they are under <sourceDirectory>, but the maven surefire plugin does expect tests to be under <testSourceDirectory>, so you have to use <testSourceDirectory> just to keep that plugin happy.
My personal opinion is that supporting a distinction between production and test subfolders within the same module adds a mind-boggling amount of completely unnecessary complication to build systems. The entire java world would be doing just fine if the feature did not exist at all. Of course this opinion is tentative, since unbeknownst to me there may exist important reasons due to which this distinction is useful. If anyone knows of any such reasons, please enlighten me in the comments.
The basic problem is as such: I've got a project that already uses multiple Maven modules for various sub-projects. However, one of the modules (the core module) could itself be split into multiple OSGi bundles when created. This is due to the core module containing several optional dependencies, each of which have isolated Java packages where they're required. For instance, support for JSON input files are optional as they require the optional dependencies from Jackson. The classes that rely on the Jackson dependencies are all isolated to certain json packages within the module. Thus, in theory, I could create a minimal bundle from core that doesn't include the packages that rely on optional dependencies.
Normally, I'd simply split up this module into more Maven modules to make life easier for creating bundles via Felix's maven-bundle-plugin. The problem here is that I still want to create a core JAR for non-OSGi users who don't want to have to include several extra JARs just to use optional functionality (which requires they provide the optional dependencies on the class path as it is). Not only that, but I don't wish to have to split up this module into more modules as it makes development on the project more tedious for the developers as well, especially when we're already splitting up code into proper package-based modules as it is.
The way we were trying to use OSGi already was to make the API module a fragment host (in order to allow it to load a provider bundle without requiring OSGi support), then make the other bundles use said fragment host. This seemed to work well for the smaller modules outside of core, but for core, we wanted to be able to provide multiple bundles from a single module so that optional dependencies wouldn't be required in the bundle itself. As it stands, for plugins, we already have a mechanism for scanning them and ignoring plugins that don't have all the required classes to load them (e.g., if a plugin requires a JPA provider but the JPA API is not available, that plugin isn't loaded). Once we can successfully split up the core module into multiple bundles, I can use declarative services as the plugin method in an OSGi environment (instead of the default class path JAR scanning mechanism in place for normal Java environments), so that isn't an issue.
Is there any way to do all this using Felix's maven-bundle-plugin? Or will I have to use the assembly plugin to copy subsets of the module where bundles can be generated from? Or will I have to resort to writing an Ant script (or Maven plugin) to do this? We've tried using separate Maven modules that simply import the core module as a dependency and generating a bundle from there, but the resultant bundle is always empty regardless of import/export package settings and embed dependencies.
Or, is there a better way to do this? We already use the <optional>true</optional> configuration for the optional dependencies, yet the Felix plugin doesn't seem to care about that and imports all of those dependencies anyways without using the optional attribute.
Well, this is what I'm ending up doing to accomplish this. I'm using the maven-assembly-plugin to copy the binaries I need and filtering out the classes I don't want to include using the <fileSets/> element similar to the <fileset/> element in Ant.
Using the generated directories for each assembly, I'm using the maven-bundle-plugin along with the <buildDirectory/> configuration option to specify where the bundle's class files are located.
It's not ideal, but it's better than writing an Ant script for a Maven project!
Currently when I am writting a bundle in that depends on a package, I have to "import" or "depend" on a whole other bundle in Maven that contains that package.
This seems like it is counter-productive to what OSGi gives me.
For example let's say I have two bundles: BundleAPI and BundleImpl.
BundleAPI provides the API interfaces:
// BundleAPI's manifest
export-package: com.service.api
BundleImpl provides the implementation:
//BundleImpl's manifest
import-package com.service.api
However, when I am coding BundleImpl in Eclipse, I am forced to "depend" in maven POM on BundleAPI itself - so that eclipse does not complain.
//BundleImpl's POM
<dependency>
<groupId>com.service</groupId>
<artifactId>com.service.api</artifactId>
[...]
</dependency>
So - on one hand, I am depending only on the package com.service.api, while on the other - I need to have the whole bundle - BundleAPI.
Is there a way to make maven or eclipse smart enough to just find the packages somewhere, instead of whole bundles?
I am very much confused as to how this works - any type of clarity here would be great. Maybe I am missing something fundamentally simple?
The key is to distinguish between build-time dependencies and runtime dependencies.
At build time you have to depend on a whole artifact, i.e. a JAR file or bundle. That's pretty much unavoidable because of the way Java compilers work. However at runtime you depend only on the packages you use in your bundle, and this is how OSGi manages runtime substitution. This is the Import-Package statement in your final bundle.
Of course as a developer you don't want to list two parallel sets of dependencies, that would be crazy. Fortunately maven-bundle-plugin is based on a tool called bnd that calculates the Import-Package statement for you based on analysing your code and discovering the actual packages used. Other tools such as bndtools (an Eclipse-based IDE for OSGi development) also use bnd in this way. Incidentally bnd is much more reliable and accurate than any human at doing this job!
So, you define only the module-level dependencies that you need at build time, and the tool generates the runtime package-level dependencies.
I would recommend against using Tycho because it forces you to use Eclipse PDE, which in turn forces you to manually manage imported packages (for the sake of full disclosure, I am the author of bndtools which competes against PDE).
You cannot develop bundles like regular Java projects with Maven and eclipse. You basically have 2 options.
Apache Felix Bundle Plugin: Basically you develop the project as a regular Java project and use Maven as you normally would. This plugin will be used to add all the OSGi specifics to the jar manifest at deployment time to OSGi enable it. The disadvantage of this aproach is that you are using a Java project in your workspace instead of a bundle, which makes running your project in the OSGi container a little extra work since Eclipse doesn't recognize it as a plugin project. Thus you have to add the jar from the Maven build as part of the target platform manually.
Tycho: This is another Maven plugin that attempts to actually bring theses two environments together and does a pretty good job of it. In this scenario, you actually create an Eclipse bundle/plugin project, which obviously makes for seamless integration in Eclipse. The pom then marks the project as being an eclipse-plugin type, which effectively makes Maven resolve the project dependencies (defined in the manifest) via the target platform instead of Maven itself.
I would take the Tycho approach as it gives a much more integrated approach with Eclipse.
Having the whole jar as a dependency shouldn't be a problem, that's how you have to do it with Maven anyway.