Develop Reference

How do optional compile-time dependencies work?

For the longest time, I thought that in Java you either had one of two types of dependencies:
Required compile-time dependencies (dependencies always required at compile time)
Possibly optional runtime dependencies (dependency that can be
resolved at runtime)
Recently, I found out that compile dependencies can be optional too. For example, commons-beanutils is listed as an optional compile dependency of JXPath.
How can this work? Can a dependency really be used at the time of compilation yet remain fully optional?
EDIT: I might have been unclear. I'm looking for a case where a dependency is used at compile-time and is at the same time fully optional, or an explanation why such a dependency is impossible.

A class can compile to an interface but the implementation of that interface is not needed during compilation. The implementation is needed during runtime.
Example commons-logging, JPA, JDBC etc which are frameworks, an application can compile based on these. At runtime an implementation is needed to execute the code. Sample implementations - Common Bean utils, Oracle thin driver, Eclipse link etc.

An extensive quote from Maven documentation describes this quite clearly:
Optional dependencies are used when it's not possible (for whatever reason) to split a project into sub-modules. The idea is that some of the dependencies are only used for certain features in the project and will not be needed if that feature isn't used. Ideally, such a feature would be split into a sub-module that depends on the core functionality project. This new subproject would have only non-optional dependencies, since you'd need them all if you decided to use the subproject's functionality.
However, since the project cannot be split up (again, for whatever reason), these dependencies are declared optional. If a user wants to use functionality related to an optional dependency, they have to redeclare that optional dependency in their own project. This is not the clearest way to handle this situation, but both optional dependencies and dependency exclusions are stop-gap solutions.
Why use optional dependencies?
Optional dependencies save space and memory. They prevent problematic jars that violate a license agreement or cause classpath issues from being bundled into a WAR, EAR, fat jar, or the like.
How do optional dependencies work?
Project-A -> Project-B
The diagram above says that Project-A depends on Project-B. When A declares B as an optional dependency in its POM, this relationship remains unchanged. It's just like a normal build where Project-B will be added in Project-A's classpath.
Project-X -> Project-A
When another project (Project-X) declares Project-A as a dependency in its POM, the optional nature of the dependency takes effect. Project-B is not included in the classpath of Project-X. You need to declare it directly in the POM of Project X for B to be included in X's classpath.
A practical example: imagine that you are a developer of a library/framework SuperLib that is built as one superlib.jar. Your library provides multiple features. Its main feature (that most of the users use) is dependency injection based on a third-party di library. However, one of your classes - EmailApi - offers features to send e-mails, using a third-party email library. Since superlib is one artifact, it needs both di and email to be compiled.
Now put yourself in the position of a user who uses superlib. They are interested in the dependency injection features. This is the core role of your library, so the dependency between superlib and di would not be optional.
However, most users are not interested in sending emails and may be bothered by having a useless email library and its dependencies added to their application (which will cause size increase of their application and may cause a dependency version clash between the dependencies of email and dependencies of the user's application). Therefore, you would mark the dependency on email as optional. As long as the user does not use your EmailApi class, everything will run fine. However, if they do use EmailApi, they will need the email dependency, otherwise the application will fail at runtime with ClassNotFoundException for whichever class from email would be referenced in EmailApi. The user of your library will need to add the email dependency explicitly in their POM.
See also When to use <optional>true</optional> and when to use <scope>provided</scope>.

What you described is actually a feature of Maven, the build tool, but not Java itself.
Without build tools, using just 'javac' you need to specify all classes or interfaces that directly used in your code. Sure there are options for dynamic class loading and even runtime compilation, but thats not on topic.
One of use-cases with separation on interface and implementation is described in previous answer, another popular case is based on classpath scanning:
if some specific class is present in classpath and/or has specific annotation - an optional module will be loaded.
That's how Spring Boot modules are loaded.

Java modules: accessibility problems for Mockito 2.20.0

I am migrating from Java 8 to Java 10, and I am running my test which now fails because of package protected classes. The build is run under maven 3.5.4 + Oracle JDK 10.0.2:
maven-compiler-plugin 3.7.0 + asm 6.2
maven-surefire-plugin 2.22.0 + asm 6.2 + junit 5.2.0
asm 6.2 is required for both compiler/surefire because of a bug in the version of ASM used by those plugins.
mockito-core 2.20.0 (but was using 2.20.0 with Java 8 before).
Eclipse Photon R
The project can be found here ide-bugs.zip (it is located at Eclipse forum because I've made this Topic on Eclipse for another problem, this time with Eclipse having local error with module).
The test is very simple: we try to mock different class, with different access level - all of which were working in Java 8.
package protected class
public class but not exported, not opened
public class not exported but opened to Mockito
public class not exported but opened to all
package protected class not exported but opened to Mockito
package protected class not exported but opened to all
In Java 8, case 1, 5 and 6 are the same (access to package protected). Case 2, 3 and 4 are the same (access to public).
The test fails because Mockito is unable to either:
class org.mockito.codegen.NotExportedOpenToMockitoProtected$MockitoMock$117073031 cannot access its superclass nodatafound.mjpmsuc.withopens.NotExportedOpenToMockitoProtected
class org.mockito.codegen.NotExportedNotOpenedPublic$MockitoMock$365628885 (in unnamed module #0x3f07b12c) cannot access class nodatafound.mjpmsuc.internal.NotExportedNotOpenedPublic (in module nodatafound.mockito_jpms_usecase) because module nodatafound.mockito_jpms_usecase does not export nodatafound.mjpmsuc.internal to unnamed module #0x3f07b12c
Mockito effectively have a Automatic-Module-Name but is seen as the unamed module because all jar found in the class path for a big "unnamed module".
While I'm fine with migrating from package-protected to non exported package, I fail to understand how I can address the problem keeping my interface/class not visible to other modules ?
[edit] updated the version of plugin/dependency one month after, no result.

I found part of answer to my problem here: https://blog.codefx.org/java/java-module-system-tutorial/#Open-Packages-And-Modules
Mockito is using reflection to access classes from module or class path.
Mockito is in the "unnamed module" because Maven adds it into the class path rather than the module path. This explains why the opens package to org.mockito never works: there is no org.mockito module.
Maven Surefire does not care to contribute to the "opens" of the module in order to allow Mockito to access it.
Mockito is (no longer?) able to mock non-private & non-final classes class. By any means package protected class are private. The error is rather explicit: Mockito create a class extending the package protected class, which now fails (it was working before, but this was probably because Mockito created the class in the same package than the one being mocked).
Nevertheless, this give a problematic configuration in the pom.xml of each module:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>
--add-opens nodatafound.mockito_jpms_usecase/nodatafound.mjpmsuc=ALL-UNNAMED
</configuration>
</plugin>
We need to explicitly add opens to an unnamed module. This should not be done in the module-info.java because it expose the module to all other modules or jars which is against encapsulation.
This is problematic because:
You need to specify it in the pom.xml for each package.
It add additional burdens to the surefire configuration which I prefer simple.
You don't have validation from the IDE; Eclipse will validate module-info.java marking invalid package.
m2e does not pass to Eclipse JUnit plugin the necessary <argLine /> making the test fail in Eclipse.
The maven approach (which is the same in Eclipse, and perhaps Gradle as far as I know) does not permits an additional module-info for the tests; eg: lets test dependency be modular (this could be probably be done using a dedicated test module per source module like Eclipse does for plugin' tests).

NetBeans 8.2 - Weblogic - Ant build

When I try to build my Enterprise Project in Netbeans with ant, i got this exception:
warning: Supported source version 'RELEASE_6' from annotation
processor
'org.eclipse.persistence.internal.jpa.modelgen.CanonicalModelProcessor'
less than -source '1.8'
Note: Creating static metadata factory ...
An annotation processor threw an uncaught exception.
Consult the following stack trace for details.
java.lang.ClassFormatError: Absent Code attribute in method that is not native or abstract in class file
javax/persistence/PersistenceException
I only have some entityclasses in my ejb module.

This error is reported as a bug in NetBeans (with a resolved status as it turns out to be considered as a "user error"). Below is the main comment that explains the problem and gives the solution:
"ClassFormatError: Absent Code attribute in method..." is always indication that javaee-api-6.0.jar is used for runtime execution. The jar contains only method signatures (method bodies are stripped) and is suitable only for compilation. In future versions of javac there might be better error message.
The problem here is that javaee-api-6.0.jar is on classpath before EclipseLink jars and when EclipseLink annotation processor is started classes from javaee-api-6.0.jar are used instead of classes from EclipseLink. First thing is that javaee-api-6.0.jar should be removed from classpath of EJB project - it should not be needed as EJB project has an Application server selected and the project takes EE 6 APIs from that server.
As I understand the comment, you should try removing any reference to javaee-api-6.0.jar (which contains only "dummy" classes) from the classpath of the project, because the EclipseLink library should provide the correct classes used by the annotation processor.
Also, this question seems to tackle the same issue.

when you use the insert code and automatic bean creation it adds the Java EE 6
API library . I deleted the library and add the Java EE 7 API.

java.lang.ExceptionInInitializerError Exception in JasperReports [duplicate]

I recompiled my classes as usual, and suddenly got the following error message. Why? How can I fix it?
java.lang.SecurityException: class "Chinese_English_Dictionary"'s signer information does not match signer information of other classes in the same package
at java.lang.ClassLoader.checkCerts(ClassLoader.java:776)

This happens when classes belonging to the same package are loaded from different JAR files, and those JAR files have signatures signed with different certificates - or, perhaps more often, at least one is signed and one or more others are not (which includes classes loaded from directories since those AFAIK cannot be signed).
So either make sure all JARs (or at least those which contain classes from the same packages) are signed using the same certificate, or remove the signatures from the manifest of JAR files with overlapping packages.

A simple way around it is just try changing the order of your imported jar files which can be done from (Eclipse). Right click on your package -> Build Path -> Configure build path -> References and Libraries -> Order and Export. Try changing the order of jars which contain signature files.

A. If you use Maven, a useful way to debug clashing jars is:
mvn dependency:tree
For example, for an exception:
java.lang.SecurityException: class "javax.servlet.HttpConstraintElement"'s signer information does not match signer information of other classes in the same package
we do:
mvn dependency:tree|grep servlet
Its output:
[INFO] +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] +- javax.servlet:jstl:jar:1.2:compile
[INFO] | +- org.eclipse.jetty.orbit:javax.servlet.jsp:jar:2.2.0.v201112011158:compile
[INFO] | +- org.eclipse.jetty.orbit:javax.servlet.jsp.jstl:jar:1.2.0.v201105211821:compile
[INFO] | +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile
[INFO] +- org.eclipse.jetty:jetty-servlet:jar:9.0.0.RC2:compile
shows clashing servlet-api 2.5 and javax.servlet 3.0.0.x.
B. Other useful hints (how to debug the security exception and how to exclude Maven deps) are at the question Signer information does not match.

In my case, I had duplicated JAR version of BouncyCastle in my library path :S

I had a similar exception:
java.lang.SecurityException: class "org.hamcrest.Matchers"'s signer information does not match signer information of other classes in the same package
The root problem was that I included the Hamcrest library twice. Once using Maven pom file. And I also added the JUnit 4 library (which also contains a Hamcrest library) to the project's build path. I simply had to remove JUnit from the build path and everything was fine.

This can occur with the cglib-instrumented proxies because CGLIB uses his own signer information instead of the signer information of the application target class.

After sign, access: dist\lib
Find extra .jar
Using Winrar, You extract for a folder (extract to "folder name") option
Access: META-INF/MANIFEST.MF
Delete each signature like that:
Name: net/sf/jasperreports/engine/util/xml/JaxenXPathExecuterFactory.c
lass
SHA-256-Digest: q3B5wW+hLX/+lP2+L0/6wRVXRHq1mISBo1dkixT6Vxc=
Save the file
Zip again
Renaime ext to .jar back
Already

I am having this problem with Eclipse and JUnit 5.
My solution is inspired by the previous answer by user2066936
It is to reconfig the ordering of the import libraries:
Right click the project.
Open [Java Build Path].
Click Order and Export.
Then push JUNIT to upper priority.

If you're running it in Eclipse, check the jars of any projects added to the build path; or do control-shift-T and scan for multiple jars matching the same namespace. Then remove redundant or outdated jars from the project's build path.

A bit of an old thread but since I was stuck for quite some time on this, here's the fix (hope it helps someone).
My scenario:
The package name is: com.abc.def. There are 2 jar files which contain classes from this package, say jar1 and jar2 i.e. some classes are present in jar1 and others in jar2. These jar files are signed using the same keystore but at different times in the build (i.e. separately). That seems to result in different signatures for the files in jar1 and jar2.
I put all the files in jar1 and built (and signed) them all together. The problem goes away.
PS: The package names and jar file names are only examples

In my case it was a package name conflict. Current project and signed referenced library had one package in common package.foo.utils. Just changed the current project error-prone package name to something else.

If you added all the jars from bouncycastle.org (in my case from crypto-159.zip), just remove the ones for the JDKs that do not apply to you. There are redundancies. You probably only need the "jdk15on" jars.

This question has lasted for a long time but I want to pitch in something. I have been working on a Spring project challenge and I discovered that in Eclipse IDE. If you are using Maven or Gradle for Spring Boot Rest APIs, you have to remove the Junit 4 or 5 in the build path and include Junit in your pom.xml or Gradle build file. I guess that applies to yml configuration file too.

This also happens if you include one file with different names or from different locations twice, especially if these are two different versions of the same file.

I could fix it.
Root Cause:
This is a common issue when using the Sun JAXB implementation with signed jars.
Essentially the JAXB implementation is trying to avoid reflection by generating a class to directly access the properties without using reflection. Unfortunately, it generates this new class in the same package as the class being accessed which is where this error comes from.
Resolution:
Add the following system property to disable the JAXB optimizations that are not compatible with signed jars:
-Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true
Ref: https://access.redhat.com/site/solutions/42149

Based on #Mohit Phougat response, if you are running a Groovy with #Grab annotations, you could try to re-order such annotations.

I was getting a similar error when trying to use Mockito:
"$$FastClassByMockitoWithCGLIB$$abb8f5a0"'s signer information does not match signer information of other classes in the same package"
I was using an old version of Mockito, and upgrading to the latest Mockito version solved this problem. The issue was with CGLIB as mentioned in one of the other answers. In newer versions, Mockito replaces CGLIB with ByteBuddy, and so the problem goes away. I also had to add the new ByteBuddy jars to the classpath in Eclipse to get Mockito working again.

I was running JUNIT 5 and was also referencing Hamcrest external jar, but Hamcrest is also a part of the JUNIT 5 library. So, I moved the order of the external Hamcrest jar file to be above the JUNIT 5 library in the build path.

This happened to me when using JUnit + REST Assured + Hamcrest. In this case, don't add JUnit to your build path. If you have a Maven project, the below pom.xml file resolved this for me:
<dependencies>
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>rest-assured</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
</dependency>
<!-- https://mvnrepository.com/artifact/junit/junit -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
</dependencies>

maven - separate modules for interfaces and implementation with Spring

We are working on Mavenizing our java project and we would like to setup a clean separation between interfaces and implementations for each module.
In order to do so, we want to split each module into two sub-modules one for interfaces and data objects used by them and another for implementations.
For example:
+commons
+commons-api
+commons-impl
The POMs of the modules will be configured such that no module depends on the impl sub-modules. This way no code from one module will be able to "see" implementation details of another module.
What we are having trouble with, is where to put our spring XMLs.
In our project we automatically import spring XML files using wildcard import like
<import resource="classpath*:**/*-beans.xml"/>
This way the location of Spring XMLs doesn't really matter at runtime, as all the modules get loaded into the same class loader and, the strict one way dependency rules in the POMs don't apply.
However, during development we want the IDE - we use Intellij IDEA - to recognize implementation classes referenced from the spring XMLs.
We also want IDEA to recognize beans defined in other modules.
If we put the spring XMLs in API sub-modules - they won't "see" the implementation classes in the impl sub-modules.
If we put them in the impl sub-modules, their beans won't be "seen" from other modules.
It is probably possible to configure the IDEA project to recognize spring XMLs from modules on which there is no dependency, but we prefer for our POMs to hold all the project structure information and not rely on IDEA project files.
We considered creating a third sub-module just to hold Spring XMLs (and perhaps hibernate xmls as well). For example:
+commons
+commons-api
+commons-impl
+commons-config
The external modules will depend on both commons-api and commons-config and commons-config will depend on both commons-api and commons-impl, with the dependency on commons-impl marked as "provided" (to prevent transitive resolution).
This however seems like a complex and awkward solution and we feel that there must be a better - simpler way to achieve interface/impl separation with Maven and Spring.

What you need is a runtime dependency scope:
runtime - This scope indicates that the dependency is not required for compilation, but is for execution. It is in the runtime and test classpaths, but not the compile classpath.
(https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html)
Define a runtime dependency from one impl module to another impl module where you use the impl classes in the *-beans.xml config. Intellij will correctly recognize this in spring configuration files, but won't auto complete them in code (but it will do that in test code).
Also if anyone used the classes in the code, compilation through maven would fail, because the runtime dependency is not on a compile class path.

You can achieve decoupling of api and impl like this:
+ commons (pom)
+ pom.xml <--- serves as a parent aggregator (see below)
+ commons-api (jar) <--- contains models, interfaces and abstract classes only
+ commons-impl (jar) <--- depends on commons-api
+ commons-config (jar) <--- depends on commons-impl only (no need to depend on commons-api as it is brought in transitively)
+ external-project (war or jar) <--- has commons-config as a dependency
Parent aggregator pom (specify build order):
<modules>
<module>commons-api</module>
<module>commons-impl</module>
<module>commons-config</module>
</modules>
The config module can be omitted if it only contains spring application context configuration. The app configuration xml should be in the classpath and folder structure of the module that contains the artifact that you are deploying. So if you are building a war artifact, the app context should be in there.
The only configuration that should be in your commons module would be in a test package of your impl module.

In short you want Idea to override maven dependency graph but avoid keeping this configuration in idea project files?
One option is to group implementation dependencies in a maven profile. This profile would not be enabled by default but you should be able to mark it as active under idea.

Two ideas come to mind:
You will have one (or more) modules where all the modules (api+impl) are dependencies, you could place your spring configuration files there.
Place the spring configuration files in the api modules and declare a dependency on the impl module with scope provided this way the implementations will be known, while there is no dependency of the api for the deployment.

commons-impl at runtime scope in external modules
commons (pom dependencyManagement) =>
+commons-api (compile)
+commons-impl (compile)
+commons-config (compile)
commons-impl (pom dependencies) =>
+commons-api (compile)
+commons-config (compile)
external modules (pom dependencies) =>
+commons-impl (runtime)
+commons-api (compile)
+commons-config (compile)

keep modules number as little as possible;
This speeds up project build time and simplifies its layout.
keep modules structure as plain as possible: single root + all sub modules in the same folder, e. g.:
pom.xml
commons-api/
commons-runtime/
module-a-api/
module-a-runtime/
...
This simplifies navigation across the project, when modules number is really high (>50)
provide runtime-scoped dependencies to the runtime modules only when they are required;
This keeps your architecture clear. Use mocks instead of explicit dependency to another runtime module.
keep your api spring contexts in api modules, define your public beans as abstract bean + interface;
keep your implementation contexts in runtime modules, override api beans with your implementations via spring profiles (use <beans profile="default").
Result: simple, transparent layout and design; full ide support; no explicit dependencies on runtime module internals.