Develop Reference

Import path in Java, Maven

Following the tutorial about Kafka Streams located at: https://github.com/confluentinc/kafka-streams-examples/blob/4.0.0-post/src/main/java/io/confluent/examples/streams/WikipediaFeedAvroExample.java
There is a line:
import io.confluent.examples.streams.avro.WikiFeed
As I suppose it relates to this file: https://github.com/confluentinc/kafka-streams-examples/blob/4.0.0-post/src/main/resources/avro/io/confluent/examples/streams/wikifeed.avsc
How does Maven knows it is in resource not java folder?
Why io/confluent/examples/streams/avro/wikifeed.avsc instead of avro/io/confluent/examples/streams/wikifeed.avsc?
The other import is even more fantastic:
import io.confluent.kafka.serializers.AbstractKafkaAvroSerDeConfig;
There is no kafka folder in the java/io/confluent folder.
https://github.com/confluentinc/kafka-streams-examples/tree/4.0.0-post/src/main/resources/avro/io/confluent.
How does all this magic suppose to work?

The magic is made by avro-maven-plugin which you can find in the pom.xml:
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>${avro.version}</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
</goals>
<configuration>
<sourceDirectory>src/main/resources/avro/io/confluent/examples/streams</sourceDirectory>
<outputDirectory>${project.build.directory}/generated-sources</outputDirectory>
<stringType>String</stringType>
</configuration>
</execution>
</executions>
</plugin>
Quoting from the documentation of the plugin:
Simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementing for statically typed languages.
This is, at pre compile time, the plugin reads the content of avsc files and generate binary sources (for this case, Java classes) that then can be used in the code.
You can see the code generated by the plugin in target/generated-sources. There will be a folder structure and proper java (not class) files there.

The WikiFeed class is created dynamically at build time using the avro-maven-plugin from the .avsc file you linked to. You can check how it's configured in the <plugins> section of pom.xml.
The AbstractKafkaAvroSerDeConfig class comes from the kafka-avro-serializer dependency. Eclipse has a nice way of navigating from the individual class in the Editor view back to the Package Explorer which includes the Maven dependencies, like this:

Java + Maven - executable jar from test class

I've got a multi-module Selenium project that's using the Page Factory and running tests through Suites by calling mvn clean verify...
I've created a JavaFX app that loads all the Suites and lets the user select a test, which is then run by the above-mentioned command.
The problem I'm facing now is that, in order to load all the test classes (I'm using test-jar in module dependencies so that my test class in the app module can see the tests in the other modules), I need to have the main method in a test class, and not in a main class.
Obviously, that doesn't work since the test classes are not included in the jar.
I'm looking for a solution, either on having that main method in a test class, or for src/main to see into src/test.
One of the recommendations people have been giving is to have a separate module for the tests. This is not an option for me though as I don't have the rights to move the tests classes around or change the project structure that much.
Here's the plugin I'm using:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>com.package.myapp.Main</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
For loading the suites, I use ClassLoader loader = Thread.currentThread().getContextClassLoader() which gets an ImmutableSet<ClassPath.ClassInfo> from ClassPath.from(loader).getTopLevelClasses()

This is a standard problem about decoupling modules: You have two modules: Dependent Module and Independent Module. You need the Independent to be invoked from the Dependent one, but without including the Independent in the Dependent's compilation.
So, you have several options to chose:
When the API and protocol of Independent modules is known at compile-time.
In this case, you should design an interface to model each Independent API. Publish these interfaces into a separated library, and retro-extend the Independent library to these interfaces. Provide also a factory which returns objects of these interfaces, by dynamic instantiation of a set of classes names (received at run time).
In the Dependent module, you just have to invoke the factory to obtain each interface implementation and use it.
When the API and protocol of independent modules is not known at compile-time.
Uh, uh...! I hope sincerely this is not your case, because it does not fit in the good practices of standard design.
Anyway, in Java you can always invoke any method in any object through the use of reflection. Though I won't advise it.

jacoco only shows coverage for classes in the same module

I have a somewhat large multi-module Maven project. I have the unit tests in each module being processed by Jacoco. I have a separate child module doing "merge" and "report-aggregate", and this appears to be generating data. I'm even using the generated data in SonarQube. Most of my tests are using PowerMock, and I'm using offline instrumentation.
However, after looking closer at the coverage data, I see that it is leaving out coverage data for classes and methods that I know are being executed during tests. The pattern I see in every module is that it only reports coverage for a single class in each module, which is a class actually in the current module. Almost all of the tests also call out to other classes in other modules in the build, and coverage for those classes are never reported.
The following plugin configurations are in the parent pom used by each child module:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<executions>
<execution>
<id>default-instrument</id>
<goals>
<goal>instrument</goal>
</goals>
</execution>
<execution>
<id>default-restore-instrumented-classes</id>
<goals>
<goal>restore-instrumented-classes</goal>
</goals>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<argLine>-Xmx1024m</argLine>
<includes>
<include>**/*Test.java</include>
</includes>
<systemPropertyVariables>
<jacoco-agent.destfile>${project.build.directory}/jacoco.exec</jacoco-agent.destfile>
<running-unit-test>true</running-unit-test>
</systemPropertyVariables>
</configuration>
</plugin>
When I inspect the generated HTML results for each module, I find that it only reports results for the single class in the current module, and not the data for classes in other modules. From this, I would assume that how I do "merge" and "report-aggregate" in the separate child module is probably irrelevant to this problem.
The generated "jacoco.exec" file is binary, but I tried "catting" out one from one module just to see what ascii text was visible, and it showed only one occurrence of anything that looked like a file name, and it was the only file name reported in the HTML coverage report for that module.
I'm not sure what other information I can report.
Update:
I guess I can see pretty clearly now that when surefire runs unit tests, it uses the instrumented classes from the current module, but the uninstrumented classes from the maven artifacts. This is why I only see coverage for classes in the current module.
So it seems like I need a way to specify that the "target/generated-classes/jacoco" folder for each module the current module depends on, is prepended to the classpath that surefire uses. I don't see a way to do that.
Alternatively, I see that the "instrument" goal has an "includes" configuration element. Should I be specifying paths to all of the "target/classes" directories for each of the modules that the current module depends on?

Recording of code coverage for some class requires its instrumentation. Goal instrument performs instrumentation of classes of current module.
all of the tests also call out to other classes in other modules
so the ones that are not instrumented. And if I correctly understood, then exactly those for which you are missing coverage.
If you don't use PowerMock for classes that come from other modules, but only for classes in current module, then you can combine offline instrumentation with on-the-fly using agent. But in this case make sure that classes instrumented offline are explicitly excluded from instrumentation by agent, otherwise agent will be throwing IllegalStateException: Class ... is already instrumented.
If you use PowerMock for classes that come from other modules, then this becomes more complex due to strictness of Maven in regards of manipulations with classpaths and dependencies. And I doubt that this can be easily achieved using one mvn comand, however seems possible using more:
instrument and run tests, but don't use restore-instrumented-classes
restore classes and generate report(s)
Unfortunately you haven't provided complete example (https://stackoverflow.com/help/mcve) and I don't have time to prepare full example to test this approach right now.
As a side note: inability to simply use agent comes from the fact that PowerMock bypasses any agent and reads class files from disk.

Maven plugin instead of javaagent for Byte Buddy?

can there be a maven plugin instead of javaagent to reduce startup time of an application? Many ORM tools have both javaagent and maven plugin, so it should be possible - is it? Or at least something like "CompiledClassFilesBuilder" similar in functionality to AgentBuilder :-)
Regards,
Pavel

It is perfectly possible to use Byte Buddy from within a Maven plugin. You can implement a ClassFileLocator for the Maven target after the compile phase and redefine the classes within this folder using a ByteBuddy instance. The API is identical to that of the AgentBuilder inside a transformer.
You would need to implement such a plugin, though. There is none currently existing.
Do however note that this static transformation has its limitations. Using a plugin, it is not possible to redefine bootstrap classes of the VM. It would also be difficult to change library classes as they are external artifacts. Finally, Byte Buddy can register live callbacks that need to be set wt runtime.
UPDATE: I just added the plugin in question and will release a Maven Byte Buddy plugin in version 1.4.21. A transformation can be added as follows:
<plugin>
<groupId>net.bytebuddy</groupId>
<artifactId>byte-buddy-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>transform</goal>
</goals>
</execution>
</executions>
<configuration>
<transformations>
<transformation>
<plugin>net.bytebuddy.test.SimplePlugin</plugin>
<!-- Optionally, specify groupId, artifactId, version of the class -->
</transformation>
</transformations>
<!-- Optionally, add 'initialization' block with EntryPoint class -->
</configuration>
</plugin>
The two relevant interfaces Plugin and EntryPoint are added to the library directly and are straight-forward to implement.

Maven compiler, only compile annotated classes

I've created a custom Java annotation (code below) in a Maven 2 project I'm working on:
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
#Target(ElementType.TYPE)
#Retention(RetentionPolicy.RUNTIME)
public #interface MYANNOTATION{}
At one part of the Maven build, I only want to compile classes annotated with this annotation, e.g:
#MYANNOTATION
public class MyClass {
// Code here
}
I'm currently using the Maven Compiler Plugin to restrict complication based on package structure. My pom.xml contains resembles the one below, restricting compilation to classes in **com.foo.bar.stuff** and **com.baz.foo.more**. This is unsatisfactory, because when I add annotated classes to com.xyz.bar.foo, I must remember to define it in the pom.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<executions>
<execution>
<id>default-compile</id>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
<configuration>
<includes>
<include>**/com/foo/bar/stuff/**</include>
<include>**/com/baz/foo/more/**</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Is there any way to define Maven to compile only classes that have been annotated with this annotation, not depending on where they are located in the package hierarchy?
(I'm trying to generate a metamodel from domain model classes so I can point to fields & methods without defining the names as String constants - and changing them manually when I refactor)
Edit: I am already doing annotation processing in another part of the build phase. The system works like this:
Compile classes in the specified packages
Using JAnnocessor, build metamodels from classes with #MYANNOTATION
Compile the rest of the classes
Dependencies from other classes to the metamodel classes prevent compiling everything in one go, unless we move the annotated classes to a different project and add a dependency to it. That's one possibility but can add complexity, because the current project structure appears to form a logical whole.

You can do something similar to what you want with annotation processing. I don't think there's any maven-specific thing you need to do, but you need to write an annotation processor that has to either be part of a separate library or compiled separately.
The concept of annotation processing is explained pretty well in this blog entry:
Code Generation using Annotation Processors in the Java language –
part 2: Annotation Processors