I would like to distribute a jar of a library I created with all my dependencies bundled inside. However I would like to avoid version conflicts of dependencies with the adopting project.
I think maven shade can do this but I could not find a way to do this with Scala / SBT. I found OneJar however from my experiments with it seems to work only for executables.
How could I achieve this?
Thanks!
You can do this with your own classloader.
The classLoader:
Write a class loader which loads class files from diferent classloader using a rewrite.
For example you could add library as a prefix to the classpath when fetching the resource.
I have created a classloader using this teqnuiqe.
https://github.com/espenbrekke/dependent/blob/master/src/main/java/no/dependent/hacks/PathRewritingClassLoader.java
It replaces the method findClass in URLClassLoader with one adding a prefix.
protected Class<?> findClass(final String name) throws ClassNotFoundException {
Class result;
try {
result = (Class)AccessController.doPrivileged(new PrivilegedExceptionAction() {
public Class<?> run() throws ClassNotFoundException {
// This is where the prefix is added:
String path = PathRewritingClassLoader.this.prefix + name.replace('.', '/').concat(".class");
Resource res = PathRewritingClassLoader.this._ucp.getResource(path, false);
if(res != null) {
try {
return PathRewritingClassLoader.this._defineClass(name, res);
} catch (IOException var4) {
throw new ClassNotFoundException(name, var4);
}
} else {
return null;
}
}
}, this._acc);
} catch (PrivilegedActionException var4) {
throw (ClassNotFoundException)var4.getException();
}
if(result == null) {
throw new ClassNotFoundException(name);
} else {
return result;
}
}
We also have to rewrite resource loading
#Override
public URL getResource(String name){
return super.getResource(prefix+name);
}
Here is how it is used:
_dependentClassLoader = new PathRewritingClassLoader("private", (URLClassLoader)DependentFactory.class.getClassLoader());
Class myImplementationClass=_dependentClassLoader.loadClass("my.hidden.Implementation");
Building your jar:
In your build you place all the library and private classes under your selected prefix. In my gradle build I have a simple loop collecting all the dependencies.
task packageImplementation {
dependsOn cleanImplementationClasses
doLast {
def paths = project.configurations.runtime.asPath
paths.split(':').each { dependencyJar ->
println "unpacking" + dependencyJar
ant.unzip(src: dependencyJar,
dest: "build/classes/main/private/",
overwrite: "true")
}
}
}
Proguard can rename packages inside jar and obfuscate code. It is a bit complicated but you can achieve you goal with it. sbt-proguard plugin is actively maintained
Also you can check answers from similar thread:
maven-shade like plugin for SBT
UPDATE:
from version 0.14.0 sbt-assembly plugin seemed to have shading ability
Have you tried sbt-assembly plugin? It has set of merging strategies in case of conflicts and has pretty much nice start guide.
Related
I want to create a custom Gradle plugin that will encapsulate Checkstyle and PMD configurations. So, other projects can just apply one custom plugin without bothering about any additional configurations.
I applied checkstyle plugin.
plugins {
id 'java-gradle-plugin'
id 'checkstyle'
}
And then I applied it inside my custom plugin.
public class CustomPlugin implements Plugin<Project> {
public void apply(Project project) {
project.getPluginManager().apply(CheckstylePlugin.class);
}
}
When I try to build the project I get an error.
Unable to find: config/checkstyle/checkstyle.xml
How can I override other plugin's properties? For example, I want to change the default checkstyle.xml path. I can do it manually inside build.gradle of the plugin project itself. But in this case, other projects that apply the plugin won't have this configurations defined by default (I tested it).
EDIT 1:
I managed to configure checkstyle plugin with ChecktyleExtension.
public class MetricCodingRulesGradlePluginPlugin implements Plugin<Project> {
public void apply(Project project) {
project.getPluginManager().apply("checkstyle");
project.getExtensions().configure(CheckstyleExtension.class, checkstyleExtension -> {
checkstyleExtension.setConfigFile(new File("style/checkstyle.xml"));
});
}
}
checkstyle.xml is placed in the plugin project. When I try to apply it within any other project, checkstyle searches it inside the current project directory but not the plugin's one. Is it possible to overcome this issue? I don't want users of that plugin to put any additional files inside their project.
EDIT 2:
I put the config files to resources folder and tried to read the content.
public class MetricCodingRulesGradlePluginPlugin implements Plugin<Project> {
public void apply(Project project) {
project.getPluginManager().apply("checkstyle");
project.getExtensions().configure(CheckstyleExtension.class, checkstyleExtension -> {
URL url = getClass().getClassLoader().getResource("style/checkstyle.xml");
System.out.println("URL: " + url);
try {
checkstyleExtension.setConfigFile(
Paths.get(url.toURI())
.toFile()
);
} catch (URISyntaxException e) {
throw new RuntimeException(e);
}
});
}
}
When I apply the plugin to another project, I get the following error:
URL: jar:file:/Users/user/.gradle/caches/jars-9/8f4176a8ae146bf601f1214b287eb805/my-plugin-0.0.1-SNAPSHOT.jar!/style/checkstyle.xml
Caused by: java.nio.file.FileSystemNotFoundException
at com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
at com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
Java cannot read the file from the jar archive for some reason. Any approaches to overcome this error?
You'd need to bundle the checkstyle.xml within your plugin's resources folder, so when you ship it, you can always access it from within the plugin code.
Basically, you need to put the config under src/main/resources/checkstyle.xml of the plugin and then access it like this:
URL resourceURL = getClass().getClassLoader().getResource("checkstyle.xml");
if (resourceURL != null) {
File resourceFile = File(resourceURL.getFile());
checkstyleExtension.setConfigFile(resourceFile);
}
Also remember, if you ship your plugin as a .jar, you'd need to unpack the checkstyle.xml into a temp file beforehand. Roughly:
File temp = File.createTempFile(".checkstyle", ".xml")
try (FileOutputStream out = new FileOutputStream(temp)) {
try (InputStream resourceStream = getClass().getClassLoader().getResourceAsStream("checkstyle.xml")) {
byte[] buffer = new byte[1024];
int bytes = resourceStream.read(buffer);
while (bytes >= 0) {
out.write(buffer, 0, bytes);
bytes = resourceStream.read(buffer);
}
}
}
Recently we had a version mismatch problem with a class org.apache.commons.beanutils.PropertyUtilsBean. We thought that mismatch is just between some dependency that brings commons-beanutils in versions 1.8 and 1.9.3 but after tracking and excluding each transitive dependency we were still facing an issue.
It turns out that the the PropertyUtilsBean was also packed inside the commons-digester3-3.2-with-deps instead declared as dependency to commons-beanutils.
Is it possible in gradle to search all dependencies (including transitive ones) for specific fully qualified classname? That way we could resolve such problems on the spot.
I tried it and it is possible using some custom gradle build logic:
Kotlin DSL
tasks {
val searchClass by creating {
doLast {
configurations.forEach { // check all configurations
if (it.isCanBeResolved) {
try {
val classLoader = configToClassloader(it)
// replace here class you are looking for
val cl = Class.forName("arrow.core.Either", false, classLoader)
println("found in Configuration $it")
println(cl.protectionDomain.codeSource.location)
} catch (e: Exception) {}
}
}
}
}
}
// Helper function: convert a gradle configuration to ClassLoader
fun configToClassloader(config: Configuration) =
URLClassLoader(
config.files.map {
it.toURI().toURL()
}.toTypedArray())
This could be further enhanced by replacing the hard coded classname with some parameter mechanism.
Sample output:
> Task :searchClass
Configuration configuration ':domain:apiDependenciesMetadata'
file:/Users/abendt/.gradle/caches/modules-2/files-2.1/io.arrow-kt/arrow-core-data/0.9.0/a5b0228eebd5ee2f233f9aa9b9b624a32f84f328/arrow-core-data-0.9.0.jar
Groovy DSL
def configToClassloader(config) {
return new URLClassLoader(
*config.files.collect {
it.toURI().toURL()
}.toArray())
}
task searchClass {
doLast {
configurations.forEach { // check all configurations
if (it.canBeResolved) {
try {
def classLoader = configToClassloader(it)
// replace here class you are looking for
def cl = Class.forName("arrow.core.Either", false, classLoader)
println("found in Configuration $it")
println(cl.protectionDomain.codeSource.location)
} catch (e) {}
}
}
}
}
Edit: I have recently created a Gradle Plugin that provides the described tasks: https://plugins.gradle.org/plugin/io.github.redgreencoding.findclass
You could do this
task findJarsForClass {
doLast {
def findMe = 'org/apache/commons/beanutils/PropertyUtilsBean.class'
def matches = configurations.runtime.findAll { f ->
f.name.endsWith('.jar') &&
!(zipTree(f).matching { include findMe }.empty)
}
println "Found $findMe in ${matches*.name}"
}
}
Just ctrl+left click class name which was imported, then you can see the jar on your ide(eclipse has that feature, probably IntelliJ has as well)
Try using the task dependencyInsight :
gradle -q dependencyInsight --configuration compile --dependency commons-beanutils
Every Gradle project provides the task dependencyInsight to render the
so-called dependency insight report from the command line. Given a
dependency in the dependency graph you can identify the selection
reason and track down the origin of the dependency selection.
I'm running into a problem with an AntTask run within the maven-antrun-plugin. Unfortunately, the AntTask uses the plugin classloader to locate a file from the project, but when run from within a plugin, the build output is not included in the plugin's classpath.
From the Guide to Maven Classloading:
Please note that the plugin classloader does neither contain the
dependencies of the current project nor its build output.
...
Plugins are free to create further classloaders on their discretion.
For example, a plugin might want to create a classloader that combines
the plugin class path and the project class path.
Can anyone point me in right direction how to create my own version of the maven-antrun-plugin in which I can create my own classloader that combines the plugin class path and the project class path? I need to update the classloader such that when a class executed by my custom antrun-plugin calls:
getClass().getClassLoader().getResource()
the classloader will search the build output folder as well.
After several hours trying to work my way around this issue with configuration, I bit the bullet and simply wrote my own plugin that extends the AntRun plugin. This was done using Maven 3.2.5:
#Mojo( name = "run", threadSafe = true, requiresDependencyResolution = ResolutionScope.TEST )
public class CustomAntMojo
extends AntRunMojo
{
#Component
private PluginDescriptor pluginDescriptor;
public void execute()
throws MojoExecutionException
{
File buildDirectory = new File( getMavenProject().getBuild().getOutputDirectory() );
// add the build directory to the classpath for the classloader
try {
ClassRealm realm = pluginDescriptor.getClassRealm();
realm.addURL(buildDirectory.toURI().toURL());
} catch (MalformedURLException e1) {
e1.printStackTrace();
}
// configure the log4j logger to output the ant logs to the maven log
BasicConfigurator.configure( new MavenLoggerLog4jBridge(getLog()));
super.execute();
}
}
With the MavenLoggerLog4jBridge class being used to convert from my Ant task's Log4j output to maven logger (https://stackoverflow.com/a/6948208/827480):
import org.apache.log4j.AppenderSkeleton;
import org.apache.log4j.Level;
import org.apache.log4j.spi.LoggingEvent;
import org.apache.maven.plugin.logging.Log;
public class MavenLoggerLog4jBridge extends AppenderSkeleton {
private Log logger;
public MavenLoggerLog4jBridge(Log logger) {
this.logger = logger;
}
protected void append(LoggingEvent event) {
int level = event.getLevel().toInt();
String msg = event.getMessage().toString();
if (level <= Level.DEBUG_INT ) {
this.logger.debug(msg);
} else if (level == Level.INFO_INT) {
this.logger.info(msg);
} else if (level == Level.WARN_INT) {
this.logger.warn(msg);
} else if (level == Level.ERROR_INT || level == Level.FATAL_INT) {
this.logger.error(msg);
}
}
public void close() {
}
public boolean requiresLayout() {
return false;
}
}
Hopefully it might be of some use or assistance to someone in the future.
So I am developing one maven plugin where I need to modify the classloaders in order to work correctly. The problem is that I am not sure that I am modifying the correct classloader. What I'm doing is the following:
#Mojo(name = "aggregate", requiresDependencyResolution = ResolutionScope.TEST)
public class AcceptanceTestMojo extends AbstractMojo {
private static final String SYSTEM_CLASSLOADER_FIELD_NAME = "scl";
#Parameter
private String property;
#Component
public PluginDescriptor pluginDescriptor;
#Component
public MavenProject mavenProject;
#Override
public void execute() throws MojoExecutionException, MojoFailureException {
ClassLoader newClassLoader = null;
List<String> runtimeClassPathElements;
try {
runtimeClassPathElements = mavenProject.getTestClasspathElements();
} catch (DependencyResolutionRequiredException e) {
throw new MojoFailureException(MojoFailureMessages.UNRESOLVED_DEPENDENCIES_MESSAGE);
}
ClassRealm realm = pluginDescriptor.getClassRealm();
ClassRealm modifiedRealm= new ClassRealm( realm.getWorld(), realm.getId(), realm.getParentClassLoader());
try {
for (String element : runtimeClassPathElements) {
File elementFile = new File(element);
modifiedRealm.addURL(elementFile.toURI().toURL());
}
} catch (MalformedURLException e) {
throw new MojoFailureException(MojoFailureMessages.UNRESOLVED_CLASSES_MESSAGE);
}
pluginDescriptor.setClassRealm(modifiedRealm);
So I am getting the ClassRealm and I'am making slight changes to the UCP(removing some jars) and after that I set the newly created ClassRealm to the project descriptor. I am also changing the ContextClassLoader and the SystemClassLoader as the project I am executing my plugin on are using them for some interactions. These two are working fine- they are changed and the plugin is working fine with them. The problem is the plugin classloader. Because for some reason when executing my plugin on one project it is looking in the plugin ClassRealm and searching for the needed jars from there. But the code I put above is not fully correct, because when I come to the part where the execution of the plugin is looking in the plugin ClassRealm it is not the modified one- it gets another reference, which I don't know where it comes from. What I think is that I am not setting the ClassRealm correctly or I am missing something else.
We have a JUnit test based on JDepend 2.9.1 in order to find illegal dependencies and cycles.
Today we found that JDepend is missing dependencies. It doesn't seem to consider A depending on B in the following piece of code:
public class A {
#SomeAnotation(value = B.class)
public String someMethod() {
...
}
}
Our test looks like this:
private JDepend setupJDepend() {
JDepend jdepend = null;
try {
jdepend = new JDepend();
jdepend.addDirectory("target/classes");
jdepend.addDirectory("target/test-classes");
} catch (final IOException ioException) {
fail("An IOException occured: " + ioException.getMessage());
}
jdepend.analyzeInnerClasses(true);
return jdepend;
}
#Test
public final void testNoCyclesOnPackageLevel() {
final JDepend jdepend = setupJDepend();
final Collection<?> packages = analyzeDependencies();
assertTrue(packages.size() > 0);
assertFalse("The code contains dependency cycles on package level!",
jdepend.containsCycles());
if (ignorePackageCycle) {
return;
}
java.util.List<String> packagesWithCycle = new ArrayList<String>();
for (Object pObject : packages) {
JavaPackage javaPackage = (JavaPackage) pObject;
if (javaPackage.containsCycle()) {
packagesWithCycle.add(javaPackage.getName());
}
}
assertTrue(packagesWithCycle.toString(), packagesWithCycle.isEmpty());
}
The JDepend4Eclipse plugin sees the dependency and reports the resulting cycle.
Is this a Bug? Is there a Workaround? Are we doing something wrong?
On a related note: jdepend.containsCycles() always returns false.
This is a missing feature!
For a #Retention(SOURCE) declared #SomeAnotation, no tool can find B in the compiled class file.
For other policies, the annotations is included in the class file but the ClassFileParser of JDepend does not support annotations of any kind, this is a missing feature so far.
EDIT: The source code checked in, support java 5 annotations and is used in the eclipse plugin. Maybe the 2.9.1 is not based on this. Not found the release notes of JDepend and its release date. The Code has been checked in 2010-05-19. The "latest" release seems to be from 2008.