I created a base image for a Java application using Jib which I want to extend using Jib.
(The Java application provides extensibility by loading additional Jars from the classpath)
In the extending gradle project, I did this:
jib {
....
container {
entrypoint = 'INHERIT'
}
...
}
It allowed me to reuse the entrypoint and args attributes added in the base image but I also want to extend/reuse the base classpath file.
As Jib creates /app/jib-classpath-file in the extending gradle project, the base layer /app/jib-classpath-file is not visible ( I would assume).
To workaround the issue, I added this in extending container configuration block.
extraClasspath = ['/app/libs/*']
Is there an idiomatic way of achieving this in Jib? One option I was thinking is to specify unique classpath files in base and extending projects and
use them like this in the Java command line:
java -cp #/app/jib-BASE-classpath-file #/app/jib-EXTENDED-classpath-file, but I am not finding the option of specifying the classpath file.
What is the recommended way? Thanks
Not a total solution but this shows how to remove the jvm args layer from the child image build altogether.
//build.gradle.kts
buildscript {
dependencies {
classpath("com.google.cloud.tools:jib-layer-filter-extension-gradle:0.1.0")
}
}
...
jib {
...
pluginExtensions {
pluginExtension {
implementation = "com.google.cloud.tools.jib.gradle.extension.layerfilter.JibLayerFilterExtension"
configuration (Action<com.google.cloud.tools.jib.gradle.extension.layerfilter.Configuration> {
filters { filter { glob ="**/jib-*-file" } }
})
}
}
}
This filters out files added in any given layer of a given jib build. If all the files from a layer are filtered out as in this case then the layer is never added to the image.
Sadly this does not let you see the contents of the java-classpath-file in the base image, which you would need to be able to extend it.
References:
https://github.com/GoogleContainerTools/jib/tree/master/jib-gradle-plugin#jib-extensions
https://github.com/GoogleContainerTools/jib-extensions/tree/master/first-party/jib-layer-filter-extension-gradle
https://github.com/GoogleContainerTools/jib-extensions
Related
i'm currently setting up a new project using java, gradle and spring boot. I'm struggling to create my planned test setup. My goal is to have separate test commands and directories located IN the test package.
What I have
I have looked into gradle sourceSets and was able to create two new sourceSets for unit and component tests. The problem I face is that I don't know if there is any way to defined the path of the sourcesets so they are inside the test directory.
src
-- main
-- test
-- unit
-- component
What I want
I just want the default test sourceSet to behave like a normal directory which itself contains my two testing sourceSets like this:
src
-- main
-- test
---- unit
---- component
Is this possible and also would this be 'against the convention' or something like that?
Any pointers are appreciated!
Basically, you should be able to set paths to your source sets any way you want to. Something like this should work:
sourceSets {
test {
java {
srcDirs = ['test/unit']
}
resources {
srcDirs = ['test/unit']
}
}
testComponent {
java {
srcDirs = ['test/component']
}
resources {
srcDirs = ['test/component']
}
}
}
Unfortunately I'm not sure if you can change the names of default source sets.
Keep in mind that you will have to provide more configuration for your new source set, e.g. create new Test task:
task testComponent(type: Test) {
testClassesDirs = sourceSets.testComponent.output
classpath += sourceSets.main.output
}
And also manage its dependencies separately or extend them from already existing configuration:
configurations {
testComponentCompile.extendsFrom(testCompile)
testComponentRuntime.extendsFrom(testRuntime)
}
As for the convention: it's best to stick to provided defaults, but if you have the need for different structure then it's also fine. Most common case for test separation is probably when you want split unit and integration test in order to be able to run them individually.
first: I'm really new to spring-boot and maven. So I still don't get how everything plugs together.
What I'm trying to achieve is some kind of plugin-feature for my application. From my research it seems the best way to do this is using ServiceLoader or the spring-boot implmentation of the SpringFactoriesLoader.
According to several instructions from the web I put two projects together
James (the main application) GitHub
TemperatureSensor (the plugin) GitHub
The JamesApplication provides an interfaces which is supposed to be implemented (de.maxrakete.james.device.domain.DeviceInterface).
The TemperatureSensor implements said class and exposes this in several ways.
For the ServiceLoader in in the file META-INF\services\de.maxrakete.james.device.domain.DeviceInterface with this content
de.maxrakete.james.plugin.TemperatureSensor.TemperatureSensor
For the SpringFactoriesLoader in the file META-INF\spring.factories with this content
de.maxrakete.james.device.domain.DeviceInterface=de.maxrakete.james.plugin.TemperatureSensor.TemperatureSensor
According to this page I tried two different implementations (see in the onApplicationEvent-function) in the MainApplication:
#SpringBootApplication
public class JamesApplication implements ApplicationListener<ApplicationReadyEvent> {
public static void main(String[] args) {
SpringApplication.run(JamesApplication.class, args);
ClassLoader cl = ClassLoader.getSystemClassLoader();
URL[] urls = ((URLClassLoader)cl).getURLs();
for(URL url: urls){
System.out.println("Classpath file: " + url.getFile());
}
}
#Override
public void onApplicationEvent(ApplicationReadyEvent event) {
ServiceLoader<DeviceInterface> loader = ServiceLoader.load(DeviceInterface.class);
loader.iterator();
List<DeviceInterface> foos = SpringFactoriesLoader.loadFactories(DeviceInterface.class, null);
}
}
I'm trying both ways to load the jar, but nothing is happening (I'm supposed to get some log-messages from the plugin) but this is not happening.
The way I'm running the application is like this:
java -cp "./plugins/TemperatureSensor-0.0.1-SNAPSHOT.jar" -jar james.war
As you see I'm trying to add the jar in the subfolder to the classpath, but in the ouput of the main-function (where I try to print all the files in the classpath) I only get Classpath file: /home/max/folder/james.war
Conclusion
So, there are three possible error-sources
Wrong cli command to add classpath files
Wrong declaration of interfaces in the META-INF folder
Wrong implementation of the Loader
Maybe I'm compiling the sources the wrong way?
Wrong configuration of the pom.xml
I really have no idea what the problem might be. I tried to provide you with as much information as possible and all the steps of my research. I hope someone finds some helpful clues, which I might have overlooked.
Thanks veryone!
I want to write custom Lombok Annotation handlers. I know http://notatube.blogspot.de/2010/12/project-lombok-creating-custom.html. But the current lombok jar file does not contain many .class files, but files named .SCL.lombok instead.
I found, the .SCL.lombok files are the .class files, the build script of Lombok does rename them while generating the jar file, and the ShadowClassLoader is capable of loading these classes -- and the acronym SCL seems to come from this. It seems the reason for this is just to "Avoid contaminating the namespace of any project using an SCL-based jar. Autocompleters in IDEs will NOT suggest anything other than actual public API."
I was only able to compile my custom handler by
unpacking the contents of the lombok.jar
renaming the .SCL.lombok files to .class
adding the resulting directory to the compile classpath
In addition, to be able to use my custom handler, I needed to create a new fat jar containing both the lombok classes and my custom handler. The custom lombok class loader essentially prevents adding custom handlers in other multiple jars.
Is this the only way to extend Lombok? Or am I missing something?
I am using the following buildscript
apply plugin: 'java'
repositories {
jcenter()
}
configurations {
lombok
compileOnly
}
def unpackedAndRenamedLombokDir = file("$buildDir/lombok")
task unpackAndRenameLombok {
inputs.files configurations.lombok
outputs.dir unpackedAndRenamedLombokDir
doFirst {
mkdir unpackedAndRenamedLombokDir
delete unpackedAndRenamedLombokDir.listFiles()
}
doLast {
copy {
from zipTree(configurations.lombok.singleFile)
into unpackedAndRenamedLombokDir
rename "(.*)[.]SCL[.]lombok", '$1.class'
}
}
}
sourceSets {
main {
compileClasspath += configurations.compileOnly
output.dir(unpackedAndRenamedLombokDir, builtBy: unpackAndRenameLombok)
}
}
tasks.compileJava {
dependsOn unpackAndRenameLombok
}
dependencies {
compile files("${System.properties['java.home']}/../lib/tools.jar")
compile "org.eclipse.jdt:org.eclipse.jdt.core:3.10.0"
compile 'javax.inject:javax.inject:1'
lombok 'org.projectlombok:lombok:1.16.6'
compileOnly files(unpackedAndRenamedLombokDir)
}
In the meantime Reinier Zwitserloot created a new git-branch sclExpansionUpdate, that contains an updated version of the ShadowClassLoader:
ShadowClassLoader is now friendlier to trying to extend lombok.
Your (separate) jar/dir should have a file named
META-INF/ShadowClassLoader. This file should contain the string
'lombok'. If you have that, any classes in that jar/dir will be loaded
in the same space as lombok classes. You can also rename the class
files to .SCL.lombok to avoid other loaders from finding them.
I guess this did not yet make it into the main branch because it certainly has not been tested that much - I just tried it out for myself and it contains a little bug that prevents loading the required META-INF/services from extensions. To fix it you should replace two method calls to partOfShadow with inOwnBase:
[... line 443]
Enumeration<URL> sec = super.getResources(name);
while (sec.hasMoreElements()) {
URL item = sec.nextElement();
if (!inOwnBase(item, name)) vector.add(item); // <<-- HERE
}
if (altName != null) {
Enumeration<URL> tern = super.getResources(altName);
while (tern.hasMoreElements()) {
URL item = tern.nextElement();
if (!inOwnBase(item, altName)) vector.add(item); // <<-- AND HERE
}
}
I tested it with the above fix and it seems to work fine (not tested much though).
On a side note: with this new extension mechanism, it is now finally also possible to have the extensions annotation handlers and annotations in a different namespace than "lombok" - nice!
Using the input from this question and from the other answer (by Balder), we managed to put together a custom Lombok annotation handler: Symbok. Feel free to use that as a sample for writing your own.
BTW, instead of writing a custom Lombok handler, you could also implement a javac plugin instead -- it might be simpler.
I need to have a jar file located in a main/assets directory within an Android project. It is important the jar file is located there.
With my main Android project is there a way to reference this jar file in my code and to use its classes?
To be clear I don't want to add the jar to the main project once compiled.
EDIT: I have tried the link below and it seems to load the Class file I've stated. But I'm strugging how to define constructor arguments for the dynamically loaded Class.
android-custom-class-loading-sample
EDIT2
Nearly there. I've confirmed the class is loaded from my classes.jar. I'm stuck instantiating it though.
On the licenseValidatorClazz.getConstructor line I get the error below. I'm guessing I'm missing something from my Interface file?
java.lang.NoSuchMethodException: [interface com.google.android.vending.licensing.Policy, interface com.google.android.vending.licensing.DeviceLimiter, interface com.google.android.vending.licensing.LicenseCheckerCallback, int, class java.lang.String, class java.lang.String]
public Class licenseValidatorClazz = null;
public LicenseValidator validator;
...
// Initialize the class loader with the secondary dex file.
DexClassLoader cl = new DexClassLoader(dexInternalStoragePath.getAbsolutePath(),
optimizedDexOutputPath.getAbsolutePath(),
null,
mContext.getClassLoader());
try {
// Load the library class from the class loader.
licenseValidatorClazz = cl.loadClass("com.google.android.vending.licensing.LicenseValidator");
validator = (LicenseValidator) licenseValidatorClazz.getConstructor(Policy.class,DeviceLimiter.class,LicenseCheckerCallback.class,int.class,String.class,String.class).newInstance(ddd, new NullDeviceLimiter(),
callback, generateNonce(), mPackageName, mVersionCode);
} catch (Exception exception) {
// Handle exception gracefully here.
exception.printStackTrace();
}
I have an Interface which contains the functions to pass to the loaded class.
public interface LicenseValidator
{
public LicenseCheckerCallback getCallback();
public int getNonce();
public String getPackageName();
public void verify(PublicKey publicKey, int responseCode, String signedData, String signature);
public void handleResponse(int response, ResponseData rawData);
public void handleApplicationError(int code);
public void handleInvalidResponse();
}
TO use an external jar to be associated with your application and use it during runtime, it needs to be in dalvik format since normal jars cannot work under dalvikVM.
Convert your files using the dx tool
using aapt cmd , add those classes.dex to your jar file.
Now this jar which contains files in dalvik format can be loaded into our project.
Here is a post which explains the procedure to accomplish it.
There are steps to accomplish this.
You have to make a copy of your JAR file into the private internal storage of your aplication.
Using the dx tool inside the android folder, you have to generate a classes.dex file associated with the JAR file. The dx tool will be at the location /android-sdks/build-tools/19.0.1 (this file is needed by the Dalvik VM, simply jar can not be read by the dalvik VM))
Using the aapt tool command which is also inside the same location, you have to add the classes.dex to the JAR file.
This JAR file could be loaded dynamically using DexClassLoader.
If you are making a JAR from any one your own library, you have to do this steps (1-4) every time when there is a change in your library source code. So you can automate this steps by creating a shell script(in Mac/Linux/Ubuntu) or batch scripts(in Windows). You can refere this link to understand how to write shell scripts.
Note : One situation for implementing this method is, when it is impossible to add the JAR files directly to the build path of core project and need to be loaded dynamically at run time. In normal cases the JAR files could be added to the build path.
please check this link for the detailed code and implementation.
How to load a jar file at runtime
Android: How to dynamically load classes from a JAR file?
Hope this helps!!
You should try out the Services API - java.util.ServiceLoader
You define a service interface and its implementations in your jar.
package com.my.project;
public interface MyService { ... }
public class MyServiceBarImpl implements MyService { ... }
public class MyServiceFooImpl implements MyService { ... }
Then you define the services contained within the jar file in the META-INF/services/ directory. For instance, in the file 'META-INF/services/com.my.project.MyService', you list the provider classes.
# Known MyService providers.
com.my.project.MyServiceBarImpl # The original implementation for handling "bar"s.
com.my.project.MyServiceFooImpl # A later implementation for "foo"s.
Then, in your main codebase, you can instantiate a MyService instance with the ServiceLoader:
for (MyService service : ServiceLoader.load(MyService.class)) {
//Perform some test to determine which is the right MyServiceImpl
//and then do something with the MyService instance
}
These examples are taken more-or-less straight from the API, although I've changed the package names to make them slightly less annoying to read.
Trying to load spring beans from custom groovy file in Grails 2.3.7. I know this question has been asked before, but after hours of searching, I'm unable to find a consistent approach that loads from the classpath.
The Goal
Modularize resources.groovy into multiple custom resource files
Put custom resource files in standard location: grails-app/conf/spring
Use plugin to do the magic; minimize overhead
Tried...
//## grails-app/conf/spring/MyBeansConfig.groovy
beans {
testsvc(TestService){
msg = 'hello'
}
}
Note above, I'm using beans {}, not beans = {}, apparently it makes a difference:
resources.groovy
This works...
beans = {
loadBeans 'file:C:\\Proj\Test1\grails-app\\conf\\spring\\MyBeansConfig.groovy'
}
...and according to docs, this should too, but doesn't:
importBeans("classpath:*MyBeansConfig.groovy")
UPDATE - WORKING OPTIONS
(working for Grails 2.3.7)
Option 1: (src/java)
Following lukelazarovic advice, this approach works since Grails automatically copies (not compiles) groovy files in src/java to the classpath; works fine in eclipse and with war deployment:
//bean config files here
src/java/MyBeansConfig.groovy
src/java/FooBeansConfig.groovy
//resources.groovy
loadBeans('classpath*:*BeansConfig.groovy')
Options 2: (grails-app/conf/spring)
This approach allows for custom bean config files in grails-app/conf/spring (credits to ideascultor and mark.esher)
//bean config files here
grails-app/conf/spring/MyBeansConfig.groovy
//## resources.groovy
//for eclipse/local testing
loadBeans('file:./grails-app/conf/spring/*BeansConfig.groovy')
//for WAR/classpath
loadBeans('classpath*:*BeansConfig.groovy')
//## BuildConfig.groovy
grails.war.resources = { stagingDir, args ->
copy(todir: "${stagingDir}/WEB-INF/classes/spring") {
fileset(dir:"grails-app/conf/spring") {
include(name: "*BeansConfig.groovy")
exclude(name: "resources.groovy")
exclude(name: "resources.xml")
}
}
}
Options 3: (Custom Plugin)
If you're using custom plugins, this approach is ideal; boiler plate config gets refactored into the plugin:
Plugin Config
//## scripts/_Events.groovy
eventCreateWarStart = { warName, stagingDir ->
def libDir = new File("${stagingDir}/WEB-INF/classes/spring")
ant.copy(todir: libDir) {
fileset(dir:"grails-app/conf/spring") {
include(name: "*BeansConfig.groovy")
exclude(name: "resources.groovy")
exclude(name: "resources.xml")
}
}
}
//## MyGrailsPlugin.groovy
def doWithSpring = {
loadBeans('file:./grails-app/conf/spring/*BeansConfig.groovy')
loadBeans('classpath*:*BeansConfig.groovy')
}
Grails App
No config!...just create your bean config files using the *BeansConfig.groovy convention, good to go!
//bean config files here
grails-app/conf/spring/MyBeansConfig.groovy
Update 9/24/2015
Found some issues with option 3:
loading of duplicate resource files
spring resources not configured correctly for test-app
We managed to fix the above issue such that any resource files in grails-app/conf/spring work the same when executing Grails in eclipse, WAR, test-app, etc.
First step: we created a class to have more control over locating and loading resource files; this was necessary as some files were being loaded more than once due to cross-referenced beans.
We're using a plugin to encapsulate this functionality, so this class lives in that plugin:
class CustomResourceLoader {
static loadSpringBeans(BeanBuilder bb){
def files = ['*'] //load all resource files
//match resources using multiple methods
def matchedResourceList = []
files.each {
matchedResourceList +=
getPatternResolvedResources("file:./grails-app/conf/spring/" + it + ".groovy").toList()
matchedResourceList +=
getPathMatchedResources("classpath*:spring/" + it + ".groovy").toList()
}
//eliminate duplicates resource loaded from recursive reference
def resourceMap = [:]
matchedResourceList.each{
if(it) resourceMap.put(it.getFilename(), it)
}
//make resources.groovy first in list
def resourcesFile = resourceMap.remove("resources.groovy")
if(!resourcesFile)
throw new RuntimeException("resources.groovy was not found where expected!")
def resourcesToLoad = [resourcesFile]
resourceMap.each{k,v -> resourcesToLoad << v }
log.debug("Spring resource files about to load: ")
resourcesToLoad.each{ bb.loadBeans(it) }
}
static def getPatternResolvedResources(path){
ResourcePatternResolver resourcePatternResolver =
new PathMatchingResourcePatternResolver();
return resourcePatternResolver.getResources(path);
}
static def getPathMatchedResources(path){
return new PathMatchingResourcePatternResolver().getResources(path)
}
}
Removed BeansConfig.groovy suffix; WAR generation now picks up any resources in grails-app/conf/spring
plugin, _Events.groovy
eventCreateWarStart = { warName, stagingDir ->
def libDir = new File("${stagingDir}/WEB-INF/classes/spring")
ant.copy(todir: libDir) {
fileset(dir:"grails-app/conf/spring") {
include(name: "*.groovy")
exclude(name: "resources.xml")
}
}
}
}
In the plugin definition file, call the loader from doWithSpring(), passing BeanBuilder (the delegate) as the argument (very important):
def doWithSpring = {
CustomResourceLoader.loadSpringBeans(delegate)
}
That's it, there is no requirement on users of the plugin aside from creating custom resource files in grails-app/conf/spring
I had a similar problem just a few days ago, with a groovy configuration file that I added into grails-app/conf. While this works with other resources (they are copied and available on the classpath), the problem with the groovy file was simply that it was compiled and the class file was included, i.e. not the groovy file itself.
I didn't find any good documentation on how this should be done and finally just added it to web-app/WEB-INF/classes. Groovy files placed here will be copied (not compiled) and available on the classpath.
I had the same problem with custom XML files in Grails 2.1.2.
Having XML resources in grails-app/conf/spring didn't work in production environment AFAIR.
To make it working both in development and production environments I finally put the resources into src/java. I think you can achieve the same result by putting your groovy files into src/groovy.
We can import beans from different groovy/xml file in the following way too : -
use the following in resources.groovy -
importBeans 'file:camel-beans.groovy'
OR
importBeans('classpath:/camel-config.xml')
Place camel-beans.groovy along with resources.groovy and provide package as "package spring" for first case, otherwise put it in web app classpath and use the second way to do it.
If your resources.groovy is at following path
grails-app/conf/spring/resources.groovy
and your camel-beans.groovy is at following path
grails-app/conf/spring/camel-beans.groovy
then you can reference camel-beans.groovy in resources.groovy file by adding following line in resources.groovy
importBeans('file:**/camel-beans.groovy')