Develop Reference

SpringLiquibase with Liquibase 4 and FileSystemResourceLoader

I'm currently facing a problem.
My project looks like this :
Project
|_ module 1
   |_ liquibase
      |_ migration.xml
      |_ file1.xml
   |_ src
      |_ main
         |_ java
         |_ resources
To be able to launch component tests, I run, using docker, a postgresql container.
I want to launch my liquibase scripts.
Here's a my code :
SpringLiquibase liquibase = new SpringLiquibase();
liquibase.setResourceLoader(new FileSystemResourceLoader());
liquibase.setDataSource(dataSource);
liquibase.setChangeLog("liquibase/migration.xml");
liquibase.setDefaultSchema("mySchema");
liquibase.setDropFirst(false);
liquibase.setShouldRun(true);
try {
liquibase.afterPropertiesSet();
log.info("Liquibase run ended");
} catch (Exception e) {
log.error(e.getMessage());
throw new RuntimeException(e.getMessage());
}
This has run well for a long time, until I made an update to Liquibase 4.
Now, I'm getting the following error : Specifying files by absolute path was removed in Liquibase 4.0. Please use a relative path or add '/' to the classpath parameter.
I searched throught the web and didn't find anything helpful.
I tried a lot of different things, and nothing worked
Someone has a clue ? (other than moving my liquibase folder inside resources)

I worked it out implementing custom SpringLiquibase and SpringResourceAcessor and moving from liquibase 4.0 to 4.6.1
If anyone is interested, here's my code :
public class CustomSpringResourceAcessor extends SpringResourceAccessor {
public CustomSpringResourceAcessor(ResourceLoader resourceLoader) {
super(resourceLoader);
}
#Override
protected String finalizeSearchPath(String searchPath) {
return super.finalizeSearchPath(searchPath).substring(11);
}
#Override
public InputStreamList openStreams(String relativeTo, String streamPath) throws IOException {
String path = this.getClass().getProtectionDomain().getCodeSource().getLocation().getPath();
path = path.substring(0, path.indexOf("/target"));
if (relativeTo == null) {
return super.openStreams(path, streamPath);
}
return super.openStreams(path + "/" + relativeTo, streamPath);
}
}
and
public class CustomSpringLiquibase extends SpringLiquibase {
#Override
protected SpringResourceAccessor createResourceOpener() {
return new CustomSpringResourceAcessor(getResourceLoader());
}
}

Can we automatically refresh spring properties file without using actuator refresh endpoint [duplicate]

Many in-house solutions come to mind. Like having the properties in a database and poll it every N secs. Then also check the timestamp modification for a .properties file and reload it.
But I was looking in Java EE standards and spring boot docs and I can't seem to find some best way of doing it.
I need my application to read a properties file(or env. variables or DB parameters), then be able to re-read them. What is the best practice being used in production?
A correct answer will at least solve one scenario (Spring Boot or Java EE) and provide a conceptual clue on how to make it work on the other

After further research, reloading properties must be carefully considered. In Spring, for example, we can reload the 'current' values of properties without much problem. But. Special care must be taken when resources were initialized at the context initialization time based on the values that were present in the application.properties file (e.g. Datasources, connection pools, queues, etc.).
NOTE:
The abstract classes used for Spring and Java EE are not the best example of clean code. But it is easy to use and it does address this basic initial requirements:
No usage of external libraries other than Java 8 Classes.
Only one file to solve the problem (~160 lines for the Java EE version).
Usage of standard Java Properties UTF-8 encoded file available in the File System.
Support encrypted properties.
For Spring Boot
This code helps with hot-reloading application.properties file without the usage of a Spring Cloud Config server (which may be overkill for some use cases)
This abstract class you may just copy & paste (SO goodies :D ) It's a code derived from this SO answer
// imports from java/spring/lombok
public abstract class ReloadableProperties {
#Autowired
protected StandardEnvironment environment;
private long lastModTime = 0L;
private Path configPath = null;
private PropertySource<?> appConfigPropertySource = null;
#PostConstruct
private void stopIfProblemsCreatingContext() {
System.out.println("reloading");
MutablePropertySources propertySources = environment.getPropertySources();
Optional<PropertySource<?>> appConfigPsOp =
StreamSupport.stream(propertySources.spliterator(), false)
.filter(ps -> ps.getName().matches("^.*applicationConfig.*file:.*$"))
.findFirst();
if (!appConfigPsOp.isPresent()) {
// this will stop context initialization
// (i.e. kill the spring boot program before it initializes)
throw new RuntimeException("Unable to find property Source as file");
}
appConfigPropertySource = appConfigPsOp.get();
String filename = appConfigPropertySource.getName();
filename = filename
.replace("applicationConfig: [file:", "")
.replaceAll("\\]$", "");
configPath = Paths.get(filename);
}
#Scheduled(fixedRate=2000)
private void reload() throws IOException {
System.out.println("reloading...");
long currentModTs = Files.getLastModifiedTime(configPath).toMillis();
if (currentModTs > lastModTime) {
lastModTime = currentModTs;
Properties properties = new Properties();
#Cleanup InputStream inputStream = Files.newInputStream(configPath);
properties.load(inputStream);
environment.getPropertySources()
.replace(
appConfigPropertySource.getName(),
new PropertiesPropertySource(
appConfigPropertySource.getName(),
properties
)
);
System.out.println("Reloaded.");
propertiesReloaded();
}
}
protected abstract void propertiesReloaded();
}
Then you make a bean class that allows retrieval of property values from applicatoin.properties that uses the abstract class
#Component
public class AppProperties extends ReloadableProperties {
public String dynamicProperty() {
return environment.getProperty("dynamic.prop");
}
public String anotherDynamicProperty() {
return environment.getProperty("another.dynamic.prop");
}
#Override
protected void propertiesReloaded() {
// do something after a change in property values was done
}
}
Make sure to add #EnableScheduling to your #SpringBootApplication
#SpringBootApplication
#EnableScheduling
public class MainApp {
public static void main(String[] args) {
SpringApplication.run(MainApp.class, args);
}
}
Now you can auto-wire the AppProperties Bean wherever you need it. Just make sure to always call the methods in it instead of saving it's value in a variable. And make sure to re-configure any resource or bean that was initialized with potentially different property values.
For now, I have only tested this with an external-and-default-found ./config/application.properties file.
For Java EE
I made a common Java SE abstract class to do the job.
You may copy & paste this:
// imports from java.* and javax.crypto.*
public abstract class ReloadableProperties {
private volatile Properties properties = null;
private volatile String propertiesPassword = null;
private volatile long lastModTimeOfFile = 0L;
private volatile long lastTimeChecked = 0L;
private volatile Path propertyFileAddress;
abstract protected void propertiesUpdated();
public class DynProp {
private final String propertyName;
public DynProp(String propertyName) {
this.propertyName = propertyName;
}
public String val() {
try {
return ReloadableProperties.this.getString(propertyName);
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException(e);
}
}
}
protected void init(Path path) {
this.propertyFileAddress = path;
initOrReloadIfNeeded();
}
private synchronized void initOrReloadIfNeeded() {
boolean firstTime = lastModTimeOfFile == 0L;
long currentTs = System.currentTimeMillis();
if ((lastTimeChecked + 3000) > currentTs)
return;
try {
File fa = propertyFileAddress.toFile();
long currModTime = fa.lastModified();
if (currModTime > lastModTimeOfFile) {
lastModTimeOfFile = currModTime;
InputStreamReader isr = new InputStreamReader(new FileInputStream(fa), StandardCharsets.UTF_8);
Properties prop = new Properties();
prop.load(isr);
properties = prop;
isr.close();
File passwordFiles = new File(fa.getAbsolutePath() + ".key");
if (passwordFiles.exists()) {
byte[] bytes = Files.readAllBytes(passwordFiles.toPath());
propertiesPassword = new String(bytes,StandardCharsets.US_ASCII);
propertiesPassword = propertiesPassword.trim();
propertiesPassword = propertiesPassword.replaceAll("(\\r|\\n)", "");
}
}
updateProperties();
if (!firstTime)
propertiesUpdated();
} catch (Exception e) {
e.printStackTrace();
}
}
private void updateProperties() {
List<DynProp> dynProps = Arrays.asList(this.getClass().getDeclaredFields())
.stream()
.filter(f -> f.getType().isAssignableFrom(DynProp.class))
.map(f-> fromField(f))
.collect(Collectors.toList());
for (DynProp dp :dynProps) {
if (!properties.containsKey(dp.propertyName)) {
System.out.println("propertyName: "+ dp.propertyName + " does not exist in property file");
}
}
for (Object key : properties.keySet()) {
if (!dynProps.stream().anyMatch(dp->dp.propertyName.equals(key.toString()))) {
System.out.println("property in file is not used in application: "+ key);
}
}
}
private DynProp fromField(Field f) {
try {
return (DynProp) f.get(this);
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return null;
}
protected String getString(String param) throws Exception {
initOrReloadIfNeeded();
String value = properties.getProperty(param);
if (value.startsWith("ENC(")) {
String cipheredText = value
.replace("ENC(", "")
.replaceAll("\\)$", "");
value = decrypt(cipheredText, propertiesPassword);
}
return value;
}
public static String encrypt(String plainText, String key)
throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException, InvalidKeySpecException {
SecureRandom secureRandom = new SecureRandom();
byte[] keyBytes = key.getBytes(StandardCharsets.US_ASCII);
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(key.toCharArray(), new byte[]{0,1,2,3,4,5,6,7}, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secretKey = new SecretKeySpec(tmp.getEncoded(), "AES");
byte[] iv = new byte[12];
secureRandom.nextBytes(iv);
final Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
GCMParameterSpec parameterSpec = new GCMParameterSpec(128, iv); //128 bit auth tag length
cipher.init(Cipher.ENCRYPT_MODE, secretKey, parameterSpec);
byte[] cipherText = cipher.doFinal(plainText.getBytes(StandardCharsets.UTF_8));
ByteBuffer byteBuffer = ByteBuffer.allocate(4 + iv.length + cipherText.length);
byteBuffer.putInt(iv.length);
byteBuffer.put(iv);
byteBuffer.put(cipherText);
byte[] cipherMessage = byteBuffer.array();
String cyphertext = Base64.getEncoder().encodeToString(cipherMessage);
return cyphertext;
}
public static String decrypt(String cypherText, String key)
throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException, InvalidKeySpecException {
byte[] cipherMessage = Base64.getDecoder().decode(cypherText);
ByteBuffer byteBuffer = ByteBuffer.wrap(cipherMessage);
int ivLength = byteBuffer.getInt();
if(ivLength < 12 || ivLength >= 16) { // check input parameter
throw new IllegalArgumentException("invalid iv length");
}
byte[] iv = new byte[ivLength];
byteBuffer.get(iv);
byte[] cipherText = new byte[byteBuffer.remaining()];
byteBuffer.get(cipherText);
byte[] keyBytes = key.getBytes(StandardCharsets.US_ASCII);
final Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(key.toCharArray(), new byte[]{0,1,2,3,4,5,6,7}, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secretKey = new SecretKeySpec(tmp.getEncoded(), "AES");
cipher.init(Cipher.DECRYPT_MODE, secretKey, new GCMParameterSpec(128, iv));
byte[] plainText= cipher.doFinal(cipherText);
String plain = new String(plainText, StandardCharsets.UTF_8);
return plain;
}
}
Then you can use it this way:
public class AppProperties extends ReloadableProperties {
public static final AppProperties INSTANCE; static {
INSTANCE = new AppProperties();
INSTANCE.init(Paths.get("application.properties"));
}
#Override
protected void propertiesUpdated() {
// run code every time a property is updated
}
public final DynProp wsUrl = new DynProp("ws.url");
public final DynProp hiddenText = new DynProp("hidden.text");
}
In case you want to use encoded properties you may enclose it's value inside ENC() and a password for decryption will be searched for in the same path and name of the property file with an added .key extension. In this example it will look for the password in the application.properties.key file.
application.properties ->
ws.url=http://some webside
hidden.text=ENC(AAAADCzaasd9g61MI4l5sbCXrFNaQfQrgkxygNmFa3UuB9Y+YzRuBGYj+A==)
aplication.properties.key ->
password aca
For the encryption of property values for the Java EE solution I consulted Patrick Favre-Bulle excellent article on Symmetric Encryption with AES in Java and Android. Then checked the Cipher, block mode and padding in this SO question about AES/GCM/NoPadding. And finally I made the AES bits be derived from a password from #erickson excellent answer in SO about AES Password Based Encryption. Regarding encryption of value properties in Spring I think they are integrated with Java Simplified Encryption
Wether this qualify as a best practice or not may be out of scope. This answer shows how to have reloadable properties in Spring Boot and Java EE.

This functionality can be achieved by using a Spring Cloud Config Server and a refresh scope client.
Server
Server (Spring Boot app) serves the configuration stored, for example, in a Git repository:
#SpringBootApplication
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
application.yml:
spring:
cloud:
config:
server:
git:
uri: git-repository-url-which-stores-configuration.git
configuration file configuration-client.properties (in a Git repository):
configuration.value=Old
Client
Client (Spring Boot app) reads configuration from the configuration server by using #RefreshScope annotation:
#Component
#RefreshScope
public class Foo {
#Value("${configuration.value}")
private String value;
....
}
bootstrap.yml:
spring:
application:
name: configuration-client
cloud:
config:
uri: configuration-server-url
When there is a configuration change in the Git repository:
configuration.value=New
reload the configuration variable by sending a POST request to the /refresh endpoint:
$ curl -X POST http://client-url/actuator/refresh
Now you have the new value New.
Additionally Foo class can serve the value to the rest of application via RESTful API if its changed to RestController and has a corresponding endpont.

I used #David Hofmann concept and made some changes because of not all was good.
First of all, in my case I no need auto-reload, I just call the REST controller for updating properties.
The second case #David Hofmann's approach not workable for me with outside files.
Now, this code can work with application.properties file from resources(inside the app) and from an outside place. The outside file I put near jar, and I use this --spring.config.location=app.properties argument when the application starts.
#Component
public class PropertyReloader {
private final Logger logger = LoggerFactory.getLogger(getClass());
#Autowired
private StandardEnvironment environment;
private long lastModTime = 0L;
private PropertySource<?> appConfigPropertySource = null;
private Path configPath;
private static final String PROPERTY_NAME = "app.properties";
#PostConstruct
private void createContext() {
MutablePropertySources propertySources = environment.getPropertySources();
// first of all we check if application started with external file
String property = "applicationConfig: [file:" + PROPERTY_NAME + "]";
PropertySource<?> appConfigPsOp = propertySources.get(property);
configPath = Paths.get(PROPERTY_NAME).toAbsolutePath();
if (appConfigPsOp == null) {
// if not we check properties file from resources folder
property = "class path resource [" + PROPERTY_NAME + "]";
configPath = Paths.get("src/main/resources/" + PROPERTY_NAME).toAbsolutePath();
}
appConfigPsOp = propertySources.get(property);
appConfigPropertySource = appConfigPsOp;
}
// this method I call into REST cintroller for reloading all properties after change
// app.properties file
public void reload() {
try {
long currentModTs = Files.getLastModifiedTime(configPath).toMillis();
if (currentModTs > lastModTime) {
lastModTime = currentModTs;
Properties properties = new Properties();
#Cleanup InputStream inputStream = Files.newInputStream(configPath);
properties.load(inputStream);
String property = appConfigPropertySource.getName();
PropertiesPropertySource updatedProperty = new PropertiesPropertySource(property, properties);
environment.getPropertySources().replace(property, updatedProperty);
logger.info("Configs {} were reloaded", property);
}
} catch (Exception e) {
logger.error("Can't reload config file " + e);
}
}
}
I hope that my approach will help somebody

As mentioned by #Boris, Spring Cloud Config is the way to go to avoid patchy solution. To keep the setup minimum, I will suggest the Embedding the Config Server Approach with native type (file type).
To support automatic config refresh without calling the actuator endpoint manually, I have created a directory listener to detect file changes and to dispatch refresh scope event.
Proof Of Concept repo (git)

For spring boot, there's a really good article on this topic here, but for multiple property files it doesn't work perfectly.
In my case I had 2 property files, one non sensitive and one containing the passwords. I proceeded with the following:
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
Extend the spring's PropertySource so that you can add the reloadable version to the environment.
public class ReloadablePropertySource extends PropertySource {
private final PropertiesConfiguration propertiesConfiguration;
public ReloadablePropertySource(String name, String path, ConfigurationListener listener) {
super(StringUtils.hasText(name) ? name : path);
try {
this.propertiesConfiguration = getConfiguration(path, listener);
} catch (Exception e) {
throw new MissingRequiredPropertiesException();
}
}
#Override
public Object getProperty(String s) {
return propertiesConfiguration.getProperty(s);
}
private PropertiesConfiguration getConfiguration(String path, ConfigurationListener listener) throws ConfigurationException {
PropertiesConfiguration configuration = new PropertiesConfiguration(path);
FileChangedReloadingStrategy reloadingStrategy = new FileChangedReloadingStrategy();
reloadingStrategy.setRefreshDelay(5000);
configuration.setReloadingStrategy(reloadingStrategy);
configuration.addConfigurationListener(listener);
return configuration;
}
}
Now add all of your properties files (now reloadable) inside the spring's env
#Configuration
public class ReloadablePropertySourceConfig {
private final ConfigurableEnvironment env;
#Value("${spring.config.location}")
private String appConfigPath;
#Value("${spring.config.additional-location}")
private String vaultConfigPath;
public ReloadablePropertySourceConfig(ConfigurableEnvironment env) {
this.env = env;
}
#Bean
#ConditionalOnProperty(name = "spring.config.location")
public ReloadablePropertySource getAppConfigReloadablePropertySource(){
ReloadablePropertySource rps = new ReloadablePropertySource("dynamicNonSensitive", appConfigPath, new PropertiesChangeListener());
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(rps);
return rps;
}
#Bean
#ConditionalOnProperty(name = "spring.config.additional-location")
public ReloadablePropertySource getVaultReloadablePropertySource(){
ReloadablePropertySource rps = new ReloadablePropertySource("dynamicVault", vaultConfigPath, new PropertiesChangeListener());
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(rps);
return rps;
}
private static class PropertiesChangeListener implements ConfigurationListener{
#Override
public void configurationChanged(ConfigurationEvent event) {
if (!event.isBeforeUpdate()){
System.out.println("config refreshed!");
}
}
}
}
From the article
We've added the new property source as the first item because we want it to override any existing property with the same key
In our case, we have 2 "reloadable" property sources and both will be looked up first.
Finally create one more class from which we can access the env's properties
#Component
public class ConfigProperties {
private final Environment environment;
public ConfigProperties(Environment environment) {
this.environment = environment;
}
public String getProperty(String name){
return environment.getProperty(name);
}
}
Now you can autowire ConfigProperties and always get the latest property in the files without requiring to restart the application.
#RestController
#Slf4j
public class TestController {
#Autowired
private ConfigProperties env;
#GetMapping("/refresh")
public String test2() {
log.info("hit");
String updatedProperty = env.getProperty("test.property");
String password = env.getProperty("db.password");
return updatedProperty + "\n" + password;
}
}
where test.property is coming from 1st file and db.password is coming from another.

If you want to change the properties at realtime and don't want to restart the server then follow the below steps:
1). Application.properties
app.name= xyz
management.endpoints.web.exposure.include=*
2). Add below dependencies in pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-context</artifactId>
<version>2.1.1.RELEASE</version>
</dependency>
3).Place application.properties in /target/config folder. Create the jar in /target folder
4).add a classas below ApplcationProperties.java
#Component
#RefreshScope
#ConfigurationProperties(prefix = "app")
public class ApplicationProperties {
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
5). Write Controller.java and inject ApplcationProperties
#RestController
public class TestController {
#Autowired
private ApplicationProperties applcationProperties;
#GetMapping("/test")
public String getString() {
return applcationProperties.getName();
}
}
6).Run the spring boot application
Call localhost:XXXX/test from your browser
Output : xyz
7). Change the value in application.properties from xyz to abc
8). Using postman send a POST request to localhost:XXXX/actuator/refresh
response: ["app.name"]
9). Call localhost:XXXX/find from your browser
Output : abc

Import of Index content with ReadOnlyPDOMProvider

This articles describes how to generate and import a PDOM index.
After invoking the generation application GeneratePDOM I got a pdom file /home/sadik/eclipse-2019-06/eclipse/pdomExample.pdom. But I have problem importing the file.
The command to generate is this:
java -jar plugins/org.eclipse.equinox.launcher_1.5.400.v20190515-0925.jar -application "org.eclipse.cdt.core.GeneratePDOM" -target /home/sadik/eclipse-2019-06/eclipse/pdomExample.pdom -source /home/sadik/my-plugin-runtime-2019-06/CDTTest_Local/ -id cdttest_01 -indexer org.eclipse.cdt.core.myfastIndexer
Note the target and source arguments.
To test the import I wrote a class that implements IReadOnlyPDOMProvider
public class MyReadOnlyPDOMProvider implements IReadOnlyPDOMProvider {
public MyReadOnlyPDOMProvider() {
System.out.println("PDOMProvider");
}
#Override
public boolean providesFor(ICProject project) throws CoreException {
return true;
}
#Override
public IPDOMDescriptor[] getDescriptors(ICConfigurationDescription config) {
final IPath fileBase = Path.fromOSString("/home/sadik/eclipse-2019-06/eclipse/");
final IPath projectBase = Path.fromOSString("/home/sadik/my-plugin-runtime-2019-06/CDTTest_Local/");
return new IPDOMDescriptor[] { new IPDOMDescriptor() {
public IIndexLocationConverter getIndexLocationConverter() {
return new URIRelativeLocationConverter(URIUtil.toURI(projectBase));
}
public IPath getLocation() {
IPath path = fileBase.append("pdomExample.pdom");
return path;
}
}};
}
Are the paths correct? I actually don't know what location is supposed to be returned here.
I defined that class in the CDT extension point CIndex in my Plugin's plugin.xml:
<extension
point="org.eclipse.cdt.core.CIndex">
<ReadOnlyPDOMProvider
class="de.blub.plugin.MyReadOnlyPDOMProvider">
</ReadOnlyPDOMProvider>
</extension>
I'm testing with this file (/home/sadik/my-plugin-runtime-2019-06/CDTTest_Local/tests/indexer/usage.cc):
#include <declaration.h>
int main() {
int a = testThis();
}
When I right click testThis() and chose go to declaration, I expect to go to the function declaration in /home/sadik/my-plugin-runtime-2019-06/CDTTest_Local/tests/indexer/declaration.h. Both files are located in the same directory.
But what happens is that an editor is opened with an empty file. The editor even tells me the path: /home/soezoguz/rtt-plugin-runtime-2019-06/tests/indexer/declaration.h.
The path is missing the project name. So I guess the pdom file stores locations below the specified source directory. How can I tell the PDOMProvider to look into the correct directory for the indexed files?

For some reason the trailing "/" has been ommited by URIUtil.toURI(...). But in the description of URIRealtiveLocationConverter it says
Note: The supplied base URI must end with a forward slash
So I create an URI instance from String and append a "/" to the String.
#Override
public IPDOMDescriptor[] getDescriptors(ICConfigurationDescription config) {
final IPath fileBase = Path.fromOSString("/home/sadik/eclipse-2019-06/eclipse/");
final IPath projectBase = config.getProjectDescription().getProject().getFullPath();
return new IPDOMDescriptor[] { new IPDOMDescriptor() {
public IIndexLocationConverter getIndexLocationConverter() {
URI baseURI;
try {
baseURI = new URI(projectBase.toString()+"/");
return new URIRelativeLocationConverter(baseURI);
} catch (URISyntaxException e) {
e.printStackTrace();
}
baseURI = URIUtil.toURI(projectBase);
return new URIRelativeLocationConverter(URIUtil.toURI(projectBase));
}
public IPath getLocation() {
IPath path = fileBase.append("pdomExample.pdom");
return path;
}
}};
}

Jersey inject instance into Resource

I'm trying to create a Jersey Resource that allows me to reuse an ElasticSearch TransportClient. So I would like to use a single instance of TransportClient over all Resources that require it. So far I've got this:
Resource:
#Path("/request")
public class ConfigurationResource {
private final TransportClient transportClient;
#Inject
public ConfigurationResource(TransportClient transportClient)
{
this.transportClient = transportClient;
}
#GET
#Produces(MediaType.TEXT_PLAIN)
public String AlarmStatus(){
if(transportClient != null)
return "Not NULL! ID: ";
else
return "NULL :(";
}
}
Binding:
public class WebMainBinder extends AbstractBinder {
#Override
protected void configure() {
TransportClient transportClient = null;
try {
transportClient = TransportClient.builder().build()
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300));
} catch (UnknownHostException e) {
e.printStackTrace();
return;
}
bind(transportClient).to(TransportClient.class);
}
}
Main Application:
#ApplicationPath("service")
public class WebMain extends ResourceConfig {
public WebMain(){
register(new WebMainBinder());
packages(true, "com.eniacdevelopment.EniacHome.Resources");
}
}
Web.xml:
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
version="3.1">
<servlet>
<servlet-name>com.eniacdevelopment.EniacHome.Application.WebMain</servlet-name>
</servlet>
<servlet-mapping>
<servlet-name>com.eniacdevelopment.EniacHome.Application.WebMain</servlet-name>
<url-pattern>/service/*</url-pattern>
</servlet-mapping>
</web-app>
I've also tried using a factory like so:
public class TransportClientFactory implements Factory<TransportClient> {
private TransportClient transportClient;
#Override
public TransportClient provide() {
if(this.transportClient == null){
try {
transportClient = TransportClient.builder().build()
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300));
} catch (UnknownHostException e) {
e.printStackTrace();
return null;
}
}
return transportClient;
}
#Override
public void dispose(TransportClient transportClient) {
}
}
And then binding it this way:
bindFactory(TransportClientFactory.class)
.to(TransportClient.class).in(Singleton.class);
But no success. I keep on getting Unsatisfied dependencies for type TransportClient with qualifiers #Default.
Help would be much appreciated!

I've found Jersey's DI container functionality pretty unpleasant. I prefer to use Guice for managing my DI, so if you're open to using Guice, you can see how to wire up Jersey and Guice to collaborate in this demo project: https://bitbucket.org/marshallpierce/guice-jaxrs-examples. The common subproject has the shared logic, and there are other subprojects for the jersey and resteasy specific parts.

Alright got it to work:
When I at first tried to deploy the app to glassfish it complained about some guava dependency. I swapped the Guava jar in glassfish/modules with the one maven had installed and got it to deploy. It may have something to do with that. No guarentees here.
I decided to drop the whole glassfish stuff and start from scratch. On the jersey introduction page there's this maven archetype that can be isntalled like so:
mvn archetype:generate -DarchetypeArtifactId=jersey-quickstart-grizzly2
-DarchetypeGroupId=org.glassfish.jersey.archetypes -DinteractiveMode=false -DgroupId=com.example -DartifactId=simple-service
-Dpackage=com.example
-DarchetypeVersion=2.24
Starting from there helped me out.

Container-level Versioned Libraries Shared by WARs

In a Java servlet container (preferably Tomcat, but if this can be done in a different container then say so) I desire something which is theoretically possible. My question here is whether tools exist to support it, and if so what tools (or what names I should research further).
Here is my problem: in one servlet container I want to run a large number of different WAR files. They share some large common libraries (such as Spring). At first blush, I have two unacceptable alternatives:
Include the large library (Spring, for example) in each WAR file. This is unacceptable because it will load a large number of copies of Spring, exhausting the memory on the server.
Place the large library in the container classpath. Now all of the WAR files share one instance of the library (good). But this is unacceptable because I cannot upgrade the Spring version without upgrading ALL of the WAR files at once, and such a large change is difficult verging on impossible.
In theory, though, there is an alternative which could work:
Put each version of the large library into the container-level classpath. Do some container level magic so that each WAR file declares which version it wishes to use and it will find that on its classpath.
The "magic" must be done at the container level (I think) because this can only be achieved by loading each version of the library with a different classloader, then adjusting what classloaders are visible to each WAR file.
So, have you ever heard of doing this? If so, how? Or tell me what it is called so I can research further.

Regarding Tomcat, for the 7th version you can use VirtualWebappLocader like so
<Context>
<Loader className="org.apache.catalina.loader.VirtualWebappLoader"
virtualClasspath="/usr/shared/lib/spring-3/*.jar,/usr/shared/classes" />
</Context>
For the 8th version Pre- & Post- Resources should be used instead
<Context>
<Resources>
<PostResources className="org.apache.catalina.webresources.DirResourceSet"
base="/usr/shared/lib/spring-3" webAppMount="/WEB-INF/lib" />
<PostResources className="org.apache.catalina.webresources.DirResourceSet"
base="/usr/shared/classes" webAppMount="/WEB-INF/classes" />
</Resources>
</Context>
Don't forget to put the corresponding context.xml into the META-INF of your webapp.
For the jetty as well as other containers the same technique may be used.
The only difference is in how to specify extra classpath elements for the webapp.
UPDATE
The samples above does not share the loaded classes, but the idea is the same - use custom classloader. Here is just the pretty ugly sample that also tries to prevent classloader leaks during undeployment.
SharedWebappLoader
package com.foo.bar;
import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.WebappLoader;
public class SharedWebappLoader extends WebappLoader {
private String pathID;
private String pathConfig;
static final ThreadLocal<ClassLoaderFactory> classLoaderFactory = new ThreadLocal<>();
public SharedWebappLoader() {
this(null);
}
public SharedWebappLoader(ClassLoader parent) {
super(parent);
setLoaderClass(SharedWebappClassLoader.class.getName());
}
public String getPathID() {
return pathID;
}
public void setPathID(String pathID) {
this.pathID = pathID;
}
public String getPathConfig() {
return pathConfig;
}
public void setPathConfig(String pathConfig) {
this.pathConfig = pathConfig;
}
#Override
protected void startInternal() throws LifecycleException {
classLoaderFactory.set(new ClassLoaderFactory(pathConfig, pathID));
try {
super.startInternal();
} finally {
classLoaderFactory.remove();
}
}
}
SharedWebappClassLoader
package com.foo.bar;
import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.ResourceEntry;
import org.apache.catalina.loader.WebappClassLoader;
import java.net.URL;
public class SharedWebappClassLoader extends WebappClassLoader {
public SharedWebappClassLoader(ClassLoader parent) {
super(SharedWebappLoader.classLoaderFactory.get().create(parent));
}
#Override
protected ResourceEntry findResourceInternal(String name, String path) {
ResourceEntry entry = super.findResourceInternal(name, path);
if(entry == null) {
URL url = parent.getResource(name);
if (url == null) {
return null;
}
entry = new ResourceEntry();
entry.source = url;
entry.codeBase = entry.source;
}
return entry;
}
#Override
public void stop() throws LifecycleException {
ClassLoaderFactory.removeLoader(parent);
}
}
ClassLoaderFactory
package com.foo.bar;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
public class ClassLoaderFactory {
private static final class ConfigKey {
private final String pathConfig;
private final String pathID;
private ConfigKey(String pathConfig, String pathID) {
this.pathConfig = pathConfig;
this.pathID = pathID;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ConfigKey configKey = (ConfigKey) o;
if (pathConfig != null ? !pathConfig.equals(configKey.pathConfig) : configKey.pathConfig != null)
return false;
if (pathID != null ? !pathID.equals(configKey.pathID) : configKey.pathID != null) return false;
return true;
}
#Override
public int hashCode() {
int result = pathConfig != null ? pathConfig.hashCode() : 0;
result = 31 * result + (pathID != null ? pathID.hashCode() : 0);
return result;
}
}
private static final Map<ConfigKey, ClassLoader> loaders = new HashMap<>();
private static final Map<ClassLoader, ConfigKey> revLoaders = new HashMap<>();
private static final Map<ClassLoader, Integer> usages = new HashMap<>();
private final ConfigKey key;
public ClassLoaderFactory(String pathConfig, String pathID) {
this.key = new ConfigKey(pathConfig, pathID);
}
public ClassLoader create(ClassLoader parent) {
synchronized (loaders) {
ClassLoader loader = loaders.get(key);
if(loader != null) {
Integer usageCount = usages.get(loader);
usages.put(loader, ++usageCount);
return loader;
}
Properties props = new Properties();
try (InputStream is = new BufferedInputStream(new FileInputStream(key.pathConfig))) {
props.load(is);
} catch (IOException e) {
throw new RuntimeException(e);
}
String libsStr = props.getProperty(key.pathID);
String[] libs = libsStr.split(File.pathSeparator);
URL[] urls = new URL[libs.length];
try {
for(int i = 0, len = libs.length; i < len; i++) {
urls[i] = new URL(libs[i]);
}
} catch (MalformedURLException e) {
throw new RuntimeException(e);
}
loader = new URLClassLoader(urls, parent);
loaders.put(key, loader);
revLoaders.put(loader, key);
usages.put(loader, 1);
return loader;
}
}
public static void removeLoader(ClassLoader parent) {
synchronized (loaders) {
Integer val = usages.get(parent);
if(val > 1) {
usages.put(parent, --val);
} else {
usages.remove(parent);
ConfigKey key = revLoaders.remove(parent);
loaders.remove(key);
}
}
}
}
context.xml of the first app
<Context>
<Loader className="com.foo.bar.SharedWebappLoader"
pathConfig="${catalina.base}/conf/shared.properties"
pathID="commons_2_1"/>
</Context>
context.xml of the second app
<Context>
<Loader className="com.foo.bar.SharedWebappLoader"
pathConfig="${catalina.base}/conf/shared.properties"
pathID="commons_2_6"/>
</Context>
$TOMCAT_HOME/conf/shared.properties
commons_2_1=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.1/commons-lang-2.1.jar
commons_2_6=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar

I was able to implement this for Tomcat (Tested on Tomcat 7.0.52). My solution involves implementing custom version of WebAppLoader which extends standard Tomcat's WebAppLoader. Thanks to this solution you can pass custom classloader to load classes for each of web application.
To utilize this new loader you need to declare it for each application (either in Context.xml file placed in each war or in Tomcat's server.xml file). This loader takes an extra custom parameter webappName which is later passed to LibrariesStorage class to determine which libraries should be used by which application.
<Context path="/pl-app" >
<Loader className="web.DynamicWebappLoader" webappName="pl-app"/>
</Context>
<Context path="/my-webapp" >
<Loader className="web.DynamicWebappLoader" webappName="myApplication2"/>
</Context>
Once this is defined you need to install this DynamicWebappLoader to Tomcat. To do this copy all copiled classes to lib directory of Tomcat (so you should have following files [tomcat dir]/lib/web/DynamicWebappLoader.class, [tomcat dir]/lib/web/LibrariesStorage.class, [tomcat dir]/lib/web/LibraryAndVersion.class, [tomcat dir]/lib/web/WebAppAwareClassLoader.class).
You need also to download xbean-classloader-4.0.jar and place it in Tomcat's lib dir (so you should have [tomcat dir]/lib/xbean-classloader-4.0.jar. NOTE:xbean-classloader provides special implementation of classloader (org.apache.xbean.classloader.JarFileClassLoader) which allowes to load needed jars at runtime.
Main trick is made in LibraryStorgeClass (full implementation is at the end). It stores a mapping between each application (defined by webappName) and libraries which this application is allowed to load. In current implementation this is hardcoded, but this can be rewritten to dynamically generate list of libs needed by each application. Each library has its own instance of JarFileClassLoader which ensures that each library is only loaded one time (the mapping between library and its classloader is stored in static field "libraryToClassLoader", so this mapping is the same for every web application because of static nature of the field)
class LibrariesStorage {
private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";
private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();
private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();
static {
try {
addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");
mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars
mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");
(...)
}
In above example, suppose that in directory with all the jars (defined here by JARS_DIR) we have only a commons-lang3-3.3.2.jar file. This would mean that application identified by "pl-app" name (the name comes from webappName attribute in tag in Context.xml as mentioned above) will be able to load classes from commons-lang jar. Application identified by "myApplication2" will get ClassNotFoundException at this point because it has access only to commons-lang3-3.3.0.jar, but this file is not present in JARS_DIR directory.
Full implementation here:
package web;
import org.apache.catalina.loader.WebappLoader;
import org.apache.xbean.classloader.JarFileClassLoader;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class DynamicWebappLoader extends WebappLoader {
private String webappName;
private WebAppAwareClassLoader webAppAwareClassLoader;
public static final ThreadLocal lastCreatedClassLoader = new ThreadLocal();
public DynamicWebappLoader() {
super(new WebAppAwareClassLoader(Thread.currentThread().getContextClassLoader()));
webAppAwareClassLoader = (WebAppAwareClassLoader) lastCreatedClassLoader.get(); // unfortunately I did not find better solution to access new instance of WebAppAwareClassLoader created in previous line so I passed it via thread local
lastCreatedClassLoader.remove();
}
// (this method is called by Tomcat because of Loader attribute in Context.xml - <Context> <Loader className="..." webappName="myApplication2"/> )
public void setWebappName(String name) {
System.out.println("Setting webapp name: " + name);
this.webappName = name;
webAppAwareClassLoader.setWebAppName(name); // pass web app name to ClassLoader
}
}
class WebAppAwareClassLoader extends ClassLoader {
private String webAppName;
public WebAppAwareClassLoader(ClassLoader parent) {
super(parent);
DynamicWebappLoader.lastCreatedClassLoader.set(this); // store newly created instance in ThreadLocal .. did not find better way to access the reference later in code
}
#Override
public Class<?> loadClass(String className) throws ClassNotFoundException {
System.out.println("Load class: " + className + " for webapp: " + webAppName);
try {
return LibrariesStorage.loadClassForWebapp(webAppName, className);
} catch (ClassNotFoundException e) {
System.out.println("JarFileClassLoader did not find class: " + className + " " + e.getMessage());
return super.loadClass(className);
}
}
public void setWebAppName(String webAppName) {
this.webAppName = webAppName;
}
}
class LibrariesStorage {
private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";
private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();
private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();
static {
try {
addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");
mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars
mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");
} catch (MalformedURLException e) {
throw new RuntimeException(e.getMessage(), e);
}
}
private static void mapApplicationToLibrary(String applicationName, String libraryName, String libraryVersion) {
LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
if (!webappLibraries.containsKey(applicationName)) {
webappLibraries.put(applicationName, new ArrayList<LibraryAndVersion>());
}
webappLibraries.get(applicationName).add(libraryAndVersion);
}
private static void addLibrary(String libraryName, String libraryVersion, String filename)
throws MalformedURLException {
LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
URL libraryLocation = new File(JARS_DIR + File.separator + filename).toURI().toURL();
libraryToClassLoader.put(libraryAndVersion,
new JarFileClassLoader("JarFileClassLoader for lib: " + libraryAndVersion,
new URL[] { libraryLocation }));
}
private LibrariesStorage() {
}
public static Class<?> loadClassForWebapp(String webappName, String className) throws ClassNotFoundException {
System.out.println("Loading class: " + className + " for web application: " + webappName);
List<LibraryAndVersion> webappLibraries = LibrariesStorage.webappLibraries.get(webappName);
for (LibraryAndVersion libraryAndVersion : webappLibraries) {
JarFileClassLoader libraryClassLoader = libraryToClassLoader.get(libraryAndVersion);
try {
return libraryClassLoader.loadClass(className); // ok current lib contained class to load
} catch (ClassNotFoundException e) {
// ok.. continue in loop... try to load the class from classloader connected to next library
}
}
throw new ClassNotFoundException("Class " + className + " was not found in any jar connected to webapp: " +
webappLibraries);
}
}
class LibraryAndVersion {
private final String name;
private final String version;
LibraryAndVersion(String name, String version) {
this.name = name;
this.version = version;
}
#Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if ((o == null) || (getClass() != o.getClass())) {
return false;
}
LibraryAndVersion that = (LibraryAndVersion) o;
if ((name != null) ? (!name.equals(that.name)) : (that.name != null)) {
return false;
}
if ((version != null) ? (!version.equals(that.version)) : (that.version != null)) {
return false;
}
return true;
}
#Override
public int hashCode() {
int result = (name != null) ? name.hashCode() : 0;
result = (31 * result) + ((version != null) ? version.hashCode() : 0);
return result;
}
#Override
public String toString() {
return "LibraryAndVersion{" +
"name='" + name + '\'' +
", version='" + version + '\'' +
'}';
}
}

JBoss has a framework called Modules that solves this problem. You can save the shared library with its version and reference it from your war-file.
I have no idea if it works on Tomcat, but it works as a charm on Wildfly.