Develop Reference

Can we automatically refresh spring properties file without using actuator refresh endpoint [duplicate]

Many in-house solutions come to mind. Like having the properties in a database and poll it every N secs. Then also check the timestamp modification for a .properties file and reload it.
But I was looking in Java EE standards and spring boot docs and I can't seem to find some best way of doing it.
I need my application to read a properties file(or env. variables or DB parameters), then be able to re-read them. What is the best practice being used in production?
A correct answer will at least solve one scenario (Spring Boot or Java EE) and provide a conceptual clue on how to make it work on the other

After further research, reloading properties must be carefully considered. In Spring, for example, we can reload the 'current' values of properties without much problem. But. Special care must be taken when resources were initialized at the context initialization time based on the values that were present in the application.properties file (e.g. Datasources, connection pools, queues, etc.).
NOTE:
The abstract classes used for Spring and Java EE are not the best example of clean code. But it is easy to use and it does address this basic initial requirements:
No usage of external libraries other than Java 8 Classes.
Only one file to solve the problem (~160 lines for the Java EE version).
Usage of standard Java Properties UTF-8 encoded file available in the File System.
Support encrypted properties.
For Spring Boot
This code helps with hot-reloading application.properties file without the usage of a Spring Cloud Config server (which may be overkill for some use cases)
This abstract class you may just copy & paste (SO goodies :D ) It's a code derived from this SO answer
// imports from java/spring/lombok
public abstract class ReloadableProperties {
#Autowired
protected StandardEnvironment environment;
private long lastModTime = 0L;
private Path configPath = null;
private PropertySource<?> appConfigPropertySource = null;
#PostConstruct
private void stopIfProblemsCreatingContext() {
System.out.println("reloading");
MutablePropertySources propertySources = environment.getPropertySources();
Optional<PropertySource<?>> appConfigPsOp =
StreamSupport.stream(propertySources.spliterator(), false)
.filter(ps -> ps.getName().matches("^.*applicationConfig.*file:.*$"))
.findFirst();
if (!appConfigPsOp.isPresent()) {
// this will stop context initialization
// (i.e. kill the spring boot program before it initializes)
throw new RuntimeException("Unable to find property Source as file");
}
appConfigPropertySource = appConfigPsOp.get();
String filename = appConfigPropertySource.getName();
filename = filename
.replace("applicationConfig: [file:", "")
.replaceAll("\\]$", "");
configPath = Paths.get(filename);
}
#Scheduled(fixedRate=2000)
private void reload() throws IOException {
System.out.println("reloading...");
long currentModTs = Files.getLastModifiedTime(configPath).toMillis();
if (currentModTs > lastModTime) {
lastModTime = currentModTs;
Properties properties = new Properties();
#Cleanup InputStream inputStream = Files.newInputStream(configPath);
properties.load(inputStream);
environment.getPropertySources()
.replace(
appConfigPropertySource.getName(),
new PropertiesPropertySource(
appConfigPropertySource.getName(),
properties
)
);
System.out.println("Reloaded.");
propertiesReloaded();
}
}
protected abstract void propertiesReloaded();
}
Then you make a bean class that allows retrieval of property values from applicatoin.properties that uses the abstract class
#Component
public class AppProperties extends ReloadableProperties {
public String dynamicProperty() {
return environment.getProperty("dynamic.prop");
}
public String anotherDynamicProperty() {
return environment.getProperty("another.dynamic.prop");
}
#Override
protected void propertiesReloaded() {
// do something after a change in property values was done
}
}
Make sure to add #EnableScheduling to your #SpringBootApplication
#SpringBootApplication
#EnableScheduling
public class MainApp {
public static void main(String[] args) {
SpringApplication.run(MainApp.class, args);
}
}
Now you can auto-wire the AppProperties Bean wherever you need it. Just make sure to always call the methods in it instead of saving it's value in a variable. And make sure to re-configure any resource or bean that was initialized with potentially different property values.
For now, I have only tested this with an external-and-default-found ./config/application.properties file.
For Java EE
I made a common Java SE abstract class to do the job.
You may copy & paste this:
// imports from java.* and javax.crypto.*
public abstract class ReloadableProperties {
private volatile Properties properties = null;
private volatile String propertiesPassword = null;
private volatile long lastModTimeOfFile = 0L;
private volatile long lastTimeChecked = 0L;
private volatile Path propertyFileAddress;
abstract protected void propertiesUpdated();
public class DynProp {
private final String propertyName;
public DynProp(String propertyName) {
this.propertyName = propertyName;
}
public String val() {
try {
return ReloadableProperties.this.getString(propertyName);
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException(e);
}
}
}
protected void init(Path path) {
this.propertyFileAddress = path;
initOrReloadIfNeeded();
}
private synchronized void initOrReloadIfNeeded() {
boolean firstTime = lastModTimeOfFile == 0L;
long currentTs = System.currentTimeMillis();
if ((lastTimeChecked + 3000) > currentTs)
return;
try {
File fa = propertyFileAddress.toFile();
long currModTime = fa.lastModified();
if (currModTime > lastModTimeOfFile) {
lastModTimeOfFile = currModTime;
InputStreamReader isr = new InputStreamReader(new FileInputStream(fa), StandardCharsets.UTF_8);
Properties prop = new Properties();
prop.load(isr);
properties = prop;
isr.close();
File passwordFiles = new File(fa.getAbsolutePath() + ".key");
if (passwordFiles.exists()) {
byte[] bytes = Files.readAllBytes(passwordFiles.toPath());
propertiesPassword = new String(bytes,StandardCharsets.US_ASCII);
propertiesPassword = propertiesPassword.trim();
propertiesPassword = propertiesPassword.replaceAll("(\\r|\\n)", "");
}
}
updateProperties();
if (!firstTime)
propertiesUpdated();
} catch (Exception e) {
e.printStackTrace();
}
}
private void updateProperties() {
List<DynProp> dynProps = Arrays.asList(this.getClass().getDeclaredFields())
.stream()
.filter(f -> f.getType().isAssignableFrom(DynProp.class))
.map(f-> fromField(f))
.collect(Collectors.toList());
for (DynProp dp :dynProps) {
if (!properties.containsKey(dp.propertyName)) {
System.out.println("propertyName: "+ dp.propertyName + " does not exist in property file");
}
}
for (Object key : properties.keySet()) {
if (!dynProps.stream().anyMatch(dp->dp.propertyName.equals(key.toString()))) {
System.out.println("property in file is not used in application: "+ key);
}
}
}
private DynProp fromField(Field f) {
try {
return (DynProp) f.get(this);
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return null;
}
protected String getString(String param) throws Exception {
initOrReloadIfNeeded();
String value = properties.getProperty(param);
if (value.startsWith("ENC(")) {
String cipheredText = value
.replace("ENC(", "")
.replaceAll("\\)$", "");
value = decrypt(cipheredText, propertiesPassword);
}
return value;
}
public static String encrypt(String plainText, String key)
throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException, InvalidKeySpecException {
SecureRandom secureRandom = new SecureRandom();
byte[] keyBytes = key.getBytes(StandardCharsets.US_ASCII);
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(key.toCharArray(), new byte[]{0,1,2,3,4,5,6,7}, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secretKey = new SecretKeySpec(tmp.getEncoded(), "AES");
byte[] iv = new byte[12];
secureRandom.nextBytes(iv);
final Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
GCMParameterSpec parameterSpec = new GCMParameterSpec(128, iv); //128 bit auth tag length
cipher.init(Cipher.ENCRYPT_MODE, secretKey, parameterSpec);
byte[] cipherText = cipher.doFinal(plainText.getBytes(StandardCharsets.UTF_8));
ByteBuffer byteBuffer = ByteBuffer.allocate(4 + iv.length + cipherText.length);
byteBuffer.putInt(iv.length);
byteBuffer.put(iv);
byteBuffer.put(cipherText);
byte[] cipherMessage = byteBuffer.array();
String cyphertext = Base64.getEncoder().encodeToString(cipherMessage);
return cyphertext;
}
public static String decrypt(String cypherText, String key)
throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException, InvalidKeySpecException {
byte[] cipherMessage = Base64.getDecoder().decode(cypherText);
ByteBuffer byteBuffer = ByteBuffer.wrap(cipherMessage);
int ivLength = byteBuffer.getInt();
if(ivLength < 12 || ivLength >= 16) { // check input parameter
throw new IllegalArgumentException("invalid iv length");
}
byte[] iv = new byte[ivLength];
byteBuffer.get(iv);
byte[] cipherText = new byte[byteBuffer.remaining()];
byteBuffer.get(cipherText);
byte[] keyBytes = key.getBytes(StandardCharsets.US_ASCII);
final Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(key.toCharArray(), new byte[]{0,1,2,3,4,5,6,7}, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secretKey = new SecretKeySpec(tmp.getEncoded(), "AES");
cipher.init(Cipher.DECRYPT_MODE, secretKey, new GCMParameterSpec(128, iv));
byte[] plainText= cipher.doFinal(cipherText);
String plain = new String(plainText, StandardCharsets.UTF_8);
return plain;
}
}
Then you can use it this way:
public class AppProperties extends ReloadableProperties {
public static final AppProperties INSTANCE; static {
INSTANCE = new AppProperties();
INSTANCE.init(Paths.get("application.properties"));
}
#Override
protected void propertiesUpdated() {
// run code every time a property is updated
}
public final DynProp wsUrl = new DynProp("ws.url");
public final DynProp hiddenText = new DynProp("hidden.text");
}
In case you want to use encoded properties you may enclose it's value inside ENC() and a password for decryption will be searched for in the same path and name of the property file with an added .key extension. In this example it will look for the password in the application.properties.key file.
application.properties ->
ws.url=http://some webside
hidden.text=ENC(AAAADCzaasd9g61MI4l5sbCXrFNaQfQrgkxygNmFa3UuB9Y+YzRuBGYj+A==)
aplication.properties.key ->
password aca
For the encryption of property values for the Java EE solution I consulted Patrick Favre-Bulle excellent article on Symmetric Encryption with AES in Java and Android. Then checked the Cipher, block mode and padding in this SO question about AES/GCM/NoPadding. And finally I made the AES bits be derived from a password from #erickson excellent answer in SO about AES Password Based Encryption. Regarding encryption of value properties in Spring I think they are integrated with Java Simplified Encryption
Wether this qualify as a best practice or not may be out of scope. This answer shows how to have reloadable properties in Spring Boot and Java EE.

This functionality can be achieved by using a Spring Cloud Config Server and a refresh scope client.
Server
Server (Spring Boot app) serves the configuration stored, for example, in a Git repository:
#SpringBootApplication
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
application.yml:
spring:
cloud:
config:
server:
git:
uri: git-repository-url-which-stores-configuration.git
configuration file configuration-client.properties (in a Git repository):
configuration.value=Old
Client
Client (Spring Boot app) reads configuration from the configuration server by using #RefreshScope annotation:
#Component
#RefreshScope
public class Foo {
#Value("${configuration.value}")
private String value;
....
}
bootstrap.yml:
spring:
application:
name: configuration-client
cloud:
config:
uri: configuration-server-url
When there is a configuration change in the Git repository:
configuration.value=New
reload the configuration variable by sending a POST request to the /refresh endpoint:
$ curl -X POST http://client-url/actuator/refresh
Now you have the new value New.
Additionally Foo class can serve the value to the rest of application via RESTful API if its changed to RestController and has a corresponding endpont.

I used #David Hofmann concept and made some changes because of not all was good.
First of all, in my case I no need auto-reload, I just call the REST controller for updating properties.
The second case #David Hofmann's approach not workable for me with outside files.
Now, this code can work with application.properties file from resources(inside the app) and from an outside place. The outside file I put near jar, and I use this --spring.config.location=app.properties argument when the application starts.
#Component
public class PropertyReloader {
private final Logger logger = LoggerFactory.getLogger(getClass());
#Autowired
private StandardEnvironment environment;
private long lastModTime = 0L;
private PropertySource<?> appConfigPropertySource = null;
private Path configPath;
private static final String PROPERTY_NAME = "app.properties";
#PostConstruct
private void createContext() {
MutablePropertySources propertySources = environment.getPropertySources();
// first of all we check if application started with external file
String property = "applicationConfig: [file:" + PROPERTY_NAME + "]";
PropertySource<?> appConfigPsOp = propertySources.get(property);
configPath = Paths.get(PROPERTY_NAME).toAbsolutePath();
if (appConfigPsOp == null) {
// if not we check properties file from resources folder
property = "class path resource [" + PROPERTY_NAME + "]";
configPath = Paths.get("src/main/resources/" + PROPERTY_NAME).toAbsolutePath();
}
appConfigPsOp = propertySources.get(property);
appConfigPropertySource = appConfigPsOp;
}
// this method I call into REST cintroller for reloading all properties after change
// app.properties file
public void reload() {
try {
long currentModTs = Files.getLastModifiedTime(configPath).toMillis();
if (currentModTs > lastModTime) {
lastModTime = currentModTs;
Properties properties = new Properties();
#Cleanup InputStream inputStream = Files.newInputStream(configPath);
properties.load(inputStream);
String property = appConfigPropertySource.getName();
PropertiesPropertySource updatedProperty = new PropertiesPropertySource(property, properties);
environment.getPropertySources().replace(property, updatedProperty);
logger.info("Configs {} were reloaded", property);
}
} catch (Exception e) {
logger.error("Can't reload config file " + e);
}
}
}
I hope that my approach will help somebody

As mentioned by #Boris, Spring Cloud Config is the way to go to avoid patchy solution. To keep the setup minimum, I will suggest the Embedding the Config Server Approach with native type (file type).
To support automatic config refresh without calling the actuator endpoint manually, I have created a directory listener to detect file changes and to dispatch refresh scope event.
Proof Of Concept repo (git)

For spring boot, there's a really good article on this topic here, but for multiple property files it doesn't work perfectly.
In my case I had 2 property files, one non sensitive and one containing the passwords. I proceeded with the following:
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
Extend the spring's PropertySource so that you can add the reloadable version to the environment.
public class ReloadablePropertySource extends PropertySource {
private final PropertiesConfiguration propertiesConfiguration;
public ReloadablePropertySource(String name, String path, ConfigurationListener listener) {
super(StringUtils.hasText(name) ? name : path);
try {
this.propertiesConfiguration = getConfiguration(path, listener);
} catch (Exception e) {
throw new MissingRequiredPropertiesException();
}
}
#Override
public Object getProperty(String s) {
return propertiesConfiguration.getProperty(s);
}
private PropertiesConfiguration getConfiguration(String path, ConfigurationListener listener) throws ConfigurationException {
PropertiesConfiguration configuration = new PropertiesConfiguration(path);
FileChangedReloadingStrategy reloadingStrategy = new FileChangedReloadingStrategy();
reloadingStrategy.setRefreshDelay(5000);
configuration.setReloadingStrategy(reloadingStrategy);
configuration.addConfigurationListener(listener);
return configuration;
}
}
Now add all of your properties files (now reloadable) inside the spring's env
#Configuration
public class ReloadablePropertySourceConfig {
private final ConfigurableEnvironment env;
#Value("${spring.config.location}")
private String appConfigPath;
#Value("${spring.config.additional-location}")
private String vaultConfigPath;
public ReloadablePropertySourceConfig(ConfigurableEnvironment env) {
this.env = env;
}
#Bean
#ConditionalOnProperty(name = "spring.config.location")
public ReloadablePropertySource getAppConfigReloadablePropertySource(){
ReloadablePropertySource rps = new ReloadablePropertySource("dynamicNonSensitive", appConfigPath, new PropertiesChangeListener());
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(rps);
return rps;
}
#Bean
#ConditionalOnProperty(name = "spring.config.additional-location")
public ReloadablePropertySource getVaultReloadablePropertySource(){
ReloadablePropertySource rps = new ReloadablePropertySource("dynamicVault", vaultConfigPath, new PropertiesChangeListener());
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(rps);
return rps;
}
private static class PropertiesChangeListener implements ConfigurationListener{
#Override
public void configurationChanged(ConfigurationEvent event) {
if (!event.isBeforeUpdate()){
System.out.println("config refreshed!");
}
}
}
}
From the article
We've added the new property source as the first item because we want it to override any existing property with the same key
In our case, we have 2 "reloadable" property sources and both will be looked up first.
Finally create one more class from which we can access the env's properties
#Component
public class ConfigProperties {
private final Environment environment;
public ConfigProperties(Environment environment) {
this.environment = environment;
}
public String getProperty(String name){
return environment.getProperty(name);
}
}
Now you can autowire ConfigProperties and always get the latest property in the files without requiring to restart the application.
#RestController
#Slf4j
public class TestController {
#Autowired
private ConfigProperties env;
#GetMapping("/refresh")
public String test2() {
log.info("hit");
String updatedProperty = env.getProperty("test.property");
String password = env.getProperty("db.password");
return updatedProperty + "\n" + password;
}
}
where test.property is coming from 1st file and db.password is coming from another.

If you want to change the properties at realtime and don't want to restart the server then follow the below steps:
1). Application.properties
app.name= xyz
management.endpoints.web.exposure.include=*
2). Add below dependencies in pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-context</artifactId>
<version>2.1.1.RELEASE</version>
</dependency>
3).Place application.properties in /target/config folder. Create the jar in /target folder
4).add a classas below ApplcationProperties.java
#Component
#RefreshScope
#ConfigurationProperties(prefix = "app")
public class ApplicationProperties {
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
5). Write Controller.java and inject ApplcationProperties
#RestController
public class TestController {
#Autowired
private ApplicationProperties applcationProperties;
#GetMapping("/test")
public String getString() {
return applcationProperties.getName();
}
}
6).Run the spring boot application
Call localhost:XXXX/test from your browser
Output : xyz
7). Change the value in application.properties from xyz to abc
8). Using postman send a POST request to localhost:XXXX/actuator/refresh
response: ["app.name"]
9). Call localhost:XXXX/find from your browser
Output : abc

Java - configure custom loggers for use

Trying to use java.util.logging and failing.
In an attempt to make use of https://stackoverflow.com/a/8249319/3322533 :
handlers = mypackage.logging.RequestFileHandler, mypackage.logging.MainFileHandler
config =
mainLogger.handlers = mypackage.logging.MainFileHandler
requestLogger.handlers = mypackage.logging.RequestFileHandler
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.ConsoleHandler.filter =
java.util.logging.ConsoleHandler.formatter = mypackage.logging.VerySimpleFormatter
java.util.logging.ConsoleHandler.encoding =
mypackage.RequestFileHandler.level = SEVERE
mypackage.RequestFileHandler.filter =
mypackage.RequestFileHandler.formatter = mypackage.logging.VerySimpleFormatter
mypackage.RequestFileHandler.encoding =
mypackage.RequestFileHandler.limit =
mypackage.RequestFileHandler.count =
mypackage.RequestFileHandler.append = false
mypackage.RequestFileHandler.pattern = REQUESTS.%u.%g.log
mypackage.MainFileHandler.level = INFO
mypackage.MainFileHandler.filter =
mypackage.MainFileHandler.formatter = mypackage.logging.VerySimpleFormatter
mypackage.MainFileHandler.encoding =
mypackage.MainFileHandler.limit =
mypackage.MainFileHandler.count =
mypackage.MainFileHandler.append = false
mypackage.MainFileHandler.pattern = MAIN.%u.%g.log
where
public class MainFileHandler extends FileHandler {
public MainFileHandler() throws IOException, SecurityException {
super();
}
}
and
public class RequestFileHandler extends FileHandler {
public RequestFileHandler() throws IOException, SecurityException {
super();
}
}
Intention: provide two loggers accessible through
Logger.getLogger("mainLogger");
or
Logger.getLogger("requestLogger");
respectively, one that will write (exclusively) to MAIN[...].log and the other to REQUESTS[...].log
(No limits on the amount of messages that can be logged to either file and if necessary, use logging level to filter out unwanted msgs to either.)
However, neither file is created when I (for example)
public static final Logger log = Logger.getLogger("mainLogger");
and then
public void configureLogger(){
try {
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
InputStream config = classLoader.getResourceAsStream("logging.properties");
LogManager.getLogManager().readConfiguration(config);
}catch(Exception ex){
throw new RuntimeException("logging properties failed");
}
}
before I
log.info("Hello World!")
I know the properties are loaded because when I include java.util.logging.ConsoleHandler in the handlers = ... list and use the global logger, instead, the formatter is applied for the console output.
So ... I guess my attempt at setting up the file loggers is faulty. How do I get this working?
EDIT
So I removed the [...].pattern = [...] lines and instead hardcoded the file names:
public class MainFileHandler extends FileHandler implements FileHandlerProperties {
public MainFileHandler() throws IOException, SecurityException {
super("MAIN_" + new SimpleDateFormat(TIME_PATTERN).format(new Date()) + ".log");
}
}
and
public class RequestFileHandler extends FileHandler implements FileHandlerProperties {
public RequestFileHandler() throws IOException, SecurityException {
super("REQUESTS_" + new SimpleDateFormat(TIME_PATTERN).format(new Date()) + ".log");
}
}
where
public interface FileHandlerProperties {
static final String TIME_PATTERN = "yyyy-MM-dd'T'HH:mm:ss.SSSXXX";
}
Both files now get created BUT they both contain exactly the same (despite their different level settings and loggers) AND what they contain is in xml:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE log SYSTEM "logger.dtd">
<log>
<record>
<date>2016-10-10T18:49:23</date>
<millis>1476118163654</millis>
<sequence>0</sequence>
<logger>mainLogger</logger>
<level>INFO</level>
<class>mypackage.main.Main</class>
<method><init></method>
<thread>1</thread>
<message>Hello World</message>
</record>
</log>
Please help ...

The problem is that the first call to Logger.getLogger during class loading reads the log configuration and your configureLogger method fails due to JDK-8033661: readConfiguration does not cleanly reinitialize the logging system.
To workaround this you have to ensure that configureLogger runs before the first call to Logger.getLogger.
public class BootMain {
static {
configureLogger();
mainLogger = Logger.getLogger("mainLogger");
requestLogger = Logger.getLogger("requestLogger");
}
private static final Logger mainLogger;
private static final Logger requestLogger;
public static void main(String[] args) throws IOException {
mainLogger.log(Level.SEVERE, "Test from main.");
requestLogger.log(Level.SEVERE, "Test from request.");
System.out.println(new File(".").getCanonicalPath());
}
private static void configureLogger() {
try {
InputStream config = config();
LogManager.getLogManager().readConfiguration(config);
} catch (Exception ex) {
throw new RuntimeException("logging properties failed");
}
}
private static String prefix() {
return "mypackage.logging";
}
private static InputStream config() throws IOException {
String p = prefix();
Properties props = new Properties();
props.put("mainLogger.handlers", p + ".MainFileHandler");
props.put("requestLogger.handlers", p + ".RequestFileHandler");
props.put(p + ".RequestFileHandler.level", "SEVERE");
props.put(p + ".MainFileHandler.level", "INFO");
props.put(p + ".RequestFileHandler.pattern", "REQUESTS.%u.%g.log");
props.put(p + ".MainFileHandler.pattern", "MAIN.%u.%g.log");
ByteArrayOutputStream out = new ByteArrayOutputStream();
props.store(out, "");
return new ByteArrayInputStream(out.toByteArray());
}
}
Also make sure you are not using a really old version of JDK or you can run into JDK-5089480: java.util.logging.FileHandler uses hardcoded classname when reading properties.
Otherwise you can use the LogManager config option to manually setup your configuration.

ServiceName is getting ignored while connecting to HA enabled HDFS from Java using RPC

Here is my build config method
private Configuration buildConfiguration() {
Configuration conf = new Configuration();
if (connectivityDetail.isSecureMode()) {
conf.set("hadoop.security.authentication", "kerberos");
conf.set("hadoop.http.authentication.type", "kerberos");
conf.set("dfs.namenode.kerberos.principal", connectivityDetail.getHdfsServicePrincipal());
}
if (isHAEnabled()) {
String hdfsServiceName = connectivityDetail.getHdfsServiceName();
conf.set("fs.defaultFS", "hdfs://" + hdfsServiceName);
conf.set("dfs.ha.namenodes." + hdfsServiceName, "nn0,nn1");
conf.set("dfs.nameservices", hdfsServiceName);
conf.set("dfs.client.failover.proxy.provider." + hdfsServiceName,
"org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider");
conf.set("dfs.namenode.rpc-address." + hdfsServiceName + ".nn1",
connectivityDetail.getNameNodeUris().get(0));
conf.set("dfs.namenode.rpc-address." + hdfsServiceName + ".nn0",
connectivityDetail.getNameNodeUris().get(1));
}
return conf;
}
hdfsService name is incorrectly set into the configuration object but I am able to get FileSystem and everything is working correctly. I am not sure why it is not using service name?
This is how I am creating Path
public static Path getHdfsResourceLocation(String resourceLocation) throws Exception {
String[] hdfsURIs = OrchestrationConfigUtil.getHdfsUri();
Path hdfsResourceLoc = null;
if (isHAEnabled()) {
hdfsResourceLoc = new Path(resourceLocation);
} else {
hdfsResourceLoc = FileContext.getFileContext(new URI(hdfsURIs[0])).makeQualified(new Path(resourceLocation));
}
return hdfsResourceLoc;
}
Everything is working fine with wrong service name and I am not sure why?

Programmatic SchemaExport / SchemaUpdate with Hibernate 5 and Spring 4

With Spring 4 and Hibernate 4, I was able to use Reflection to get the Hibernate Configuration object from the current environment, using this code:
#Autowired LocalContainerEntityManagerFactoryBean lcemfb;
EntityManagerFactoryImpl emf = (EntityManagerFactoryImpl) lcemfb.getNativeEntityManagerFactory();
SessionFactoryImpl sf = emf.getSessionFactory();
SessionFactoryServiceRegistryImpl serviceRegistry = (SessionFactoryServiceRegistryImpl) sf.getServiceRegistry();
Configuration cfg = null;
try {
Field field = SessionFactoryServiceRegistryImpl.class.getDeclaredField("configuration");
field.setAccessible(true);
cfg = (Configuration) field.get(serviceRegistry);
} catch (NoSuchFieldException | SecurityException | IllegalArgumentException | IllegalAccessException e) {
e.printStackTrace();
}
SchemaUpdate update = new SchemaUpdate(serviceRegistry, cfg);
With Hibernate 5, I must use some MetadataImplementor, which doesn't seems to be available from any of those objects. I also tried to use MetadataSources with the serviceRegistry. But it did say that it's the wrong kind of ServiceRegistry.
Is there any other way to get this working?

Basic idea for this problem is:
implementation of org.hibernate.integrator.spi.Integrator which stores required data to some holder. Register implementation as a service and use it where you need.
Work example you can find here https://github.com/valery-barysok/spring4-hibernate5-stackoverflow-34612019
create org.hibernate.integrator.api.integrator.Integrator class
import hello.HibernateInfoHolder;
import org.hibernate.boot.Metadata;
import org.hibernate.engine.spi.SessionFactoryImplementor;
import org.hibernate.service.spi.SessionFactoryServiceRegistry;
public class Integrator implements org.hibernate.integrator.spi.Integrator {
#Override
public void integrate(Metadata metadata, SessionFactoryImplementor sessionFactory, SessionFactoryServiceRegistry serviceRegistry) {
HibernateInfoHolder.setMetadata(metadata);
HibernateInfoHolder.setSessionFactory(sessionFactory);
HibernateInfoHolder.setServiceRegistry(serviceRegistry);
}
#Override
public void disintegrate(SessionFactoryImplementor sessionFactory, SessionFactoryServiceRegistry serviceRegistry) {
}
}
create META-INF/services/org.hibernate.integrator.spi.Integrator file
org.hibernate.integrator.api.integrator.Integrator
import org.hibernate.boot.spi.MetadataImplementor;
import org.hibernate.tool.hbm2ddl.SchemaExport;
import org.hibernate.tool.hbm2ddl.SchemaUpdate;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
#SpringBootApplication
public class Application implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Override
public void run(String... args) throws Exception {
new SchemaExport((MetadataImplementor) HibernateInfoHolder.getMetadata()).create(true, true);
new SchemaUpdate(HibernateInfoHolder.getServiceRegistry(), (MetadataImplementor) HibernateInfoHolder.getMetadata()).execute(true, true);
}
}

I would like to add up on Aviad's answer to make it complete as per OP's request.
The internals:
In order to get an instance of MetadataImplementor, the workaround is to register an instance of SessionFactoryBuilderFactory through Java's ServiceLoader facility. This registered service's getSessionFactoryBuilder method is then invoked by MetadataImplementor with an instance of itself, when hibernate is bootstrapped. The code references are below:
Service Loading
Invocation of getSessionFactoryBuilder
So, ultimately to get an instance of MetadataImplementor, you have to implement SessionFactoryBuilderFactory and register so ServiceLoader can recognize this service:
An implementation of SessionFactoryBuilderFactory:
public class MetadataProvider implements SessionFactoryBuilderFactory {
private static MetadataImplementor metadata;
#Override
public SessionFactoryBuilder getSessionFactoryBuilder(MetadataImplementor metadata, SessionFactoryBuilderImplementor defaultBuilder) {
this.metadata = metadata;
return defaultBuilder; //Just return the one provided in the argument itself. All we care about is the metadata :)
}
public static MetadataImplementor getMetadata() {
return metadata;
}
}
In order to register the above, create simple text file in the following path(assuming it's a maven project, ultimately we need the 'META-INF' folder to be available in the classpath):
src/main/resources/META-INF/services/org.hibernate.boot.spi.SessionFactoryBuilderFactory
And the content of the text file should be a single line(can even be multiple lines if you need to register multiple instances) stating the fully qualified class path of your implementation of SessionFactoryBuilderFactory. For example, for the above class, if your package name is 'com.yourcompany.prj', the following should be the content of the file.
com.yourcompany.prj.MetadataProvider
And that's it, if you run your application, spring app or standalone hibernate, you will have an instance of MetadataImplementor available through a static method once hibernate is bootstraped.
Update 1:
There is no way it can be injected via Spring. I digged into Hibernate's source code and the metadata object is not stored anywhere in SessionFactory(which is what we get from Spring). So, it's not possible to inject it. But there are two options if you want it in Spring's way:
Extend existing classes and customize all the way from
LocalSessionFactoryBean -> MetadataSources -> MetadataBuilder
LocalSessionFactoryBean is what you configure in Spring and it has an object of MetadataSources. MetadataSources creates MetadataBuilder which in turn creates MetadataImplementor. All the above operations don't store anything, they just create object on the fly and return. If you want to have an instance of MetaData, you should extend and modify the above classes so that they store a local copy of respective objects before they return. That way you can have a reference to MetadataImplementor. But I wouldn't really recommend this unless it's really needed, because the APIs might change over time.
On the other hand, if you don't mind building a MetaDataImplemetor from SessionFactory, the following code will help you:
EntityManagerFactoryImpl emf=(EntityManagerFactoryImpl)lcemfb.getNativeEntityManagerFactory();
SessionFactoryImpl sf=emf.getSessionFactory();
StandardServiceRegistry serviceRegistry = sf.getSessionFactoryOptions().getServiceRegistry();
MetadataSources metadataSources = new MetadataSources(new BootstrapServiceRegistryBuilder().build());
Metadata metadata = metadataSources.buildMetadata(serviceRegistry);
SchemaUpdate update=new SchemaUpdate(serviceRegistry,metadata); //To create SchemaUpdate
// You can either create SchemaExport from the above details, or you can get the existing one as follows:
try {
Field field = SessionFactoryImpl.class.getDeclaredField("schemaExport");
field.setAccessible(true);
SchemaExport schemaExport = (SchemaExport) field.get(serviceRegistry);
} catch (NoSuchFieldException | SecurityException | IllegalArgumentException | IllegalAccessException e) {
e.printStackTrace();
}

Take a look on this one:
public class EntityMetaData implements SessionFactoryBuilderFactory {
private static final ThreadLocal<MetadataImplementor> meta = new ThreadLocal<>();
#Override
public SessionFactoryBuilder getSessionFactoryBuilder(MetadataImplementor metadata, SessionFactoryBuilderImplementor defaultBuilder) {
meta.set(metadata);
return defaultBuilder;
}
public static MetadataImplementor getMeta() {
return meta.get();
}
}
Take a look on This Thread which seems to answer your needs

Well, my go to on this:
public class SchemaTranslator {
public static void main(String[] args) throws Exception {
new SchemaTranslator().run();
}
private void run() throws Exception {
String packageName[] = { "model"};
generate(packageName);
}
private List<Class<?>> getClasses(String packageName) throws Exception {
File directory = null;
try {
ClassLoader cld = getClassLoader();
URL resource = getResource(packageName, cld);
directory = new File(resource.getFile());
} catch (NullPointerException ex) {
throw new ClassNotFoundException(packageName + " (" + directory + ") does not appear to be a valid package");
}
return collectClasses(packageName, directory);
}
private ClassLoader getClassLoader() throws ClassNotFoundException {
ClassLoader cld = Thread.currentThread().getContextClassLoader();
if (cld == null) {
throw new ClassNotFoundException("Can't get class loader.");
}
return cld;
}
private URL getResource(String packageName, ClassLoader cld) throws ClassNotFoundException {
String path = packageName.replace('.', '/');
URL resource = cld.getResource(path);
if (resource == null) {
throw new ClassNotFoundException("No resource for " + path);
}
return resource;
}
private List<Class<?>> collectClasses(String packageName, File directory) throws ClassNotFoundException {
List<Class<?>> classes = new ArrayList<>();
if (directory.exists()) {
String[] files = directory.list();
for (String file : files) {
if (file.endsWith(".class")) {
// removes the .class extension
classes.add(Class.forName(packageName + '.' + file.substring(0, file.length() - 6)));
}
}
} else {
throw new ClassNotFoundException(packageName + " is not a valid package");
}
return classes;
}
private void generate(String[] packagesName) throws Exception {
Map<String, String> settings = new HashMap<String, String>();
settings.put("hibernate.hbm2ddl.auto", "drop-create");
settings.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQL94Dialect");
MetadataSources metadata = new MetadataSources(
new StandardServiceRegistryBuilder()
.applySettings(settings)
.build());
for (String packageName : packagesName) {
System.out.println("packageName: " + packageName);
for (Class<?> clazz : getClasses(packageName)) {
System.out.println("Class: " + clazz);
metadata.addAnnotatedClass(clazz);
}
}
SchemaExport export = new SchemaExport(
(MetadataImplementor) metadata.buildMetadata()
);
export.setDelimiter(";");
export.setOutputFile("db-schema.sql");
export.setFormat(true);
export.execute(true, false, false, false);
}
}

Embedded Jetty application

I a newbie in Jetty. I have created an application in which I embed the jetty web container. When I run the the application from eclipse it runs perfectly without any issues. However when I export the project with all the required libraries and run it from command line I cannot access the index.jsp web page like I used to in eclispe. This is the file that run the jetty web container.
public class JettyServer {
// The folder containing all the .jsp files
private final static String WEB_ROOT = "src/WebContent";
// Instance of the Jetty server
private final static Server SRV = new Server();
// Context Path
private final static String CONTEXT_PATH = "/smpp";
// Logging
private final static org.slf4j.Logger logger = LoggerFactory.getLogger(JettyServer.class);
/**
* #param args
* #throws ConfigurationException
*/
public static void main(String[] args) throws ConfigurationException {
logger.info("Initializing Web Server......");
// Servlet Context
final ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
// Set the security constraints
context.setContextPath(CONTEXT_PATH);
context.setResourceBase(WEB_ROOT);
context.setClassLoader(Thread.currentThread().getContextClassLoader());
context.addServlet(DefaultServlet.class, "/");
context.setInitParameter("org.eclipse.jetty.servlet.Default.dirAllowed", "false");
String [] welcomeFiles = {"index.jsp"};
context.setWelcomeFiles(welcomeFiles);
// Set the .jsp servlet handlers
final ServletHolder jsp = context.addServlet(JspServlet.class, "*.jsp");
jsp.setInitParameter("classpath", context.getClassPath());
// Session Manager
SessionHandler sh = new SessionHandler();
context.setSessionHandler(sh);
/* Http Request Handlers */
context.addServlet(HttpRequestProcessor.class, "/HttpHandler");
// Server configuration setup
// Connector setup
// We explicitly use the SocketConnector because the SelectChannelConnector locks files
Connector connector = new SocketConnector();
connector.setHost("localhost");
connector.setPort(Integer.parseInt(System.getProperty("jetty.port", new PropertiesConfiguration("smpp-config.properties").getString("http_port").trim())));
connector.setMaxIdleTime(60000);
JettyServer.SRV.setConnectors(new Connector[] { connector });
JettyServer.SRV.setHandler(context);
JettyServer.SRV.setAttribute("org.mortbay.jetty.Request.maxFormContentSize", 0);
JettyServer.SRV.setGracefulShutdown(5000);
JettyServer.SRV.setStopAtShutdown(true);
logger.info("Starting Jetty Web Container....");
try{
JettyServer.SRV.start();
}
catch(Exception ex){
logger.error("Jetty Web Container failed to start [CAUSE : " + ex.getMessage() + "]");
return;
}
logger.info("Jetty Web Container running....");
while(true){
try{
JettyServer.SRV.join();
}
catch(InterruptedException iex){
logger.error("Jetty Web Container interrupted [CAUSE : " + iex.getMessage() + "]");
}
}
}
}
code formatted properly

Your use of relative paths in the context.setResourceBase("src/WebContent"); will cause you problems.
Use a full, and absolute, URI reference with context.setResourceBase(String).
Note that you can use the following URI schemes: file, ftp, jar, and even http

Instead of this
final ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
Can you use this ?
WebAppContext root = new WebAppContext();
and rest of the code as example :
String webappDirLocation = "src/Webcontent/";
Server server = new Server(8080);
root.setContextPath(CONTEXT_PATH);
root.setDescriptor(webappDirLocation + "/WEB-INF/web.xml");
root.setResourceBase(webappDirLocation);
root.setParentLoaderPriority(true);
server.setHandler(root);