Develop Reference

Constant path replacement for different environments using gradle

This is a Apache Storm based project. I have a Constants file which looks something like this
public class Constant {
public static final String CONTEXT_PATH ="<some path to a context.xml file>";
public static final String APP_PROPERTIES_PATH = "<path to the properties file>";
//...More static properties
}
This CONTEXT_PATH variable is different for different environments (dev, test, prod).
I have a gradle task which generates the JAR file for deployment
task stormJar(type: Jar) {
baseName = 'diagnostic'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
I was looking to dynamically change or refer the CONTEXT_PATH variable so that we can create builds for different environments without making any changes to this file.
I believe there are solutions to do so in the Android territory (BuildConfig), but not able to find a solution for a plain Java project.
The solution should be such, it should work for IDE (IntelliJ and Eclipse) as well as create environment specific build.
Doing something like below should get the work done
gradlew build -pEnvironment=prod
Not much experience with gradle. Please point me in the right direction.

Note there is a working example here.
One technique is to generate Constants.java as part of the build. Consider this template (stored as a resource, not as Java code):
public class Constants {
public static final String CONTEXT_PATH = "__CONTEXT_PATH";
public static final String APP_PROPERTIES_PATH = "__APP_PROPERTIES_PATH";
}
and the following generates Constants.java early in the compileJava task:
compileJava.doFirst {
def newConstantsFile = new File("${projectDir}/src/main/java/net/codetojoy/util/Constants.java")
def templateConstantsFile = new File("${projectDir}/resources/TemplateConstants.java")
newConstantsFile.withWriter { def writer ->
templateConstantsFile.eachLine { def line ->
def newLine = line.replace("__PACKAGE", "net.codetojoy.util")
.replace("__CONTEXT_PATH", getContextPath())
.replace("__APP_PROPERTIES_PATH", getAppPropertiesPath())
writer.write(newLine + "\n");
}
}
}
and then the crucial env-specific predicates:
def getContextPath = { ->
def result = "default"
if (project.Environment == "prod") {
result = "PROD context path here"
} else if (project.Environment == "uat") {
result = "UAT context path here"
} else if (project.Environment == "dev") {
result = "DEV context path here"
}
return result
}
def getAppPropertiesPath = { ->
def result = "default"
if (project.Environment == "prod") {
result = "PROD app properties path here"
} else if (project.Environment == "uat") {
result = "UAT app properties path here"
} else if (project.Environment == "dev") {
result = "DEV app properties path here"
}
return result
}
Note that the ENV-specific values could easily be abstracted into config files, ENV vars, etc.
Also note that the example addresses: Java package used, project version, and build timestamp as well. I write something like this for most projects.

Why I'm not able to connect to HBase running as Docker container?

I have my Java Spring app that deals with HBase.
Here is my configuration:
#Configuration
public class HbaseConfiguration {
#Bean
public HbaseTemplate hbaseTemplate(#Value("${hadoop.home.dir}") final String hadoopHome,
#Value("${hbase.zookeeper.quorum}") final String quorum,
#Value("${hbase.zookeeper.property.clientPort}") final String port)
throws IOException, ServiceException {
System.setProperty("hadoop.home.dir", hadoopHome);
org.apache.hadoop.conf.Configuration configuration = HBaseConfiguration.create();
configuration.set("hbase.zookeeper.quorum", quorum);
configuration.set("hbase.zookeeper.property.clientPort", port);
HBaseAdmin.checkHBaseAvailable(configuration);
return new HbaseTemplate(configuration);
}
}
#HBASE
hbase.zookeeper.quorum = localhost
hbase.zookeeper.property.clientPort = 2181
hadoop.home.dir = C:/hadoop
Before asking the question I tried to figure out the problem on my own and found this link https://github.com/sel-fish/hbase.docker
But still, I get an error
org.apache.hadoop.net.ConnectTimeoutException: 10000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=myhbase/192.168.99.100:60000]
Could I ask you to help me and clarify how can I connect my local Java app with HBase running in Docker?

Spring Boot - How to start my Spring Boot Application from sister module in a multi-module project?

I have a multi-module project with two projects: backend and client. The backend is a normal Spring Boot Rest API, nothing special. The client module is just a Java Library using the Rest API.
The backend has packaging of "war" as the backend as it uses JSPs, too and needs to be deployed to a servlet container. The backend is still easily testable with #SpringBootTest.
Now I want to have some integration tests inside the client module using the backend module as a sandbox server.
To use all the backend classes in the client module I added
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<attachClasses>true</attachClasses>
</configuration>
</plugin>
and configured the backend as a test dependency in client with classes
In my client/src/test/java I have a helper class which starts up the backend module
#Configuration
public class SandboxServer {
#Bean
public ConfigurableApplicationContext backend() {
return
new SpringApplicationBuilder(BackendApplication.class)
.sources(SandboxServerConfig.class)
.run("spring.profiles.active=sandbox")
}
}
The profile "sandbox" is used to setup a test database etc. But I had more problems. First problem was regarding the document root, so I configured it:
public class SandboxServerConfig
implements WebServerFactoryCustomizer<TomcatServletWebServerFactory> {
#Override
public void customize(TomcatServletWebServerFactory factory) {
factory.setDocumentRoot(new File("../backend/src/main/webapp"));
}
}
But it still does not work as Spring is not picking up backend/src/main/resources/application.properties
That might be correct as it is not in the root classpath of the client module.
So it does not really work. I guess it is not possible to just start up the sibling module in an Integration test.
How can I achieve to start up the sibling spring boot module for integration testing? What is the best practice for szenarios like this?

You can override the application.properties location using TestPropertySource like this:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = BlaApplication.class)
#TestPropertySource(locations="/path/to/backend/src/main/resources/application.properties")
public class ExampleApplicationTests {
}

I found a much more solid solution. In my sibling Project "frontend" I have a Component which is starting up the backend server in integration mode if and only if it is not already running.
Benefits:
The real WAR is tested
You can start the WAR before in your IDE and let the tests run fast
If you run it with maven it is started up before all tests only once
No build configuration needed (like pre-integration in maven)
process is seperated from Junit runtime so no hassle with complex setups.
Drawbacks:
You need to build the package before you can run any integration test in the frontend. But hey, you should build your package before you test it. That's what integration test is all about.
And here is my SandboxServerProcess.class.
import org.springframework.stereotype.Component;
import javax.annotation.*;
import javax.annotation.*;
import java.io.*;
import java.net.*;
import java.util.*;
#Component
#Profile("integration")
public class SandboxServerProcess {
private static final String WAR = "../backend/target/backend.war";
private final static int PORT = 8081;
private boolean startedByMe;
#PostConstruct
public void start() throws Exception {
if (isStarted()) {
return;
}
testWarExists();
packagedWar("start");
if (waitForStartup()) {
startedByMe = true;
return;
}
throw new RuntimeException("Sandbox Server not started");
}
private void testWarExists() {
File file = new File(WAR);
if (!file.exists()) {
throw new RuntimeException("WAR does not exist:" + file.getAbsolutePath());
}
}
#PreDestroy
public void stop() throws IOException {
if (startedByMe) {
packagedWar("stop");
}
}
private void packagedWar(String command) throws IOException {
ProcessBuilder builder = new ProcessBuilder();
builder.environment().put("MODE", "service");
builder.environment().put("SPRING_PROFILES_ACTIVE", "integration");
builder.environment().put("APP_NAME", "backend");
builder.environment().put("PID_FOLDER", "./");
builder.environment().put("LOG_FOLDER", "./");
List<String> commands = new ArrayList<>();
commands.add(WAR);
commands.add(command);
builder.command(commands);
builder.inheritIO();
builder.redirectErrorStream(true);
builder.start();
}
private boolean isStarted() {
try {
Socket socket = new Socket();
InetSocketAddress sa = new InetSocketAddress("localhost", PORT);
socket.connect(sa, 500);
logger.warn("SandboxServer is started");
return true;
} catch (IOException e) {
return false;
}
}
private boolean waitForStartup() throws InterruptedException {
for (int i = 1; i < 30; i++) {
if (isStarted()) {
return true;
}
logger.warn("SandboxServer not yet ready, tries: " + i);
Thread.sleep(1000);
}
return false;
}
}

Configuration depending on launch mode

Play can be launched in dev mode (via run), in production mode (via start) or in test mode. Is there a way to provide a different config file (conf/application.conf) depending on which mode it is launched in?

I usually have a base configuration (application.conf) and three extra configs per environment. In Play Framework 2.4 it can be done by extending GuiceApplicationLoader and merging base conf with your environment specific conf. You can go one step forward and provide different guice modules per environment.
Scala version:
class CustomApplicationLoader extends GuiceApplicationLoader {
override protected def builder(context: Context): GuiceApplicationBuilder = {
val builder = initialBuilder.in(context.environment).overrides(overrides(context): _*)
context.environment.mode match {
case Prod =>
// start mode
val prodConf = Configuration(ConfigFactory.load("prod.conf"))
builder.loadConfig(prodConf ++ context.initialConfiguration).bindings(new ProdModule());
case Dev =>
// run mode
val devConf = Configuration(ConfigFactory.load("dev.conf"))
builder.loadConfig(devConf ++ context.initialConfiguration).bindings(new DevModule());
case Test =>
// test mode
val testConf = Configuration(ConfigFactory.load("test.conf"))
builder.loadConfig(testConf ++ context.initialConfiguration).bindings(new TestModule());
}
}
}
Java version:
public class CustomApplicationLoader extends GuiceApplicationLoader {
#Override
public GuiceApplicationBuilder builder(ApplicationLoader.Context context) {
final Environment environment = context.environment();
GuiceApplicationBuilder builder = initialBuilder.in(environment);
Configuration config = context.initialConfiguration();
if (environment.isTest()) {
config = merge("test.conf", config);
builder = builder.bindings(new TestModule());
} else if (environment.isDev()) {
config = merge("dev.conf", config);
builder = builder.bindings(new DevModule());
} else if (environment.isProd()) {
config = merge("prod.conf", config);
builder = builder.bindings(new DevModule());
} else {
throw new IllegalStateException("No such mode.");
}
return builder.in(environment).loadConfig(config);
}
private Configuration merge(String configName, Configuration currentConfig) {
return new Configuration(currentConfig.getWrappedConfiguration().$plus$plus(new play.api.Configuration(ConfigFactory.load(configName))));
}
}
Do not forget to include play.application.loader = "modules.CustomApplicationLoader" to your application.conf.
In lower versions of Play something similar can be achieved by using GlobalSettings class and overriding onLoadConfig. Mind GlobalSettings in Play 2.4 is depracted.
If you don't like including test.conf and test mocks from TestModule to your production build, you can filter the files with sbt.

You can set a different config file using one of the 3 ways play gives to you:
1 - Using -Dconfig.resource
It will search for an alternative configuration file in the
application classpath (you usually provide these alternative
configuration files into your application conf/ directory before
packaging). Play will look into conf/ so you don’t have to add conf/.
$ /path/to/bin/ -Dconfig.resource=prod.conf
2 - Using -Dconfig.file
You can also specify another local configuration file not packaged
into the application artifacts:
$ /path/to/bin/ -Dconfig.file=/opt/conf/prod.conf
3 - Using -Dconfig.url
You can also specify a configuration file to be loaded from any URL:
$ /path/to/bin/
-Dconfig.url=http://conf.mycompany.com/conf/prod.conf
Checkout more on:
https://www.playframework.com/documentation/2.3.x/ProductionConfiguration

This thing can be done by having loading config files based on the environment which can be supplied via -Dmode=staging/dev/prod, and for loading the files I override onLoadConfig of GlobalSettings in Global.java.
Java snippet-
#Override
public Configuration onLoadConfig(Configuration config, File file,ClassLoader classLoader) {
Configuration updatedConfig = config;
String mode = config.getString("mode");
if (StringUtils.isNotEmpty(mode)) {
try {
File modeFolder = FileUtils.getFile(file, "conf/" + mode);
if (modeFolder.exists()) {
play.api.Configuration modeConfig = config.getWrappedConfiguration();
IOFileFilter fileFilter = new WildcardFileFilter("*.conf");
Collection<File> fileList = FileUtils.listFiles(modeFolder, fileFilter, null);
for (File confFile : fileList) {
modeConfig = modeConfig
.$plus$plus(new play.api.Configuration(ConfigFactory.parseFile(confFile)));
}
updatedConfig = new Configuration(modeConfig);
}
} catch (Exception e) {
Logger.error("Exception while loading configuration for mode : " + mode, e);
}
} else {
Logger.error("Please provide mode in which play application has to start (Ex. play -Dmode=<mode>) ");
}
For each mode, create a folder(name same as environment) and keep environment specific config in that folder.

JMX process, is it possible to call an external application to handle access rights when client attempts access

I have an application that is running on localhost:1234, I am using jconsole to connect to this. The application has a password file to handle login.
I need to allow logging in based on different AD groups of the windows user. So for example, if they are in Group1 they will be given readwrite access, if they are Group2 they are given readonly access, and group3 is not given and access.
I have created an AD group handling application that can query a list of AD groups and return the required user access level and login details.
My problem: I want to connect to the application using jconsole via the command line using something like:
jconsole localhost:1234
Obviously this will fail to connect, because it's expecting a username and password.
Is there a way in which I can have my JMX application that's running on localhost:1234 wait for an incoming connection request and run my AD group handling application to determine their access level?
My application on localhost:1234 is very basic and looks like this:
import java.lang.management.ManagementFactory;
import javax.management.InstanceAlreadyExistsException;
import javax.management.MBeanRegistrationException;
import javax.management.MBeanServer;
import javax.management.MalformedObjectNameException;
import javax.management.NotCompliantMBeanException;
import javax.management.ObjectName;
public class SystemConfigManagement {
private static final int DEFAULT_NO_THREADS = 10;
private static final String DEFAULT_SCHEMA = "default";
public static void main(String[] args)
throws MalformedObjectNameException, InterruptedException,
InstanceAlreadyExistsException, MBeanRegistrationException,
NotCompliantMBeanException{
//Get the MBean server
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
//register the mBean
SystemConfig mBean = new SystemConfig(DEFAULT_NO_THREADS, DEFAULT_SCHEMA);
ObjectName name = new ObjectName("com.barc.jmx:type=SystemConfig");
mbs.registerMBean(mBean, name);
do{
Thread.sleep(2000);
System.out.println(
"Thread Count = " + mBean.getThreadCount()
+ ":::Schema Name = " + mBean.getSchemaName()
);
}while(mBean.getThreadCount() != 0);
}
}
and
package com.test.jmx;
public class SystemConfig implements SystemConfigMBean {
private int threadCount;
private String schemaName;
public SystemConfig(int numThreads, String schema){
this.threadCount = numThreads;
this.schemaName = schema;
}
#Override
public void setThreadCount(int noOfThreads) {
this.threadCount = noOfThreads;
}
#Override
public int getThreadCount() {
return this.threadCount;
}
#Override
public void setSchemaName(String schemaName) {
this.schemaName = schemaName;
}
#Override
public String getSchemaName() {
return this.schemaName;
}
#Override
public String doConfig() {
return "No of Threads=" + this.threadCount + " and DB Schema Name = " + this.schemaName;
}
}
[source : http://www.journaldev.com/1352/what-is-jmx-mbean-jconsole-tutorial]
Is there somewhere in main() where I can create this query to validate the user details using the AD group handling application?

The default RMI connector server cannot do that very well (you can provide your own JAAS module (UC3) or Authenticator (UC4)).
You might be better off using another protocol/implementation which does already delegate authentication. There are some webservice, REST- and even jboss remoting connectors and most of them can be authenticated via a container mechanism. However I think most of them are not easy to integrate.
If you use for example Jolokia (servlet), you could also use hawt.io as a very nice "AJAX" console. (I am not sure if jolokia actually ships a JMX client connector which you can use in JConsole but there are many alternative clients which are most of the time better for integration/automation).