Where is ${SHDP_AMSERVICE_PORT} for DefaultMindAppmasterServiceClient defined? - java

In the YARN Documentation, Section 11.12.2 Partitioning on Configuring Container the
DefaultMindAppmasterServiceClient is setup by the following:
<yarn-int:amservice-client
service-impl="org.springframework.yarn.integration.ip.mind.DefaultMindAppmasterServiceClient"
host="${SHDP_AMSERVICE_HOST}"
port="${SHDP_AMSERVICE_PORT}" />
How is the port SHDP_AMSERVICE_PORT defined for the amservice?

Your link includes this,
Through Spring's property placeholder support, SpEL and the environment abstraction (available in Spring 3.1). one can externalize environment specific properties from the main code base easing the deployment across multiple machines.
It also mentions support for Properties files. But it appears that it supports environment variables. On *nix type systems you might define it like
export SHDP_AMSERVICE_PORT=1234 # for example
On Windows that would look like
set SHDP_AMSERVICE_PORT 1234
Edit
Based on your comment I downloaded it and it's used in container-context.xml under
yarn/yarn/custom-amservice/src/main/resources/container-context.xml
yarn/yarn/batch-files/src/main/resources/container-context.xml
yarn/yarn/batch-partition/src/main/resources/container-context.xml
Nothing defines it, so it uses a default value. You can override it as above.

Related

12 factor app config and Java

I was reading the 12 factor app manifesto http://12factor.net/. The manifesto recommends storing the configuration data for the application in Enviornment variables. Does this mean that properties like the DB username / password, resource URL should be stored as a part of Java Env variables rather than as property files ? Is this a secure way of storing the information ? To me this seems to be a pretty clunky way of storing the information.
Are there any best practices / experiences around this that can be shared ?
One option that I can think of is to have a separate configuration service running in the landscape, and use Env property to connect to the config service and then query the config service for further detailed configuration data.
12 factor apps are designed to run on platforms that orchestrate isolated UNIX processes. UNIX processes are configured via environment variables. While property files are a well-established Java convention, UNIX processes are a language-agnostic way to configure processes.
To support multiple configuration methods, a good best practice is to:
Read from process environment with System.getenv('CONFIG'), if null
Read from property file with properties.getProperty('CONFIG'), if null
Fall back to a default value
For more details, see Heroku's instructions on defining config vars for Java apps.
We can use Spring Centralized Configuration to do that, using centralized configuration we can commit configuration of all of our projects into a single repository and later on while writing build scripts we can override our local configuration from that repository to use that centralised configuration.
By clicking on below link you will find getting started guide to do so
https://spring.io/guides/gs/centralized-configuration/
This piece on How To Implement 12 Factor Configuration In Java maybe helpful: https://blog.codacy.com/12-factor-config-for-java/
It's published by Codacy, the automated code review tool.

Logback: out-of-the-box properties

Where should I look to see the default logback properties, e.g. user.home?
It is great that I can use external file, System and environment to load properties, but are there any out-of-box ones?
user.home is a Java system property, not a logback property. You can see the rest of them with System.out.println(System.getProperties()); or using JConsole on a running JVM.
Apart from that, I think there are only HOSTNAME and CONTEXT_NAME defined by Logback: http://logback.qos.ch/manual/configuration.html#variableSubstitution
user.home is not logback specific, they come from java.
See http://docs.oracle.com/javase/tutorial/essential/environment/sysprop.html

property-placeholder location from another property (Spring 3.1)

I need to load an environment-specific property file, and I'd like to be able to both set it from the JVM (using -D) and to provide a default value in the main properties file or, failing that, somewhere else (like the applicationContext.xml)
I'm using the new hotness Spring 3.1 with its unified property management, but I can't find a lot of info on the property system.
UPDATE:
To clarify:
<context:property-placeholder location="/WEB-INF/myapp.properties,
/WEB-INF/myapp-${deploy.env}.properties"/>
You can do it using Spring 3.1, JVM property will be put into placeholders and you can define default values using ":", for example:
${property1:defValue}
where defValue is default value, it can be overridden by JVM option -Dproperty1=newValue
You should read this API - it is pretty informative. Example here.
EDIT
As the example points to outdated version of Spring the more modern approach is illustrated here

Existing implementations of OSGi Configuration Admin Service?

We are considering to use Configuration Admin Service as a primary API for configuring components in our OSGi-based application. It would be nice if we could reuse some existing implementation so I'm trying to investigate and evaluate the most popular ones. I know there is:
Apache Felix Config Admin (org.apache.felix.cm)
Equinox Config Admin (org.eclipse.equinox.cm)
Are there any other implementations to be considered?
Also I was not able to find any good documentation for these implementations. I would be mainly interested in the implementation-specific details. For example I was wondering how different implementations persist the configuration data (e.g. multiple property files? XML file? multiple XML files? database?, ...).
Felix's Configuration Admin has a default implementation that persists to the file system, but they define a service interface (org.apache.felix.cm.PersistenceManager) for alternative backends that you could plug in instead.
The default implementation does the following:
The FilePersistenceManager class stores configuration data in
properties-like files inside a given directory. All configuration files are
located in the same directory.
Configuration files are created in the configuration directory by appending
the extension ".config" to the PID of the configuration. The PID
is converted into a relative path name by replacing enclosed dots to slashes.
Non-symbolic-name characters in the PID are encoded with their
Unicode character code in hexadecimal.
The three public implementations I know of are
Apache Felix
Equinox …source (this has moved recently)
Knopflerfish …front page and …source
Equinox's implementation of the ConfigurationAdmin service appears not to support fine control over the persistence policy, as Felix's does, and the Knopflerfish implementation looks (I've only read the source briefly) similar to Equinox's.
The Felix one appears to be the most recently updated and the most reliable.
At present these are the only ones I can find; at dm Server we made the decision to use Felix's bundle, and this is now obtainable from the SpringSource Enterprise Bundle Repository, where you can quick-search for Apache Felix or ConfigAdmin.
Just to complete the answer further: I personally also prefer the Felix implementation. For an example of how to change the way storage occurs at the back-end using a PersistenceManager, see also this implementation that uses standard Java property files as backing storage. Has some limitations, but at least allows you to store your configuration with your application and apart from your OSGi framework implementation.

How to Use Different META-INF/context.xml Files For Development and Production Environments

In Tomcat (and some other servlet containers) I can store information about my JDBC DataSource in META-INF/context.xml. This is very useful.
However, the settings for my JDBC DataSource can be different in my development and production environments. I'd like to know how other people deal with these differences in an elegant way, specifically how can I set up a context.xml for my development environment and one for my production environment in the most hassle-free manner.
You can create different files for specific builds. For example, create:
development.context.xml
production.context.xml
Then, you can control which context file is used in your build.xml file. Basically, setup a prompt for which type of build you would like to use. When you select development, it uses the development context file. When you select production, it uses the production context file.
I would do the same as Kevin mentioned. If you're using Maven you would use "profiles".
If you want to learn more about Maven profiles read this: Introduction to Build Profiles
Personally I wouldn't store configuration information like that in context.xml (perhaps in another properties file or something), but the general way for something like this is to have your build script package different versions of the configuration file into the WAR/EAR/whatever. You could have your build script decide whether to use the "dev" or "production" configuration file based on parameters you pass in, running different targets, etc.
Something I use often is the task in ant to replace certain tokens in files with values from a filters file; and swap which filters file is used depending on which environment I am targeting.

Categories

Resources