Develop Reference

Parse a single POJO from multiple YAML documents representing different classes

I want to use a single YAML file which contains several different objects - for different applications. I need to fetch one object to get an instance of MyClass1, ignoring the rest of docs for MyClass2, MyClass3, etc. Some sort of selective de-serializing: now this class, then that one... The structure of MyClass2, MyClass3 is totally unknown to the application working with MyClass1. The file is always a valid YAML, of course.
The YAML may be of any structure we need to implement such a multi-class container. The preferred parsing tool is snakeyaml.
Is it sensible? How can I ignore all but one object?
UPD: replaced all "document" with "object". I think we have to speak about the single YAML document containing several objects of different structure. More of it, the parser knows exactly only 1 structure and wants to ignore the rest.
UDP2: I think it is impossible with snakeyaml. We have to read all objects anyway - and select the needed one later. But maybe I'm wrong.
UPD2: sample config file
---
-
exportConfiguration781:
attachmentFieldName: "name"
baseSftpInboxPath: /home/user/somedir/
somebool: false
days: 9999
expected:
- ABC w/o quotes
- "Cat ABC"
- "Some string"
dateFormat: yyyy-MMdd-HHmm
user: someuser
-
anotherConfiguration:
k1: v1
k2:
- v21
- v22

This is definitely possible with SnakeYAML, albeit not trivial. Here's a general rundown what you need to do:
First, let's have a look what loading with SnakeYAML does. Here's the important part of the YAML class:
private Object loadFromReader(StreamReader sreader, Class<?> type) {
Composer composer = new Composer(new ParserImpl(sreader), resolver, loadingConfig);
constructor.setComposer(composer);
return constructor.getSingleData(type);
}
The composer parses YAML input into Nodes. To do that, it doesn't need any knowledge about the structure of your classes, since every node is either a ScalarNode, a SequenceNode or a MappingNode and they just represent the YAML structure.
The constructor takes a root node generated by the composer and generates native POJOs from it. So what you want to do is to throw away parts of the node graph before they reach the constructor.
The easiest way to do that is probably to derive from Composer and override two methods like this:
public class MyComposer extends Composer {
private final int objIndex;
public MyComposer(Parser parser, Resolver resolver, int objIndex) {
super(parser, resolver);
this.objIndex = objIndex;
}
public MyComposer(Parser parser, Resolver resolver, LoaderOptions loadingConfig, int objIndex) {
super(parser, resolver, loadingConfig);
this.objIndex = objIndex;
}
#Override
public Node getNode() {
return strip(super.getNode());
}
private Node strip(Node input) {
return ((SequenceNode)input).getValue().get(objIndex);
}
}
The strip implementation is just an example. In this case, I assumed your YAML looks like this (object content is arbitrary):
- {first: obj}
- {second: obj}
- {third: obj}
And you simply select the object you actually want to deserialize by its index in the sequence. But you can also have something more complex like a searching algorithm.
Now that you have your own composer, you can do
Constructor constructor = new Constructor();
// assuming we want to get the object at index 1 (i.e. second object)
Composer composer = new MyComposer(new ParserImpl(sreader), new Resolver(), 1);
constructor.setComposer(composer);
MyObject result = (MyObject)constructor.getSingleData(MyObject.class);

The answer of #flyx was very helpful for me, opening the way to workaround the library (in our case - snakeyaml) limitations by overriding some methods. Thanks a lot! It's quite possible there is a final solution in it - but not now. Besides, the simple solution below is robust and should be considered even if we'd found the complete library-intruding solution.
I've decided to solve the task by double distilling, sorry, processing the configuration file. Imagine the latter consisting of several parts and every part is marked by the unique token-delimiter. For the sake of keeping the YAML-likenes, it may be
---
#this is a unique key for the configuration A
<some YAML document>
---
#this is another key for the configuration B
<some YAML document
The first pass is pre-processing. For the given String fileString and String key (and DELIMITER = "\n---\n". for example) we select a substring with the key-defined configuration:
int begIndex;
do {
begIndex= fileString.indexOf(DELIMITER);
if (begIndex == -1) {
break;
}
if (fileString.startsWith(DELIMITER + key, begIndex)) {
fileString = fileString.substring(begIndex + DELIMITER.length() + key.length());
break;
}
// spoil alien delimiter and repeat search
fileString = fileString.replaceFirst(DELIMITER, " ");
} while (true);
int endIndex = fileString.indexOf(DELIMITER);
if (endIndex != -1) {
fileString = fileString.substring(0, endIndex);
}
Now we feed the fileString to the simple YAML parsing
ExportConfiguration configuration = new Yaml(new Constructor(ExportConfiguration.class))
.loadAs(fileString, ExportConfiguration.class);
This time we have a single document that must co-respond to the ExportConfiguration class.
Note 1: The structure and even the very content of the rest of configuration file plays absolutely no role. This was the main idea, to get independent configurations in a single file
Note 2: the rest of configurations may be JSON or XML or whatever. We have a method-preprocessor that returns a String configuration - and the next processor parses it properly.

Creating configurations interface for Model Project (JAVA)

Currently I am creating a java Project that will be used by many people to (in theory) create a product, each with their own set of configurations, in a straightforward, fast and with standardized mechanisms.
If I were using a GUI, it would be easier to handle the configurations in different files, however, because I am making it just an editable project, I see myself in the need of putting all the constants in a single class/file so as to centralize the area where the respective engineer configures the product he is creating. Thus I have the following Constants interface:
public interface Constants {
//ROUTE OF FOLDERS
String PATH_ACTUALIZ = "..\\actualiz\\";
String PATH_ENTRADAS = "..\\entries\\";
String PATH_SALIDAS = "..\\outputs\\";
String PATH_ENTRADAS_CARTAS = PATH_ENTRADAS + "CARTAS\\";
String PATH_ENTRADAS_ANEXOS = PATH_ENTRADAS + "ANEXOS\\";
String PATH_SALIDAS_PDF = PATH_SALIDAS + "PDF\\";
//ROUTE OF FILES
String PATH_FILE_SPOOL_CLIENTE = PATH_ENTRADAS + "spool.txt";
String PATH_FILE_SPOOL_ERRORES = PATH_SALIDAS + "Reporte_de_Errores.txt";
String PATH_FILE_BASE_EMAIL = PATH_SALIDAS + "BaseEmail_[OP].txt";
String PATH_FILE_DATA_VAR = PATH_SALIDAS + "DataVar_[OP].txt";
String PATH_FILE_BASE = PATH_SALIDAS + "Base.obj";
String NAME_FILE_LOG = "Log.txt";
//DESIGNER
String DESIGNER_DELIMITER = "\t"; //Other options "»", "|"
String DESIGNER_CANAL = "AA01";
//GUI INFORMATION
String NAME_RESPONSABLE = "JOHN DOE";
String TITULO_MENSAJE_ERROR = "Error when processing the data.";
String TITULO_MENSAJE_INFORMACION = "Processing data";
//HTML
String HTML = "123";
//SEGMENTS
HashMap<String, String> HMSEGMENTO = new HashMap<String, String>(); //Initialization missing.
}
The last line of which is a HashMap because the data involves several pairs of String to String, including one that is returned by default if the key provided isn't found, however I find myself at a standstill because there is no elegant/simple way add data to a Hashmap inside an Interface.
Other solutions I've looked into include the use of Java Properties, but I also wouldn't be able to add data to it in this interface.
So my larger question becomes if I'm even doing this right at all. Any sources or ideas on software development that would direct to the right path would be greatly appreciated.
Finally I understand my question is slightly broad. If there is a way to make it more specific, I would appreciate the help too.

Serious non-answer: don't do this.
Java invented Properties to exactly give you that: the ability to specify runtime configuration information in text files. And when you need multiple layers - that is also possible using property files (you could for example have a base property file that defines useful defaults, and then different people can provide their own property files that "override" those properties that they need to be changed).
And when you insist on using interfaces to "wrap" your constants (although exactly that is considered a bad practice) - please don't stuff everything into the same interface. Instead identify the different "categories" you need and have one file per category.
Your approach is determined to end up with monolithic spaghetti code.

Stanford-NER customization to classify software programming keywords

I am new in NLP and I used Stanford NER tool to classify some random text to extract special keywords used in software programming.
The problem is, I don't no how to do changes to the classifiers and text annotators in Stanford NER to recognize software programming keywords. For example:
today Java used in different operating systems (Windows, Linux, ..)
the classification results should such as:
Java "Programming_Language"
Windows "Operating_System"
Linux "Operating_system"
Would you please help on how to customize the StanfordNER classifiers to satisfied my needs?

I think it is quite well documented in Stanford NER faq section http://nlp.stanford.edu/software/crf-faq.shtml#a.
Here are the steps:
In your properties file change the map to specify how your training data is annotated (or
structured)
map = word=0,myfeature=1,answer=2
In src\edu\stanford\nlp\sequences\SeqClassifierFlags.java
Add a flag stating that you want to use your new feature, let's call it useMyFeature
Below public boolean useLabelSource = false , Add
public boolean useMyFeature= true;
In same file in setProperties(Properties props, boolean printProps) method after
else if (key.equalsIgnoreCase("useTrainLexicon")) { ..} tell tool, if this flag is on/off for you
else if (key.equalsIgnoreCase("useMyFeature")) {
useMyFeature= Boolean.parseBoolean(val);
}
In src/edu/stanford/nlp/ling/CoreAnnotations.java, add following
section
public static class myfeature implements CoreAnnotation<String> {
public Class<String> getType() {
return String.class;
}
}
In src/edu/stanford/nlp/ling/AnnotationLookup.java in
public enumKeyLookup{..} in bottom add
MY_TAG(CoreAnnotations.myfeature.class,"myfeature")
In src\edu\stanford\nlp\ie\NERFeatureFactory.java, depending on the
"type" of feature it is, add in
protected Collection<String> featuresC(PaddedList<IN> cInfo, int loc)
if(flags.useRahulPOSTAGS){
featuresC.add(c.get(CoreAnnotations.myfeature.class)+"-my_tag");
}
Debugging:
In addition to this, there are methods which dump the features on file, use them to see how things are getting done under hood. Also, I think you would have to spend some time with debugger too :P

Seems you want to train your custom NER model.
Here is a detailed tutorial with full code:
https://dataturks.com/blog/stanford-core-nlp-ner-training-java-example.php?s=so
Training data format
Training data is passed as a text file where each line is one word-label pair. Each word in the line should be labeled in a format like "word\tLABEL", the word and the label name is separated by a tab '\t'. For a text sentence, we should break it down into words and add one line for each word in the training file. To mark the start of the next line, we add an empty line in the training file.
Here is a sample of the input training file:
hp Brand
spectre ModelName
x360 ModelName
home Category
theater Category
system 0
horizon ModelName
zero ModelName
dawn ModelName
ps4 0
Depending upon your domain, you can build such a dataset either automatically or manually. Building such a dataset manually can be really painful, tools like a NER annotation tool can help make the process much easier.
Train model
public void trainAndWrite(String modelOutPath, String prop, String trainingFilepath) {
Properties props = StringUtils.propFileToProperties(prop);
props.setProperty("serializeTo", modelOutPath);
//if input use that, else use from properties file.
if (trainingFilepath != null) {
props.setProperty("trainFile", trainingFilepath);
}
SeqClassifierFlags flags = new SeqClassifierFlags(props);
CRFClassifier<CoreLabel> crf = new CRFClassifier<>(flags);
crf.train();
crf.serializeClassifier(modelOutPath);
}
Use the model to generate tags:
public void doTagging(CRFClassifier model, String input) {
input = input.trim();
System.out.println(input + "=>" + model.classifyToString(input));
}
Hope this helps.

Access to MessageResources

What's the analogous of
Resources.getMessage(request,"key");
(it's in org.apache.struts.validator package)?
I need to take the right string according to the language in use and put that string in an array.
These strings have a variable part. For example:
The field %s is required
I want to take the The field and is required from the MessageResources
EDIT
I've used
ResourceBundle myResources = ResourceBundle.getBundle("MessageResources");
String msg = MessageFormat.format(myResources.getString("errors.required"),
new Object[] { title });
It works, but only with the default language

You can use a resource bundle for this in idiomatic Java. The bundles are properties files, and they are named according to their locale's.
Here is a link to ResourceBundle in the Java API documentation.
http://docs.oracle.com/javase/7/docs/api/java/util/ResourceBundle.html
Let me know if you need any more info.

Do resource bundles in Java support runtime string substitution?

Can you do the following with a Java ResourceBundle?
In the properties file...
example.dynamicresource=You currently have {0} accounts.
At runtime...
int accountAcount = 3;
bundle.get("example.dynamicresource",accountCount,param2,...);
To give a result of
"You currently have 3 accounts."

Not without using the MessageFormat class, such as:
String pattern = bundle.getString("example.dynamicresource");
String message = MessageFormat.format(pattern, accountCount);

On their own, ResourceBundle does not support property placeholders. The usual idea is to take the String you get from the bundle, and stick it into a MessageFormat, and then use that to get your parameterized message.
If you're using JSP/JSTL, then you can combine <fmt:message> and <fmt:param> to do this, which uses ResourceBundle and MessageFormat under the covers.
If you happen to be using Spring, then it has the ResourceBundleMessageSource which does something similar, and can be used anywhere in your program. This MessageSource abstraction (combined with MessageSourceAccessor) is much nicer to use than ResourceBundle.

There are various ways, depending on the view technology you're using. If you're using "plain vanilla" Java (e.g. Swing), then use MessageFormat API as answered before. If you're using a webapplication framework (which is true, if I judge your question history here correctly), then the way depends on the view technology and/or MVC framework you're using. If it is for example "plain vanilla" JSP, then you can use JSTL fmt:message for this.
<fmt:message key="example.dynamicresource">
<fmt:param value="${bean.accountCount}">
</fmt:message>
If it is for example JSF, you can use h:outputFormat for this.
<h:outputFormat value="#{bundle['example.dynamicresource']}">
<f:param value="#{bean.accountCount}">
</h:outputFormat>
Best place is to just consult the documentation of the technology/framework you're using (or to tell it here so that we can give better suited and more detailed answers).

Struts have a nice util called MessageResources which does exactly what you ask for....
e.g.
MessageResources resources = getResources(request, "my_resource_bundle"); // Call your bundle exactly like ResourceBundle.getBundle() method
resources.getMessage("example.dynamicresource",accountCount,param2,...);
Limitation
It only allows maximum of 3 parameters (i.e. resource attribute, param1, ..., param3).
I suggest using MessageFormat (if you want to use more than 3 parameter values) as suggested by David Sykes.
PS the getResources method is available only in the Struts Action class.

I don't think you can make this work for Non-English properties file.
My message.properties file has the following line:
info.fomat.log.message.start=Starting to parse log message in {0} format.
And my message_fr_FR.properties file has the following line:
info.fomat.log.message.start=A partir d'analyser le message connecter {0} format.
This code works only for the English one
String.format((String) messages .getString(GlobalConstants.MESSAGE_FORMAT_START), GlobalConstants.STR_JSON));
It does NOT replace the placeholder with the value when my language / locale is French :-(
Even MessageFormat.fomat() is no good

I don't believe ResourceBundle can do that itself, but String can:
String.format(bundle.getString("example.dynamicresource"), accountCount);

Remember that when using MessageFormat.format() you need to use a double quote ('') in your resource bundle if you want to express single quote (').

MessageFormoat#format will work for the case like:
greetingTo=Have Param, saying hello {0}
You can declare two methods like this where RB is a instance of ResourceBundle:
/**This is a method that takes the param to substitute the placeholder**/
public String getString(String key, Object... params ) {
try {
return MessageFormat.format(this.RB.getString(key), params);
} catch (MissingResourceException e) {
return "[" + key + "]";
}
}
/**Without a param, this will derectly delegate to ResourceBundle#getString**/
public String getString(String key) {
try {
return this.RB.getString(key);
} catch (MissingResourceException e) {
return "[" + key + "]";
}
}