Develop Reference

Modifying YAML in java while preserving comments

How can we modify an existing YAML and preserve comments in it.
Is there any Java parser which does that ?
For example if i have following YAML:
#This is a test YAML
name: abcd
age: 23
#Test YAML ends here.
Is there a way I can edit this Yaml using a java parser and preserve the comments.

As of the time of writing, there is no round-tripping YAML parser for Java. There is the well-known SnakeYAML, which does not preserve comments (see the author's comment here), and a newer project named camel, which I know little of; but it definitely is not round-tripping.
What you can theoretically do is to use SnakeYaml's Yaml.parse and then iterate over the events. Each event has a start and an end mark, giving the start and end line & column of the event. This makes it possible to map the events back into the source and discover the portions of the source that were not parsed into events (presumably comments). Having this mapping, you can now modify the event list and write it back. Finally, you read the result in a second time and discover the gaps between your events where there were comments in the original YAML, but not in the modified YAML, and re-insert those comments, giving you the final YAML with your modifications and the comments.
Of course, this is very complex. I would not advice to do it unless you a) have either a solid understanding of how YAML is structured or are willing to learn it, and b) your use-case justifies this amount of work.

I wrote a groovy script to solve this. The Java version is very similar:
def key = "name"
def value = "efgh"
def yamlFile = new File("file.yaml")
def yamlFileLines = new StringBuilder()
def foundKey = false
yamlFile.text.eachLine { line ->
if (!foundKey && line.contains("$key:")) {
line = line.replaceAll(/$key:.*/, "$key: $value")
foundKey = true
}
yamlFileLines.append("$line\n")
}
if (foundKey) {
yamlFile.text = yamlFileLines.toString()
} else {
throw new StopExecutionException("Could not find key '$key' in file ${yamlFile.getAbsolutePath()}")
}

if you use snakeyaml , you should modify the ScannerImpl file.
notice: read the in-line comment as text
private Token scanPlain() {
StringBuilder chunks = new StringBuilder();
Mark startMark = reader.getMark();
Mark endMark = startMark;
int indent = this.indent + 1;
String spaces = "";
while (true) {
int c;
int length = 0;
// A comment indicates the end of the scalar.
// read the in-line comment as text
// if (reader.peek() == '#' && ) {
// break;
// }
while (true) {
c = reader.peek(length);
if (Constant.NULL_BL_T_LINEBR.has(c)
|| (c == ':' && Constant.NULL_BL_T_LINEBR.has(reader.peek(length + 1), flowLevel != 0 ? ",[]{}":""))
|| (this.flowLevel != 0 && ",?[]{}".indexOf(c) != -1)) {
break;
}
length++;
}
if (length == 0) {
break;
}
this.allowSimpleKey = false;
chunks.append(spaces);
chunks.append(reader.prefixForward(length));
endMark = reader.getMark();
spaces = scanPlainSpaces();
// System.out.printf("spaces[%s]\n", spaces);
if (spaces.length() == 0
// read the in-line comment as text
// || reader.peek() == '#'
|| (this.flowLevel == 0 && this.reader.getColumn() < indent)) {
break;
}
}
return new ScalarToken(chunks.toString(), startMark, endMark, true);
}

can't delete from getContentResolver().delete

I can't delete a conversation from getContentResolver, I don't know in which part am doing mistakes, as I also searched about these but can't help myself and I also tried different sols which were given on stackoverflow but same result & thanks a lot in advance.
Here is the code:
public static boolean deleteSmsofContact(Context context, String number,
boolean deleteLocked)
{
int result;
if (deleteLocked) {
//changes values
String[] selectionArgs=new String[]{number};
String selection= ""+"address=?";
//
result = context.getContentResolver().delete(Uri.parse("content://sms/"),selection,selectionArgs);
// Log.d("UF","WOW "+result+" " +number);
} else {
result = context.getContentResolver().delete(Constants.URI_SMS,
"address=? AND locked=?", new String[] { number, "1" });
}
if (result > 0) {
return true;
}
return false;
}
Here is the method from which I am calling:
boolean result = Utils.deleteSmsofContact(InboxActivity.this, sms.getNumber(), true);
if (result) {
dataList.remove(threadPosition);
iAdapter.notifyDataSetChanged();
Toast.makeText(InboxActivity.this,"Removed",Toast.LENGTH_LONG).show();
}else
{
Toast.makeText(InboxActivity.this,"cant removed",Toast.LENGTH_LONG).show();
}

Well I posted it but did not get the answer so finally I searched a lot on this and the correct answer is until or unless your app is not set a default you can't delete any sms or whole conversation.
Follow this link it will make your app set a default OR you will be able to delete.

Getting skipped whitespases in ANTLR parser

I have some grammar, that ignores whitespaces in following way
WS : [ \r\t\n]+ -> channel(HIDDEN) ;
It's ok, 'cos whitespace isn't part of my grammar. But in parser I need to know where whitespaces was. For now I unable to find any straight way to do this.
I use last version of ANTLR4
Thanks in advance.

in v3 you would do something like that if you're looking for a token while parsing the tree:
getPreviousTokenInHiddenChannel(retval, input);
public String getPreviousTokenInHiddenChannel(TreeRuleReturnScope retval, TreeNodeStream input) {
try {
TokenStream tstream = input.getTokenStream();
CommonTree node = (CommonTree) retval.start;
int boundary = node.getTokenStopIndex();
if (boundary <= 0) { // fix for antlr 3.3 bug, from 3.5 getTokenStartIndex should itself resolve parent's boundaries if <= 0
while (node.getTokenStartIndex() == -1) { // if node is imaginary
node = (CommonTree) node.getParent();
if (node == null) return ""; // means we are root
boundary = node.getTokenStopIndex();
if (boundary > 0) break;
}
}
int i = boundary;
while (true) {
i--;
Token tok = tstream.get(i);
if (tok.getChannel() == HIDDEN) {
// do what you want to do https://www.youtube.com/watch?v=JgRBkjgXHro
}
}
} catch (Exception e) {
// handle e
}
}
You can easily adapt that piece of code for v4 with something like that (pseudocode):
BufferedTokenStream bts;
// retrieve bts
List<Token> hiddenTokens = bts.getHiddenTokensToLeft(bts.index(), HIDDEN);
// loop backwards over the list
for (int i = hiddenTokens.size(); i--; i >= 0) {
Token t = hiddenTokens.get(i)
// process your hidden token
}

See Token stream API
You must get used to looking at the API and source code. You can also buy the book cheaply. Page 206: Accessing Hidden Channels.

Parsing XML with StAX with non-unique tag paths, design suggestions

I need to parse a large XML file (probably going to use StAX in Java) and output it into a delimited text file and I have a couple of design questions. First here is an example of the XML
<demographic>
<value>001</value>
<question>Name?</question>
<value>Bob</value>
<question>Last Name?</question>
<value>Smith</value>
<followUpQuestions>
<question>Middle Init.</question>
<value>J</value>
</followUpQuestions>
</demographic>
this would need to be outputted (in the delimited output file) as
001~Bob~Smith~J
so here are my questions:
How can I distinguish between all the different "value" tags, since the tag names are not unique. Currently I tried to resolve this by having 'state' variables that turn on once they pass question-text such as "Name?", however this approach doesnt really work for the first value since I have to check to make sure the 'name' and 'lastName' states are off to ensure I'm getting the first value.
Everytime the client changes the text of the questions (which happens) I have to change the code and recompile it. Is there anyway to avoid this? Maybe save the questions-text in a text file that the program reads in?
Can this be scalable? I need to extract over 100 values and the XML files are usually about 2 gigs large.
Thank you, in advance, for your help (from a Java and XML newbie)!!
UPDATE: here is my attempt to code the solution, can someone please help to streamline? There has to be a less messy way to do this:
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamConstants;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamReader;
import java.io.*;
class TestJavaForStackOverflow{
boolean nameState = false,
lastNameState = false,
middleInitState = false;
String name = "",
lastName = "",
middleInit = "",
value = "";
public void parse() throws IOException, XMLStreamException{
XMLInputFactory factory = XMLInputFactory.newInstance();
XMLStreamReader streamReader = factory.createXMLStreamReader(
new FileReader("/n04/data/revmgmt/anthony/scripts/Java_Programs/TestJavaForStackOverflow.xml"));
while(streamReader.hasNext()){
streamReader.next();
if(streamReader.getEventType() == XMLStreamReader.START_ELEMENT){
if("demographic".equals(streamReader.getLocalName())){
parseDemographicInformation(streamReader);
}
}
}
System.out.println(value + "~" + name + "~" + lastName + "~" + middleInit);
}
public void parseDemographicInformation(XMLStreamReader streamReader) throws XMLStreamException {
while(streamReader.hasNext()){
streamReader.next();
if(streamReader.getEventType() == XMLStreamReader.END_ELEMENT){
if("demographic".equals(streamReader.getLocalName())){
return;
}
}
else if(streamReader.getEventType() == XMLStreamReader.START_ELEMENT){
if("question".equals(streamReader.getLocalName())){
streamReader.next();
if("Name?".equals(streamReader.getText())){
nameState = true;
}
else if("Last Name?".equals(streamReader.getText())){
lastNameState = true;
}
else if("Middle Init.".equals(streamReader.getText())){
middleInitState = true;
}
}
else if("value".equals(streamReader.getLocalName())){
streamReader.next();
if(nameState){
name = streamReader.getText();
nameState = false;
}
else if (lastNameState){
lastName = streamReader.getText();
lastNameState = false;
}
else if (middleInitState){
middleInit = streamReader.getText();
middleInitState = false;
}
else {
value = streamReader.getText();
}
}
}
}
}
public static void main(String[] args){
TestJavaForStackOverflow t = new TestJavaForStackOverflow();
try{t.parse();}
catch(IOException e1){}
catch(XMLStreamException e2){}
}
}

I think the flags are not very scalable if you have a lot of different questions to parse, and neither are the global variables to hold the results... if you have 100 questions then you'll need 100 variables, and when they change over time it will be a bear to keep them up to date. I would use a map structure to hold the result, and another one to hold the correspondence between each question text and the corresponding field you are trying to capture (this is not actual Java, just an approximation):
public Map parseDemographicInformation(XmlStream xml, Map questionMap) {
Map record = new Map();
String field = "id";
while((elem = xml.getNextElement())) {
if(elem.tagName == "question") {
field = questionMap[elem.value];
} else if(elem.tagName == "value") {
record[field] = elem.value;
}
}
return record;
}
Then you have something like this to output the result:
String[] fieldsToOutput = { "id", "firstName", "lastName" }; // ideally read this from a file too so it can be changed dynamically
// ...
for(int i=0; i < fieldsToOutput.length; i++){
if(i > 0)
System.out.print("~");
System.out.print(record[fieldsToOutput[i]]);
}
System.out.println();

EMV TLV Java Function

I'm looking for a way to translate an EMV response with Java like with this online option:
http://www.emvlab.org/tlvutils/
where you put something like this EMV response:
6f3a8407a0000000031010a52f500b56495341204352454449548701015f2d086573656e707466729f12074352454449544f9f1101019f38039f1a02
and it will show you everything perfectly, I started doing something by myself but then I realize that maybe we could have two 9F38(PDOL) Strings not neccesary two same tags cuz I know it's impossible but maybe the value of a tag end in 9F and the start of the next tag would be 38 and that would give me an error... Now that I mention it, is that possible? cuz that was one of the main reasons why I stopped doing my own function..
Does any of you have written a function to do this already?
Thanks!

https://github.com/binaryfoo/emv-bertlv should do the trick.
Using your example, the following code:
List<DecodedData> decoded = new RootDecoder().decode("6f3a8407a0000000031010a52f500b56495341204352454449548701015f2d086573656e707466729f12074352454449544f9f1101019f38039f1a02", "EMV", "constructed");
new DecodedWriter(System.out).write(decoded, "");
Will output:
[6F (FCI template)] 8407A0000000031010A52F500B56495341204352454449548701015F...1A02
[84 (dedicated file name)] A0000000031010
[A5 (FCI proprietary template)] 500B56495341204352454449548701015F2D086573656E707466729F...1A02
[50 (application label)] VISA CREDIT
[87 (application priority indicator)] 01
[5F2D (language preference)] esenptfr
[9F12 (application preferred name)] CREDITO
[9F11 (issuer code table index)] 01
[9F38 (PDOL - Processing data object list)] 9F1A02
9F1A (terminal country code) 2 bytes

This project has code to deal with EMV data http://code.google.com/p/javaemvreader/

You are on the right track. You can easily build your own EMV parser using the technique call TLV (Tag Length Value). Your raw data always comes back with a Tag, then after the tag is the length, using the length can get you the value.
So create three methods
method 1: Contains all the short tags
method 2: Contains all the long tags
method 3: Contains all the proprietary tags
So when you pass in your raw emv tag:
6f3a8407a0000000031010a52f500b56495341204352454449548701015f2d086573656e707466729f12074352454449544f9f1101019f38039f1a02
Loop through all those three methods, it will give you all the nice information that you need.

Use below function which will gives you hashmap of TLV value
public LinkedHashMap parseBERTLVTag(String tlv) throws DecoderException
{
if(tlv==null || "".equalsIgnoreCase(tlv)){
return null;
}
System.out.println("============= START ["+tlv+"]==================");
boolean inTagRead= true;
Map<String,String> tags= new HashMap<>();
StringBuilder _tmp = new StringBuilder();
String lastTag = "";
int old_index = 0;
boolean isFirstTagByte = true;
int len = 0;
boolean more=true;
String data = "";
while (more)
{
len = 0;
String hByte = tlv.substring(old_index,(old_index = old_index+2));
if(inTagRead)
{
if(isLastTagByte(hByte, isFirstTagByte))
{
inTagRead=false;
_tmp.append(hByte);
lastTag = _tmp.toString();
System.out.println("Tag["+lastTag+"]");
tags.put(lastTag, null);
_tmp= new StringBuilder();
}else
{
_tmp.append(hByte);
}
isFirstTagByte = false;
}else//Length
{
isFirstTagByte = true;
if(isLastLengthByte(hByte)) {
inTagRead=true;
_tmp.append(hByte);
len = Integer.parseInt(_tmp.toString(), 16 );
//read len*2
System.out.println(" Length ["+len+"]");
data = tlv.substring(old_index, (old_index = old_index+len*2));
String tmpData= lastTag+":"+_tmp.toString()+":h"+data;
System.out.println(" Data ["+tmpData+"]");
_tmp = new StringBuilder();
tags.put(lastTag, tmpData);
}else
{
_tmp.append(hByte);
}
}
more= tlv.length()<=old_index?false:true;
System.out.println("tag "+lastTag+" value "+data+" length "+len);
if(lastTag.length() > 0 && data.length() > 0 && len > 0){
if(!map.containsKey(lastTag)){
map.put(lastTag,new TLVModel().setTag(lastTag).setLength(len).setValue(data));
}
}
}//END OF WHILE
System.out.println("------------ as MAP ---------------------");
System.out.println("size "+map.size());
for (Map.Entry mp:map.entrySet()){
System.out.println("key "+mp.getKey()+" value "+mp.getValue());
}
return map.size() > 0 ? map : null;
}