Develop Reference

How to properly handle comma inside a quoted string using opencsv?

I'm trying to read csv file that contains strings both quoted and not.
If string is quoted, it should save it's quote chars.
Beside that, if string contains comma, it should not be split.
I've tried multiple ways but nothing works as of now.
Current test data:
"field1 (with use of , we lose the other part)",some description
field2,"Dear %s, some text"
Getting 1st field of mapped bean
Expected result:
"field1 (with use of , we lose the other part)"
field2
Current result:
"field1 (with use of
field2
Here is the code:
public class CsvToBeanReaderTest {
#Test
void shouldIncludeDoubleQuotes() {
String testData =
"\"field1 (with use of , we lose the other part)\",some description\n"
+
"field2,\"Dear %s, some text\"";
RFC4180ParserBuilder rfc4180ParserBuilder = new RFC4180ParserBuilder();
rfc4180ParserBuilder.withQuoteChar(ICSVWriter.NO_QUOTE_CHARACTER);
ICSVParser rfc4180Parser = rfc4180ParserBuilder.build();
CSVReaderBuilder builder = new CSVReaderBuilder(new StringReader(testData));
CSVReader reader = builder
.withCSVParser(rfc4180Parser)
.build();
List<TestClass> result = new CsvToBeanBuilder<TestClass>(reader)
.withType(TestClass.class)
.withEscapeChar('\"')
.build()
.parse();
result.forEach(testClass -> System.out.println(testClass.getField1()));
}
private List<TestClass> readTestData(String testData) {
return new CsvToBeanBuilder<TestClass>(new StringReader(testData))
.withType(TestClass.class)
.withSeparator(',')
.withSkipLines(0)
.withIgnoreEmptyLine(true)
.build()
.parse();
}
public static final class TestClass {
#CsvBindByPosition(position = 0)
private String field1;
#CsvBindByPosition(position = 1)
private String description;
public String toCsvFormat() {
return String.join(",",
field1,
description);
}
public String getField1() {
return field1;
}
}
}
I've found out that if I comment or remove rfc4180ParserBuilder.withQuoteChar(ICSVWriter.NO_QUOTE_CHARACTER); the string will be parsed correctly, but I will lose the quote char which should not be lost. Is there any suggestions what can be done? (I would prefer not to switch on other csv libraries)

Java GSON check data

I'm having trouble with gson:
For example I have this output from website:
[["connected"], ["user1":"Hello"], ["user2":"Hey"], ["disconnected"]]
But I want parse this JSON and output something like this:
connected
user1 says: Hello
user2 says: Hey
disconnected
I quicly wrote this code:
public static void PrintEvents(String id){
String response = Post.getResponse(Server()+"events?id="+id,"");
// response is [["connected"],["user1":"Hello"],["user2":"Hey"],["disconnected"]]
JsonElement parse = (new JsonParser()).parse(response); //found this in internet
int bound = ????????????; // Should be 4
for (int i=1;i<=bound;i++){
String data = ???????????;
if (data == "connected" || data == "disconnected") then {
System.out.println(data);
}else if(?????==2){// to check how many strings there is, if it's ["abc","def"] or ["abc"]
String data2 = ??????????????;
System.out.println(data+" says: "+data2);
}else{
//something else
}
};
}
What should I insert to these parts with question marks to make code work?
I cannot find any way to make it work...
Sorry for my bad English.
EDIT: Changed response to [["connected"], ["user1","Hello"], ["user2","Hey"], ["disconnected"]]. Earlier response was not valid JSON.

The response that you have pasted is not a valid json. paste it in http://www.jsoneditoronline.org/ and see the error.
Please find the below code snippet:
public static void printEvents(String id)
{
String response = "[[\"connected\"] ,[\"user1:Hello\"],[\"user2:Hey\"],[\"disconnected\"]]";
JsonElement parse = (new JsonParser()).parse(response); //found this in internet
int bound = ((JsonArray)parse).size(); // Should be 4
for (int i = 0; i < bound; i++) {
String data = ((JsonArray)parse).get(0).getAsString();
if (data.equals("connected") || data.equals("disconnected")) {
System.out.println(data);
continue;
}
String[] splittedData = data.split(":");
if (splittedData.length
== 2) {// to check how many strings there is, if it's ["abc","def"] or ["abc"]
System.out.println(splittedData[0] + " says: " + splittedData[1]);
}
/*
*else{
* your else logic goes here
* }
* */
}
}
Couple of suggestions:
If you are new to json world, use jackson instead of Gson.
the response is not a good design. Slightly correct json:
{
"firstKey": "connected",
"userResponses": [
{
"user1": "hey"
},
{
"user2": "hi"
}
],
"lastKey": "disconnected"
}
Also try to define pojos , instead of working inline with json.

You need to define a separate class like this:
class MyClass{
String name;
String value;
}
and then:
List<MyClass> myclasses = new Gson().fromJson(response, new TypeToken<List<MyClass>>(){}.getType());
then
for(MyClass myclass: myclasses){
...
}

Android Json Parsing with multipartEntity

I need to parse this as an JsonArray instead of objects
but it dosent seem to work correctly.
my output is ["{"TransactionVolume":"34","TransactionitemID":"2"}"]
I need it to be [{"TransactionVolume":"34","TransactionitemID":"2"}]
this is part of my code
public class TransactionItem {
public String TransactionitemID;
public String TransactionVolume;
public String getTransactionVolume() {
return TransactionVolume;
}
public void setTransactionVolume(String transactionVolume) {
TransactionVolume = transactionVolume;
}
public String getTransactionitemID() {
return TransactionitemID;
}
public void setTransactionitemID(String transactionitemID) {
TransactionitemID = transactionitemID;
}
}
TransactionItem transactionItem = new TransactionItem();
transactionItem.setTransactionitemID(article.toString());
transactionItem.setTransactionVolume(volume.toString());
String transitemjson = gson.toJson(transactionItem);
JSONArray ja = new JSONArray();
ja.put(transitemjson);
String test = ja.toString().replaceAll("\\\\\"", "\"");
Ja seems to be right where the output is
["{\"TransactionVolume\":\"34\",\"TransactionitemID\":\"2\"}"]
So I've tried to replace all \ with " but still the " at the begging and end.
Trimming the String doesn't seem like an good idea, but it could work. Is there any other way of doing this?
EDIT
This got me to the goal!
ArrayList<TransactionItem> Transactionlist = new ArrayList<>();
for (int i=0; i < CompleteArticle.size(); i++) {
String id = CompleteArticle.get(i);
String Volume = CompleteVolume.get(i);
TransactionItem transactionItem = new TransactionItem();
transactionItem.setTransactionitemID(id);
transactionItem.setTransactionVolume(Volume);
//transactionitems.transactionitems.add(transactionItem);
Transactionlist.add(transactionItem);
}
JsonElement transitemjson = gson.toJsonTree(transactionItems);
Output :
[{"TransactionVolume":"1","TransactionitemID":"5"},{"TransactionVolume":"3","TransactionitemID":"3"}]

You need to use a JSONElement instead of a String.
Replace this line:
String transitemjson = gson.toJson(transactionItem);
With this:
JSONElement transitemjson = gson.toJsonTree(transactionItem);

Using Antlr to get identifiers and function names

I'm trying to use and understand AntLR, this is new to me. My purpose is to read a source code file written in C and extract from it the identifiers (variables and function names).
In my C grammar (file C.g4) consider:
identifierList
: Identifier
| identifierList Comma Identifier
;
Identifier
: IdentifierNondigit
( IdentifierNondigit
| Digit
)*
;
After generation of parser and listener I create my own listener to the identifierList.
Note that MyCListener class extends CBaseListener:
public class MyCListener extends CBaseListener {
#Override
public void enterIdentifierList(CParser.IdentifierListContext ctx) {
List<ParseTree> children = ctx.children;
for (ParseTree parseTree : children) {
System.out.println(parseTree.getText());
}
}
Then I have this in main class:
String fileurl = "C:/example.c";
CLexer lexer;
try {
lexer = new CLexer(new ANTLRFileStream(fileurl));
CommonTokenStream tokens = new CommonTokenStream(lexer);
CParser parser = new CParser(tokens);
CParser.IdentifierListContext identifierContext = parser.identifierList();
ParseTreeWalker walker = new ParseTreeWalker();
MyCListener listener = new MyCListener();
walker.walk(listener, identifierContext);
} catch (IOException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
Where example.c is:
int main() {
// this is C
int i=0; // i is int
/* double j=0.0;
C
*/
}
What am I doing wrong?
Maybe I didn't write MyCListener properly, or identifierList is not what I need to listen... Really don't know. I'm sorry, but I didn't even understand my output, why is there a lexical error?:
line 3:4 mismatched input '(' expecting {<EOF>, ','}
main
(
)
{
int
i
=
0
;
}
As you see, I'm very confused about this. Can somebody help me ? Please...

With this line:
CParser.IdentifierListContext identifierContext = parser.identifierList();
you're trying to parse your entire input as an identifierList. But your input isn't just that.
Assuming you're using the C.g4 from the ANTLR4 Github repository, try to let the parser start at the entry point of the grammar (which is the rule compilationUnit):
MyCListener listener = new MyCListener();
ParseTreeWalker.DEFAULT.walk(listener, parser.compilationUnit());
EDIT
Here's a quick demo:
public class Main {
public static void main(String[] args) throws Exception {
final List<String> identifiers = new ArrayList<String>();
String source = "int main() {\n" +
"\n" +
"// this is C\n" +
"\n" +
" int i=0; // i is int\n" +
" /* double j=0.0;\n" +
" C\n" +
" */\n" +
"}";
CLexer lexer = new CLexer(new ANTLRInputStream(source));
CParser parser = new CParser(new CommonTokenStream(lexer));
ParseTreeWalker.DEFAULT.walk(new CBaseListener(){
#Override
public void enterDirectDeclarator(#NotNull CParser.DirectDeclaratorContext ctx) {
if (ctx.Identifier() != null) {
identifiers.add(ctx.Identifier().getText());
}
}
// Perhaps override other rules that use `Identifier`
}, parser.compilationUnit());
System.out.println("identifiers -> " + identifiers);
}
}
which would print:
identifiers -> [main, i]

Lucene wrong match

I have a csvfile
id|name
1|PC
2|Activation
3|USB
public class TESTResult
{
private Long id;
private String name;
private Float score;
// with setters & getters
}
public class TEST
{
private Long id;
private String name;
// with setters & getters
}
public class JobTESTTagger {
private static Version VERSION;
private static CharArraySet STOPWORDS;
private static RewriteMethod REWRITEMETHOD;
private static Float MINSCORE = 0.0001F;
static {
BooleanQuery.setMaxClauseCount(100000);
VERSION = Version.LUCENE_44;
STOPWORDS = StopAnalyzer.ENGLISH_STOP_WORDS_SET;
REWRITEMETHOD = MultiTermQuery.CONSTANT_SCORE_FILTER_REWRITE;
}
public static ArrayList<TESTResult> searchText(String text, String keyId,
List<TEST> TESTs) {
ArrayList<TESTResult> results = new ArrayList<TESTResult>();
MemoryIndex index = new MemoryIndex();
EnglishAnalyzer englishAnalyzer = new EnglishAnalyzer(VERSION,STOPWORDS);
QueryParser parser = new QueryParser(VERSION, "text", englishAnalyzer);
parser.setMultiTermRewriteMethod(REWRITEMETHOD);
index.addField("text", text, englishAnalyzer);
for (int i = 0; i < TESTs.size(); i++) {
TEST TEST = TESTs.get(i);
String criteria = "\"" + TEST.getName().trim() + "\"";
if (criteria == null || criteria.isEmpty())
continue;
criteria = criteria.replaceAll("\r", " ");
criteria = criteria.replaceAll("\n", " ");
try {
Query query = parser.parse(criteria);
Float score = index.search(query);
if (score > MINSCORE) {
int result = new TESTResult(TEST.getId(), TEST.getName(),score);
results.add(result);
}
} catch (ParseException e) {
System.out.println("Could not parse article.");
}
}
return results;
}
public static void main(String[] args) {
ArrayList<TESTResult> testresults = searchText(text, keyId, iths);
CsvReader reader = new CsvReader("C:\a.csv");
reader.setDelimiter('|');
reader.readHeaders();
List<TEST> result = new ArrayList<TEST>();
while (reader.readRecord()) {
Long id = Long.valueOf(reader.get("id").trim());
String name = reader.get("name").trim();
TEST concept = new TEST(id, name);
result.add(concept);
}
String text = "These activities are good. I have a good PC in my house.";
}
I am matching 'activities' to Activation. How is it possible. Can anybody tell me how Lucene matches the words.
Thanks
R

EnglishAnalyzer, along with most language-specific analyzers, uses a stemmer. This means that it reduces terms to a stem (or root) of the term, in order to attempt to match more loosely. Mostly this works well, removing suffixes and matching up derived words to a common root. So when I search for "fish", I also find "fished", "fishing" and "fishes".
In this case though, both "activities" and "activation" both reduce to the root of "activ", resulting in the match you are seeing. Another example: "organ", "organic" and "organize" all have the common stem "organ".
You can stem or not, neither approach is perfect. If you don't stem you'll miss relevant results. If you do, you'll hit some odd irrelevant results.
To deal with specific problematic cases, you can define a stemmer exclusion set in EnglishAnalyzer to prevent stemming just on those specific problematic terms. In this case, I would think of "activation" as the probable term to prevent stemming on, though you could go either way. So I could do something like:
CharArraySet stemExclusionSet = new CharArraySet(VERSION, 1, true);
stemExclusionSet.add("activation");
EnglishAnalyzer englishAnalyzer = new EnglishAnalyzer(VERSION, STOPWORDS, stemExclusionSet);