Develop Reference

Can not identify text in Spanish with Lingpipe

Some days ago, I am developing an java server to keep a bunch of data and identify its language, so I decided to use lingpipe for such task. But I have facing an issue, after training code and evaluating it with two languages(English and Spanish) by getting that I can't identify spanish text, but I got a successful result with english and french.
The tutorial that I have followed in order to complete this task is:
http://alias-i.com/lingpipe/demos/tutorial/langid/read-me.html
An the next steps I have made in order to complete the task:
Steps followed to train a Language Classifier
~1.First place and unpack the english and spanish metadata inside a folder named leipzig, as follow (Note: Metadata and Sentences are provided from http://wortschatz.uni-leipzig.de/en/download):
leipzig //Main folder
1M sentences //Folder with data of the last trial
eng_news_2015_1M
eng_news_2015_1M.tar.gz
spa-hn_web_2015_1M
spa-hn_web_2015_1M.tar.gz
ClassifyLang.java //Custom program to try the trained code
dist //Folder
eng_news_2015_300K.tar.gz //unpackaged english sentences
spa-hn_web_2015_300K.tar.gz //unpackaged spanish sentences
EvalLanguageId.java
langid-leipzig.classifier //trained code
lingpipe-4.1.2.jar
munged //Folder
eng //folder containing the sentences.txt for english
sentences.txt
spa //folder containing the sentences.txt for spanish
sentences.txt
Munge.java
TrainLanguageId.java
unpacked //Folder
eng_news_2015_300K //Folder with the english metadata
eng_news_2015_300K-co_n.txt
eng_news_2015_300K-co_s.txt
eng_news_2015_300K-import.sql
eng_news_2015_300K-inv_so.txt
eng_news_2015_300K-inv_w.txt
eng_news_2015_300K-sources.txt
eng_news_2015_300K-words.txt
sentences.txt
spa-hn_web_2015_300K //Folder with the spanish metadata
sentences.txt
spa-hn_web_2015_300K-co_n.txt
spa-hn_web_2015_300K-co_s.txt
spa-hn_web_2015_300K-import.sql
spa-hn_web_2015_300K-inv_so.txt
spa-hn_web_2015_300K-inv_w.txt
spa-hn_web_2015_300K-sources.txt
spa-hn_web_2015_300K-words.txt
~2.Second unpack the language metadata compressed into a unpack folder
unpacked //Folder
eng_news_2015_300K //Folder with the english metadata
eng_news_2015_300K-co_n.txt
eng_news_2015_300K-co_s.txt
eng_news_2015_300K-import.sql
eng_news_2015_300K-inv_so.txt
eng_news_2015_300K-inv_w.txt
eng_news_2015_300K-sources.txt
eng_news_2015_300K-words.txt
sentences.txt
spa-hn_web_2015_300K //Folder with the spanish metadata
sentences.txt
spa-hn_web_2015_300K-co_n.txt
spa-hn_web_2015_300K-co_s.txt
spa-hn_web_2015_300K-import.sql
spa-hn_web_2015_300K-inv_so.txt
spa-hn_web_2015_300K-inv_w.txt
spa-hn_web_2015_300K-sources.txt
spa-hn_web_2015_300K-words.txt
~3.Then Munge the sentences of each one in order to remove the line numbers, tabs and replacing line breaks with single space characters. The output is uniformly written using the UTF-8 unicode encoding (Note:the munge.java at Lingpipe site).
/-----------------Command line----------------------------------------------/
javac -cp lingpipe-4.1.2.jar: Munge.java
java -cp lingpipe-4.1.2.jar: Munge /home/samuel/leipzig/unpacked /home/samuel/leipzig/munged
----------------------------------------Results-----------------------------
spa
reading from=/home/samuel/leipzig/unpacked/spa-hn_web_2015_300K/sentences.txt charset=iso-8859-1
writing to=/home/samuel/leipzig/munged/spa/spa.txt charset=utf-8
total length=43267166
eng
reading from=/home/samuel/leipzig/unpacked/eng_news_2015_300K/sentences.txt charset=iso-8859-1
writing to=/home/samuel/leipzig/munged/eng/eng.txt charset=utf-8
total length=35847257
/---------------------------------------------------------------/
<---------------------------------Folder------------------------------------->
munged //Folder
eng //folder containing the sentences.txt for english
sentences.txt
spa //folder containing the sentences.txt for spanish
sentences.txt
<-------------------------------------------------------------------------->
~4.Next we start by training the language(Note:the TrainLanguageId.java at Lingpipe LanguageId tutorial).
/---------------Command line--------------------------------------------/
javac -cp lingpipe-4.1.2.jar: TrainLanguageId.java
java -cp lingpipe-4.1.2.jar: TrainLanguageId /home/samuel/leipzig/munged /home/samuel/leipzig/langid-leipzig.classifier 100000 5
-----------------------------------Results-----------------------------------
nGram=100000 numChars=5
Training category=eng
Training category=spa
Compiling model to file=/home/samuel/leipzig/langid-leipzig.classifier
/----------------------------------------------------------------------------/
~5. We evaluated our trained code with the next result, having some issues on the confusion matrix (Note:the EvalLanguageId.java at Lingpipe LanguageId tutorial).
/------------------------Command line---------------------------------/
javac -cp lingpipe-4.1.2.jar: EvalLanguageId.java
java -cp lingpipe-4.1.2.jar: EvalLanguageId /home/samuel/leipzig/munged /home/samuel/leipzig/langid-leipzig.classifier 100000 50 1000
-------------------------------Results-------------------------------------
Reading classifier from file=/home/samuel/leipzig/langid-leipzig.classifier
Evaluating category=eng
Evaluating category=spa
TEST RESULTS
BASE CLASSIFIER EVALUATION
Categories=[eng, spa]
Total Count=2000
Total Correct=1000
Total Accuracy=0.5
95% Confidence Interval=0.5 +/- 0.02191346617949794
Confusion Matrix
reference \ response
,eng,spa
eng,1000,0 <---------- not diagonal sampling
spa,1000,0
Macro-averaged Precision=NaN
Macro-averaged Recall=0.5
Macro-averaged F=NaN
Micro-averaged Results
the following symmetries are expected:
TP=TN, FN=FP
PosRef=PosResp=NegRef=NegResp
Acc=Prec=Rec=F
Total=4000
True Positive=1000
False Negative=1000
False Positive=1000
True Negative=1000
Positive Reference=2000
Positive Response=2000
Negative Reference=2000
Negative Response=2000
Accuracy=0.5
Recall=0.5
Precision=0.5
Rejection Recall=0.5
Rejection Precision=0.5
F(1)=0.5
Fowlkes-Mallows=2000.0
Jaccard Coefficient=0.3333333333333333
Yule's Q=0.0
Yule's Y=0.0
Reference Likelihood=0.5
Response Likelihood=0.5
Random Accuracy=0.5
Random Accuracy Unbiased=0.5
kappa=0.0
kappa Unbiased=0.0
kappa No Prevalence=0.0
chi Squared=0.0
phi Squared=0.0
Accuracy Deviation=0.007905694150420948
Random Accuracy=0.5
Random Accuracy Unbiased=0.625
kappa=0.0
kappa Unbiased=-0.3333333333333333
kappa No Prevalence =0.0
Reference Entropy=1.0
Response Entropy=NaN
Cross Entropy=Infinity
Joint Entropy=1.0
Conditional Entropy=0.0
Mutual Information=0.0
Kullback-Liebler Divergence=Infinity
chi Squared=NaN
chi-Squared Degrees of Freedom=1
phi Squared=NaN
Cramer's V=NaN
lambda A=0.0
lambda B=NaN
ONE VERSUS ALL EVALUATIONS BY CATEGORY
CATEGORY[0]=eng VERSUS ALL
First-Best Precision/Recall Evaluation
Total=2000
True Positive=1000
False Negative=0
False Positive=1000
True Negative=0
Positive Reference=1000
Positive Response=2000
Negative Reference=1000
Negative Response=0
Accuracy=0.5
Recall=1.0
Precision=0.5
Rejection Recall=0.0
Rejection Precision=NaN
F(1)=0.6666666666666666
Fowlkes-Mallows=1414.2135623730949
Jaccard Coefficient=0.5
Yule's Q=NaN
Yule's Y=NaN
Reference Likelihood=0.5
Response Likelihood=1.0
Random Accuracy=0.5
Random Accuracy Unbiased=0.625
kappa=0.0
kappa Unbiased=-0.3333333333333333
kappa No Prevalence=0.0
chi Squared=NaN
phi Squared=NaN
Accuracy Deviation=0.011180339887498949
CATEGORY[1]=spa VERSUS ALL
First-Best Precision/Recall Evaluation
Total=2000
True Positive=0
False Negative=1000
False Positive=0
True Negative=1000
Positive Reference=1000
Positive Response=0
Negative Reference=1000
Negative Response=2000
Accuracy=0.5
Recall=0.0
Precision=NaN
Rejection Recall=1.0
Rejection Precision=0.5
F(1)=NaN
Fowlkes-Mallows=NaN
Jaccard Coefficient=0.0
Yule's Q=NaN
Yule's Y=NaN
Reference Likelihood=0.5
Response Likelihood=0.0
Random Accuracy=0.5
Random Accuracy Unbiased=0.625
kappa=0.0
kappa Unbiased=-0.3333333333333333
kappa No Prevalence=0.0
chi Squared=NaN
phi Squared=NaN
Accuracy Deviation=0.011180339887498949
/-----------------------------------------------------------------------/
~6.Then we tried to make a real evaluation with spanish text:
/-------------------Command line----------------------------------/
javac -cp lingpipe-4.1.2.jar: ClassifyLang.java
java -cp lingpipe-4.1.2.jar: ClassifyLang
/-------------------------------------------------------------------------/
<---------------------------------Result------------------------------------>
Text: Yo soy una persona increíble y muy inteligente, me admiro a mi mismo lo que me hace sentir ansiedad de lo que viene, por que es algo grandioso lleno de cosas buenas y de ahora en adelante estaré enfocado y optimista aunque tengo que aclarar que no lo haré por querer algo, sino por que es mi pasión.
Best Language: eng <------------- Wrong Result
<----------------------------------------------------------------------->
Code for ClassifyLang.java:
import com.aliasi.classify.Classification;
import com.aliasi.classify.Classified;
import com.aliasi.classify.ConfusionMatrix;
import com.aliasi.classify.DynamicLMClassifier;
import com.aliasi.classify.JointClassification;
import com.aliasi.classify.JointClassifier;
import com.aliasi.classify.JointClassifierEvaluator;
import com.aliasi.classify.LMClassifier;
import com.aliasi.lm.NGramProcessLM;
import com.aliasi.util.AbstractExternalizable;
import java.io.File;
import java.io.IOException;
import com.aliasi.util.Files;
public class ClassifyLang {
public static String text = "Yo soy una persona increíble y muy inteligente, me admiro a mi mismo"
+ " estoy ansioso de lo que viene, por que es algo grandioso lleno de cosas buenas"
+ " y de ahora en adelante estaré enfocado y optimista"
+ " aunque tengo que aclarar que no lo haré por querer algo, sino por que no es difícil serlo. ";
private static File MODEL_DIR
= new File("/home/samuel/leipzig/langid-leipzig.classifier");
public static void main(String[] args)
throws ClassNotFoundException, IOException {
System.out.println("Text: " + text);
LMClassifier classifier = null;
try {
classifier = (LMClassifier) AbstractExternalizable.readObject(MODEL_DIR);
} catch (IOException | ClassNotFoundException ex) {
// Handle exceptions
System.out.println("Problem with the Model");
}
Classification classification = classifier.classify(text);
String bestCategory = classification.bestCategory();
System.out.println("Best Language: " + bestCategory);
}
}
~7.I tried with a 1 million metadata file, but it got the same result and also changing the ngram number by getting the same results.
I will be so thankfull for your help.

Well, after days working in Natural Language Processing I found a way to determine the language of one text using OpenNLP.
Here is the Sample Code:
https://github.com/samuelchapas/languagePredictionOpenNLP/tree/master/TrainingLanguageDecOpenNLP
and over here is the training Corpus for the model created to make language predictions.
I decided to use OpenNLP for the issue described in this question, really this library has a complete stack of functionalities.
Here is the sample for model training>
https://mega.nz/#F!HHYHGJ4Q!PY2qfbZr-e0w8tg3cUgAXg

Parsing simple times in hh:mmaa format

DateFormat formatter = new SimpleDateFormat("hh:mmaa");
formatter.parse("01:20pm")
I'm trying to parse times in the format of 01:20pm. If I run the above code, I get the following exception:
java.text.ParseException: Unparseable date: "01:20pm"
at java.text.DateFormat.parse(DateFormat.java:366)
As far as the format I put in the SimpleDateFormat constructor, I don't see anything wrong. What went wrong here?

Your system locale must not recognize AM/PM. Use a Locale that does. Something like,
DateFormat formatter = new SimpleDateFormat("hh:mmaa", Locale.US);
Or, in Java 8+, use the new java.time API like
LocalTime lt = LocalTime.parse("01:20pm",
DateTimeFormatter.ofPattern("hh:mmaa", Locale.US));

Number and date parsing in Java uses the Locale to specify, well, locale-specific symbols. In this case, it is mostly the pm value that is being rejected.
To confirm this, here is a piece of code to exercise all available locales in the VM.
For locales that don't work, I was curious to see why, so instead of parsing a time, I format a valid time instead. Had to enable UTF-8 output, but it's interesting to see.
The really interesting part is that all Spanish (es) locales, except the United States variant (es_US) works fine. Hmmm........
Set<String> good = new TreeSet<>();
Set<String> bad = new TreeSet<>();
for (Locale locale : Locale.getAvailableLocales()) {
try {
new SimpleDateFormat("hh:mmaa", locale).parse("01:20pm");
good.add(locale.toLanguageTag());
} catch (ParseException e) {
bad.add(locale.toLanguageTag());
}
}
System.out.println("Good locales: " + good);
Date time = new SimpleDateFormat("hh:mmaa", Locale.ENGLISH).parse("01:20pm");
System.out.println("Bad locales:");
for (String languageTag : bad)
System.out.printf(" %-5s: %s%n", languageTag, new SimpleDateFormat("hh:mmaa", Locale.forLanguageTag(languageTag)).format(time));
OUTPUT
Good locales: [be, be-BY, bg, bg-BG, ca, ca-ES, da, da-DK, de, de-AT, de-CH, de-DE, de-GR, de-LU, en, en-AU, en-CA, en-GB, en-IE, en-IN, en-MT, en-NZ, en-PH, en-SG, en-US, en-ZA, es, es-AR, es-BO, es-CL, es-CO, es-CR, es-CU, es-DO, es-EC, es-ES, es-GT, es-HN, es-MX, es-NI, es-PA, es-PE, es-PR, es-PY, es-SV, es-UY, es-VE, et, et-EE, fr, fr-BE, fr-CA, fr-CH, fr-FR, fr-LU, he, he-IL, hi, hr, hr-HR, id, id-ID, is, is-IS, it, it-CH, it-IT, lt, lt-LT, lv, lv-LV, mk, mk-MK, ms, ms-MY, nl, nl-BE, nl-NL, nn-NO, no, no-NO, pl, pl-PL, pt, pt-BR, pt-PT, ro, ro-RO, ru, ru-RU, sk, sk-SK, sl, sl-SI, sr, sr-BA, sr-CS, sr-Latn, sr-Latn-BA, sr-Latn-ME, sr-Latn-RS, sr-ME, sr-RS, tr, tr-TR, uk, uk-UA, und]
Bad locales:
ar : 01:20م
ar-AE: 01:20م
ar-BH: 01:20م
ar-DZ: 01:20م
ar-EG: 01:20م
ar-IQ: 01:20م
ar-JO: 01:20م
ar-KW: 01:20م
ar-LB: 01:20م
ar-LY: 01:20م
ar-MA: 01:20م
ar-OM: 01:20م
ar-QA: 01:20م
ar-SA: 01:20م
ar-SD: 01:20م
ar-SY: 01:20م
ar-TN: 01:20م
ar-YE: 01:20م
cs : 01:20odp.
cs-CZ: 01:20odp.
el : 01:20μμ
el-CY: 01:20ΜΜ
el-GR: 01:20μμ
es-US: 01:20p.m.
fi : 01:20ip.
fi-FI: 01:20ip.
ga : 01:20p.m.
ga-IE: 01:20p.m.
hi-IN: ०१:२०अपराह्न
hu : 01:20DU
hu-HU: 01:20DU
ja : 01:20午後
ja-JP: 01:20午後
ja-JP-u-ca-japanese-x-lvariant-JP: 01:20午後
ko : 01:20오후
ko-KR: 01:20오후
mt : 01:20WN
mt-MT: 01:20WN
sq : 01:20MD
sq-AL: 01:20MD
sv : 01:20em
sv-SE: 01:20em
th : 01:20หลังเที่ยง
th-TH: 01:20หลังเที่ยง
th-TH-u-nu-thai-x-lvariant-TH: ๐๑:๒๐หลังเที่ยง
vi : 01:20CH
vi-VN: 01:20CH
zh : 01:20下午
zh-CN: 01:20下午
zh-HK: 01:20下午
zh-SG: 01:20下午
zh-TW: 01:20下午

How to read the contents of (.bib) file format using Java

I need to read .bib file and insert it tags into an objects of bib-entries
the file is big (almost 4000 lines) , so my first question is what to use (bufferrReader or FileReader)
the general format is
#ARTICLE{orleans01DJ,
author = {Doug Orleans and Karl Lieberherr},
title = {{{DJ}: {Dynamic} Adaptive Programming in {Java}}},
journal = {Metalevel Architectures and Separation of Crosscutting Concerns 3rd
Int'l Conf. (Reflection 2001), {LNCS} 2192},
year = {2001},
pages = {73--80},
month = sep,
editor = {A. Yonezawa and S. Matsuoka},
owner = {Administrator},
publisher = {Springer-Verlag},
timestamp = {2009.03.09}
}
#ARTICLE{Ossher:1995:SOCR,
author = {Harold Ossher and Matthew Kaplan and William Harrison and Alexander
Katz},
title = {{Subject-Oriented Composition Rules}},
journal = {ACM SIG{\-}PLAN Notices},
year = {1995},
volume = {30},
pages = {235--250},
number = {10},
month = oct,
acknowledgement = {Nelson H. F. Beebe, University of Utah, Department of Mathematics,
110 LCB, 155 S 1400 E RM 233, Salt Lake City, UT 84112-0090, USA,
Tel: +1 801 581 5254, FAX: +1 801 581 4148, e-mail: \path|beebe#math.utah.edu|,
\path|beebe#acm.org|, \path|beebe#computer.org| (Internet), URL:
\path|http://www.math.utah.edu/~beebe/|},
bibdate = {Fri Apr 30 12:33:10 MDT 1999},
coden = {SINODQ},
issn = {0362-1340},
keywords = {ACM; object-oriented programming systems; OOPSLA; programming languages;
SIGPLAN},
owner = {Administrator},
timestamp = {2009.02.26}
}
As you can see , there are some entries that have more than line, entries that end with }
entries that end with }, or }},
Also , some entries have {..},{..}.. in the middle
so , i am a little bit confused on how to start reading this file and how to get these entries and manipulate them.
Any help will be highly appreciated.

We currently discuss different options at JabRef.
These are the current options:
JBibTeX
ANTLRv3 Grammar
JabRef's BibtexParser.java

JsonParser ArrayIndexOutOfBoundsException when parsing json

The following code should work as a JsonParser but it is returning an ArrayIndexOutOfBoundsException even though I check that there are still elements to parse.
public void parseJson(String url) {
try {
InputStream is = new URL(url).openStream();
JsonParser parser = Json.createParser(is);
while (parser.hasNext()) {
Event e = parser.next();
if (e == Event.KEY_NAME) {
if (parser.getString().equals("name") && parser.hasNext()) {
parser.next();
writer = new BufferedWriter(new FileWriter("names.txt"));
writer.write(parser.getString());
}
}
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
The error produced is such:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 4096
at org.glassfish.json.JsonTokenizer.readChar(JsonTokenizer.java:388)
at org.glassfish.json.JsonTokenizer.read(JsonTokenizer.java:98)
at org.glassfish.json.JsonTokenizer.readString(JsonTokenizer.java:124)
at org.glassfish.json.JsonTokenizer.nextToken(JsonTokenizer.java:310)
at org.glassfish.json.JsonParserImpl$StateIterator.nextToken(JsonParserImpl.java:157)
at org.glassfish.json.JsonParserImpl$StateIterator.next(JsonParserImpl.java:182)
at org.glassfish.json.JsonParserImpl.next(JsonParserImpl.java:150)
at proj.Proj2.parseJson(Proj2.java:43)
at proj.Proj1.main(Proj1.java:78)
It parses everything up to when the error is thrown.
Sample JSON that produces error:
[{"id":774,"na":"Hungry Leaf","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1200,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Sandwiches"],"addr":"2608 Erwin Rd, Durham, NC","full_addr":{"addr":"2608 Erwin Rd","addr2":"","city":"Durham","state":"NC","postal_code":"27705-3843"},"city":"Durham","latitude":36.008621,"longitude":-78.94474,"del":1200,"mino":100,"is_delivering":0},{"id":780,"na":"Armadillo Grill","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1080,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["Mexican","Tex-Mex"],"addr":"439 Glenwood Ave, Raleigh, NC","full_addr":{"addr":"439 Glenwood Ave","addr2":"","city":"Raleigh","state":"NC","postal_code":"27603-1219"},"city":"Raleigh","latitude":35.785754,"longitude":-78.647293,"del":1080,"mino":100,"is_delivering":0},{"id":783,"na":"Chef Mario's Bistro","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":2880,"mino":150,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Sandwiches"],"addr":"2610 Wycliff Rd, Raleigh, NC","full_addr":{"addr":"2610 Wycliff Rd","addr2":"","city":"Raleigh","state":"NC","postal_code":"27607-3060"},"city":"Raleigh","latitude":35.818683,"longitude":-78.692145,"del":2880,"mino":150,"is_delivering":0},{"id":784,"na":"Chef Mario's Catering","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":4320,"mino":150,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Catering","Italian","Sandwiches"],"addr":"2610 Wycliff Rd, Raleigh, NC","full_addr":{"addr":"2610 Wycliff Rd","addr2":"","city":"Raleigh","state":"NC","postal_code":"27607-3060"},"city":"Raleigh","latitude":35.818683,"longitude":-78.692145,"del":4320,"mino":150,"is_delivering":0},{"id":786,"na":"D'Nardys Caterers","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1320,"mino":6.55,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Catering","Italian","Sandwiches"],"addr":"314 Madison Grove Pl, Cary, NC","full_addr":{"addr":"314 Madison Grove Pl","addr2":"","city":"Cary","state":"NC","postal_code":"27519-8161"},"city":"Cary","latitude":35.82135,"longitude":-78.854469,"del":1320,"mino":6.55,"is_delivering":0},{"id":787,"na":"Danny's Bar-B-Que","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":2640,"mino":100,"can":1}},"allow_tip":1,"allow_asap":1,"cu":["American","Barbecue"],"addr":"9561 Chapel Hill Rd, Morrisville, NC","full_addr":{"addr":"9561 Chapel Hill Rd","addr2":"","city":"Morrisville","state":"NC","postal_code":"27560-7359"},"city":"Morrisville","latitude":35.805972,"longitude":-78.809315,"del":2640,"mino":100,"is_delivering":1},{"id":792,"na":"Flying Biscuit","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1440,"mino":150,"can":1}},"allow_tip":1,"allow_asap":1,"cu":["Breakfast","Sandwiches"],"addr":"2016 Clark Ave, Raleigh, NC","full_addr":{"addr":"2016 Clark Ave","addr2":"","city":"Raleigh","state":"NC","postal_code":"27605-1604"},"city":"Raleigh","latitude":35.788996,"longitude":-78.659642,"del":1440,"mino":150,"is_delivering":1},{"id":793,"na":"Gateway Restaurant","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1320,"mino":50,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast"],"addr":"2411 Crabtree Blvd, Raleigh, NC","full_addr":{"addr":"2411 Crabtree Blvd","addr2":"","city":"Raleigh","state":"NC","postal_code":"27604-2232"},"city":"Raleigh","latitude":35.805222,"longitude":-78.611873,"del":1320,"mino":50,"is_delivering":0},{"id":796,"na":"Hibachi Xpress","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":720,"mino":150,"can":0}},"allow_tip":1,"allow_asap":1,"addr":"2470 Walnut St, Cary, NC","full_addr":{"addr":"2470 Walnut St","addr2":"","city":"Cary","state":"NC","postal_code":"27518-9212"},"city":"Cary","latitude":35.75113,"longitude":-78.741205,"del":720,"mino":150,"is_delivering":0},{"id":798,"na":"HoneyBaked Ham","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1080,"mino":60,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Sandwiches"],"addr":"5275 Six Forks Rd, Raleigh, NC","full_addr":{"addr":"5275 Six Forks Rd","addr2":"","city":"Raleigh","state":"NC","postal_code":"27609-4431"},"city":"Raleigh","latitude":35.85326,"longitude":-78.642593,"del":1080,"mino":60,"is_delivering":0},{"id":804,"na":"Moe's Southwest Grill","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1080,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["Mexican","Tex-Mex"],"addr":"506 Daniels St, Raleigh, NC","full_addr":{"addr":"506 Daniels St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27605-1317"},"city":"Raleigh","latitude":35.791061,"longitude":-78.66118,"del":1080,"mino":100,"is_delivering":0},{"id":806,"na":"Ole Time Barbecue","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1080,"mino":75,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["Barbecue"],"addr":"6309 Hillsborough St, Raleigh, NC","full_addr":{"addr":"6309 Hillsborough St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27606-1148"},"city":"Raleigh","latitude":35.787922,"longitude":-78.73851,"del":1080,"mino":75,"is_delivering":0},{"id":811,"na":"The Q Shack","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":180,"mino":40,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Barbecue"],"addr":"North Hills, Raleigh, NC","full_addr":{"addr":"North Hills","addr2":"","city":"Raleigh","state":"NC","postal_code":"27609"},"city":"Raleigh","latitude":35.837521,"longitude":-78.64312,"del":180,"mino":40,"is_delivering":0},{"id":812,"na":"We Cook For You","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":720,"mino":125,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Caribbean","Catering","Sandwiches"],"addr":"1125 Fuller St, Raleigh, NC","full_addr":{"addr":"1125 Fuller St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27603-2217"},"city":"Raleigh","latitude":35.765381,"longitude":-78.650581,"del":720,"mino":125,"is_delivering":0},{"id":813,"na":"WhichWich Superior Sandwiches","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1080,"mino":75,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Sandwiches"],"addr":"4025 Lake Boone Trl, Raleigh, NC","full_addr":{"addr":"4025 Lake Boone Trl","addr2":"","city":"Raleigh","state":"NC","postal_code":"27607-2928"},"city":"Raleigh","latitude":35.815604,"longitude":-78.696541,"del":1080,"mino":75,"is_delivering":0},{"id":820,"na":"Greens","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1080,"mino":6.55,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Italian","Seafood"],"addr":"4120 Main at North Hills St, Raleigh, NC","full_addr":{"addr":"4120 Main at North Hills St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27609-5754"},"city":"Raleigh","latitude":35.837546,"longitude":-78.642515,"del":1080,"mino":6.55,"is_delivering":0},{"id":861,"na":"Catering By Design","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":240,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Catering","Italian","Pizza","Sandwiches"],"addr":"132 Kilmayne Dr, Cary, NC","full_addr":{"addr":"132 Kilmayne Dr","addr2":"","city":"Cary","state":"NC","postal_code":"27511-4465"},"city":"Cary","latitude":35.769451,"longitude":-78.783912,"del":240,"mino":100,"is_delivering":0},{"id":3498,"na":"Z Pizza","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":900,"mino":75,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Pizza","Sandwiches"],"addr":"421 Fayetteville St, Raleigh, NC","full_addr":{"addr":"421 Fayetteville St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27601-1778"},"city":"Raleigh","latitude":35.77467,"longitude":-78.639389,"del":900,"mino":75,"is_delivering":0},{"id":3500,"na":"Artisan Sandwiches & Salads","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":900,"mino":50,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Sandwiches"],"addr":"421 Fayetteville St, Raleigh, NC","full_addr":{"addr":"421 Fayetteville St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27601-1778"},"city":"Raleigh","latitude":35.77467,"longitude":-78.639389,"del":900,"mino":50,"is_delivering":0},{"id":3795,"na":"Chef Mario's Catering Dinner","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":4320,"mino":150,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Asian","Barbecue","Breakfast","Catering","Italian","Jamaican","Mediterranean","Sandwiches","Seafood","Thai"],"addr":"2610 Wycliff Rd, Raleigh, NC","full_addr":{"addr":"2610 Wycliff Rd","addr2":"","city":"Raleigh","state":"NC","postal_code":"27607-3060"},"city":"Raleigh","latitude":35.818683,"longitude":-78.692145,"del":4320,"mino":150,"is_delivering":0},{"id":3895,"na":"A Catered Affair","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":720,"mino":125,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Asian","Breakfast","Catering","Italian","Jamaican","Sandwiches"],"addr":"1125 Fuller St, Raleigh, NC","full_addr":{"addr":"1125 Fuller St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27603-2217"},"city":"Raleigh","latitude":35.765381,"longitude":-78.650581,"del":720,"mino":125,"is_delivering":0},{"id":4011,"na":"Neomonde Mediterranean","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1260,"mino":125,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["Breakfast","Mediterranean","Sandwiches"],"addr":"9650 Strickland Rd, Raleigh, NC","full_addr":{"addr":"9650 Strickland Rd","addr2":"","city":"Raleigh","state":"NC","postal_code":"27615-1937"},"city":"Raleigh","del":1260,"mino":125,"is_delivering":0},{"id":4425,"na":"Dorry's Downtown Deli","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1320,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Sandwiches"],"addr":"211 E Chatham St, Cary, NC","full_addr":{"addr":"211 E Chatham St","addr2":"","city":"Cary","state":"NC","postal_code":"27511-3427"},"city":"Cary","latitude":35.787534,"longitude":-78.777666,"del":1320,"mino":100,"is_delivering":0},{"id":4427,"na":"Pogo","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":180,"mino":40,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Sandwiches"],"addr":"201 E Hargett St, Raleigh, NC","full_addr":{"addr":"201 E Hargett St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27601-1437"},"city":"Raleigh","latitude":35.778141,"longitude":-78.635834,"del":180,"mino":40,"is_delivering":0},{"id":4428,"na":"Chick-fil-A","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1200,"mino":200,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Sandwiches"],"addr":"1803 N Harrison Ave, Cary, NC","full_addr":{"addr":"1803 N Harrison Ave","addr2":"","city":"Cary","state":"NC","postal_code":"27513-2408"},"city":"Cary","latitude":35.828831,"longitude":-78.769402,"del":1200,"mino":200,"is_delivering":0},{"id":6142,"na":"Spartacus Restaurant","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1200,"mino":300,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["Mediterranean","Sandwiches"],"addr":"4139 Old Chapel Hill Rd, Durham, NC","full_addr":{"addr":"4139 Old Chapel Hill Rd","addr2":"","city":"Durham","state":"NC","postal_code":"27707-5057"},"city":"Durham","del":1200,"mino":300,"is_delivering":0},{"id":7776,"na":"Firehouse Subs","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":120,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Sandwiches"],"addr":"1539 Us Highway 70 E, Garner, NC","full_addr":{"addr":"1539 Us Highway 70 E","addr2":"","city":"Garner","state":"NC","postal_code":"27529"},"city":"Garner","del":120,"mino":100,"is_delivering":0},{"id":7985,"na":"Ben & Jerry's Ice Cream","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":2880,"mino":200,"can":1}},"allow_tip":1,"allow_asap":1,"cu":["American","Bakery","Ice Cream"],"addr":"102 W Franklin St, Chapel Hill, NC","full_addr":{"addr":"102 W Franklin St","addr2":"","city":"Chapel Hill","state":"NC","postal_code":"27516-2516"},"city":"Chapel Hill","latitude":35.91304,"longitude":-79.056091,"del":2880,"mino":200,"is_delivering":1},{"id":10509,"na":"Catering By Design - Dinner","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":240,"mino":6.55,"can":1}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Catering","Italian","Sandwiches","Seafood"],"addr":"132 Kilmayne Dr, Cary, NC","full_addr":{"addr":"132 Kilmayne Dr","addr2":"","city":"Cary","state":"NC","postal_code":"27511-4465"},"city":"Cary","latitude":35.769451,"longitude":-78.783912,"del":240,"mino":6.55,"is_delivering":1},{"id":12715,"na":"Top This Cafe # The Atrium","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":45,"mino":0,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Breakfast","Sandwiches"],"addr":"2501 Blue Ridge Rd, Raleigh, NC","full_addr":{"addr":"2501 Blue Ridge Rd","addr2":"","city":"Raleigh","state":"NC","postal_code":"27607-6436"},"city":"Raleigh","latitude":35.814449,"longitude":-78.705399,"del":45,"mino":0,"is_delivering":0},{"id":12724,"na":"Donovan's Dish","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1200,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Catering","Sandwiches"],"addr":"1040 Buck Jones Rd, Raleigh, NC","full_addr":{"addr":"1040 Buck Jones Rd","addr2":"","city":"Raleigh","state":"NC","postal_code":"27606-3323"},"city":"Raleigh","latitude":35.770224,"longitude":-78.739941,"del":1200,"mino":100,"is_delivering":0},{"id":12739,"na":"Lubrano's Ristorante Italiano","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1200,"mino":125,"can":1}},"allow_tip":1,"allow_asap":1,"cu":["Italian"],"addr":"101 Keybridge Dr, Morrisville, NC","full_addr":{"addr":"101 Keybridge Dr","addr2":"","city":"Morrisville","state":"NC","postal_code":"27560-5911"},"city":"Morrisville","latitude":35.819771,"longitude":-78.822189,"del":1200,"mino":125,"is_delivering":1},{"id":23709,"na":"DeMarco's Restaurant & Bar","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":1260,"mino":125,"can":1}},"allow_tip":1,"allow_asap":1,"cu":["Italian"],"addr":"3607 Falls River Ave, Raleigh, NC","full_addr":{"addr":"3607 Falls River Ave","addr2":"","city":"Raleigh","state":"NC","postal_code":"27614-7359"},"city":"Raleigh","latitude":35.930141,"longitude":-78.564522,"del":1260,"mino":125,"is_delivering":1},{"id":23710,"na":"Green Planet Catering","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":4200,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Catering","Sandwiches"],"addr":"218 S Blount St, Raleigh, NC","full_addr":{"addr":"218 S Blount St","addr2":"","city":"Raleigh","state":"NC","postal_code":"27601-1408"},"city":"Raleigh","latitude":35.777495,"longitude":-78.63662,"del":4200,"mino":100,"is_delivering":0},{"id":23815,"na":"Capital Center Catering","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":960,"mino":150,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Bakery","Catering","Italian","Sandwiches"],"addr":"217 E Main St, Clayton, NC","full_addr":{"addr":"217 E Main St","addr2":"","city":"Clayton","state":"NC","postal_code":"27520-2449"},"city":"Clayton","latitude":35.652088,"longitude":-78.458515,"del":960,"mino":150,"is_delivering":0},{"id":23817,"na":"Firewurst","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":960,"mino":100,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["American","Sandwiches"],"addr":"8531 Brier Creek Pky, Raleigh, NC","full_addr":{"addr":"8531 Brier Creek Pky","addr2":"","city":"Raleigh","state":"NC","postal_code":"27617-7333"},"city":"Raleigh","latitude":35.905433,"longitude":-78.786068,"del":960,"mino":100,"is_delivering":0},{"id":32244,"na":"Don Rio's Southwest Catering","cs_phone":"919-234-7755","rds_info":{"id":125,"name":"Triangle Food Guy","logo":""},"services":{"deliver":{"time":720,"mino":125,"can":0}},"allow_tip":1,"allow_asap":1,"cu":["Mexican","Tex-Mex"],"addr":"3607 Falls River Ave, Raleigh, NC","full_addr":{"addr":"3607 Falls River Ave","addr2":"","city":"Raleigh","state":"NC","postal_code":"27614-7359"},"city":"Raleigh","latitude":35.930141,"longitude":-78.564522,"del":720,"mino":125,"is_delivering":0}]

It seems this was a bug in version 1.0.1. Here's another thread that talks about it. The JsonTokenizer class has changed pretty dramatically since then. Upgrade to 1.0.2 and you will be fine.

Sort Japanese data in java

I need to sort list of Japanese Strings.As of now I am using Java's Collator API.That is working fine for all the languages.But for Japanese it is not giving the expected results.How can I achieve this ??
`Collator collator = Collator.getInstance(Locale.JAPAN);
collator.setStrength(Collator.PRIMARY);
Collections.sort(Words, collator);`
Here Words is the list of Japanese String.

There are three for Japan, I would make sure you are using the right locale. (they could be aliases for the same Collator)
final Locale[] availableLocales = Collator.getAvailableLocales();
Arrays.sort(availableLocales, new Comparator<Locale>() {
#Override
public int compare(Locale o1, Locale o2) {
return o1.toString().compareTo(o2.toString());
}
});
for(Locale locale : availableLocales)
System.out.println(locale) ;
prints
ar
ar_AE
ar_BH
ar_DZ
ar_EG
ar_IQ
ar_JO
ar_KW
ar_LB
ar_LY
ar_MA
ar_OM
ar_QA
ar_SA
ar_SD
ar_SY
ar_TN
ar_YE
be
be_BY
bg
bg_BG
ca
ca_ES
cs
cs_CZ
da
da_DK
de
de_AT
de_CH
de_DE
de_LU
el
el_CY
el_GR
en
en_AU
en_CA
en_GB
en_IE
en_IN
en_MT
en_NZ
en_PH
en_SG
en_US
en_ZA
es
es_AR
es_BO
es_CL
es_CO
es_CR
es_DO
es_EC
es_ES
es_GT
es_HN
es_MX
es_NI
es_PA
es_PE
es_PR
es_PY
es_SV
es_US
es_UY
es_VE
et
et_EE
fi
fi_FI
fr
fr_BE
fr_CA
fr_CH
fr_FR
fr_LU
ga
ga_IE
hi_IN
hr
hr_HR
hu
hu_HU
in
in_ID
is
is_IS
it
it_CH
it_IT
iw
iw_IL
ja
ja_JP
ja_JP_JP_#u-ca-japanese
ko
ko_KR
lt
lt_LT
lv
lv_LV
mk
mk_MK
ms
ms_MY
mt
mt_MT
nl
nl_BE
nl_NL
no
no_NO
no_NO_NY
pl
pl_PL
pt
pt_BR
pt_PT
ro
ro_RO
ru
ru_RU
sk
sk_SK
sl
sl_SI
sq
sq_AL
sr
sr_BA
sr_BA_#Latn
sr_CS
sr_ME
sr_ME_#Latn
sr_RS
sr_RS_#Latn
sr__#Latn
sv
sv_SE
th
th_TH
th_TH_TH_#u-nu-thai
tr
tr_TR
uk
uk_UA
vi
vi_VN
zh
zh_CN
zh_HK
zh_SG
zh_TW