Develop Reference

java.io.NotSerializableException: Custom JSQL Parser Implementation

I have a Flink application where I run a custom SQL parser and parse the SQL provided by user. This generates a Predicate tree which gets evaluated once the streaming application starts.
I use JSQL parser to parse the SQL. When running the application, I get a weird Serialization error which I didn't get before. I haven't changed anything in the code, yet I get this error suddenly using Flink API.
Below is the code snippet where it is failing and I get a java.io.NotSerializableException: com.eventwatch.query.OperationType$$Lambda$1272/411594792:
private Predicate<RowMapPair> checkEquals(String type, String column, Object value) {
log.debug("string: " + value);
StringValue stringValue = new StringValue((""+value).trim());
log.debug("stringValue: " + stringValue.toString());
return (rowMapPair) -> { // PLACE OF RUNTIME EXCEPTION
MapState<String, Pair> mapState = rowMapPair.getMapState();
Row row = rowMapPair.getRow();
String _id = String.valueOf(row.getFieldAs("_id"));
String key = String.join("__", _id, column);
String curVal = null;
try {
Pair val = mapState.get(key);
log.debug("** val: {}", mapState.get(key));
if (Objects.nonNull(val)) {
curVal = String.valueOf(val.getValue());
} else {
return false;
}
} catch (Exception e) {
log.error("Error: {}", e.getMessage(), e);
throw new RuntimeException(e);
}
return curVal.equalsIgnoreCase(stringValue.getValue());
};
}
Below is the stack trace:
Caused by: java.io.NotSerializableException: com.eventwatch.query.OperationType$$Lambda$1272/411594792
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632)
at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:143)

error when get value from marketdataincremental refresh

I got an error in my quickfixj Application. First, I got an error like this:
Out of order repeating group members
After that, I added this text into my initiator.config:
ValidateUserDefinedFields=N
ValidateIncomingMessage=N
But now I got another error in my application:
quickfix.FieldNotFound: Field was not found in message, field=55
at quickfix.FieldMap.getField(FieldMap.java:223)
at quickfix.FieldMap.getString(FieldMap.java:237)
at com.dxtr.fastmatch.marketdatarequestapps.TestMarketdataRequest.fromApp(TestMarketdataRequest.java:38)
at quickfix.Session.fromCallback(Session.java:1847)
at quickfix.Session.verify(Session.java:1791)
at quickfix.Session.verify(Session.java:1862)
at quickfix.Session.next(Session.java:1047)
at quickfix.Session.next(Session.java:1204)
at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:163)
at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:113)
at quickfix.mina.SingleThreadedEventHandlingStrategy.lambda$blockInThread$1(SingleThreadedEventHandlingStrategy.java:145)
at quickfix.mina.SingleThreadedEventHandlingStrategy$ThreadAdapter$RunnableWrapper.run(SingleThreadedEventHandlingStrategy.java:267)
at java.lang.Thread.run(Thread.java:748)
My code for get value of symbols is :
public void fromApp(quickfix.Message message, SessionID sessionID)
throws FieldNotFound, IncorrectDataFormat, IncorrectTagValue, UnsupportedMessageType {
try {
String symbol = message.getString(Symbol.FIELD);
System.out.println(" FromApp " + message);
message.getString(TransactTime.FIELD);
// String seqNo = message.getString(MsgSeqNum.FIELD);
double bid = message.getDouble(MDEntryPx.FIELD);
double ask = message.getDouble(MDEntryPx.FIELD);
// System.out.println(seqNo + " " + message);
} catch (FieldNotFound fieldNotFound) {
fieldNotFound.printStackTrace();
}
}
I have also using this code
public void onMessage (MarketDataIncrementalRefresh message, SessionID sessionID) throws FieldNotFound{
try
{
MDReqID mdreqid = new MDReqID();
SendingTime sendingtime = new SendingTime();
NoMDEntries nomdentries = new NoMDEntries();
quickfix.fix42.MarketDataIncrementalRefresh.NoMDEntries group
= new quickfix.fix42.MarketDataIncrementalRefresh.NoMDEntries();
MDUpdateAction mdupdateaction = new MDUpdateAction();
DeleteReason deletereason = new DeleteReason();
MDEntryType mdentrytype = new MDEntryType();
MDEntryID mdentryid = new MDEntryID();
Symbol symbol = new Symbol();
MDEntryOriginator mdentryoriginator = new MDEntryOriginator();
MDEntryPx mdentrypx = new MDEntryPx();
Currency currency = new Currency();
MDEntrySize mdentrysize = new MDEntrySize();
ExpireDate expiredate = new ExpireDate();
ExpireTime expiretime = new ExpireTime();
NumberOfOrders numberoforders = new NumberOfOrders();
MDEntryPositionNo mdentrypositionno = new MDEntryPositionNo();
message.getField(nomdentries);
message.getField(sendingtime);
message.getGroup(1, group);
int list = nomdentries.getValue();
for (int i = 0; i < list; i++)
{
message.getGroup(i + 1, group);
group.get(mdupdateaction);
if (mdupdateaction.getValue() == '2')
System.out.println("Enter");
group.get(deletereason);
group.get(mdentrytype);
group.get(mdentryid);
group.get(symbol);
group.get(mdentryoriginator);
if (mdupdateaction.getValue() == '0')
group.get(mdentrypx);
group.get(currency);
if (mdupdateaction.getValue() == '0')
group.get(mdentrysize);
}
System.out.printf("Got Symbol {0} Price {1}",
symbol.getValue(), mdentrypx.getValue());
}catch (Exception ex)
{
System.out.println("error" + ex);
}
but i also get error like this
quickfix.FieldNotFound: Field was not found in message, field=55
at quickfix.FieldMap.getField(FieldMap.java:223)
at quickfix.FieldMap.getString(FieldMap.java:237)
at com.dxtr.fastmatch.marketdatarequestapps.TestMarketdataRequest.fromApp(TestMarketdataRequest.java:39)
at quickfix.Session.fromCallback(Session.java:1847)
at quickfix.Session.verify(Session.java:1791)
at quickfix.Session.verify(Session.java:1862)
at quickfix.Session.next(Session.java:1047)
at quickfix.Session.next(Session.java:1204)
at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:163)
at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:113)
at quickfix.mina.SingleThreadedEventHandlingStrategy.lambda$blockInpacket_write_wait: Connection to 3.13.235.241 port 22: Broken pipe
and here the value i check in my message.log
8=FIX.4.2^A9=0217^A35=X^A34=7291^A49=Fastmatch1^A52=20200401-10:47:59.833^A56=MDValueTrade2UAT1^A262=VT_020^A268=02^A279=2^A55=GBP/CHF^A269=0^A278=1140851192^A270=1.19503^A271=02000000^A279=0^A55=GBP/CHF^A269=0^A278=1140851194^A270=1.19502^A271=06000000^A10=114^A
my broker have send to me the price and etc
My question is: how to fix my problem from this code ?

First, I got an error like this:
Out of order repeating group members
Your data dictionary doesn't match your counterparty's. Fix that and this will go away.
After that, I added this text into my initiator.config:
ValidateUserDefinedFields=N
ValidateIncomingMessage=N
This did not fix anything -- it HIDES your actual problem and has you looking at a new fake problem.
What you need to do:
Your configuration has this, right?
UseDataDictionary=Y
DataDictionary=path/to/FIXnn.xml
# or if FIX5:
AppDataDictionary=path/to/FIX5n.xml
TransportDataDictionary=path/to/FIXT.xml
Find your counterparty's documentation, and make sure your xml file's messages and fields match what they say they're going to send you. Make sure all repeating groups have the same fields in the same order.
Here is some documentation about how the Data Dictionary xml file is structured. It's pretty easy.

Java GSON check data

I'm having trouble with gson:
For example I have this output from website:
[["connected"], ["user1":"Hello"], ["user2":"Hey"], ["disconnected"]]
But I want parse this JSON and output something like this:
connected
user1 says: Hello
user2 says: Hey
disconnected
I quicly wrote this code:
public static void PrintEvents(String id){
String response = Post.getResponse(Server()+"events?id="+id,"");
// response is [["connected"],["user1":"Hello"],["user2":"Hey"],["disconnected"]]
JsonElement parse = (new JsonParser()).parse(response); //found this in internet
int bound = ????????????; // Should be 4
for (int i=1;i<=bound;i++){
String data = ???????????;
if (data == "connected" || data == "disconnected") then {
System.out.println(data);
}else if(?????==2){// to check how many strings there is, if it's ["abc","def"] or ["abc"]
String data2 = ??????????????;
System.out.println(data+" says: "+data2);
}else{
//something else
}
};
}
What should I insert to these parts with question marks to make code work?
I cannot find any way to make it work...
Sorry for my bad English.
EDIT: Changed response to [["connected"], ["user1","Hello"], ["user2","Hey"], ["disconnected"]]. Earlier response was not valid JSON.

The response that you have pasted is not a valid json. paste it in http://www.jsoneditoronline.org/ and see the error.
Please find the below code snippet:
public static void printEvents(String id)
{
String response = "[[\"connected\"] ,[\"user1:Hello\"],[\"user2:Hey\"],[\"disconnected\"]]";
JsonElement parse = (new JsonParser()).parse(response); //found this in internet
int bound = ((JsonArray)parse).size(); // Should be 4
for (int i = 0; i < bound; i++) {
String data = ((JsonArray)parse).get(0).getAsString();
if (data.equals("connected") || data.equals("disconnected")) {
System.out.println(data);
continue;
}
String[] splittedData = data.split(":");
if (splittedData.length
== 2) {// to check how many strings there is, if it's ["abc","def"] or ["abc"]
System.out.println(splittedData[0] + " says: " + splittedData[1]);
}
/*
*else{
* your else logic goes here
* }
* */
}
}
Couple of suggestions:
If you are new to json world, use jackson instead of Gson.
the response is not a good design. Slightly correct json:
{
"firstKey": "connected",
"userResponses": [
{
"user1": "hey"
},
{
"user2": "hi"
}
],
"lastKey": "disconnected"
}
Also try to define pojos , instead of working inline with json.

You need to define a separate class like this:
class MyClass{
String name;
String value;
}
and then:
List<MyClass> myclasses = new Gson().fromJson(response, new TypeToken<List<MyClass>>(){}.getType());
then
for(MyClass myclass: myclasses){
...
}

org.json.JSONException: Missing value at character 2

I am trying to parse the below json response using following code but the code is working fine on normal eclipse program but failing on tomcat (java 1.8)
2016-12-20 00:18:17,146 DEBUG c.a.f.u.s.UpdateFileUploadData [Thread-35]
sTempOutput::"[{\"Status\":\"Success\",\"MonthlyReportId\":\"R/I -
000913\"},{\"Status\":\"Success\",\"MonthlyReportId\":\"R/I -
001473\"},{\"Status\":\"Success\",\"MonthlyReportId\":\"R/I -
001474\"}]"
2016-12-20 00:18:17,146 DEBUG c.a.f.u.s.UpdateFileUploadData [Thread-35]
transformed:[{\"Status\":\"Success\",\"MonthlyReportId\":\"R/I -
000913\"},{\"Status\":\"Success\",\"MonthlyReportId\":\"R/I -
001473\"},{\"Status\":\"Success\",\"MonthlyReportId\":\"R/I -
001474\"}]
jsonObject = jsonObject.replaceAll("\\\"", "\"");
if(jsonObject.startsWith("\"")){
jsonObject = jsonObject.substring(1, jsonObject.length()-1);
} // replacing the " from start and end.
if (logger.isDebugEnabled()) {
logger.debug("transformed:{}", jsonObject);
}
org.json.JSONArray array = new org.json.JSONArray(jsonObject);
if (logger.isDebugEnabled()) {
logger.debug("array:::{}", array);
}
DBAdapter dbAdapter = new DBAdapter();
for (int start = 0 ;start<array.length(); start++){
JSONObject object = array.getJSONObject(start);
String status = (String)object.get("Status");
String monthlyReportId = (String)object.get("MonthlyReportId");
if (logger.isDebugEnabled()) {
logger.debug("status:::{}", status);
logger.debug("monthlyReportId:::{}", monthlyReportId);
}
}
This is killing me. I have also tried it -
jsonObject = jsonObject.replaceAll("(\\r|\\n|\\r\\n)+", "");
jsonObject = jsonObject.replaceAll("\\\"", "\"");
rest remains the same.

getJSONObject and subsequent getString returns null

This is a very straightforward question, but this error is very mysterious to me as I have not been able to find a solution or anyone else who has had this problem. I've also used a very similar technique in another activity and it worked just fine. I am making an android application which makes a POST request to a server. The response is a JSONObject that must be parsed into a number and another JSONObject which must also be parsed, and its values assigned to an array of CurrentGame objects. The first call to getJSONObject works fine, but calling getString on that JSONObject returns the following error:
java.lang.NullPointerException: Attempt to write to field 'java.lang.String com.xxxxx.xxxxx.CurrentGame.oppEmail' on a null object reference
Here is my java code:
private void handleResponse(JSONObject response){
int numGroups = 0;
try{
numGroups = response.getInt("Number");
}catch(JSONException e){
e.printStackTrace();
}
Log.i("Number of Groups", String.valueOf(numGroups));
CurrentGame[] currentGames = new CurrentGame[numGroups];
JSONObject current;
int yourTurn = 0;
for(int i = 0; i < numGroups; i++){
try{
current = response.getJSONObject(String.valueOf(i));
Log.i("Current JSONObject: ", String.valueOf(current));
if(current.has("OppEmail")){
currentGames[i].oppEmail = current.getString("OppEmail");
}
if(current.has("OppName")) {
currentGames[i].oppName = current.getString("OppName");
}
if(current.has("Group")) {
currentGames[i].group = current.getString("Group");
}
if(current.has("YourTurn")) {
yourTurn = current.getInt("YourTurn");
}
if(yourTurn == 0){
currentGames[i].yourTurn = true;
}
else{
currentGames[i].yourTurn = false;
}
}
catch (JSONException e){
e.printStackTrace();
}
}
}
Shouldn't the JSONObject.has() check at least be preventing this error?
I know the first getInt() and getJSONObject are working. Heres the Log:
06-21 21:58:56.644 20116-20116/com.xxxxx.xxxxx D/Response:﹕ {"Number":2,"0":{"Group":"Test Group 1","OppEmail":"xxxxx#xxxxx.edu","OppName":"MikeyP","YourTurn":0},"1":{"Group":"Test Group 2","OppEmail":"xxxxx#xxxxx.edu","OppName":"MikeyP","YourTurn":1}}
06-21 21:58:56.644 20116-20116/com.xxxxxx.xxxxxt I/Number of Groups﹕ 2
06-21 21:58:56.644 20116-20116/com.xxxxx.xxxxx I/Current JSONObject﹕ {"Group":"Test Group 1","OppEmail":"xxxxxx#xxxxx.edu","OppName":"MikeyP","YourTurn":0}
Here's the server code:
$games['Number'] = $numgames;
if($numgames > 0){
$i = 0;
while($row = mysqli_fetch_array($getgames)){
$currGame['Group'] = $row['GroupName'];
// Get the opponent's email and username
if($row['Player1'] != $email){
$opponent = $row['Player1'];
$currGame['OppEmail'] = $opponent;
$sql = "SELECT Username FROM users WHERE Email = '".$opponent."'";
$username = mysqli_query($conn, $sql);
$row2 = mysqli_fetch_assoc($username);
$currGame['OppName'] = $row2['Username'];
}
else if($row['Player2'] != $email){
$opponent = $row['Player2'];
$currGame['OppEmail'] = $opponent;
$sql = "SELECT Username FROM users WHERE Email = '".$opponent."'";
$username = mysqli_query($conn, $sql);
$row2 = mysqli_fetch_assoc($username);
$currGame['OppName'] = $row2['Username'];
}
// Determine if it is this player's turn
if($row['CurrentPlayer'] != $email){
$currGame['YourTurn'] = 0;
}
else{
$currGame['YourTurn'] = 1;
}
$games[$i] = $currGame;
$i++;
}
}
//Echo array of groups
header('Content-Type: application/json');
$response = json_encode($games);
echo $response;
Thank you in advance for any ideas as to what I'm doing wrong here. I know similar questions have been asked about getString() returning null, but having read them all I'm still very stumped.

Problem is caused by :
currentGames[i].oppEmail = current.getString("OppEmail");
line.
Because currentGames Array is initialized with size 2 but not added any item of type CurrentGame.
Instead of using currentGames[i].oppEmail create a object of CurrentGame class add all values then add it in currentGames Array like:
CurrentGame objCurrentGame=new CurrentGame();
if(current.has("OppEmail")){
objCurrentGame.oppEmail = current.getString("OppEmail");
}
... same for other fields
...
//Add objCurrentGame to Array
currentGames[i]=objCurrentGame;

Parsing json this way is not robust and error prone, it is recommended to use such libraries as
Gson
Jackson
Retrofit
as these open source libraries offer stable implementation for such purposes and there is no need to reinvent the wheel yourself.
example:
YourPojoClass obj = new Gson().fromJson("{SomeJsonString}", YourPojoClass.class);
In this way, you get the strongly typed pojo instance.You don't even need write the POJO class yourself, and there are many online service that can generate the POJO class out of json strings:
http://www.jsonschema2pojo.org/
http://pojo.sodhanalibrary.com/