We are working on Spring based application which needs to access and read certain credentials stored in the yaml file which use the format below:
#yaml file content
...
secrets: username1:pw1, username2:pw2
...
we load the yaml file using PropertySource annotation like this
#PropertySources({
#PropertySource(value = "file:/someSecretPath/secrets.yaml", ignoreResourceNotFound = true) // secret yaml file
})
I have tried using the following code to parse it into array of map object since it is a list contains 2 object type,
#Repository
public class getSecretClass {
#Value("${secrets}")
private Map<String,String>[] credentials;
}
but it fails with this error:
... 'java.lang.string' cannot be converted into java.lang.map[] ....
On the other hand,
I tested reading from simple string array like this one:
#yaml file content
...
secrets: text1, text2
...
and the code below would work with no issue:
#Repository
public class getSecretClass {
#Value("${secrets}")
private String[] credentials;
...
private someReadSecretMethod(){
System.out.println( credentials[0] );
}
}
it will print "text1" as expected. But we do need to store the credentials as pairs in my use case, a plain text would not work for me.
Any suggestion or advice would be much appreciated !
Why not do something easier on the eyes, easier to program if not a little verbose:
secrets:
- username: username1
password: pwd1
- username: username2
password: pwd2
Have a class for your secret:
public class Secret {
private String username;
private String password;
public String toString() { return username + ":" password; }
}
And then injects as
#Value("${secrets}")
List<Secret> secrets;
If you must use map. I think array of map are probably not what you desired. Maybe you need one map which contains two keys, username1 and username2. The following might let you know how to obtain the map.
The yml file content is like that.
secrets: "{username1:pw1,username2:pw1}"
The Java code field is like that.
#Value("#{${secrets}}")
public Map<Integer,Integer> secrets;
For example, we are getting a json message like this from our partner:
{
"message": "Dear client,\nWe'd like to offer you.."
}
The partner wants the client to receive the message like this (without newline but with \n)
Dear client,\nWe'd like to offer you
But we have a chain of microservices in our ecosystem and that json goes through 4 or 5 microservices which proccesing it before client can get it. So, our partner should give us \\\\\\\\n instead of \n in order to client got \n in the result. But I'm wondering, is adding 8 backslashes in the source message to escape "\n" for every microservice the only way to solve this problem. I think it's not really good solution, because we have to make changes in source message if count of microservices in chain changes (Moreover, we will face he problem if count of microservices in the chain start changing dynamically)? Is there way to use \n in source message (from partner) without replacing every \n with \\n in our microservies?
There is an example how I process the json in one of the microservices:
private String replace(String sourceJson, List<String> properties, DocumentContext context) {
StringBuilder stringBuilder = new StringBuilder(sourceJson);
for (String property : properties) {
String newValue = Pattern.compile("ABC")
.matcher(stringBuilder)
.replaceAll(context.read(property, String.class));
stringBuilder.replace(0, stringBuilder.length(), newValue);
}
return stringBuilder.toString();
}
Here's an example of creating a JSON object, turning it into a String and and then back into a JsonObject, adding a property, and turning it back into a String again.
package org.example;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
public class App {
public static void main(String[] args) {
JsonObject o = new JsonObject();
o.addProperty("message","foo\nbar");
Gson gson = new Gson();
String stringRep = gson.toJson(o);
System.out.println(stringRep);
JsonObject o2 = gson.fromJson(stringRep, JsonObject.class);
o2.addProperty("newProp", 42);
String messageValue = o2.get("message").getAsString();
System.out.println(messageValue);
String newMessageValue = messageValue.replace("foo", "baz");
o2.addProperty("message", newMessageValue);
stringRep = gson.toJson(o2);
System.out.println(stringRep);
}
}
The output of this program is:
{"message":"foo\nbar"}
foo
bar
{"message":"baz\nbar","newProp":42}
So you can see the the Java representations of strings contain newline characters, but the JSON representations contain the character sequence '\', 'n'.
The maven dependency you need is:
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.9</version>
</dependency>
I have this method:
public static Object parseStringToObject(String json) {
String Object = json;
Gson gson = new Gson();
Object objects = gson.fromJson(object, Object.class);
parseConfigFromObjectToString(object);
return objects;
}
And I want to parse a JSON with:
public static void addObject(String IP, Object addObject) {
try {
String json = sendPostRequest("http://" + IP + ":3000/config/add_Object", ConfigJSONParser.parseConfigFromObjectToString(addObject));
addObject = ConfigJSONParser.parseStringToObject(json);
} catch (Exception ex) {
ex.printStackTrace();
}
}
But I get an error message:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException:
Expected BEGIN_OBJECT but was STRING at line 1 column 1
Even without seeing your JSON string you can tell from the error message that it is not the correct structure to be parsed into an instance of your class.
Gson is expecting your JSON string to begin with an object opening brace. e.g.
{
But the string you have passed to it starts with an open quotes
"
Invalid JSON from the server should always be an expected use case. A million things can go wrong during transmission. Gson is a bit tricky, because its error output will give you one problem, and the actual exception you catch will be of a different type.
With all that in mind, the proper fix on the client side is
try
{
gson.fromJSON(ad, Ad.class);
//...
}
catch (IllegalStateException | JsonSyntaxException exception)
{
//...
If you want to know why the JSON you received from the server is wrong, you can look inside your catch block at the exception. But even if it is your problem, it's not the client's responsibility to fix JSON it is receiving from the internet.
Either way, it is the client's responsibility to decide what to do when it gets bad JSON. Two possibilities are rejecting the JSON and doing nothing, and trying again.
If you are going to try again, I highly recommend setting a flag inside the try / catch block and then responding to that flag outside the try / catch block. Nested try / catch is likely how Gson got us into this mess with our stack trace and exceptions not matching up.
In other words, even though I'll admit it doesn't look very elegant, I would recommend
boolean failed = false;
try
{
gson.fromJSON(ad, Ad.class);
//...
}
catch (IllegalStateException | JsonSyntaxException exception)
{
failed = true;
//...
}
if (failed)
{
//...
I had a similar problem recently and found an interesting solution. Basically I needed to deserialize following nested JSON String into my POJO:
"{\"restaurant\":{\"id\":\"abc-012\",\"name\":\"good restaurant\",\"foodType\":\"American\",\"phoneNumber\":\"123-456-7890\",\"currency\":\"USD\",\"website\":\"website.com\",\"location\":{\"address\":{\"street\":\" Good Street\",\"city\":\"Good City\",\"state\":\"CA\",\"country\":\"USA\",\"postalCode\":\"12345\"},\"coordinates\":{\"latitude\":\"00.7904692\",\"longitude\":\"-000.4047208\"}},\"restaurantUser\":{\"firstName\":\"test\",\"lastName\":\"test\",\"email\":\"test#test.com\",\"title\":\"server\",\"phone\":\"0000000000\"}}}"
I ended up using regex to remove the open quotes from beginning and the end of JSON and then used apache.commons unescapeJava() method to unescape it. Basically passed the unclean JSON into following method to get back a cleansed one:
private String removeQuotesAndUnescape(String uncleanJson) {
String noQuotes = uncleanJson.replaceAll("^\"|\"$", "");
return StringEscapeUtils.unescapeJava(noQuotes);
}
then used Google GSON to parse it into my own Object:
MyObject myObject = new.Gson().fromJson(this.removeQuotesAndUnescape(uncleanJson));
In Retrofit2, When you want to send your parameters in raw you must use Scalars.
first add this in your gradle:
compile 'com.squareup.retrofit2:retrofit:2.3.0'
compile 'com.squareup.retrofit2:converter-gson:2.3.0'
compile 'com.squareup.retrofit2:converter-scalars:2.3.0'
public interface ApiInterface {
String URL_BASE = "http://10.157.102.22/rest/";
#Headers("Content-Type: application/json")
#POST("login")
Call<User> getUser(#Body String body);
}
my SampleActivity :
public class SampleActivity extends AppCompatActivity implements Callback<User> {
#Override
protected void onCreate(#Nullable Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_sample);
Retrofit retrofit = new Retrofit.Builder()
.baseUrl(ApiInterface.URL_BASE)
.addConverterFactory(ScalarsConverterFactory.create())
.addConverterFactory(GsonConverterFactory.create())
.build();
ApiInterface apiInterface = retrofit.create(ApiInterface.class);
// prepare call in Retrofit 2.0
try {
JSONObject paramObject = new JSONObject();
paramObject.put("email", "sample#gmail.com");
paramObject.put("pass", "4384984938943");
Call<User> userCall = apiInterface.getUser(paramObject.toString());
userCall.enqueue(this);
} catch (JSONException e) {
e.printStackTrace();
}
}
#Override
public void onResponse(Call<User> call, Response<User> response) {
}
#Override
public void onFailure(Call<User> call, Throwable t) {
}
}
Reference: [How to POST raw whole JSON in the body of a Retrofit request?
I have come to share an solution. The error happened to me after forcing the notbook to hang up. possible solution clean preject.
Maybe your JSON Object is right,but the response that you received is not your valid data.Just like when you connect the invalid WiFi,you may received a strange response < html>.....< /html> that GSON can not parse.
you may need to do some try..catch.. for this strange response to avoid crash.
Make sure you have DESERIALIZED objects like DATE/DATETIME etc. If you are directly sending JSON without deserializing it then it can cause this problem.
In my situation, I have a "model", consist of several String parameters, with the exception of one: it is byte array byte[].
Some code snippet:
String response = args[0].toString();
Gson gson = new Gson();
BaseModel responseModel = gson.fromJson(response, BaseModel.class);
The last line above is when the
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column
is triggered. Searching through the SO, I realised I need to have some form of Adapter to convert my BaseModel to and fro a JsonObject. Having mixed of String and byte[] in a model does complicate thing. Apparently, Gson don't really like the situation.
I end up making an Adapter to ensure byte[] is converted to Base64 format. Here is my Adapter class:
public class ByteArrayToBase64Adapter implements JsonSerializer<byte[]>, JsonDeserializer<byte[]> {
#Override
public byte[] deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
return Base64.decode(json.getAsString(), Base64.NO_WRAP);
}
#Override
public JsonElement serialize(byte[] src, Type typeOfSrc, JsonSerializationContext context) {
return new JsonPrimitive(Base64.encodeToString(src, Base64.NO_WRAP));
}
}
To convert JSONObject to model, I used the following:
Gson customGson = new GsonBuilder().registerTypeHierarchyAdapter(byte[].class, new ByteArrayToBase64Adapter()).create();
BaseModel responseModel = customGson.fromJson(response, BaseModel.class);
Similarly, to convert the model to JSONObject, I used the following:
Gson customGson = new GsonBuilder().registerTypeHierarchyAdapter(byte[].class, new ByteArrayToBase64Adapter()).create();
String responseJSon = customGson.toJson(response);
What the code is doing is basically to push the intended class/object (in this case, byte[] class) through the Adapter whenever it is encountered during the convertion to/fro JSONObject.
Don't use jsonObject.toString on a JSON object.
In my case, I am Returning JSON Object as
{"data":"","message":"Attendance Saved
Successfully..!!!","status":"success"}
Resolved by changing it as
{"data":{},"message":"Attendance Saved
Successfully..!!!","status":"success"}
Here data is a sub JsonObject and it should starts from { not ""
Don't forget to convert your object into Json first using Gson()
val fromUserJson = Gson().toJson(notificationRequest.fromUser)
Then you can easily convert it back into an object using this awesome library
val fromUser = Gson().fromJson(fromUserJson, User::class.java)
if your json format and variables are okay then check your database queries...even if data is saved in db correctly the actual problem might be in there...recheck your queries and try again.. Hope it helps
I had a case where I read from a handwritten json file. The json is perfect. However, this error occurred. So I write from a java object to json file, then read from that json file. things are fine. I could not see any difference between the handwritten json and the one from java object. Tried beyondCompare it sees no difference.
I finally noticed the two file sizes are slightly different, and I used winHex tool and detected extra stuff.
So the solution for my situation is, make copy of the good json file, paste content into it and use.
In my case, my custom http-client didn't support the gzip encoding. I was sending the "Accept-Encoding: gzip" header, and so the response was sent back as a gzip string and couldn't be decoded.
The solution was to not send that header.
I was making a POST request with some parameters using Retrofit in Android
WHAT I FACED:
The error I was getting in Android Studio logcat:
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING
at line 2 column 1 path $
[but it was working fine with VOLLY library]
when I googled it...
you know[ Obviously json is expecting a OBJECT but...]
BUT when I changed my service to return a simple string [ like print_r("don't lose hope") ] or
Noting at all
It was getting printed fine in Postman
but in Android studio logcat, it was still SAME ERROR [
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING
at line 2 column 1 path $
]
Hold up now, I am sending a simple message or not sending anything in response and still studio is
telling me "...Expected BEGIN_OBJECT but was STRING..."
SOMETHING IS WRONG
On 4th day:
I finally stopped for looking "QUICK SOLUTIONS" and REALLY READ some stack overflow questions
and articles carefully.
WHAT I GOT:
Logging interceptor
It will show you whatever data comes from your server[even eco messages] which are not shown in
Andorid studios logcat,
that way you can FIND THE PROBLEM.
What I found is I was sending data with #Body like-
#Headers("Content-Type: application/json")
#POST("CreateNewPost")
Call<Resp> createNewPost(#Body ParaModel paraModel);
but no parameter was reaching to server, everything was null [I found using Logging interceptor]
then I simply searched an article "how to make POST request using Retrofit"
here's one
SOLUTION:
from here I changed my method to:
#POST("CreateNewPost")
#FormUrlEncoded
Call<Resp> createNewPost(
#Field("user_id") Integer user_id,
#Field("user_name") String user_name,
#Field("description") String description,
#Field("tags") String tags);
and everything was fine.
CONCLUSION:
I don't understand why Retrofit gave this error
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING
at line 2 column 1 path $
it doesn't make any sense at all.
So ALWAYS DEBUG in detail then find WHERE THINGS ARE LEAKING and then FIX.
This error solved for by replacing .toString method to .string on the response
toString => string (add in try{...code..}catche(IOException e))
below code is working for me
try {
MainModelResponse model;
Gson gson = new GsonBuilder().create();
if (response.code() == ConstantValues.SUCCESS_OK) {
model = gson.fromJson(response.body().string(), MainModelResponse.class);
} else {
model = gson.fromJson(response.errorBody().string(), MainModelResponse.class);
}
moduleData.postValue(model);
}catch (IllegalStateException | JsonSyntaxException | IOException exception){
exception.printStackTrace();
}
}
use a string begin & end with {}.
such as
final String jsStr = "{\"metric\":\"opentsdb_metric\",\"tags\":{\"testtag\":\"sunbotest\"},\"aggregateTags\":[],\"dps\":{\"1483399261\":18}}";
DataPoint dataPoint = new Gson().fromJson(jsStr, DataPoint.class);
this works for me.
In my case the object was all fine even the Json Validator was giving it a valid resposne but I was using Interface like this
#POST(NetworkConstants.REGISTER_USER)
Call<UserResponse> registerUser(
#Query("name") String name,
#Query("email") String email,
#Query("password") String password,
#Query("created_date") Long creationDate
);
Then I changed the code to
#FormUrlEncoded
#POST(NetworkConstants.REGISTER_USER)
Call<UserResponse> registerUser(
#Field("name") String name,
#Field("email") String email,
#Field("password") String password,
#Field("created_date") Long creationDate
);
And everything was resolved.
my problem not related to my codes
after copy some files from an other project got this issue
in the stack pointed to Gson library
in android studio 4.2.1 this problem not solved when I try file-> invalidate and restart
and
after restart in first time build got same error but in second build this problem solved
I don't understand why this happened
I was using an old version of retrofit library. So what I had to do was to change my code from this after upgrading it to com.squareup.retrofit2:retrofit:2.9.0:
#POST(AppConstants.UPLOAD_TRANSACTION_DETAIL)
fun postPremiumAppTransactionDetail(
#Query("name") planName:String,
#Query("amount") amount:String,
#Query("user_id") userId: String,
#Query("sub_id") planId: String,
#Query("folder") description:String,
#Query("payment_type") paymentType:String):
Call<TransactionResponseModel>
To this:
#FormUrlEncoded
#POST(AppConstants.UPLOAD_TRANSACTION_DETAIL)
fun postPremiumAppTransactionDetail(
#Field("name") planName:String,
#Field("amount") amount:String,
#Field("user_id") userId: String,
#Field("sub_id") planId: String,
#Field("folder") description:String,
#Field("payment_type") paymentType:String):
Call<TransactionResponseModel>
For me it turned out that I was trying to deserialize to an object that used java.time.ZonedDateTime for one of the properties. It worked as soon as I changed it to a java.util.Date instead.
I am trying to deserialize a JSON String using Jackson 2 with RestAssured (java tool for IT tests).
I have a problem. The String I am trying to deserialize is :
{"Medium":{"uuid":"2","estimatedWaitTime":0,"status":"OPEN_AVAILABLE","name":"Chat","type":"CHAT"}}
There is the object type "Medium" at the begining of the String. This cause Jackson failing during deserialization:
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "Medium"
I've set the "IGNORE_ON_UNKNOWN_PROPERTIES" to false and then I got no exception during deserialisation. However, all of my properties are 'null' in java.
Response getAvailability -> {"Medium":{"uuid":"2","estimatedWaitTime":0,"status":"OPEN_AVAILABLE","name":"Chat","type":"CHAT"}}
### MEDIUM name -> null
### MEDIUM uuid -> null
### MEDIUM wait time -> null
### MEDIUM wait time -> null
### MEDIUM status -> null
Does anyone can help me ? (note: I can't change my input JSON string).
{
"Medium": {
"uuid": "2",
"estimatedWaitTime": 0,
"status": "OPEN_AVAILABLE",
"name": "Chat",
"type": "CHAT"
}
}
as you can see uuid and other params are part of medium object , so class in which it can be deserialized is.
class Medium
{
string name;
// specify other params also.
}
class BaseObject
{
Medium Medium;
}
and then use jackson.deserialize('json', BaseObject.class)
above i had given pseudo code
You need to put annotation
#JsonRootName("Medium")
on your bean class and configure object mapper to
mapper.enable(DeserializationFeature.UNWRAP_ROOT_VALUE).
You need a way to remove the Object name that is the part of the input JSON. Since you cannot change the input string, Use this code to change this input string to a tree and get the value of "Medium" node.
ObjectMapper m = new ObjectMapper();
JsonNode root = m.readTree("{\"Medium\":{\"uuid\":\"2\",\"estimatedWaitTime\":0,\"status\":\"OPEN_AVAILABLE\",\"name\":\"Chat\",\"type\":\"CHAT\"}}");
JsonNode obj = root.get("Medium");
Medium medium = m.readValue(obj.asText, Medium.class);
Need a quick help. I am a newbie in QuickFixJ. I have a FIX message in a txt file. I need to convert that into FIX50SP2 format. I am enclosing the code snippet.
String fixMsg = "1128=99=25535=X49=CME34=47134052=20100318-03:21:11.36475=20120904268=2279=122=848=336683=607400107=ESU2269=1270=140575271=152273=121014000336=2346=521023=1279=122=848=336683=607401107=ESU2269=1270=140600271=206273=121014000336=2346=681023=210=159";
System.out.println("FixMsg String:"+fixMsg);
Message FIXMessage = new Message();
DataDictionary dd = new DataDictionary("FIX50SP2.xml");
FIXMessage.fromString(fixMsg, dd, false);
System.out.println("FIXMessage Output:" + FIXMessage.toString()); // Print message after parsing
MsgType msgType = new MsgType();
System.out.println(FIXMessage.getField(msgType));
Here is the output:
FixMsg String:1128=99=15835=X49=CME34=47164052=2012090312102051175=20120904268=1279=122=848=336683=607745107=ESU2269=1270=140575271=123273=121020000336=2346=501023=110=205
FIXMessage Output:9=6135=X34=47164049=CME52=2012090312102051175=20120904268=110=117
quickfix.FieldNotFound: Field [35] was not found in message.
at quickfix.FieldMap.getField(FieldMap.java:216)
at quickfix.FieldMap.getFieldInternal(FieldMap.java:353)
at quickfix.FieldMap.getField(FieldMap.java:349)
at MainApp.main(MainApp.java:52)
I want to extract MsgType field (field 35). Could you please tell me where I am wrong? The thing I have observed is that after parsing to FIX50SP2 format, the convert FIX message is missing many data element (for details see the output)
Thanks
Like others mentioned the MsgType is an header field and you get it by using the following
String msgType = null;
if(FIXMessage.getHeader().isSetField(MsgType.FIELD)) {
msgType = FIXMessage.getHeader().getString(MsgType.FIELD);
}
System.out.println("MsgType is " + msgType);`
The reason you are missing many data element after parsing is, probably your message have some custom tags(like tag 2346), which is not defined in your data dictionary(FIXSP02.xml). hence the parsing of those tags failed and missing in the output.
To fix this, get the data dictionary from the party that is sending you the message and use it to parse the message
I'm not familiar with FIX messages and QuickFixJ, but glancing at the Javadoc, it seems like you should use the identifyType method :
String fixMsg = "1128=99=25535=X49=CME34=47134052=20100318-03:21:11.36475=20120904268=2279=122=848=336683=607400107=ESU2269=1270=140575271=152273=121014000336=2346=521023=1279=122=848=336683=607401107=ESU2269=1270=140600271=206273=121014000336=2346=681023=210=159";
MsgType msgType = Message.identifyType(fixMsg);
You may find FixB framework useful as it deals well with non-standard use cases of FIX.
As in your case, to extract only data you are interested in, you need to define a class that will represent this data and to bind it to FIX using annotations. E.g.:
#FixBlock
public class MDEntry {
#FixField(tag=269) public int entryType; // you could define an enum type for it as well
#FixField(tag=278) public String entryId;
#FixField(tag=55) public String symbol;
}
...
FixFieldExtractor fixExtractor = new NativeFixFieldExtractor();
List<MDEntry> mdEntries = fixExtractor.getGroups(fixMsg, List.class, 268, FixMetaScanner.scanClass(MDEntry.class))
In more common cases, FixSerializer interface should be used, but it requires a message with MsgType(35) tag and a class annotated with #FixMessage(type="...") accordingly. E.g.:
#FixMessage(type="X")
public class MarketData {
#FixGroup(tag=268) public List<MDEntry> entries;
}
...
FixMetaDictionary fixMetaDictionary = FixMetaScanner.scanClassesIn("my.fix.classes.package");
FixSerializer fixSerializer = new NativeFixSerializer("FIX.5.0.SP2", fixMetaDictionary);
MarketData marketData = fixSerializer.deserialize(fixMsg);
I hope you will find it useful.
If you need just a MsgTyp, you're sure the message is correct and you do not need any other field from the message, then I would recommend extracting MsgType from string using regexp.
e.g.: \u000135=(\w+)\u0001
It is MUCH FASTER than parsing (and validating) a string via QuickFix.