Develop Reference

Gson: Expected begin_array but was STRING how to control this

Im learning how to produce and consume JSON in rest services, but I wanna learn it well so im trying all possible cases of objects, one of them is an object that has an List attribute like this class:
import java.util.List;
public class PruebaJSON {
private String nombre;
private List atributos;
private String descripcion;
public String getNombre() {
return nombre;
}
public void setNombre(String nombre) {
this.nombre = nombre;
}
public List getAtributos() {
return atributos;
}
public void setAtributos(List atributos) {
this.atributos = atributos;
}
public String getDescripcion() {
return descripcion;
}
public void setDescripcion(String descripcion) {
this.descripcion = descripcion;
}
}
Then all what im doing on my rest service method is this:
#POST
#Path("/prueba")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public PruebaJSON prueba(String data) {
try {
JSONObject json = new JSONObject(data);
Gson convertir = new GsonBuilder().create();
PruebaJSON pruebaJson = convertir.fromJson(json.toString(), PruebaJSON.class);
return pruebaJson;
} catch (Exception e) {
System.out.println("error " + e);
return null;
}
}
Then in POSTMAN I pass this:
{
"descripcion": "Primera prueba",
"nombre": "Prueba 1",
"atributos": [
"hello",
"kek",
"lul"
]
}
And it works fine, the problem is when I try to do the same by Java, for example:
List atributos = new ArrayList<>();
atributos.add("hello");
atributos.add("kek");
atributos.add("lul");
System.out.println(bus.prueba("Prueba 1", "Primera Prueba", atributos));
bus.prueba just executes the service but then in console I get this error:
14:16:56,567 INFO [stdout] (default task-2) error com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was STRING at line 1 column 66 path $.atributos
I did the search of the error and found this:
Gson: Expected begin_array but was STRING
I understand the error but whats the solution?
I can't really control how the JSON builds the arraylist can I?
This is the method prueba in my client:
public String prueba(String nombre, String descripcion, List atributos) {
HashMap map = new HashMap<>();
map.put("nombre", nombre);
map.put("descripcion", descripcion);
map.put("atributos", atributos);
String respuesta = utilidadesRestSeguridad.consumir("prueba", map);
return respuesta;
}
In my client component this is the method that builds the json:
public static JsonObject generateJSON(HashMap map) throws MalformedURLException {
JsonObject json = new JsonObject();
for (Object key : map.keySet()) {
json.addProperty(key.toString(), map.get(key).toString());
}
return json;
}
And thats it guys if you wanna see more code or me to explain something tell me I appreciate any help.
I think maybe the error is in the method generateJSON because of the .toString(), but then how I should handle that case?

Assuming that the line utilidadesRestSeguridad.consumir("prueba", map) ends up calling your generateJSON method downstream, then your issue is likely in the generateJSON() method as you suspect. Basically, you are just adding all elements as strings. If one of the elements in your map is an instance of a List, then you need to call JsonObject#add("atributos", value). For example, you will need something like the following code:
if (map.get(key) instanceof List) {
json.add(key.toString(), map.get(key);
} else {
json.addProperty(key.toString(), map.get(key).toString());
}

As I suspected, the error was in the generateJSON method, needed to add this validation that entpnerd suggested:
public static JsonObject generateJSON(HashMap map) throws MalformedURLException {
JsonObject json = new JsonObject();
for (Object key : map.keySet()) {
if (map.get(key) instanceof List) {
JsonParser parser = new JsonParser();
parser.parse((map.get(key).toString()));
json.add(key.toString(), parser.parse((map.get(key).toString())));
} else {
json.addProperty(key.toString(), map.get(key).toString());
}
}
return json;
}
Notice that I had to use JsonParser, not sure how is it working but at the end that made it work.
Source: How to parse this JSON String with GSON?
Anyways Im gonna try the solution entpnerd is suggesting and post it too.
Here is the implementation of entpnerd suggestion:
public static JsonObject generateJSON(HashMap map) throws MalformedURLException {
JsonObject json = new JsonObject();
for (Object key : map.keySet()) {
if (map.get(key) instanceof List) {
JsonArray jsonArray = new JsonArray();
for (Object object : (ArrayList<Object>) map.get(key)) {
jsonArray.add(object.toString());
}
json.add(key.toString(), jsonArray);
} else {
json.addProperty(key.toString(), map.get(key).toString());
}
}
return json;
}
it works too, you guys decide which one to use, thanks very much.
My only question is, what if the element that is an array, has more arrays inside it, what would you do?

You don't need to get that json value manually, add requestbody annotation to your method parameter
public PruebaJSON prueba(#RequestBody PruebaJSON json){
System.out.println(json);
};

How to parse json in Spring Boot and write to a repository

I am trying to parse a JSON API response of historical time series data with potentially thousands of line. The response is in the following format:
{
"name": "AAPL",
"history": {
"2019-03-05": {
"open": "175.94",
"close": "175.53",
"high": "176.00",
"low": "174.54",
"volume": "19163899"
},
"2019-03-04": {
"open": "175.69",
"close": "175.85",
"high": "177.75",
"low": "173.97",
"volume": "27436203"
}
}
}
I would like to write the response to a Spring repository. I have a simple code to do this and a section is shown below:
RestTemplate restTemplate = new RestTemplate();
JsonParser jsonParser = new JsonParser();
JsonObject jsonObject = (JsonObject) jsonParser.parse(result);
JsonElement jsonElement = jsonObject.get("history");
Set<Map.Entry<String, JsonElement>> entrySet = jsonElement.getAsJsonObject().entrySet();
for (Map.Entry<String, JsonElement> entry : entrySet) {
StockHistory stockHistory = new StockHistory();
stockHistory.setSymbol(stk);
// .... Other code
}
I set the object properties as per JSON response, add the object to a list, and finally save the list to a repository. This process is very slow presumably because I am creating a new StockHistory object for every element in the JSON return. I was wondering if there is a better way of doing it.

As you cannot modify the JSON structure. I would like to add the following code that can parse the JSON that you provided in a simple class called Repo. In order to do that you need to add the library from here that I have used.
Now you need to add the following classes in your code.
public class Repo {
public String name;
public ArrayList<History> histories;
public Repo() {
histories = new ArrayList<History>();
}
}
public class History {
public String date;
public HistoryElements elements;
}
public class HistoryElements {
public String volume;
public String high;
public String low;
public String close;
public String open;
}
Hence I have written a RepoParser and tested it with your JSON String and it parses the JSON into the classes.
import com.oracle.javafx.jmx.json.JSONException;
import org.json.JSONObject;
import java.text.ParseException;
import java.util.ArrayList;
import java.util.Iterator;
public class RepoParser {
public static Repo parseRepo(String jsonString) throws ParseException, JSONException {
JSONObject jsonObject = new JSONObject(jsonString);
Iterator<?> iterator = jsonObject.keys();
Repo repo = new Repo();
while (iterator.hasNext()) {
String obj = iterator.next().toString();
if (obj.equals("name")) repo.name = obj;
else repo.histories = parseHistory((JSONObject) jsonObject.get(obj));
}
return repo;
}
public static ArrayList<History> parseHistory(JSONObject jsonObject) throws ParseException, JSONException {
Iterator<?> iterator = jsonObject.keys();
ArrayList<History> historyList = new ArrayList<>();
while (iterator.hasNext()) {
String obj = iterator.next().toString();
History history = new History();
history.date = obj;
history.elements = parseHistoryElement((JSONObject) jsonObject.get(obj));
historyList.add(history);
}
return historyList;
}
public static HistoryElements parseHistoryElement(JSONObject jsonObject) throws ParseException, JSONException {
Iterator<?> iterator = jsonObject.keys();
HistoryElements historyElements = new HistoryElements();
while (iterator.hasNext()) {
String obj = iterator.next().toString();
if (obj.equals("open")) historyElements.open = jsonObject.getString("open");
if (obj.equals("close")) historyElements.close = jsonObject.getString("close");
if (obj.equals("high")) historyElements.high = jsonObject.getString("high");
if (obj.equals("low")) historyElements.low = jsonObject.getString("low");
if (obj.equals("volume")) historyElements.volume = jsonObject.getString("volume");
}
return historyElements;
}
}
Just use the RepoParser class like the following.
try {
Repo repo = RepoParser.parseRepo(jsonString);
} catch (Exception e) {
e.printStackTrace();
}
I have created a Gist as well for convenience.
Update
You might consider adding all the Repo in a list and then save all of them into your database at once using the save method of the repository.
Hence the code should be something like the following.
try {
while(there is no repo left for parsing) {
Repo repo = RepoParser.parseRepo(jsonString);
repoList.add(repo)
}
yourRepository.save(repoList); // Save all values at once
} catch (Exception e) {
e.printStackTrace();
}
Hope that helps!

After some research, I found that the problem was with hibernate. As far as I understand it, a useful feature of hibernate is that it caches objects, but this causes a problem when a large number of objects are created for insertion. The issue can be resolved by batch processing using the spring.jpa.properties.hibernate.jdbc.batch_size property and using a sequence generator in the entity class. Now saving the list of many thousands of lines is much faster.

Getting error "Expected BEGIN_OBJECT but was STRING at line 1" from a multi layer json

Here is the entire class:
public class Item {
static class Page {
Map <String,String> other_data;
Map <String,Map<String,List<Map<String,String>>>> specification;
}
public static String showName() throws Exception {
String json = Json.fetch(jsonurl);
Gson gson = new Gson();
Page result = gson.fromJson(json, Page.class);
return result.specification.get("result").get("feature").get(0).get("value");
// not working.
//return result.other_data.get("id"); <-- this one working
}
}
Here's how I fetch the json:
public class Json {
public static String fetch(String urlString) throws Exception {
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(Connection.auth(urlString)));
StringBuffer buffer = new StringBuffer();
int read;
char[] chars = new char[1024];
while ((read = reader.read(chars)) != -1)
buffer.append(chars, 0, read);
//return buffer.toString();
return buffer.toString();
} finally {
if (reader != null)
reader.close();
}
}
}
I have been struggling to get a specific value from a mixed-type JSON feed using gson.
{
"other_data":{"id":"150","name":"AA"},
"specification":{"result":{"feature":[{"name":"attribute A","value":"50"}]}}
}
The feed should be valid since I can get 150 from other_data
`return result.other_data.get("id");`
However I can't get the value 50 from the first object of the array feature:
return result.specification.get("result").get("feature").get(0).get("value");
I'm receiving this error:
Expected BEGIN_OBJECT but was STRING at line 1 column 37948 path $.specification.
I think the declaration Map <String,Map<String,List<Map<String,String>>>> specification; is incorrect. I did a little debugging by changing it to Map <String,Object> specification. I managed to get the stringified object
{"feature":[{"name":"attribute A","value":"50"}]}
public class Item {
static class Page {
String page_type;
String name;
Map <String,String> submit_user_data;
Map <String,Object> specification;
}
public static String showName() throws Exception {
String json = Json.fetch(jsonurl);
Gson gson = new Gson();
Page td = gson.fromJson(json, Page.class);
return td.specification.get("result").toString(); // this one works!
}
}
Would anyone tell me what's wrong with the class getting the error?

Apparently escaping the json string solves the issue:
public static void main(String args[]) {
String json = "{\"other_data\":{\"id\":\"150\",\"name\":\"AA\"},\"specification\":{\"result\":{\"feature\":[{\"name\":\"attribute A\",\"value\":\"50\"}]}}}";
Gson gson = new Gson();
Page result = gson.fromJson(json, Page.class);
System.out.println(result.specification.get("result").get("feature").get(0).get("value"));
}

How to check if stream in twitch.tv is on?

Here is code of my getStream method:
public static Twitch_Stream getStream(String channelname) {
try {
String json = API.readJsonFromUrl("https://api.twitch.tv/kraken/streams?channel=" + channelname);
Twitch_Stream stream = new Twitch_Stream();
if (json.equalsIgnoreCase("[]")) {
stream.setOnline(false);
return stream;
}
JsonArray jb = gson.fromJson(json, JsonArray.class);
if (jb.size() != 0) {
JsonObject jo = (JsonObject) jb.get(0);
stream.setOnline(true);
stream.load(jo);
}
return stream;
} catch (Exception error) {
error.printStackTrace();
}
return null;
}
and here is code of Twitch_Stream class http://pastebin.com/3RX1L1cv
When I make something like this
Twitch_Stream streamer = Twitch_API.getStream("Jankos");
Bukkit.broadcastMessage("getName " + streamer.getName());
Bukkit.broadcastMessage(streamer.isOnline() + "");
streamer.getName() return null and streamer.isOnline() returns false, even when stream is on.
Where did I make a mistake?

I don't know what problem is in your code but simple workaround would be reading content from "https://api.twitch.tv/kraken/streams/" + channel which is JSON in format:
{
"_links" : {
//links to stream and channel
},
"stream" : {
//details about current stream
}
}
Now if value of stream key is null stream is off-line. If it is not null, it is on-line.
So your code can look like
public static void main(String[] argv) throws IOException {
System.out.println(checkIfOnline("Jankos"));
System.out.println(checkIfOnline("nightblue3"));
}
public static boolean checkIfOnline(String channel) throws IOException {
String channerUrl = "https://api.twitch.tv/kraken/streams/" + channel;
String jsonText = readFromUrl(channerUrl);// reads text from URL
JSONObject json = new JSONObject(jsonText);
return !json.isNull("stream");
}
private static String readFromUrl(String url) throws IOException {
URL page = new URL(url);
try (Stream<String> stream = new BufferedReader(new InputStreamReader(
page.openStream(), StandardCharsets.UTF_8)).lines()) {
return stream.collect(Collectors.joining(System.lineSeparator()));
}
}
I used JSONObject from org.json library. I am also using Java 8 and its streams.
If you want to use gson you can use instead something like
public static boolean checkIfOnline(String channel) throws IOException {
String channerUrl = "https://api.twitch.tv/kraken/streams/" + channel;
String jsonText = readFromUrl(channerUrl);// reads text from URL
JsonParser parser = new JsonParser();
JsonObject json = parser.parse(jsonText).getAsJsonObject();
return !json.get("stream").isJsonNull();
}
If you don't have Java 8 you can rewrite code reading text from URL to something like
private static String readFromUrl(String url) throws IOException {
URL page = new URL(url);
StringBuilder sb = new StringBuilder();
Scanner scanner = null;
try{
scanner = new Scanner(page.openStream(), StandardCharsets.UTF_8.name());
while (scanner.hasNextLine()){
sb.append(scanner.nextLine());
}
}finally{
if (scanner!=null)
scanner.close();
}
return sb.toString();
}
or from what I see you can use your API.readJsonFromUrl instead of readFromUrl.

JSON jsonObject.optString() returns String "null"

I'm developing an Android App which uses JSON for the server communication and I've got a weird problem when I'm trying to parse my json file.
This is my json from the server
{
"street2": null,
"province": null,
"street1": null,
"postalCode": null,
"country": null,
"city": null
}
I'm getting the value for City by calling String city = address.optString("city", "") on my address Json-object. For this situation I'm expecting cityto be empty (that's what optString is here for isn't it?) but in fact it contains the String "null". So further null- or isEmpty-checks will return false as the String contains text. If I call address.isNull("city") it returns true which is correct. Only optString fails.
I couldn't find anything on Google or Stackoverflow for this problem. I don't really understand how it can happen as I thought optString would do exactly what I expected. Anybody knows what's going wrong here?

You're not alone in running into this problem and scratching your head, thinking "Could they really have meant this?" According to an AOSP issue, the Google engineers did consider this a bug, but they had to be compatible with the org.json implementation, even bug-compatible.
If you think about it, it makes sense, because if the same code which uses the same libraries run in other Java environments behaves differently in Android, there would be major compatibility problems when using 3rd party libraries. Even if the intentions were good and it truly fixed bugs, it would open up a whole new can of worms.
According to the AOSP issue:
The behavior is intentional; we went out of our way to be bug-compatible with org.json. Now that that's fixed, it's unclear whether we should fix our code as well. Applications may have come to rely on this buggy behavior.
If this is causing you grief, I recommend you workaround by using a different mechanism to test for null, such as json.isNull().
Here's a simple method to help you out:
/** Return the value mapped by the given key, or {#code null} if not present or null. */
public static String optString(JSONObject json, String key)
{
// http://code.google.com/p/android/issues/detail?id=13830
if (json.isNull(key))
return null;
else
return json.optString(key, null);
}

You basically have 2 choices:
1) Send a JSON payload with null values
{
"street2": "s2",
"province": "p1",
"street1": null,
"postalCode": null,
"country": null,
"city": null
}
You will have to check for null values and parse them accordingly:
private String optString_1(final JSONObject json, final String key) {
return json.isNull(key) ? null : json.optString(key);
}
2) Do not send the keys with null values and use optString(key, null) directly (should save you bandwidth).
{
"street2": "s2",
"province": "p1"
}

Got rid off this situation by simply replacing "null" with "".
String city = address.optString("city").replace("null", "");

Using Matt Quigley's answer as a basis, here is the code if you desire to mimic the full functionality of optString, including the fallback portion, written in Kotlin and Java.
Kotlin:
fun optString(json: JSONObject, key: String, fallback: String?): String? {
var stringToReturn = fallback
if (!json.isNull(key)) {
stringToReturn = json.optString(key, null)
}
return stringToReturn
}
Java:
public static String optString(JSONObject json, String key, String fallback) {
String stringToReturn = fallback;
if (!json.isNull(key)) {
stringToReturn = json.optString(key, null);
}
return stringToReturn;
}
Simply pass in null for the fallback parameter if you don't need the fallback.

I ended up creating a utility class for this purpose:
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import org.json.JSONException;
import org.json.JSONObject;
final public class JsonUtil {
#Nullable
public static String optString(
#NonNull JSONObject object,
#NonNull String key
) throws JSONException {
if (object.has(key) && !object.isNull(key)) {
return object.getString(key);
}
return null;
}
}

I do like this...
String value;
if(jsonObject.get("name").toString().equals("null")) {
value = "";
}else {
value = jsonObject.getString("name");
}

if (json != null && json.getString(KEY_SUCCESS) != null){
// PARSE RESULT
}else{
// SHOW NOTIFICIATION: URL/SERVER NOT REACHABLE
}
that is for checking json null with there key word.
JSONObject json = new JSONObject("{\"hello\":null}");
json.getString("hello");
this you get is String "null" not null.
your shoud use
if(json.isNull("hello")) {
helloStr = null;
} else {
helloStr = json.getString("hello");
}
first check with isNull()....if cant work then try belows
and also you have JSONObject.NULL to check null value...
if ((resultObject.has("username")
&& null != resultObject.getString("username")
&& resultObject.getString("username").trim().length() != 0)
{
//not null
}
and in your case also check
resultObject.getString("username").trim().eqauls("null")
If you must parse json first and handle object later, let try this
Parser
Object data = json.get("username");
Handler
if (data instanceof Integer || data instanceof Double || data instanceof Long) {
// handle number ;
} else if (data instanceof String) {
// hanle string;
} else if (data == JSONObject.NULL) {
// hanle null;
}

My Josn parser was long and had to create a new class to fix that,
then just had to add 1 extra line in each method and rename current JSONObject property name, so all other calls were referencing to my new class instead to JSONObject.
public static ArrayList<PieceOfNews> readNews(String json) {
if (json != null) {
ArrayList<PieceOfNews> res = new ArrayList<>();
try {
JSONArray jsonArray = new JSONArray(json);
for (int i = 0; i < jsonArray.length(); i++) {
//before JSONObject jo = jsonArray.getJSONObject(i);
JSONObject joClassic = jsonArray.getJSONObject(i);
//facade
FixJsonObject jo = new FixJsonObject(joClassic);
PieceOfNews pn = new PieceOfNews();
pn.setId(jo.getInt("id"));
pn.setImageUrl(jo.getString("imageURL"));
pn.setText(jo.getString("text"));
pn.setTitle(jo.getString("title"));
pn.setDate(jo.getLong("mills"));
res.add(pn);
}
return res;
} catch (JSONException e) {
e.printStackTrace();
}
}
return null;
}
Here is my class with the methods I needed, you can add more
public class FixJsonObject {
private JSONObject jsonObject;
public FixJsonObject(JSONObject jsonObject) {
this.jsonObject = jsonObject;
}
public String optString(String key, String defaultValue) {
if (jsonObject.isNull(key)) {
return null;
} else {
return jsonObject.optString(key, defaultValue);
}
}
public String optString(String key) {
return optString(key, null);
}
public int optInt(String key) {
if (jsonObject.isNull(key)) {
return 0;
} else {
return jsonObject.optInt(key, 0);
}
}
public double optDouble(String key) {
return optDouble(key, 0);
}
public double optDouble(String key, double defaultValue) {
if (jsonObject.isNull(key)) {
return 0;
} else {
return jsonObject.optDouble(key, defaultValue);
}
}
public boolean optBoolean(String key, boolean defaultValue) {
if (jsonObject.isNull(key)) {
return false;
} else {
return jsonObject.optBoolean(key, defaultValue);
}
}
public long optLong(String key) {
if (jsonObject.isNull(key)) {
return 0;
} else {
return jsonObject.optLong(key, 0);
}
}
public long getLong(String key) {
return optLong(key);
}
public String getString(String key) {
return optString(key);
}
public int getInt(String key) {
return optInt(key);
}
public double getDouble(String key) {
return optDouble(key);
}
public JSONArray getJSONArray(String key) {
if (jsonObject.isNull(key)) {
return null;
} else {
return jsonObject.optJSONArray(key);
}
}
}

If values for key is null like below
{
"status": 200,
"message": "",
"data": {
"totalFare": null,
},
}
check with "isNull" , for Eg:
String strTotalFare;
if (objResponse.isNull("totalFare"))
{
strTotalFare = "0";
} else {
strTotalFare = objResponse.getString("totalFare");
}
if value is "null" for key "totalFare", above function will enter in if and assign value zero else it will get actual value from key.