I have to retrieve data from some site that sends back responses with edn bodies. I am trying to convert the sent back Edn to Json so I can parse it with Jsoup.
I found a website that was able to do the conversion, but how do I implement something like this in java?
I tried something like this, but it didn't a full job:
public static String edmToJson(String edm) {
String json = edm;
json = json.replaceFirst("(\\(\\{).*?(}\\))", "1").replace("(", "").replace("})", "").replace("} {", "},{");
return json;
}
Is there a way to do it without using closure?
You can parse EDN data in java by using a library like edn-java.
Sample usage:
#Test
public void simpleUsageExample() throws IOException {
Parseable pbr = Parsers.newParseable("{:x 1, :y 2}");
Parser p = Parsers.newParser(defaultConfiguration());
Map<?, ?> m = (Map<?, ?>) p.nextValue(pbr);
assertEquals(m.get(newKeyword("x")), 1L);
assertEquals(m.get(newKeyword("y")), 2L);
assertEquals(Parser.END_OF_INPUT, p.nextValue(pbr));
}
Complete docs available at edn-java
Related
I'm new to using protobuf, and was wondering if there is a simple way to convert a json stream/string to a protobuf stream/string in Java?
For example,
protoString = convertToProto(jsonString)
I have a json string that I want to parse into a protobuf message. So, I want to first convert the json string to protobuf, and then call Message.parseFrom() on it.
With proto3 you can do this using JsonFormat. It parses directly from the JSON representation, so there is no need for separately calling MyMessage.parseFrom(...). Something like this should work:
JsonFormat.parser().merge(json_string, builder);
//You can use this for converting your input json to a Struct / any other Protobuf Class
import com.google.protobuf.Struct.Builder;
import com.google.protobuf.Struct;
import com.google.protobuf.util.JsonFormat;
import org.json.JSONObject;
JSONObject parameters = new JSONObject();
Builder structBuilder = Struct.newBuilder();
JsonFormat.parser().merge(parameters.toString(), structBuilder);
// Now use the structBuilder to pass below (I used it for Dialog Flow V2 Context Management)
Since someone asked about getting the exception "com.google.protobuf.InvalidProtocolBufferException: JsonObject" when following Adam's advice--I ran into the same issue. Turns out it was due to the google protobuf timestamps. They are being serialized as an object containing two fields "seconds" and "nanos", since this isn't production code, I just got around this by parsing the JSON using jackson, going through the JSON object recursively and changing every timestamp from an object to a string formatted as per RFC 3339, I then serialized it back out and used the protobuf JSON parser as Adam has shown. This fixed the issue. This is some throwaway code I wrote (in my case all timestamp fields contain the word "timestamp", this could be more robust, but I don't care):
public Map<String, Object> fixJsonTimestamps(Map<String, Object> inMap) {
Map<String, Object> outMap = new HashMap<>();
for(String key : inMap.keySet()) {
Object val = inMap.get(key);
if(val instanceof Map) {
Map<String, Object> valMap = (Map<String, Object>)val;
if(key.toLowerCase().contains("timestamp") &&
valMap.containsKey("seconds") && valMap.containsKey("nanos")) {
if(valMap.get("seconds") != null) {
ZonedDateTime d = ZonedDateTime.ofInstant(Instant.ofEpochSecond((int)valMap.get("seconds")).plusNanos((int)valMap.get("nanos")),
ZoneId.of("UTC"));
val = d.format(DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"));
}
} else {
val = fixJsonTimestamps(valMap);
}
} else if(val instanceof List && ((List) val).size() > 0 &&
((List) val).get(0) instanceof Map) {
List<Map<String, Object>> outputList = new ArrayList<>();
for(Map item : (List<Map>)val) {
outputList.add(fixJsonTimestamps(item));
}
val = outputList;
}
outMap.put(key, val);
}
return outMap;
}
Not the most ideal solution but it works for what I am doing, I think I saw someone recommend using a different timestamp class.
You can convert json string to Proto using builder and json String
Example :
YourProto.Builder protoBuilder = YourProto.newBuilder();
JsonFormat.parser().merge(JsonString, protoBuilder);
If you want to ignore unknown json field then
YourProto.Builder protoBuilder = YourProto.newBuilder();
JsonFormat.parser()..ignoringUnknownFields().merge(JsonString, protoBuilder);
Another way is, to use mergeFrom method from ProtocolBuffer
Example :
YourProto.Builder protoBuilder = YourProto.newBuilder();
protoBuilder.mergeFrom(JsonString.getBytes());
Once it execute, you will get all the data in protoBuilder from json String
online service:
https://json-to-proto.github.io/
This tool instantly converts JSON into a Protobuf. Paste a JSON structure on the left and the equivalent Protobuf will be generated to the right, which you can paste into your program. The script has to make some assumptions, so double-check the output!
I have the following:
I notice that at the end of running the code, if I print out aggregations.asMap().get('subjects');
I am getting:
org.elasticsearch.search.aggregations.bucket.terms.StringTerms#6cff59fa
Printing out "aggregations" gives me: org.elasticsearch.search.aggregations.InternalAggregations#65cf321d
What I really want is the entire string/json response that is normally returned if you were to curl on elasticsearch to get aggregations. How do I get to the raw response from the aggregation query? Also, is there a way to iterate and print out what's in those "wrapped up" objects?
https://github.com/spring-projects/spring-data-elasticsearch/blob/ab7e870d5f82f6c0de236048bd7001e8e7d2a680/src/test/java/org/springframework/data/elasticsearch/core/aggregation/ElasticsearchTemplateAggregationTests.java
#Test
public void shouldReturnAggregatedResponseForGivenSearchQuery() {
// given
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withSearchType(COUNT)
.withIndices("articles").withTypes("article")
.addAggregation(terms("subjects").field("subject"))
.build();
// when
Aggregations aggregations = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<Aggregations>() {
#Override
public Aggregations extract(SearchResponse response) {
return response.getAggregations();
}
});
// then
System.out.println(aggregations); // gives me some cryptic InternalAggregations object, how do I get to the raw JSON normally returned by elasticsearch?
System.out.println(aggregations.asMap().get("subjects")); // gives me some StringTerms object I have no idea how to iterate over to get results
}
You cannot get the raw JSON response this way, since Spring Data Elasticsearch will take care of parsing it for you, that's the whole point.
If you need to parse those buckets, you can do it like this easily:
...
StringTerms subjects = aggregations.asMap().get("subjects");
for (Terms.Bucket bucket : subjects.getBuckets()) {
String key = bucket.getKey();
long docCount = bucket.getDocCount();
// do something with the key and the doc count
}
If you really want to see the JSON being returned, what you can do is to re-write the parsed Aggregations object into JSON using serialization, but that won't really be helpful:
InternalAggregations aggregations = ...;
XContentBuilder jsonBuilder = JsonXContent.contentBuilder();
aggregations.toXContent(jsonBuilder, ToXContent.EMPTY_PARAMS);
String rawJson = jsonBuilder.string();
Set Size of EsRequest to Zero
Get Esresponse.toString()
Convert String to Json
Get aggregation field from Json.
I need to create a JSON response with some dynamic fields in java. Here is an example of the JSON response I want to return :
{
"success": true,
"completed_at": 1400515821,
"<uuid>": {
type: "my_type",
...
},
"<uuid>": {
type: "my_type",
...
}
}
The "success" and the "completed_at" fields are easy to format. How can I format the fields? What would be the corresponding java object?
Basically I want to work with 2 java objects :
public class ApiResponseDTO {
private boolean success;
private DateTime completedAt;
...
}
and
public class AuthenticateResponseDTO extends ApiResponseDTO {
public List<ApplianceResponseDTO> uuids = new ArrayList<ApplianceResponseDTO>();
}
These java objects don't correspond to the expected JSON format. It would work if I could change the JSON format to have a list, but I can't change it.
Thanks a lot!
You can massage your data into JSON form using the javax.json library, specifically the JsonObjectBuilder and the JsonArrayBuilder. You'll probably want to nest a few levels of a toJson() method which will either give you the string representation you're looking for, or the JsonObject/JsonArray you desire. Something like this:
JsonArray value = null;
JsonArrayBuilder builder = Json.createArrayBuilder();
for (ApplianceResponseDTO apr : uuids) {
builder.add(apr.toJson());
}
value = builder.build();
return value;
I am connecting to a third party API and getting back a long JSON string. I only need one value from it, but it is located pretty deep inside the hierarchy. Is there a simple way to get it, without going through the whole thing? I looked all over but nothing seems easy.
Here's my example:
"response":{"status":1,"httpStatus":200,"data":{"myDesiredInfo":"someInfo"},"errors":[],"errorMessage":null}}
I've been trying to use Gson so I can get this blob as a JsonObject. I was sure there's something simple, like this:
jsonObject.get("myDesiredInfo")
or at the minimum something like this:
jsonObject.get("response.data.myDesiredInfo")
But it doesn't seem to exist.
So is there any parser out there that will allow me to do this?
This is my json string
String s="{"age":0,"name":"name","email":"emailk","address":{"housename":"villa"}}";
I use following code to get housename
JsonElement je = new JsonParser().parse(s);
JsonObject asJsonObject = je.getAsJsonObject();
JsonElement get = asJsonObject.get("address");
System.out.println(s + "\n" + get);
JsonObject asJsonObject1 = get.getAsJsonObject();
JsonElement get1 = asJsonObject1.get("housename");
System.out.println(get1);
The Following is my output :
{"age":0,"name":"name","email":"emailk","address":{"housename":"villa"}}
{"housename":"villa"}
"villa"
I don't think there is another way to do this. I also tried to do in other ways but i didn't get any output.
The following way you can retrieve from your jsonObject.
JSONObject jObject = new JSONObject(yourresponse);
Log.i("Desired Info is ",jObject.getJSONObject("response").getJSONObject("data").getString("myDesiredInfo"));
I wrote a little utility method that uses Gson's API to get a value as String from a JSON object, based on a java.util.List of values. So for my original question the list objects will be "response", "data", "myDesiredInfo."
Surely this can be improved on, but it's a start.
/*
* Takes a JsonObject and parses it for a primitive value, going level by level
* according to the values in #infos
*/
public static String parseJson(JsonObject json, List<String> infos) {
try {
if(infos.size() == 0) {
return json.toString();
}
JsonElement je = json.get((String)infos.get(0));
infos.remove(0);
if(je instanceof JsonObject) {
return parseJson(je.getAsJsonObject(), infos);
} else {
return je.getAsString();
}
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
Most languages have a JSON decoding library, a lot of them native. No idea what language you're using so here's PHP as an example:
$jsonObj = json_decode($json);
$json->response->data->myDesiredInfo;
Ruby, Python, Java - all these languages have good libraries.
I have the JSON String of the below format which I get as a http request in Java. I need to get the name and values of the below JSON string, but I am not able to get the correct solution.
Can any one tell me how to parse this? Also, let me know if we will be able to format this string because there is no property names from the 3 element.
The string format is
{
'appname':'application',
'Version':'0.1.0',
'UUID':'300V',
'WWXY':'310W',
'ABCD':'270B',
'YUDE':'280T'
}
edit#1 formatted the question.
In JavaScript, you can do something like
var v = eval("("+data_from_server+")");
var aName = v.appname;
For example this script will alert appname.
<script>
var serverdata = "{'appname':'application', 'Version':'0.1.0', 'UUID':'300V', 'WWXY':'310W', 'ABCD':'270B', 'YUDE':'280T'}";
var v = eval("("+serverdata+")");
alert(v.appname);
</script>
Based on your comment on this answer, here is a way to parse in Java
In Java, you may want to leverage GSon. See here.
You need to define a Java class that maps the JSON object one-to-one. Then ask GSon to create a Java object using the JSON String. Here is the example.
Your Java class that maps JSON should look like this
public class MyData{
public String appname;
public String Version;
public String UUID;
public String WWXY;
public String ABCD;
public String YUDE;
public MyData(){}
}
And you parse in Java like this.
String jsons = "{'appname':'application', 'Version':'0.1.0', 'UUID':'300V', 'WWXY':'310W', 'ABCD':'270B', 'YUDE':'280T'}";
Gson gson = new Gson();
MyData obj = gson.fromJson(jsons, MyData.class);
System.out.println("ada "+ obj.appname);
With which language do you want to do that ?
Here is the solution for PHP :
$data = json_decode('the json');