So in Java, I have this as my json String:
public static void main(String[] args) {
String stringJson = "{\n" +
"nodes: {\n" +
"Random-key-Here: {\n" +
"name: \"PRO-cnt-elastic2-4-i-0a414518a5b67\",\n" +
"transport_address: \"172.18.7.104:9300\",\n" +
"host: \"189.88.7.884\",\n" +
"ip: \"188.88.8.884:8880\",\n" +
"roles: [\n" +
"\"ingest\"\n" +
"],\n" +
"attributes: {\n" +
"zone: \"us-east-1a\"\n" +
"},\n" +
"tasks: {\n" +
"Random-key-Here: {\n" +
"node: \"-2688888mRPYHywWA\",\n" +
"id: 37202060,\n" +
"type: \"transport\",\n" +
"action: \"indices:data/write/reindex\",\n" +
"status: {\n" +
"total: 4869544,\n" +
"updated: 13920,\n" +
"created: 3654080,\n" +
"deleted: 0\n" +
"},\n" +
"description: \"blaaa\",\n" +
"start_time_in_millis: 1596456902705,\n" +
"running_time_in_nanos: 647855785005,\n" +
"cancellable: true,\n" +
"headers: { }\n" +
"}\n" +
"}\n" +
"}\n" +
"}\n" +
"}";
JsonObject jo = new JsonParser().parse(stringJson).getAsJsonObject();
Set s = jo.get("nodes").getAsJsonObject().entrySet().stream().map(e ->
e.getValue().getAsJsonObject().get("tasks").collect(toSet());
}
Important: notice inside the Json some of the Keys are actually a random string that I couldn't know what it is in advance... - so I Wrote them as "Random-key-Here"
In short: My goal is to retrieve the number of this field, "total: 4869544".
Above that's what I've tried. I've managed to have a list that contains one object member of a set, and it's the "Random-key-Here" as the key, and the value is the rest of my nested json.
I thought if I would repeat the same logic as I did to overcome the first "Random-key-Here" thing, like I did with
.entrySet().stream().map(e ->
e.getValue().getAsJsonObject().get("tasks")
then I would get to another level below, so instead of "tasks" I wrote "status"
and I tought then I would be able to get the rest of the nested json which is easier from then on... but instead, nothing is returned when doing that:
jo.get("nodes").getAsJsonObject().entrySet()
.stream()
.map(e -> e.getValue().getAsJsonObject().get("tasks").getAsJsonObject().entrySet()
.stream()
.map(ent-> ent.getValue().getAsJsonObject().get("status").getAsJsonObject().get("total").getAsJsonObject()))
would appreciate any help to get to the nested part I need which is: "total: 4869544".
and keep in mind, I can't simply just use this:
jo.get("nodes").getAsJsonObject().get("Random-key-Here").getAsJsonObject().get("tasks")
.getAsJsonObject().get("Random-key-Here").getAsJsonObject()
.get("status").getAsJsonObject().get("total").getAsInt()
because of "Random-key-Here".
Tnx for the helpers!
Answer to myself: So, on every unknown key that you have, you can use entrySet()
and then loop over it until you get to your desired field member.
so this usage solves the problem and can get you the nested Json even when your key is unknown in advance:
jo.get("nodes").getAsJsonObject().entrySet()
.forEach(node -> node.getValue().getAsJsonObject().get("tasks").getAsJsonObject().entrySet()
.forEach(task -> { int i = task.getValue().getAsJsonObject().get("status").getAsJsonObject().get("total").getAsInt();
System.out.println(i); }))
pls share if someone has a more efficient way to do so
Related
I'm using Java High level rest client in my project and i want to limit my indices based on document count. I'm using rollover api but its not creating indices automatically.
The code will be given below.
I'm creating index pattern so that my custom analyzer will be applied to all other indices that follow the respective pattern.
PutIndexTemplateRequest request = new PutIndexTemplateRequest("testingtemplate");
request.source("{\n" +
" \"index_patterns\":[\n" +
" \"test_log-*\"\n" +
" ],\n" +
" \"settings\": {\n" +
" \"analysis\": {\n" +
" \"analyzer\": { \n" +
" \"my_analyzer\": {\n" +
" \"type\": \"custom\",\n" +
" \"tokenizer\": \"whitespace\",\n" +
" \"filter\": []\n" +
" }\n" +
" }\n" +
" }\n" +
" },\n" +
" \"mappings\": {\n" +
" \"properties\": {\n" +
" \"fullLog\": {\n" +
" \"type\": \"text\",\n" +
" \"analyzer\": \"my_analyzer\"\n" +
" }\n" +
" }\n" +
" }\n" +
" }",XContentType.JSON);
return client.indices().putTemplate(request,RequestOptions.DEFAULT).isAcknowledged();
My Rollover code. Here i want to rollover one when index gets one or document.
final RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
boolean isIndexTemplateCreated = createIndexTemplate(client);
System.out.println(isIndexTemplateCreated);
CreateIndexRequest request = new CreateIndexRequest("test_log-1");
request.alias(new Alias("temp_alias_new"));
CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);
RolloverRequest roll_req = new RolloverRequest("temp_alias_new",null);
roll_req.addMaxIndexAgeCondition(new TimeValue(7, TimeUnit.DAYS));
roll_req.addMaxIndexDocsCondition(1);
roll_req.addMaxIndexSizeCondition(new ByteSizeValue(5, ByteSizeUnit.GB));
RolloverResponse rolloverResponse = client.indices().rollover(roll_req, RequestOptions.DEFAULT);
Map<String,Object> map = new HashMap<>();
map.put("fullLog","Hi");
client.index(new IndexRequest("temp_alias_new").source(map), RequestOptions.DEFAULT);
map.put("fullLog","hello");
client.index(new IndexRequest("temp_alias_new").source(map), RequestOptions.DEFAULT);
But the code is not working and the rollover api is not creating indices automatically. All the 2 documents are stored only in test_log-1 index.
Is there any mistake in my code?
Thanks
Note: Rollover will not happen automatically.
Elasticsearch tries to rollover the index on getting a rollover request.
For example consider the following sequences:
You have a new index test_log-1 which is empty and is being pointed by the alias temp_alias_new.
If you try to rollover now, none of the conditions mentioned along with rollover request holds good as the index is new and empty. So this time rollover fails.
Add some documents to the index.
Now try the rollover with the condition maxIndexDocsCondition(1), it will rollover. Because the condition holds good.
Update
With the latest release of elastic search, you can use ILM (Index Life cycle Management) to automate the rollover.
Here is doc link for more info: https://www.elastic.co/guide/en/elasticsearch/reference/7.x/getting-started-index-lifecycle-management.html
I have a string object representing a json object returning for a network task. I need to convert it into a Map (or HashMap). I've been using gson, but it has been unsuccessful. Here is the json string (please excuse indentation, for I had to manually add newline spaces):
{
"plans":{
"Ankle Recovery":{
"StartDate":"09/24/2018",
"Progress":0.6666666666666666,
"Tasks":[
{
"date":"10/16/2018",
"amount":200,
"task":"ice ankle for 30 min",
"completed":true,
"requirementType":"steps"},
{
"date":"10/17/2018",
"amount":200,
"task":"ice ankle for 30 min",
"completed":true,
"requirementType":"steps"
},
{
"date":"10/18/2018",
"amount":200,
"task":"ice ankle for 30 min",
"completed":false,
"requirementType":"steps"
}
],
"Username":"email#site.com",
"Doctor":"Mike Michaels",
"EndDate":"12/24/2018"}},
"status":true
}
This is the code I've been using to make the transformation:
private Map<String, String> plans;
plans = new Gson().fromJson(result, new TypeToken<Map<String, String>>() {}.getType());
Neither nor has worked. I've tried some different solutions across Stack Overflow, but none yield success to this point.
I'm also getting an exception thrown that I don't quite understand:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 11
(Column 11 is just before the first quote in "AnkleRecovery")
I'd like to use simple gson to make this work if possible. But I'm open to alternative solutions.
The JSON you posted is not valid, line 3:
"Ankle Recovery" : {
// / \
// this is what you are missing
This tool will help you verify the JSON structure and format it as well: https://jsonlint.com/
Now to the actual problem. Your JSON has a following structure:
{
"plans": Object,
"status": Boolean,
}
Neither of these are strings ( object != string, boolean != string ).
Such a structure can not be mapped to Map<String, String> as this requires the value to be a string.
You will need to create multiple POJOs to define your structure and then map to these, e.g.:
class Project {
public Map<String,Plan> plans;
public Boolean status;
}
class Plan {
public String StartDate;
public Double Progress;
public List<Task> tasks;
...
}
class Task {
...
}
Disclaimer...
I would always investigate using one or more POJOs which can be used to represent the data structure if at all possible.
Without more information, it's impossible to know if keys like Ankle Recovery are stable or not, or if they might change.
"A" possible solution
Generally, JSON is in the form of key/value pairs, where the value might be another JSON object, array or list of other values, so you "could" process the structure directly, for example...
String text = "{\n"
+ " \"plans\":{\n"
+ " \"Ankle Recovery\":{\n"
+ " \"StartDate\":\"09/24/2018\",\n"
+ " \"Progress\":0.6666666666666666,\n"
+ " \"Tasks\":[\n"
+ " {\n"
+ " \"date\":\"10/16/2018\",\n"
+ " \"amount\":200,\n"
+ " \"task\":\"ice ankle for 30 min\",\n"
+ " \"completed\":true,\n"
+ " \"requirementType\":\"steps\"\n"
+ " },\n"
+ " {\n"
+ " \"date\":\"10/17/2018\",\n"
+ " \"amount\":200,\n"
+ " \"task\":\"ice ankle for 30 min\",\n"
+ " \"completed\":true,\n"
+ " \"requirementType\":\"steps\"\n"
+ " },\n"
+ " {\n"
+ " \"date\":\"10/18/2018\",\n"
+ " \"amount\":200,\n"
+ " \"task\":\"ice ankle for 30 min\",\n"
+ " \"completed\":false,\n"
+ " \"requirementType\":\"steps\"\n"
+ " }\n"
+ " ],\n"
+ " \"Username\":\"email#site.com\",\n"
+ " \"Doctor\":\"Mike Michaels\",\n"
+ " \"EndDate\":\"12/24/2018\"\n"
+ " }\n"
+ " },\n"
+ " \"status\":true\n"
+ "}";
Gson gson = new Gson();
Map<String, Object> fromJson = gson.fromJson(text, Map.class);
Map<String, Object> plans = (Map<String, Object>) fromJson.get("plans");
Map<String, Object> recovery = (Map<String, Object>) plans.get("Ankle Recovery");
List<Map<String, Object>> tasks = (List<Map<String, Object>>) recovery.get("Tasks");
for (Map<String, Object> taks : tasks) {
for (Map.Entry<String, Object> entry : taks.entrySet()) {
System.out.println(entry.getKey() + " = " + entry.getValue());
}
}
Now, this would get you the output of ...
date = 10/16/2018
amount = 200.0
task = ice ankle for 30 min
completed = true
requirementType = steps
date = 10/17/2018
amount = 200.0
task = ice ankle for 30 min
completed = true
requirementType = steps
date = 10/18/2018
amount = 200.0
task = ice ankle for 30 min
completed = false
requirementType = steps
Having said all that, your own parsing might be a lot more involved, have to inspect if certain keys exist or not and taking appropriate action as required
I am trying to implement a function to be able to remove or modify a json object base on a specified json path. For example, if i have a below json string/object:
{
"PersonalDetailsDTO": {
"FirstName": "Mark",
"LastName": "Sully",
"TotalDependent": "2",
"DOB": "19811212",
"SecQuestion": "Some Que",
"SecAnswer": "Some-Ans",
"Mobile": "0123456789",
"Email": "some#validemail.com",
"Title": "Mr",
"EmploymentListDTO": [
{
"Type": "Full-time",
"Probation": true
}
],
"AddressListDTO": [
{
"AddressType": "BUS",
"PostCode": "1234",
"State": "NSW",
"StreetName": "miller",
"StreetNumber": "111",
"StreetType": "Invalid",
"Suburb": "Sydney",
"UnitNumber": "Maximum"
}
]
}
}
And i want to remove element $.PersonalDetailsDTO.AddressListDTO.PostCode.
I've done quite some search, and the one lib i found is JsonPath: http://static.javadoc.io/com.jayway.jsonpath/json-path/2.2.0/com/jayway/jsonpath/JsonPath.html
So i wrote the below code:
public static void main(String[] args) {
// Prints "Hello, World" to the terminal window.
String jsonString = "{\n" +
" \"PersonalDetailsDTO\": {\n" +
" \"FirstName\":\"Mark\",\n" +
" \"LastName\":\"Sully\",\n" +
" \"Title\":\"Mr\",\n" +
" \"DOB\":\"19811201\",\n" +
" \"SecQuestion\":\"Some Ques\",\n" +
" \"SecAnswer\":\"Some-Ans\",\n" +
" \"Email\":\"some#validemail.com\",\n" +
" \"EmploymentListDTO\": [\n" +
" {\n" +
" \"Type\": \"Full-time\",\n" +
" \"Probation\": true\n" +
" }\n" +
" ],\n" +
" \"AddressListDTO\": [\n" +
" {\n" +
" \"AddressType\": \"Residential\",\n" +
" \"PostCode\": \"2345\",\n" +
" \"State\": \"NSW\",\n" +
" \"StreetName\": \"MEL\",\n" +
" \"StreetNumber\": \"2\",\n" +
" \"StreetType\": \"Boulevard\",\n" +
" \"Suburb\": \"Melbourne\",\n" +
" \"UnitNumber\": \"345\"\n" +
" }\n" +
" ]\n" +
" } \n" +
"}";
JSONObject jsonObject = new JSONObject(jsonString);
System.out.println("Before: " + jsonObject.toString());
JsonPath jp = JsonPath.compile("$.PersonalDetailsDTO.AddressListDTO[0].PostCode");
Configuration conf = Configuration.defaultConfiguration();
Object json = conf.jsonProvider().parse(jsonString);
System.out.println("After: " + jp.delete(json, conf).toString());
}
And the console log displays:
Before: {"PersonalDetailsDTO":{"EmploymentListDTO":[{"Type":"Full-time","Probation":true}],"SecAnswer":"Some-Ans","Email":"some#validemail.com","SecQuestion":"Some Ques","FirstName":"Mark","DOB":"19811201","AddressListDTO":[{"StreetName":"MEL","Suburb":"Melbourne","State":"NSW","StreetNumber":"2","UnitNumber":"345","AddressType":"Residential","PostCode":"2345","StreetType":"Boulevard"}],"Title":"Mr","LastName":"Sully"}}
After: {PersonalDetailsDTO={FirstName=Mark, LastName=Sully, Title=Mr, DOB=19811201, SecQuestion=Some Ques, SecAnswer=Some-Ans, Email=some#validemail.com, EmploymentListDTO=[{"Type":"Full-time","Probation":true}], AddressListDTO=[{"AddressType":"Residential","State":"NSW","StreetName":"MEL","StreetNumber":"2","StreetType":"Boulevard","Suburb":"Melbourne","UnitNumber":"345"}]}}
Looks like JsonPath is doing it's job and removing $.PersonalDetailsDTO.AddressListDTO.PostCode. However, there's something very obvious that bothers me:
Looking at the json string produced by .toString() in before and after case, JSONObject API printed a nice string in true json standard format with every double quotes "" present, while the JsonPath .toString produce a customer string format that has some elements in double quote "" while others are not and i can not use it further like JSONObject.
And what i noticed is that although JsonPath claim to accept "java.lang.Object" as parameter in many of its function, what it truely accept is something called "jsonProvider". Not sure if it's causing the weird .toString() behavior.
Anyway, does anyone know how get a nice formatted json string out of JsonPath APIs like remove(), put(), read() and many other? Or to convert the return value to something like JSONObject?
If you know any other Java lib that can do remove/modify element by json path, please feel free to recommand. Thank you!
I don't know JsonPath.
I think you should use jackson which is defacto standard lib when work with JSON in java
aproximate what you are going to do is:
String jsonString = "{"k1": {"k2":"v2"}";
ObjectMapper mapper = new ObjectMapper();
JsonNode actualObj = mapper.readTree(jsonString);
actualObj.at("/k1/k2").getValueAsInt()
and replace getValueAsInt with any other function
I'm not sure how to properly pass the multiple attributes needed for an OpenLDAP insert via UnboundID. I have omitted the objectClass attributes & received a "no objectClass" error. I have also tried comma-separated & the bracket/array route like below & received the "value #0 invalid per syntax" error.
String[] ldifLines = {"dn: ou=users,dc=sub,dc=domain,dc=com", "cn: " + uid, "userPassword: " + pw, "description: user", "uidNumber: " + lclDT, "gidNumber: 504", "uid: " + uid, "homeDirectory: " + File.separator + "home" + File.separator + this.getStrippedUser(), "objectClass: {posixAccount, top}"};
LDAPResult ldapResult = lclLC.add(new AddRequest(ldifLines));
So, the question is, how do I successfully pass these objectClass attributes in the string array included above? Again, I have tried: "objectClass: top, posixAccount" as well. Thanks in advance!
It uses an LDIF representation, so if an attribute has multiple values, then the attribute appears multiple times. Like:
String[] ldifLines =
{
"dn: ou=users,dc=sub,dc=domain,dc=com",
"objectClass: top",
"objectClass: posixAccount"
"cn: " + uid,
"userPassword: " + pw,
"description: user",
"uidNumber: " + lclDT,
"gidNumber: 504",
"uid: " + uid,
"homeDirectory: " + File.separator + "home" +
File.separator + this.getStrippedUser(),
};
LDAPResult ldapResult = lclLC.add(new AddRequest(ldifLines));
Also, the LDAP SDK allows you to use a shortcut and just do it in a single call without the need to create the array or the AddRequest object, like:
LDAPResult ldapResult = lclLC.add(
"dn: ou=users,dc=sub,dc=domain,dc=com",
"objectClass: top",
"objectClass: posixAccount"
"cn: " + uid,
"userPassword: " + pw,
"description: user",
"uidNumber: " + lclDT,
"gidNumber: 504",
"uid: " + uid,
"homeDirectory: " + File.separator + "home" +
File.separator + this.getStrippedUser());
I want some help to find a quick solution for my problem. Given a json object that is large with a recursive model. I want to list the JSON sub elements & its immediate parent Object( only the sub object which satisfies the given key value condition).
Ex :
{
Object : {
id : "0001",
parent:"A",
child: {
id:"0001A",
Country:"US",
parent:"B",
child:{
id:"0001AA",
Country:"UK",
parent:"C",
child:{
id:"0000AAA",
Country:"US",
parent:"D",
child:{
.........
}
}
}
}
}
}
I want to list the id's of the subObject whose country is 'US' and it's parent id..
is there available any readymade plugins to handle these kind of scenarios in JAVA , without using object mappers/custom class objects..
Ps provide any possible idea ..
Yes, it is possible write code using the Jackson Tree Model API which would traverse a JSON tree and select the nodes that satisfy criteria. Here is an example:
public class JacksonTree2 {
public static final String JSON = "{\"Ex\" : {\"Object\" : {\n" +
" \"id\" : \"0001\",\n" +
" \"parent\":\"A\",\n" +
" \"child\": {\n" +
" \"id\":\"0001A\",\n" +
" \"Country\":\"US\",\n" +
" \"parent\":\"B\",\n" +
" \"child\":{\n" +
" \"id\":\"0001AA\",\n" +
" \"Country\":\"UK\",\n" +
" \"parent\":\"C\",\n" +
" \"child\":{\n" +
" \"id\":\"0000AAA\",\n" +
" \"Country\":\"US\",\n" +
" \"parent\":\"D\",\n" +
" \"child\":{\n" +
" \n" +
" }\n" +
" }\n" +
" }\n" +
"\t}\n" +
"}}}";
public static void main(String[] args) throws IOException {
ObjectMapper mapper = new ObjectMapper();
JsonNode root = mapper.readTree(JSON);
for (JsonNode node : root.findParents("Country")) {
if ("UK".equals(node.get("Country").asText())) {
System.out.println(node.get("id"));
break;
}
}
}
}
Output:
"0001AA"