Java ResiSearch FT.SEARCH results to json - java

I am new to RedisSearch. I have a Java client. What is the easiest way to parse this sample FT.SEARCH result into JSON or POJO or something more useful?
Sample result from FT.SEARCH (actually a string):
[
3,
movie_json: 1, [$, { "id": 1, "title": "Game of Thrones" } ],
movie_json: 3, [$, { "id": 3, "title": "Looking for Sugarman" } ],
movie_json: 2, [$, { "id": 2, "title": "Inception" } ]
]
Something like this would be useful:
{
"count": 3,
"docs": [
{ "id": 1, "title": "Game of Thrones" },
{ "id": 3, "title": "Looking for Sugarman" },
{ "id": 2, "title": "Inception" }
]
}
The most obvious is a RegEx matcher as below (I am no regex expert).
This is the code generated by the https://regex101.com/ site where I can get the right groups on their site as long as I use a global flag - but it seems that Java doesn't have a GLOBAL pattern / flag! Is that true?
The code the site generated is below and sure enough matcher.find() shows no match, presumably due to the absence of the global flag.
final String regex = "(?<=\\[\\$, ).*?(?= \\])";
final String string = respContent; // The rediSearch result string shown above
final Pattern pattern = Pattern.compile(regex);
final Matcher matcher = pattern.matcher(string);
while (matcher.find()) {
System.out.println("Full match: " + matcher.group(0));
for (int i = 1; i <= matcher.groupCount(); i++) {
System.out.println("Group " + i + ": " + matcher.group(i));
}
}
I could use the String.split() dance too.
However, is there an existing solution that is probably more robust for multiple FT.SEARCH results use-cases?
I imagined someone would have written a RedisSearch results parser by now but I cannot find one.
Thanks,
Murray


The high level Redis API for Quarkus only exposes the plain Redis commands as a set of java APIs. To handle Redis extensions, you can always refer to the low-level API: https://quarkus.io/guides/redis-reference
Once you choose the low-level API, you are, in fact, using the underlying driver that Quarkus uses. This is Vert.x Redis client.
In this mode, you can use any Redis extension and work with JSON directly, for example:
// set a JSON value
lowLevelClient
.send(cmd(Command.create("JSON.SET")).arg("foo").arg(".").arg("\"bar\""))
.compose(response -> {
// OK
// get a JSON value
return lowLevelClient.send(cmd(Command.create("JSON.GET")).arg("foo"));
})
.compose(response -> {
// verify that it is correct
should.assertEquals("\"bar\"", response.toString());
// do another call...
return lowLevelClient.send(cmd(Command.create("JSON.TYPE")).arg("foo").arg("."));
})
.compose(response -> {
should.assertEquals("string", response.toString());
return Future.succeededFuture();
})
.onFailure(should::fail)
.onSuccess(v -> {
test.complete();
});
While this mode is much more verbose, it gives you full control to the Redis extension you're using.
If the response can be mapped to JSON or is JSON already, you can get the content from its holder directly without need to parse the response, for example:
response.getKeys(); // returns the set of keys
response.get("key1"); // returns the JSON value for key "key1"
response.get(0); // returns the JSON value for array index 0
...

Related

Regex in a iterative manner only for Json Keys

I have been trying to replace the dots with a different symbol using regex. But unable to add all the dots in capturing groups to replace. Basically this dots needs to be replaced only for keys.
I have tried with [\w\d]*([.])]* which gets me all the instances where dots exists with words. But unable to set it just for keys.
If at all try ending the expression with " and : like [\w\d]+?([.])]*?[\w\d]+?[":] it selects only the last part of the key or value.
{
"header": {
"dataModelVersion": "3"
},
"content": {
"sasl.kerberos.service.name": {
"displayName": "Kerberos Service Name",
"identifiesControllerService": false,
"name": "sasl.kerberos.service.name",
"sensitive": false
}
}
}

From your input json, here I have assume that there is no digit present on json key. So, there is no need to add \d on regex.
Here, I have replace dot (.) on json key with character underscore (_). Yes, you can replace with any character if you want.
Matcher matcher = Pattern.compile("\"([\\w\\.]*)\"[:]", Pattern.CASE_INSENSITIVE)
.matcher(json);
while(matcher.find()) {
if(matcher.group().contains(".")) {
json = json.substring(0, matcher.start())
+ matcher.group().replace(".", "_")
+ json.substring(matcher.end());
}
}
System.out.println(json);
Result of this program as below:
{
"header": {
"dataModelVersion": "3"
},
"content": {
"sasl_kerberos_service_name": {
"displayName": "Kerberos Service Name",
"identifiesControllerService": false,
"name": "sasl.kerberos.service.name",
"sensitive": false
}
}
}

How To Extract Json Data Filed With RestApi?

i make a post to an api with rest assured. and than i try to make sure expected data from responsed data ,
but i got some errors like this -> "java.lang.IllegalArgumentException: The parameter "data" was used but not defined. Define parameters using the JsonPath.params(...) function"
my code:
String payload_data = "{" +
"\"Time\":1638057600, " +
"\"exampleType\":example, " +
"\"Id\":[2]}";
RestAssured.defaultParser = Parser.JSON;
given().
contentType(ContentType.JSON).
body(payload_data).
when().
post(api_url).
then().
statusCode(200).
body("data.examples.2.exampleData", equalTo("33"));
}
my json data
{
"success": true,
"data": {
"examples": {
"2": {
"ex_data": 0,
"exampleData": 33,
"data_ex": 0,
}
}
}

First, I tested path "data.examples.2.exampleData" with your json, it works fine. No problem.
You made some mistakes here.
Your payload is invalid json.
{
"Time": 1638057600,
"exampleType": example, //it must be number or String with double quote
"Id": [
2
]
}
You are comparing 2 things with different data types.
"data.examples.2.exampleData" --> int 33
equalTo("33") --> String "33"
Fix: body("data.examples.2.exampleData", equalTo(33));

Keep double quotes when getting element from JSONArray

I am using JsonPath to retrieve a value from a JSON file. The JSON file looks something like this:
[
{
"username": "John",
"password": {
"passwordValue": "passwordjohn",
"secret_key": "123"
}
},
{
"username": "Nick",
"password": {
"passwordValue": "XXX",
"secret_key": "ZZZ",
"other_key": "YYY"
}
}
]
The JsonPath I am using is to retrieve the password from a particular user. Example:
fun getPassword() {
val passwords: JSONArray = read(jsonFile, "\$.[?(#.name==\"John\")].password")
}
However, I found two obstacles. Firstly, I get back a net.minidev.json.JSONArray always, and the same path with appended [0] doesn't work.
Therefore, I try to get the only element from the JSONArray I get back, like this: credentials[0]. Unfortunately, this removes the double quotes in the field names, resulting in something like this:
{passwordValue: passwordjohn, secret_key: 123}
Which is impossible to work with.
I am looking for a way to get this back:
{"passwordValue": "passwordjohn", "secret_key": "123"}

What I ended up doing was to remove the [ and ] symbols from the beginning of the JSONArray after converting it to a String:
private fun JSONArray.toCredentialString(): String {
val credentialString = this.toString()
return credentialString.substring(1, credentialString.length - 1)
}
Any better solution is welcome.

Elasticsearch - Deleting nested object using java api not working

I have an elasticsearch documents which contain nested objects within them, I want to be able to remove them via the java update api. Here is the code containing the script:
UpdateRequest updateRequest = new UpdateRequest(INDEX, "thread", String.valueOf(threadId));
updateRequest.script("for (int i = 0; i < ctx._source.messages.size(); i++){if(ctx._source.messages[i]._message_id == " + messageId + ")" +
"{ctx._source.messages.remove(i);i--;}}", ScriptService.ScriptType.INLINE);
client.update(updateRequest).actionGet();
This is the mapping of my document:
{
"thread_and_messages": {
"mappings": {
"thread": {
"properties": {
"messages": {
"type": "nested",
"include_in_parent": true,
"properties": {
"message_id": {
"type": "string"
},
"message_nick": {
"type": "string"
},
"message_text": {
"type": "string"
}
}
},
"thread_id": {
"type": "long"
}
}
}
}
}
}
I'm not receiving any error messages, but when I run a query on the index to find that nested document it hasn't been removed. Could someone let me know what I am doing wrong?

Since message_id is a string your script needs to account for it and be modified like this (see the escaped double quotes around the message_id field). There is a second typo, in that your mapping declares a message_id field but you name it _message_id in your script:
"for (int i = 0; i < ctx._source.messages.size(); i++){if(ctx._source.messages[i].message_id == \"" + messageId + "\")"
^ ^ ^
| | |
no underscore here add escaped double quotes
Finally also make sure that you have dynamic scripting enabled in your ES config
UPDATE
You can try a "groovy-er" way of removing elements from lists, i.e. no more for loop and if, just use the groovy power:
"ctx._source.messages.removeAll{ it.message_id == \"" + messageId + "\"}"
Normally, that will modify the messages array by removing all elements whose message_id field matches the messageId value.

Extract specific value from complex JSON in java

I have got a valid complex json and I need to parse this json and print the values of only ak, dt and mi from this complex json in java... hope you can help me...
{
"CP": "{\"e\":{\"h\":{\"ak\":\"1c8d1d7eaa32ff3f58a882\",\"at\":\"app\"},\"c\":{\"dt\":\"MEmulator\",\"mi\":\"DD278047D56BF292F1FC16F\",\"ui\":\"m4J\/2s=\",\"av\":\"0.2\",\"pn\":\"WP\",\"pv\":\"7.10\",\"nv\":\"C# 1.1.0\",\"al\":\"en\"},\"b\":[{\"ts\":139658547,\"tz\":-400,\"s\":\"StartUpScreen\",\"et\":8,\"ev\":\"sessionStart\",\"si\":\"19477682-de55-414f-82c9-19bec331dc33\",\"tt\":{\"DaySessionStarted\":\"Tuesday\"}},{\"ts\":1319549658751,\"tz\":-400,\"s\":\"StartUpScreen\",\"et\":3,\"ev\":\"AutomaticFeedRefresh\",\"si\":\"19477682-de5ec331dc33\",\"tt\":{}},{\"ts\":1319549675609,\"tz\":-400,\"s\":\"MainScreen\",\"et\":3,\"ev\":\"MainScreen Event\",\"si\":\"19477682-de55-414f-82c9-19bec331dc33\",\"tt\":{}},{\"ts\":1319549677179,\"tz\":-400,\"s\":\"MainScreen\",\"et\":3,\"ev\":\"MainScreen Event\",\"si\":\"19477682-de55-414f-82c9-19bec331dc33\",\"tt\":{}},{\"ts\":1319549678401,\"tz\":-400,\"s\":\"MainScreen\",\"et\":3,\"ev\":\"MainScreen Event\",\"si\":\"19477682-de55-414f-82c9-19bec331dc33\",\"tt\":{}},{\"ts\":1319549679973,\"tz\":-400,\"s\":\"MainScreen\",\"et\":3,\"ev\":\"MainScreen Event\",\"si\":\"19477682-c9-19bec331dc33\",\"tt\":{}}],\"tt\":{\"OSV\":\"ME\"}}}",
"SP": {
"httpHeaders": {
"x-bluecoat-via": [
"35D3468F4D5F18"
],
"content-type": [
"application\/x-form-ur"
],
"connection": [
"Keep-Alive"
],
"host": [
"20.198.134.198:8080"
],
"accept": [
"text\/html, image\/gif, image\/jpeg, *; q=.2, *\/*; q=.2"
],
"content-length": [
"1791"
],
"user-agent": [
"Java\/1.6.0_23"
]
},
"senderIp": [
"112.101.216.113"
],
"receiveTimeStamp": "2012-06-26T06:29:36+0000"
}
}

Use json-path.
It's like xpath for JSON, and will allow you to write string queries on JSON objects.
There are a lot of examples on the project site showing possible usages, but in your case it's probably just a simple dot notation.
An example for the provided JSON:
// First extract the CP value, as its JSON-string-inside-JSON:
String cp = JsonPath.read(yourJsonString, "$.CP");
// Treat the `cp` as another JSON-string, and extract the ak value:
String ak = JsonPath.read(cp, "$.e.h.ak");
// Do the rest yourself...

You can use JsonPath to extract the value. I recommend JsonSurfer library.
<dependency>
<groupId>com.github.jsurfer</groupId>
<artifactId>jsurfer-simple</artifactId>
<version>1.2.1</version>
</dependency>
The sample code solving your problem in two steps:
1) Extract plain string from "$.CP" node.
2) Parse the "CP" string and extract value for "ak", "dt" and "mi".
JsonSurfer jsonSurfer = JsonSurfer.simple();
String cp = jsonSurfer.collectOne(new StringReader(yourString), String.class, "$.CP");
Collection<Object> result = jsonSurfer.collectAll(new StringReader(cp), "$..ak", "$..dt", "$..mi");

Like the others have suggested there are numerous libs out there that you can use (npe suggestion seems really nice). On the other hand, if you only have those simple cases and you don't really need to do anything else with JSON, maybe all you need is a regex. In essence JSON is just text, so you can do something like this:
Pattern akPattern = Pattern.compile("ak\":\"([^\"]+)");
Matcher matcher = akPattern.matcher(jsonAsString);
matcher.find();
String akValue = matcher.group(1);
System.out.println(akValue);
This prints out the value for "ak".
But again, I would only do this if I didn't have any other JSON requirements. Otherwise, go with a JSON lib.
My 2 cents.

Categories

Resources