Hide some array elements in Elasticsearch query result - java

I index objects in Elasticsearch 2.4 where one of the fields is an array of other objects, for example, something like this:
{
A : 1,
B : [{BB : 1},{BB : 2},{BB : 4},{BB : 5}]
}
Is it possible to query Elasticsearch and apply a filter to hide some of the arrays's B values, for example, to hide values where BB is less than 3 such that a result would be
{
A : 1,
B : [{BB : 4},{BB : 5}]
}
P.S.
I'm using Java API

I didn't find a flexible built in way of doing it so I decided to filter manually. Here's my implementation:
SearchResponse resp = request.execute().actionGet();
for (SearchHit hit : resp.getHits().getHits()) {
Map<String, Object> source = hit.getSource();
transformHitSource(source); // a method where you work with a map and can modify it as you want
XContentBuilder xContentBuilder = jsonBuilder().startObject();
for (Map.Entry<String, Object> entry : source.entrySet()) {
xContentBuilder.field(entry.getKey(), entry.getValue());
}
BytesReference bytes = xContentBuilder.endObject().bytes();
((InternalSearchHit) hit).sourceRef(bytes);
}

Related

How to add JSON values to an ArrayList in Java

I have the following JSON file:
{
"meta" : {
"stock" : "AWS",
"date modified" : 90
},
"roles" : [ "Member", "Admin" ],
"name" : "John Doe",
"admin" : true,
"email" : "john.doe#example.com"
}
I wanted to both read the values of the keys and add them to an Array List.
try {
// create object mapper instance
ObjectMapper mapper = new ObjectMapper();
// convert JSON file to map
Map<?, ?> map = mapper.readValue(Paths.get("user.json").toFile(), Map.class);
ArrayList<String> data = new ArrayList<String>();
// print map entries
for (Map.Entry<?, ?> entry : map.entrySet()) {
System.out.println((entry.getClass()) + " " + entry.getValue());
data.add((String)entry.getValue()); // trying to add entry values to arraylist
}
} catch (Exception ex) {
ex.printStackTrace();
}
I'm able to print out the data type of the value along with the value itself. All the values are part of class java.util.LinkedHashMap$Entry. I'm not able to cast the values to a String to add them to an ArrayList. How should I go about doing this? Thanks
From the jackson-databind documentation you can convert your json to a Map<String, Object> map with the following line (you have boolean, list, number and string values in your json) :
Map<String, Object> map = mapper.readValue(json, Map.class);
// it prints {meta={stock=AWS, date modified=90}, roles=[Member, Admin], name=John Doe, admin=true, email=john.doe#example.com}
System.out.println(map);
If you want to save your map values string representation into an ArrayList data you can iterate over them with a loop :
List<String> data = new ArrayList<>();
for (Object value : map.values()) {
data.add(value.toString());
}
//it will print [{stock=AWS, date modified=90}, [Member, Admin], John Doe, true, john.doe#example.com]
System.out.println(data);
Your data type of entries will be like:
meta: Map<String:Object>
roles: List<String>
admin: Boolean
So you will get an exception when casting to string for each entry value.
You should handle different data type and convert it according to your request:
Object value = entry.getValue();
I highly recommend you write more few functions to check and convert map/list/primitive variables to expected data (String):
boolean isList(Object obj);
boolean isMap(Object obj);
...
public List<String> convertMap(Map<String,Object> map);
...

Java remove "parent":"somevalue" from flattened json if “parent”:“somevalue” and there also exists a “parent.new”: “somevalue” in same json

I Have json like this :
{
"lastModifiedBy" :"value",
"lastModifiedBy.$oid": "1234567189",
"displayedBy" : 0,
"displayedBy.one" : "Abhi",
"displayedBy.one.new" : "Sammy",
"displayedBy.two":"random_value3",
"a.b":null,
"b.c":null,
"d.e.f":null
}
I only want to keep the longest keys and not the previous state parent without affecting the other keys
output I need:
{
"lastModifiedBy.$oid": "1234567189",
"displayedBy.one.new" : "Sammy",
"displayedBy.two":"random_value3",
"a.b":null,
"b.c":null,
"d.e.f":null
}
Is there a way to do this in Jackson or any other java package.?
If you are able to convert it into an ArrayList, remove all keys you dont need. For example with foreach.
If you are able to convert it into an String split at ":" and "," remove the "s create an ArrayList and populate it by using index 0,2,4... as key and 1,3,5... as value and do the thing mentioned above.
Try this.
Map<String, String> map = new HashMap<>();
map.put("lastModifiedBy", "value");
map.put("lastModifiedBy.$oid", "1234567189");
map.put("displayedBy", "0");
map.put("displayedBy.one", "Abhi");
map.put("displayedBy.one.new", "Sammy");
map.put("displayedBy.two", "random_value3");
map.put("a.b", null);
map.put("b.c", null);
map.put("d.e.f", null);
List<String> removes = new ArrayList<>();
for (String parent : map.keySet())
for (String child : map.keySet())
if (parent != child && child.startsWith(parent)) {
removes.add(parent);
break;
}
for (String key : removes)
map.remove(key);
System.out.println(map);
output:
{a.b=null, d.e.f=null, displayedBy.one.new=Sammy, b.c=null, displayedBy.two=random_value3, lastModifiedBy.$oid=1234567189}

Parsing unnamed nested arrays with minimal-json?

So I'm working on a fairly simple Java program which grabs market data from cryptocurrency exchanges and displays information to the user. I am using the minimal-json library.
Here is my current code:
public class Market {
static JsonArray arrayBittrex;
public static void startTimer(){
Timer timer = new Timer();
timer.scheduleAtFixedRate(new TimerTask() {
#Override
public void run() {
String url = "https://bittrex.com/api/v1.1/public/getmarketsummaries";
try {
URL url2 = new URL(url);
URLConnection con = url2.openConnection();
InputStream in = con.getInputStream();
String encoding = "UTF-8";
String body = IOUtils.toString(in, encoding);
arrayBittrex = Json.parse(body).asObject().get("result").asArray();
}
catch(MalformedURLException e) {}
catch(IOException e) {}
}
}, 0,5000);
}
public static float getPrice(String exchange, String market) {
for (JsonValue item : arrayBittrex) {
float last = item.asObject().getFloat("Last", 0);
System.out.println(last);
return last;
}
return 0;
}
}
This code works with simple json, for example (from https://bittrex.com/api/v1.1/public/getmarketsummary?market=btc-ltc):
{
"success" : true,
"message" : "",
"result" : [{
"MarketName" : "BTC-LTC",
"High" : 0.01350000,
"Low" : 0.01200000,
"Volume" : 3833.97619253,
"Last" : 0.01349998
}
]
}
It will properly return the "Last" value in the array.
However, this cant work when the json has multiple arrays (like in https://bittrex.com/api/v1.1/public/getmarketsummaries):
{
"success" : true,
"message" : "",
"result" : [{
"MarketName" : "BTC-888",
"High" : 0.00000919,
"Low" : 0.00000820,
"Volume" : 74339.61396015,
"Last" : 0.00000820
}, {
"MarketName" : "BTC-A3C",
"High" : 0.00000072,
"Low" : 0.00000001,
"Volume" : 166340678.42280999,
"Last" : 0.00000005
}
]
}
So my question is: how can I get the "Last" value by searching for the array by the "MarketName" value?
Here is a direct & null-safe way to tackle this using Java 8 library Dynamics. We're going to parse the json into a Map, read that map dynamically to what we want.
So first we can use Jackson, Gson or something to convert json -> map.
// com.fasterxml.jackson.core:jackson-databind json -> map
Map jsonMap = new ObjectMapper()
.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
.readValue(jsonStringOrInputSourceEtc, Map.class);
We can now get a Dynamic instance. And, for example, grab the BTC-A3C - Last value.
Dynamic json = Dynamic.from(jsonMap);
BigDecimal a3cLast = json.get("result").children()
.filter(data -> data.get("MarketName").asString().equals("BTC-A3C"))
.findAny()
.flatMap(data -> data.get("Last").maybe().convert().intoDecimal())
.orElse(BigDecimal.ZERO);
// 5E-8
Or perhaps convert the whole lot into a map of MarketName -> Last value
Map<String, BigDecimal> marketNameLastValue = json.get("result").children()
// assume fields are always present, otherwise see #maybe() methods
.collect(toMap(
data -> data.get("MarketName").asString(),
data -> data.get("Last").convert().intoDecimal()
));
// {BTC-A3C=5E-8, BTC-888=0.00000820}
See more examples https://github.com/alexheretic/dynamics

Retrieving a subset of a JSON object stored in Dynamo

Is it currently possible to retrieve a subset of a json object stored in Dynamo? For example, I have an attribute named record of JSON type which is an array of JSON objects:
records:
[
{"K1": "V1" },
{"K2": "V2" },
{"K3": "V3" },
{"K4": "V4" }
]
I store them in the JSON format in Dynamo. I wanted to know if I can only retrieve key value pairs 1 to 2 and not the 3rd and 4th one? I am unsure if I can provide a specific filter expression to do this operation.
If it is possible, I would love to hear the methodology as to how it can be done?
Thanks!
Firstly, there is no JSON data type in DynamoDB. If you mean the data is stored as DynamoDB data type MAP, then the below solution should work for you.
In short, the filter expression should be something like below:-
FilterExpression : 'records.K1 = :recordsK1Value and records.K2 = :recordsK2Value'
If you need to have only "records.K1" and "records.K2" in output, you can use project expression for that.
ProjectionExpression : 'records.K1, records.K2'
Full code:-
public List<String> queryMoviesAndFilterByMapAttribute() {
List<String> moviesJsonList = new ArrayList<>();
DynamoDB dynamoDB = new DynamoDB(dynamoDBClient);
Table table = dynamoDB.getTable("Movies");
QuerySpec querySpec = new QuerySpec();
querySpec.withKeyConditionExpression("yearkey = :yearval and title = :titleval")
//.withProjectionExpression("records.K1, records.K2")
.withFilterExpression("records.K1 = :recordsK1Value and records.K2 = :recordsK2Value").withValueMap(
new ValueMap().withNumber(":yearval", 1991).withString(":titleval", "Movie with map attribute")
.withString(":recordsK1Value", "V1").withString(":recordsK2Value", "V2"));
IteratorSupport<Item, QueryOutcome> iterator = table.query(querySpec).iterator();
while (iterator.hasNext()) {
Item movieItem = iterator.next();
System.out.println("Movie data ====================>" + movieItem.toJSONPretty());
moviesJsonList.add(movieItem.toJSON());
}
return moviesJsonList;
}
Sample output with all fields (i.e. without project expression):-
Movie data ====================>{
"yearkey" : 1991,
"records" : {
"K1" : "V1",
"K2" : "V2",
"K3" : "V3",
"K4" : "V4"
},
"title" : "Movie with map attribute"
}
Sample output after un-commenting the project expression:-
Please note that other fields such as yearkey, title, K3 and K4 are not present in the output.
Movie data ====================>{
"records" : {
"K1" : "V1",
"K2" : "V2"
}
}

how to insert multiple values of an attribute into mongoDB using java with Map?

I am writing a java code to insert form values into mongoDB using java code. I am using map to retrieve all the values from the map and inserting it into mongoDB. However, if an attribute is having multiple values, it is only inserting only one value. My code is:
Map<String, String[]> articleData = request.getParameterMap();
for(String key : articleData.keySet())
{
for(int i=0; i<articleData.get(key).length;i++)
{
document.put(key,articleData.get(key)[i]);
}
}
table.insert(document);
However, right now, it is overriding the values of the attribute having multiple values.
How can I resolve it?
Try this, It will give you a basic idea. Adjust code according to your program:
Map<String, String[]> articleData = request.getParameterMap();
for(String key : articleData.keySet())
{
BasicDBObject data =new BasicDBObject();
for(int i=0; i<articleData.get(key).length;i++)
{
data.put("",articleData.get(key)[i]);
}
document.put(key,data);
}
table.insert(document);
Encode a JSON object .
Try this out.
Map<String, String[]> articleData = request.getParameterMap();
for(String key : articleData.keySet())
{
JSONObject out = new JSONObject();
out.put("key", key);
out.put("value", articleData.get(key));
System.out.println(out);
}
dbobj.put("multiple",out);
collection.insert(dbobj);

Categories

Resources