as title of the questions says; is it possible in DynamoDB to query for a particular attribute value within a nested JSON document? I've been looking for the answer on official docs, SO, forums, etc., however I'm still unable to provide a valid query.
Example of the entity objects in Java:
#DynamoDBTable(tableName = "Car")
public class Car {
#DynamoDBHashKey
String id;
String color;
String type;
Engine engine;
}
#DynamoDBDocument
public class Engine {
String manufacturer;
int hp;
String model;
}
Records in database:
{
"id" : "43-asdasda4214ads13-13",
"color" : "blue",
"type" : "sedan",
"engine" : {
"manufacturer" : "ford",
"hp" : 450,
"model" : "av15"
}
}
{
"id" : "24ads123-adad123-das1",
"color" : "red",
"type" : "cabrio",
"engine" : {
"manufacturer" : "peugeot",
"hp" : 130,
"model" : "ig31-2"
}
}
Can I query, let's say all cars which have engine provided by ford? If so, how could I do that? I've already tried keyConditionExpression in following format engine.manufacturer = ford, but no success. What I'd like to avoid is using FilterExpression.
Thanks in advance!
Related
collection:
{
"username" : "gorim99949#x1post.com",
"channels" : [
{
"channelName" : "yyyyy",
"id" : 383,
"url" : "https://xxxxxxxxx",
}
],
}
#Document(collection = "user_masters")
public class UserMasters {
#Field("username")
private String userName;
#DBRef
private List<UserChannels> UserChannels userChannels;
******getter and setter*********
#Document(collection="channels")
public class UserChannels {
private String url;
************getter/setter****************
Repository class
public UserMasters findByUserName(String username);
#Test method
Query: userRepository.findByUserName(userName).getUserChannels().getUrl();
how can i get value of url from array in the collection in mongodb. I am unable to read values in the array in the collection. thanks in advance for your help
You are going to need a custom query to do that.
Run this query:
db.collection.aggregate({$match: {username: "gorim99949#x1post.com"}}, {$project: {"userChannels": "$channels.url"}});
It will return:
[{"_id": ObjectId("5a934e000102030405000000"),"userChannels": ["https://xxxxxxxxx"]}].
You can remove _id from returning document by setting it to 0 in projection.
I am creating a new endpoint in springboot that will return simple stats on users generated from an aggregate query in a mongo database. However I get a PropertyReferenceException. I have read multiple stackoverflow questions about it, but didn't find one that solved this problem.
We have a mongo data scheme like this:
{
"_id" : ObjectId("5d795993288c3831c8dffe60"),
"user" : "000001",
"name" : "test",
"attributes" : {
"brand" : "Chrome",
"language" : "English" }
}
The database is filled with multiple users and we want using Springboot aggregate the stats of users per brand. There could be any number of attributes in the attributes object.
Here is the aggregation we are doing
Aggregation agg = newAggregation(
group("attributes.brand").count().as("number"),
project("number").and("type").previousOperation()
);
AggregationResults<Stats> groupResults
= mongoTemplate.aggregate(agg, Profile.class, Stats.class);
return groupResults.getMappedResults();
Which produces this mongo query which works:
> db.collection.aggregate([
{ "$group" : { "_id" : "$attributes.brand" , "number" : { "$sum" : 1}}} ,
{ "$project" : { "number" : 1 , "_id" : 0 , "type" : "$_id"}} ])
{ "number" : 4, "type" : "Chrome" }
{ "number" : 2, "type" : "Firefox" }
However when running a simple integration test we get this error:
org.springframework.data.mapping.PropertyReferenceException: No property brand found for type String! Traversed path: Profile.attributes.
From what I understand, it seems that since attributes is a Map<String, String> there might be a schematic problem. And in the mean time I can't modify the Profile object.
Is there something I am missing in the aggregation, or anything I could change in my Stats object?
For reference, here are the data models we're using, to work with JSON and jackson.
The Stats data model:
#Document
public class Stats {
#JsonProperty
private String type;
#JsonProperty
private int number;
public Stats() {}
/* ... */
}
The Profile data model:
#Document
public class Profiles {
#NotNull
#JsonProperty
private String user;
#NotNull
#JsonProperty
private String name;
#JsonProperty
private Map<String, String> attributes = new HashMap<>();
public Stats() {}
/* ... */
}
I found a solution, which was a combination of two problems:
The PropertyReferenceException was indeed caused because attributes is a Map<String, String> which means there is no schemes for Mongo.
The error message No property brand found for type String! Traversed path: Profile.attributes. means that the Map object doesn't have a brand property in it.
In order to fix that without touching my orginal Profile class, I had to create a new custom class which would map the attributes to an attributes object having the properties I want to aggreate on like:
public class StatsAttributes {
#JsonProperty
private String brand;
#JsonProperty
private String language;
public StatsAttributes() {}
/* ... */
}
Then I created a custom StatsProfile which would leverage my StatsAttributes and would be similar to the the original Profile object without modifying it.
#Document
public class StatsProfile {
#JsonProperty
private String user;
#JsonProperty
private StatsAttributes attributes;
public StatsProfile() {}
/* ... */
}
With that I made disapear my problem with the PropertyReferenceException using my new class StatsAggregation in the aggregation:
AggregationResults<Stats> groupResults
= mongoTemplate.aggregate(agg, StatsProfile.class, Stats.class);
However I would not get any results. It seems the query would not find any document in the database. That's where I realied that production mongo objects had the field "_class: com.company.dao.model.Profile" which was tied to the Profile object.
After some research, for the new StatsProfile to work it would need to be a #TypeAlias("Profile"). After looking around, I found that I also needed to precise a collection name which would lead to:
#Document(collection = "profile")
#TypeAlias("Profile")
public class StatsProfile {
/* ... */
}
And with all that, finally it worked!
I suppose that's not the prettiest solution, I wish I would not need to create a new Profile object and just consider the attributes as a StatsAttributes.class somehow in the mongoTemplate query. If anyone knows how to, please share 🙏
I am using spring-boot-starter-parent 1.4.1.RELEASE.
Spring boot #ResponseBody doesn't serialize entity id by default.
If I use exposeIdsFor returns the id but I need to configure this for each class
e.g
RepositoryRestConfiguration.exposeIdsFor(User.class);
RepositoryRestConfiguration.exposeIdsFor(Address.class);
Is there a simpler configuration to do this without configuring each entity class.
Refer: https://jira.spring.io/browse/DATAREST-366
The REST resource will render the attribute as a URI to it’s corresponding associated resource. We need to return the associated object instead of URI.
If I use Projection, it will returns the associated objects but I need to configure this for each class
e.g
#Entity
public class Person {
#Id #GeneratedValue
private Long id;
private String firstName, lastName;
#ManyToOne
private Address address;
…
}
PersonRepository:
interface PersonRepository extends CrudRepository<Person, Long> {}
PersonRepository returns,
{
"firstName" : "Frodo",
"lastName" : "Baggins",
"_links" : {
"self" : {
"href" : "http://localhost:8080/persons/1"
},
"address" : {
"href" : "http://localhost:8080/persons/1/address"
}
}
}
My Projection:
#Projection(name = "inlineAddress", types = { Person.class })
interface InlineAddress {
String getFirstName();
String getLastName();
Address getAddress();
}
After adding projection, it returns..
{
"firstName" : "Frodo",
"lastName" : "Baggins",
"address" : {
"street": "Bag End",
"state": "The Shire",
"country": "Middle Earth"
},
"_links" : {
"self" : {
"href" : "http://localhost:8080/persons/1"
},
"address" : {
"href" : "http://localhost:8080/persons/1/address"
}
}
}
If some other classes having the association as an address, then I need to add projection for those classes also.
But I don't want to configure for each classes. How to implement the dynamic Projection? So that all my entities will return the nested object in response.
Refer: https://jira.spring.io/browse/DATAREST-221
Regarding your first question, which was already answered here, exposeIdsFor accepts an arbitrary number of arguments. Also, as mentionned in this thread If all your entity classes are located in the same package, you could get a list of your classes using the Reflections library and feed it to exposeIdsFor().
I can't seem to get Nested Fields to return when I use the NativeSearchQueryBuilder.withFields(...) method.
Here is my parent Object:
#Document(indexName = "inventory")
public class Inventory
{
#Id
private String id;
#Field(type=FieldType.String)
private String name;
#Field(type=FieldType.Nested, index=FieldIndex.not_analyzed, store=true)
private List<Model> models;
}
And here is the nested Object:
public class Model
{
#Field(type=FieldType.String, index=FieldIndex.not_analyzed, store=true)
private String model;
#Field(type=FieldType.String, index=FieldIndex.not_analyzed, store=true)
private Set<String> series;
}
And the Query
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
nativeSearchQueryBuilder.withQuery(QueryBuilders.matchAllQuery());
nativeSearchQueryBuilder.withFields("models.series");
NativeSearchQuery nativeSearchQuery = nativeSearchQueryBuilder.build();
FacetedPage<Inventory> results = inventoryRepository.search(nativeSearchQuery);
Resulting TotalElements = 529
But each Object in the Content Looks like this (in JSON format):
{
"id":"d5f82880-15bc-45ed-8abb-ff97d0e45da9",
"name": null,
"models": null
}
If I remove the withFields(...) setting, I get back:
{
"id":"d5f82880-15bc-45ed-8abb-ff97d0e45da9",
"name": "Cool Beans",
"models": [
{
"model" : "foo",
"series" : ["bar"]
}
]
}
I've tried models, models.model, models.series, model, series. I can't get withFields working with NestedFields.
Any thoughts?
My understanding of elastic search fields was incorrect. rahulroc tipped me off.
withFields is not the same as source filtering.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-fields.html
So i'm effectively telling Spring Data ES to do this:
curl localhost:9200/inventory/_search?pretty=true -d '
{
"fields" : ["models.series"],
"query" : {
"match" : {"name" : "cool"}
}
}'
When this is what I want
curl localhost:9200/inventory/_search?pretty=true -d '
{
"_source" : ["models.series"],
"query" : {
"match" : {"name" : "cool"}
}
}'
The withFields approach worked for what I was doing up till I started adding NestedFields. And the current implementation of Spring Data ES that I'm using does not support source filtering.
Source Filtering was just recently added to Spring Data ES 2.0.0.RC1
I am working on a web application using Spring + Jersey + Mongo. I am using elemMatch from spring mongo for some specific result.
Here is my class that will be mapped with the output of the mongo query.
public class ABC {
private String id;
private String dId;
private String lId;
private String hId;
private String pId;
private List<GeneralData> generalRecords;
private List<String> fhistory;
private List<String> mhistory;
private List<String> specialNotes;
}
public class GeneralData {
private Object data;
private HistoryFilter dataType;
}
Now, this is how my data looks like in mongo :
{
"_id" : ObjectId("55ccda7fe4b03cc159c2ce17"),
"_class" : "com.dpdocter.collections.HistoryCollection",
"dId" : "5525ef96e4b077dfc168369b",
"lId" : "5525ef96e4b077dfc16836a1",
"hId : "5525ef96e4b077dfc16836a0",
"pId" : "55ccd9f1e4b03cc159c2ce0b",
"generalRecords" : [
{
"data" : "55ccdde7e4b03cc159c2ce18",
"dataType" : "REPORTS"
},
{
"data" : "55ccda63e4b03cc159c2ce14",
"dataType" : "CLINICAL_NOTES"
}
]
}
I want record based on dId, lId, hId, pId along with elemMatch on generalRecords for a specific dataType (say REPORTS).
And, below is my criteria code to do that.
Criteria matchCriteria = Criteria.where("dId").is(dId).and("lId").is(lId).and("hId").is(hId).and("pId").is(pId).and("generalRecords.dataType").is("REPORTS");
Criteria elementMatchCriteria = Criteria.where("generalRecords").elemMatch(Criteria.where("dataType").is("REPORTS"))
BasicQuery query = new BasicQuery(matchCriteria.getCriteriaObject(), elementMatchCriteria.getCriteriaObject());
ABC abc = mongoOperations.findOne(query, ABC.class);
Result of the above
ABC [id=55ccda7fe4b03cc159c2ce17, dId=null, lId=null, hId=null, pId=null, generalRecords=[GeneralData [data=55ccdde7e4b03cc159c2ce18, dataType=REPORTS]], fhistory=null, mhistory=null, specialNotes=null]
generalRecords is just perfect and as desired, but I just want to include dId, lId, hId, pId, fHistory, mHistory, and specialNotes also. I have already tried :
query.fields().include('fieldName')
also, some other weird query combos, but nothing helps.
Can anyone suggest something.