Develop Reference

Mongotemplate custom converter not working

I have a list of documents called customers that I retrieved using mongotemplate, bellow some of the documents:
{"customer": {"entityPerimeter": "abp", "name": "ZERZER", "siren": "6154645", "enterpriseId": "546456", "ic01": "", "marketingOffer": "qlksdjf", "irType": "Router", "offerSpecificationOfferLabel": "2Mb"}}
{"customer": {"entityPerimeter": "sdf", "name": "qazer", "siren": "156", "enterpriseId": "546456", "ic01": "", "marketingOffer": "qlksdjddddsqf", "irType": "Ruter", "offerSpecificationOfferLabel": "2Mb"}}
{"customer": {"entityPerimeter": "zer", "name": "fazdsdfsdgg", "siren": "sdfs", "enterpriseId": "1111", "ic01": "", "marketingOffer": "qsdfqsd", "irType": "Router", "offerSpecificationOfferLabel": "2Mb"}}
That what I did in mongodb to have this result:
public List<DBObject> findAllCustomersByExtractionDateMongo(LocalDate extractionDate) {
Aggregation aggregation = newAggregation(
match(Criteria.where(EXTRACTION_DATE).is(extractionDate)),
project(CUSTOMER).andExclude("_id"),
group().addToSet("$customer").as("distinct_customers"),
unwind("distinct_customers"),
project().andExclude("_id").and("distinct_customers").as("customer"),
project().andExclude("distinct_customers")
);
return template
.aggregate(aggregation, COLLECTION, DBObject.class)
.getMappedResults();
}
Now what I really want is to map those Documents to a Class called Customer:
#Data
#NoArgsConstructor
#AllArgsConstructor
#Builder
public class Customer {
private String entityPerimeter;
private String name;
private String siren;
private String enterpriseId;
private String ic01;
private String marketingOffer;
private String product;
private String irType;
}
I tried to do that by creating a DTO interface:
public interface DocumentToCustomerMapper {
String NULL = "null";
static Customer getFilter(DBObject document) {
var customer = new Customer();
customer.setSiren(Optional.ofNullable((String) document.get(CustomerAttributes.SIREN.value())).orElse(NULL));
customer.setEnterpriseId(Optional.ofNullable((String) document.get(CustomerAttributes.ENTERPRISE_ID.value())).orElse(NULL));
customer.setEntityPerimeter(Optional.ofNullable((String) document.get(CustomerAttributes.ENTITY_PERIMETER.value())).orElse(NULL));
customer.setName(Optional.ofNullable((String) document.get(CustomerAttributes.NAME.value())).orElse(NULL));
customer.setIc01(Optional.ofNullable((String) document.get(CustomerAttributes.IC_01.value())).orElse(NULL));
customer.setMarketingOffer(Optional.ofNullable((String) document.get(CustomerAttributes.MARKETING_OFFER.value())).orElse(NULL));
customer.setProduct(Optional.ofNullable((String) document.get(CustomerAttributes.PRODUCT.value())).orElse(NULL));
customer.setIrType(Optional.ofNullable((String) document.get(CustomerAttributes.IR_TYPE.value())).orElse(NULL));
return customer;
}
}
Then in the findAllCystomersByExtractionDateMongo() I'm doing this:
public List<Customer> findAllCustomersByExtractionDateMongo(LocalDate extractionDate) {
Aggregation aggregation = newAggregation(
match(Criteria.where(EXTRACTION_DATE).is(extractionDate)),
project(CUSTOMER).andExclude("_id"),
group().addToSet("$customer").as("distinct_customers"),
unwind("distinct_customers"),
project().andExclude("_id").and("distinct_customers").as("customer"),
project().andExclude("distinct_customers")
);
final Converter<DBObject, Customer> converter = DocumentToCustomerMapper::getFilter;
MongoCustomConversions cc = new MongoCustomConversions(List.of(converter));
((MappingMongoConverter) template.getConverter()).setCustomConversions(cc);
return template
.aggregate(aggregation, COLLECTION, Customer.class)
.getMappedResults();
}
But unfortunately it's giving me an exception:
Couldn't resolve type arguments for class com.obs.dqsc.api.repository.mongo_template.CustomerRepositoryImpl$$Lambda$1333/0x00000008012869a8!
I tried to remove this code:
final Converter<DBObject, Customer> converter = DocumentToCustomerMapper::getFilter;
MongoCustomConversions cc = new MongoCustomConversions(List.of(converter));
((MappingMongoConverter) template.getConverter()).setCustomConversions(cc);
Then all what I'm getting is some null values in my customer objects:
Customer(entityPerimeter=null, name=null, siren=null, enterpriseId=null, ic01=null, marketingOffer=null, product=null, irType=null)
Customer(entityPerimeter=null, name=null, siren=null, enterpriseId=null, ic01=null, marketingOffer=null, product=null, irType=null)
Customer(entityPerimeter=null, name=null, siren=null, enterpriseId=null, ic01=null, marketingOffer=null, product=null, irType=null)
Note: for performance issues, I don't want to do any mapping in the java side, also I don't want to use a global converter in my mongo configuration.

The problem is that you are using a method reference to express your converter:
final Converter<DBObject, Customer> converter = DocumentToCustomerMapper::getFilter;
(Expanding the method reference to a lambda won't work either.)
Try rewriting that snippet to something else (such as an anonymous inner class).
Here is a very similar issue reported, including info on how to work around this problem: https://github.com/arangodb/spring-data/issues/120

MongoDB and Spring Data - Aggregation with group returns incorrect id

I have following MongoDB document:
#Data
#EqualsAndHashCode(callSuper = true)
#NoArgsConstructor
#AllArgsConstructor
#Accessors(chain = true)
#SuperBuilder
#Document(collection = ReasonDocument.COLLECTION)
public class ReasonDocument extends BaseDocument<ObjectId> {
public static final String COLLECTION = "reasons";
#Id
private ObjectId id;
#Indexed
private ObjectId ownerId;
#Indexed
private LocalDate date;
private Type type;
private String reason;
}
I would like to get all rows for ownerId with latest date and additionally filter some of them out. I wrote custom repository for that, where I use aggregation with a group statement:
public class ReasonsRepositoryImpl implements ReasonsRepository {
private final MongoTemplate mongoTemplate;
#Autowired
public ReasonsRepositoryImpl(MongoTemplate mongoTemplate) {
this.mongoTemplate = mongoTemplate;
}
public List<ReasonDocument> findReasons(LocalDate date) {
final Aggregation aggregation = Aggregation.newAggregation(
sort(Direction.DESC, "date"),
group("ownerId")
.first("id").as("id")
.first("reason").as("reason")
.first("type").as("type")
.first("date").as("date")
.first("ownerId").as("ownerId"),
match(Criteria.where("date").lte(date).and("type").is(Type.TYPE_A))
);
return mongoTemplate.aggregate(aggregation, "reasons", ReasonDocument.class).getMappedResults();
}
}
It is smart query but unfortunately it returns corrupted rows while testing:
java.lang.AssertionError:
Expecting:
<[ReasonDocument(id=5dd5500960483c1b2d974eed, ownerId=5dd5500960483c1b2d974eed, date=2019-05-14, type=TYPA_A, reason=14),
ReasonDocument(id=5dd5500960483c1b2d974ee8, ownerId=5dd5500960483c1b2d974ee8, date=2019-05-15, type=TYPA_A, reason=1)]>
to contain exactly in any order:
<[ReasonDocument(id=5dd5500960483c1b2d974eef, ownerId=5dd5500960483c1b2d974ee8, date=2019-05-15, type=TYPA_A, reason=1),
ReasonDocument(id=5dd5500960483c1b2d974efc, ownerId=5dd5500960483c1b2d974eed, date=2019-05-14, type=TYPA_A, reason=14)]>
elements not found:
<[ReasonDocument(id=5dd5500960483c1b2d974eef, ownerId=5dd5500960483c1b2d974ee8, date=2019-05-15, type=TYPA_A, reason=1),
ReasonDocument(id=5dd5500960483c1b2d974efc, ownerId=5dd5500960483c1b2d974eed, date=2019-05-14, type=TYPA_A, reason=14)]>
and elements not expected:
<[ReasonDocument(id=5dd5500960483c1b2d974eed, ownerId=5dd5500960483c1b2d974eed, date=2019-05-14, type=TYPA_A, reason=14),
ReasonDocument(id=5dd5500960483c1b2d974ee8, ownerId=5dd5500960483c1b2d974ee8, date=2019-05-15, type=TYPA_A, reason=1)]>
The id returned is the same as ownerId.
Could anyone say what is wrong with the query?

Im not entirely sure whether or not this may be the problem. But did you check how mongo has saved the ID? because even if you're grouping by ownerID. IF mongo has saved the item under the _id header in your Json. Then you need to refer it as _id
Ex: If it looks like this
{
"_id" : "2893u4jrnjnwfwpfn",
"name" : "Jenkins"
}
then your groupBy should be groupBy(_id) and not what you've written.

This happens to be limitation of MongoDB and ORM, unless I'm not aware of something.
According to documentation https://docs.mongodb.com/manual/reference/operator/aggregation/group/, native mongo query looks like this:
{
$group:
{
_id: <expression>, // Group By Expression
<field1>: { <accumulator1> : <expression1> },
...
}
}
So grouping itself creates new _id - if I group by ownerId that value will end up in _id field.
One way of solving this is by using:
.first("_id").as("oldId")
and creating a new type with oldId as a field that can be later used to map back to original Document.

How to aggregate in spring data mongo db a nested object and avoid a PropertyReferenceException?

I am creating a new endpoint in springboot that will return simple stats on users generated from an aggregate query in a mongo database. However I get a PropertyReferenceException. I have read multiple stackoverflow questions about it, but didn't find one that solved this problem.
We have a mongo data scheme like this:
{
"_id" : ObjectId("5d795993288c3831c8dffe60"),
"user" : "000001",
"name" : "test",
"attributes" : {
"brand" : "Chrome",
"language" : "English" }
}
The database is filled with multiple users and we want using Springboot aggregate the stats of users per brand. There could be any number of attributes in the attributes object.
Here is the aggregation we are doing
Aggregation agg = newAggregation(
group("attributes.brand").count().as("number"),
project("number").and("type").previousOperation()
);
AggregationResults<Stats> groupResults
= mongoTemplate.aggregate(agg, Profile.class, Stats.class);
return groupResults.getMappedResults();
Which produces this mongo query which works:
> db.collection.aggregate([
{ "$group" : { "_id" : "$attributes.brand" , "number" : { "$sum" : 1}}} ,
{ "$project" : { "number" : 1 , "_id" : 0 , "type" : "$_id"}} ])
{ "number" : 4, "type" : "Chrome" }
{ "number" : 2, "type" : "Firefox" }
However when running a simple integration test we get this error:
org.springframework.data.mapping.PropertyReferenceException: No property brand found for type String! Traversed path: Profile.attributes.
From what I understand, it seems that since attributes is a Map<String, String> there might be a schematic problem. And in the mean time I can't modify the Profile object.
Is there something I am missing in the aggregation, or anything I could change in my Stats object?
For reference, here are the data models we're using, to work with JSON and jackson.
The Stats data model:
#Document
public class Stats {
#JsonProperty
private String type;
#JsonProperty
private int number;
public Stats() {}
/* ... */
}
The Profile data model:
#Document
public class Profiles {
#NotNull
#JsonProperty
private String user;
#NotNull
#JsonProperty
private String name;
#JsonProperty
private Map<String, String> attributes = new HashMap<>();
public Stats() {}
/* ... */
}

I found a solution, which was a combination of two problems:
The PropertyReferenceException was indeed caused because attributes is a Map<String, String> which means there is no schemes for Mongo.
The error message No property brand found for type String! Traversed path: Profile.attributes. means that the Map object doesn't have a brand property in it.
In order to fix that without touching my orginal Profile class, I had to create a new custom class which would map the attributes to an attributes object having the properties I want to aggreate on like:
public class StatsAttributes {
#JsonProperty
private String brand;
#JsonProperty
private String language;
public StatsAttributes() {}
/* ... */
}
Then I created a custom StatsProfile which would leverage my StatsAttributes and would be similar to the the original Profile object without modifying it.
#Document
public class StatsProfile {
#JsonProperty
private String user;
#JsonProperty
private StatsAttributes attributes;
public StatsProfile() {}
/* ... */
}
With that I made disapear my problem with the PropertyReferenceException using my new class StatsAggregation in the aggregation:
AggregationResults<Stats> groupResults
= mongoTemplate.aggregate(agg, StatsProfile.class, Stats.class);
However I would not get any results. It seems the query would not find any document in the database. That's where I realied that production mongo objects had the field "_class: com.company.dao.model.Profile" which was tied to the Profile object.
After some research, for the new StatsProfile to work it would need to be a #TypeAlias("Profile"). After looking around, I found that I also needed to precise a collection name which would lead to:
#Document(collection = "profile")
#TypeAlias("Profile")
public class StatsProfile {
/* ... */
}
And with all that, finally it worked!
I suppose that's not the prettiest solution, I wish I would not need to create a new Profile object and just consider the attributes as a StatsAttributes.class somehow in the mongoTemplate query. If anyone knows how to, please share 🙏

spring data mongo geospatial query

I have been trying to use a geospatial query to fetch data into a pojo with no success.
Here is an example data in my monogdb collection
{
"_id" : ObjectId("597b8c9a21871eeabd5a1cf5"),
"amb_id" : 10,
"amb_number" : "KL25 8945",
"driver_name" : "Amal Shaji",
"driver_licence_id" : "12/4562/2017",
"ownership" : "Amrita Institute of Medical Science",
"ownership_address" : "Peeliyadu Road, Ponekkara, Edappally, Ernakulam",
"location" : {
"type" : "Point",
"coordinates" : [
76.293485,
10.032871
]
}
}
The below mongo query works perfectly fine in the mongoshell
db.trial.find(
{ location :
{ $near :{
$geometry :{
type : "Point" ,
coordinates : [ 76.2 , 9.9 ] } ,
$maxDistance : 20000 }
}
}
)
.pretty();
Here is the pojo that I have been trying to fetch the data into
#Document(collection = "trial")
public class Ambulance {
#Id
String id;
#Field("amb_id")
String ambulanceId;
#Field("amb_number")
String licensePlateNumber;
#Field("driver_name")
String driverName;
#Field("driver_licence_id")
String driverLicenseNumber;
#Field("ownership")
String ownerShip;
#Field("ownership_address")
String ownerShipAddress;
#GeoSpatialIndexed(name="Location")
Double[] location;
//setters and getters
}
Here is the repository I have been using
#ComponentScan
#Repository
public interface AmbulanceRepo extends MongoRepository<Ambulance, String> {
GeoResults<Ambulance> findByLocationNear(Point p, Distance d);
}
and the controller
#RestController
#CrossOrigin(origins = "http://localhost:4200")
#RequestMapping("/")
public class NearbyAmbulanceController {
private AmbulanceRepo ambulanceRepo;
#Autowired
public NearbyAmbulanceController(AmbulanceRepo repo){
this.ambulanceRepo = repo;
}
#RequestMapping(value="/nearbyAmbulance",method = RequestMethod.POST)
#ResponseBody
public GeoResults<Ambulance> getAmbulanceDetails(
#RequestBody LocationRequest locationRequest){
System.out.println("lati "+locationRequest.getLatitude()+ " long "+locationRequest.getLongitude()+" d "+locationRequest.getDistance());
// List<Ambulance> ambulanceList=this.ambulanceRepo.findByLocationNear(new Point(Double.valueOf(locationRequest.getLongitude()),Double.valueOf(locationRequest.getLatitude())),new Distance(locationRequest.getDistance(), Metrics.KILOMETERS));
Point point = new Point(locationRequest.getLatitude(), locationRequest.getLongitude());
Distance distance = new Distance(locationRequest.getDistance(), Metrics.KILOMETERS);
GeoResults<Ambulance> ambulanceList=this.ambulanceRepo.findByLocationNear(point,distance);
System.out.println(ambulanceList);
return ambulanceList;
}
}
Every time I try it I get no results. I am sure that I have data in the given point and nearby locations and I was even able to use the mongoshell to fetch those. I have feeling that the problem is with the way that I have annotated the location field in the entity. Is there anything that I am missing? Any help is appreiciated.

Spring MongoDb Geo spatial query: It is tested well.
public List<MachineDetails> searchbylocation(String lat, String longt, String maxDistance) {
BasicQuery query1 = new BasicQuery("{geoLocation:{ $near: { $geometry: { type: 'Point',coordinates: ["+ lat+","+ longt+" ] }, $minDistance: 10, $maxDistance: "+maxDistance+"}}}");
List<MachineDetails> venues1 = mtemplate.find(query1, MachineDetails.class);
return venues1;
}

Seems like there were more than one database in mongodb that had collections by the name 'trial'. I tried removing all those and now it is working fine. Spring data wasn't able to figure out which collection to be queried on as there were multiple collection with the same name in different databases. Hope this helps someone, cause I have just wasted a couple of days on this.

Spring Boot + MongoRepository, get all rows but only certain fields

I'm trying to fetch all the entries in my database, but only the reference fields. In my MongoRepository, I want to use a custom query, so it doesn't use the method name to build up the query. The following doesn't seem to work:
public interface JvRepository extends MongoRepository<Jv, String> {
#Query(value = "{}", fields = "{ id : 0, reference : 1 }")
public List<String> findAllJvReferences();
}
public class Jv {
#Id
private String id;
private String reference;
}
The error is:
Jv cannot be cast to java.lang.String
I tell it to not get the id, and do get the reference right? Why does it not return a String, but still a Jv?
I guess I should pass something in the value that tells Mongo to select everything, but I have no idea what. Or is there something else wrong with this code?