Develop Reference

Firestore with complex filter and composite indexes

I use Cloud Firestore and this is my data structure :
Firestore-root
|
--- collection (collection)
|
--- uid (document)
|
--- name: some_name
|
--- some_fields: some_data
|
--- origin: [origin_1, origine_2, ...] <- List
|
--- category: [category_1, category_2, ...] <- List
|
--- meals: [meals_1, meals_2, ...] <- List
|
--- some_other_filters: [.....] <- List
|
--- all_categories: [origin_1:true, origine_2:true, meals_1:true, ....]
I created the Map field all_categories to hold all different filters (origin, category, ...) so that I can perform queries easily like: if user select meals_4 and category_2, category_6 and some_other_filter I can do:
List<String> filters = getFilters();
query = recipeRepository.getUsersRecipeCollection()
.orderBy(sorted_filter, DESCENDING).limit(5);
for (String item: filters)
query = query.whereEqualTo("all_categories."+item, true);
query.get().addOnCompleteListener(...)
I cannot use the whereArrayContainsAny function since I need to get the exact items based on selected filters. What I need is something like "whereArrayContainsExactly.."
So, for example, if a user select 1, 2,.. or 25 different items from my filter page (like: category_1, meals_2, origin_1,... ), does it mean I need to create all 25 combinations of the composite indexes (through the link I get from the console) ? or I create 25 indexes of that map fields?
Second attempt:
I denormalised the data as following:
The collection that hold all the data:
The filter collections:
If I select meals_1 and origin_1, With whenAllSuccess I can get all document ids like the following:
Iterator<String> iterator = filters.iterator();
List<Task> tasks = new ArrayList<>();
while (iterator.hasNext()) {
String filter = iterator.next();
tasks.add(recipeRepository.getFilterCollection("filter_"+filter).get());
}
Tasks.whenAllSuccess(tasks.toArray(new Task[tasks.size()]))
.addOnSuccessListener(new OnSuccessListener<List<Object>>() {
#Override
public void onSuccess(List<Object> objects) {
if (!objects.isEmpty()) {
for (Object querySnapshot: objects) {
QuerySnapshot querySnap = (QuerySnapshot)querySnapshot;
for (DocumentSnapshot id: querySnap.getDocuments()) {
doc_ids.add(id.getId());
}
}
CollectionReference collectionReference = recipeRepository.getUsersRecipeCollection();
List<DocumentReference> listDocRef = new ArrayList<>();
for (String id: doc_ids) {
DocumentReference docRef = collectionReference.document(id);
listDocRef.add(docRef);
}
List<Task<DocumentSnapshot>> tasks = new ArrayList<>();
for (DocumentReference documentReference : listDocRef) {
Task<DocumentSnapshot> documentSnapshotTask = documentReference.get();
tasks.add(documentSnapshotTask);
}
Tasks.whenAllSuccess(tasks).addOnSuccessListener(new OnSuccessListener<List<Object>>() {
#Override
public void onSuccess(List<Object> list) {
Log.e(TAG, ">> list objects: " + list.size() + " data: " + list);
for (Object object : list) {
Recipes recipes = ((DocumentSnapshot) object).toObject(Recipes.class);
if (recipes == null) continue;
Log.e(TAG, ">> recipe name: " + recipes.getName());
}
}
});
}
}
});
With this I get all data containing meals_1 OR origin_1 (same result as using whereArrayContainsAny).
What I need is to get the list of docs ids that exists in both meals_1 and origin_1 collections.

So, for example, if a user select 1, 2,.. or 25 different items from my filter page (like: category_1, meals_2, origin_1,... ), does it mean I need to create all 25 combinations of the composite indexes (through the link I get from the console) ? or I create 25 indexes of that map fields?
As I see in the for loop, at each iteration you are creating a new Query object, and this is because Cloud Firestore queries are immutable. Because you are using in the same query a .orderBy() call along with a .whereEqualTo() call, an index is required.
And to answer your question, yes, an index is required for each and very different query that you perform.
Edit:
An alternative solution might be to denormalize the data, by adding separate collections of each type of meal, and every time a filter is added you should create a new query for the selected collection. This is a common practice when it comes to NoSQL databases.

Merge data by date

I have below data which I fetched from database by using Hibernate NamedQuery
TXN_ID END_DATE
---------- ------------
121 15-JUN-16
122 15-JUN-16
123 16-MAY-16
Each row data can be store in Java class Object.
Now I want to combined data depending on the END_DATE. If END_DATE are same then merge TXN_ID data.
From the above data output would be :
TXN_ID END_DATE
---------- ------------
121|122 15-JUN-16
123 16-MAY-16
I want to do this program in java. What is the easy program for that?

Using the accepted function printMap, to iterate through the hashmap in order to see if output is correct.
With the code below:
public static void main(String[] args) {
String[][] b = {{"1","15-JUN-16"},{"2","16-JUN-16"},{"3","13-JUN-16"},{"4","16-JUN-16"},{"5","17-JUN-16"}};
Map<String, String> mapb = new HashMap<String,String>();
for(int j=0; j<b.length; j++){
String c = mapb.get(b[j][1]);
if(c == null)
mapb.put(b[j][1], b[j][0]);
else
mapb.put(b[j][1], c+" "+b[j][0]);
}
printMap(mapb);
}
You get the following output:
13-JUN-16 = 3
16-JUN-16 = 2 4
17-JUN-16 = 5
15-JUN-16 = 1
I think this will solve your problem.

With hibernate you can put query result in a list of object
Query q = session.createSQLQuery( sql ).addEntity(ObjDataQuery.class);
List<ObjDataQuery> res = q.list();
Now you can create an hashmap to storage final result, to populate this object you can iterate over res
Map<String, String> finalResult= new HashMap<>();
for (int i=0; i<res.size(); i++){
if (finalResult.get(res.get(i).date!=null){
//new element
finalResult.put(res.get(i).date,res.get(i).txn)
} else {
//update element
finalResult.put(res.get(i).date,
finalResult.get(res.get(i).date) + res.get(i).txn)
}
}
I've not tested it by logic should be correct.
Another way is to change the query to obtain direct the final result (in oracle see LISTAGG)

How to Iterate column with multiple values using cypher?

The code I tried was
firstNode = graphDb.createNode();//creating nodes like this
firstNode.setProperty( "person", "Andy " );
Label myLabel = DynamicLabel.label("person");
firstNode.addLabel(myLabel); ...
relationship = firstNode.createRelationshipTo( secondNode, RelTypes.emails );// creating relationships like this
relationship.setProperty( "relationship", "email " );....
Transaction tx1 = graphDb.beginTx();
try{
ExecutionEngine engine = new ExecutionEngine(graphDb);
ExecutionResult result = engine.execute("MATCH (sender:person)-[:emails]-(receiver) RETURN sender, count(receiver)as count, collect(receiver) as receivers ORDER BY count DESC ");..
and the result I obtained was:
sender | count | receivers
Node[2]{person:"Chris"} | 3 | [Node[4]{person:"Elsa "},Node[0]{person:"Andy "},Node[1]{person:"Bobby"}]
Node[4]{person:"Elsa "} | 3 | [Node[5]{person:"Frank"},Node[2]{person:"Chris"},Node[3]{person:"David"}]
Node[1]{person:"Bobby"} | 3 | [Node[2]{person:"Chris"},Node[3]{person:"David"},Node[0]{person:"Andy "}]
Node[5]{person:"Frank"} | 2 | [Node[3]{person:"David"},Node[4]{person:"Elsa "}
I want to iterate the receivers. so I tried the following :
for (Map<String,Object> row : result) {
Node x = (Node)row.get("receivers");
System.out.println(x);
for (String prop : x.getPropertyKeys()) {
System.out.println(prop +": "+x.getProperty(prop));
}
But it throws Exception in thread "main" java.lang.ClassCastException: scala.collection.convert.Wrappers$SeqWrapper cannot be cast to org.neo4j.graphdb.Node.
How can I do this?

The problem is that in your cypher query you collect the receiver.person property into an array called receivers. Receivers isn't a Node, its an array of properties. You can't cast that to strings. If you were looking to get the actual receiver nodes then you need to change your query statement to:
MATCH (sender:person)-[:emails]-(receiver) RETURN sender, count(receiver)as count, receiver as receivers ORDER BY count DESC
Alternatively, if you want to use the array of properties then you can do something like the code below:
Object receivers = row.get("receivers")
if(receivers instanceof String[]) {
for(String receiver in receivers) {
.. do something
}
} else {
// do something with receiver as a single string value
}
Clearly, you will need to change to type of receivers from String to the appropriate type if it isn't a String.

It is a simple thing.
try{
ExecutionEngine engine = new ExecutionEngine(graphDb);
ExecutionResult result = engine.execute("MATCH (sender:person)-[:emails]-(receiver) RETURN sender, count(receiver)as count, collect(receiver.person) as receivers ORDER BY count DESC ");
//ExecutionResult result = engine.execute("MATCH (sender:person)-[:emails]->(receiver) WITH sender, collect(receiver.person) as receivers, RETURN {sender: sender.person, receivers: receivers) ORDER BY size(receivers) DESC");
//System.out.println(result.dumpToString());
LinkedList list_prop = new LinkedList();
for (Map<String,Object> row : result) {
Node x = (Node)row.get("sender");
Object y = row.get("receivers");
System.out.println(y);
for (String prop_x : x.getPropertyKeys()) {
System.out.println(prop_x +": "+x.getProperty(prop_x));
}
}
tx1.success();
}
finally {
tx1.close();
}
In the match query, I used "collect(receiver.person) as receivers" instead of "collect(receiver) as receivers". It worked.

How do I get the values of the List<Map<Integer, String>>?

Hello community have a query, is that I have a query that returns me the data in a List <Map <Integer, String>>
Query HQL Hibernate:
#SuppressWarnings({ "unchecked" })
#Transactional(readOnly = true)
#Override
public List<Map<Integer, String>> obtenerPermisosForm(int icodUsu, int icodRol) throws Exception {
try {
StringBuilder hql = new StringBuilder();
hql.append("select new Map(opc.id.icodMaeMenu as icodMaeMenu, ");
hql.append("opc.videObj as videObj) ");
hql.append("from Sgmaeopc opc ");
hql.append("inner join opc.sgmaemenu men ");
hql.append("inner join men.sgusurol usr ");
hql.append("inner join usr.sgusuario us ");
hql.append("inner join usr.sgrol rl ");
hql.append("where opc.id.icodUsuario = :icodUsu ");
hql.append("and opc.id.icodRol = :icodRol ");
hql.append("and men.bactivo = :bactivo ");
hql.append("and us.bactivo = :bactivo ");
Query query = super.getSession().createQuery(hql.toString());
query.setParameter("icodUsu", icodUsu);
query.setParameter("icodRol", icodRol);
query.setParameter("bactivo", Constantes.ESTADO_ACTIVO_TRUE);
return query.list();
} catch (Exception e) {
throw new Exception( getGenerarError(Thread.currentThread().getStackTrace()[1].getMethodName(),
Constantes.NIVEL_APP_DAO,
this.getClass().getName(),
e.getMessage()) );
}
}
And I try to capture controller data in this way.
List<Map<Integer, String>> permisos = userService.obtenerPermisosForm(getCVariableSesion().getIcodUsu(),
getCVariableSesion().getIcodRol());
for(Map<Integer, String> map : permisos) {
System.out.println("===> "+map+" | "+map.get("videObj"));
}
But the problem that I have is to capture the value of the integer code for the description of "icodMaeMenu"
===> {videObj=menuItemMarcas001, icodMaeMenu=14} | menuItemMarcas001
===> {videObj=menuItemMarcas002, icodMaeMenu=14} | menuItemMarcas002
===> {videObj=menuItemMarcas003, icodMaeMenu=14} | menuItemMarcas003
===> {videObj=menuItemMarcas005, icodMaeMenu=14} | menuItemMarcas005
If I try to access the map.get("icodMaeMenu"), it generates an error that can not cast an Integer to a String, you should consult that do in this case.

You're not storing entirely homogeneous objects into that map. I'll give you that you're at least storing a String as a key, but the values could either be String or Integer.
You could change your Map to be List<Map<String, Object>>. This has the distinct disadvantage of you not being able to accurately predict what exactly is in that mapping, and it forces you to cast based on what string you're working with.
Alternatively, you could ditch the map altogether and create some sort of wrapper object that encapsulates both of those objects neatly, and removes the need to do any casting at all.
I won't write the class since I don't know what the best name for it would be, but I can give you the fragment of HQL you'd require:
"select new Wrapper(opc.id.icodMaeMenu, opc.videObj)" +
// rest of append here...

try to change List<Map<Integer, String>> permisos to List<Map<String, Object>> permisos. It looks like the key of your maps is a String and the value can be either Integer or String.

Add elements to HashMap<String, List> - .containsKey() always returns false

I have List<HashMap> of mail addressess, along with users ids (each entry in list looks like: id: 123, mail: "john#doe.com").
I want to make a HashMap in which every key is a domain name of e-mail address, and value is list of e-mails from that domain:
"foo.com":
[1]
id: 123,
mail: a#foo.com
[2]
id: 345
mail: b#foo.com
"bar.com":
[1]
id: 787,
mail: a#bar.com
[2]
id: 456
mail: b#bar.com
To achieve that, I do what's below. Problem is, when I try to add new list entry to list existing on domain entry, java adds new record to sortedAddresses instead of using present one. My prediction is containsKey() method always returns false.
HashMap<String, List> sortedAddresses = new HashMap<String, List>();
for(HashMap<String, String> r : this.lightUsersList){
String mail = r.get("email");
Integer uid = Integer.parseInt(r.get("id"));
try{
String[] mailSplit = mail.split("#");
String domain = mailSplit[1];
//if domain key doesn't exist, add it to hashmap
if(!sortedAddresses.containsKey(domain)){
List<HashMap> domainAddr = new ArrayList<HashMap>();
sortedAddresses.put(domain, domainAddr);
}
List<HashMap> domainAddr = sortedAddresses.get(domain);
sortedAddresses.remove(domain);
domainAddr.add(r);
sortedAddresses.put(domain, domainAddr);
}
catch(Exception e){
//to be implemented
System.out.println("Nie udalo sie dodac adresu " + mail + " do tablicy domenowej (" + e.getMessage() + ")");
}
//displaying hashmap summary (source from another SO thread)
Iterator it = sortedAddresses.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pairs = (Map.Entry)it.next();
System.out.println(pairs.getKey() + " = " + sortedAddresses.get(pairs.getKey()).size());
it.remove(); // avoids a ConcurrentModificationException
}
}
Output I get:
foo = 1
bar = 1
foo = 1
bar = 1
Should be:
foo = 2
bar = 2

Okay, it seems that I see where I made a mistake. Of course Iterator part should be after for loop. My bad. Hate mondays.