Develop Reference

How to search in firebase database

I'm trying to filter the data from my database using this code:
fdb.orderByChild("title").startAt(searchquery).endAt(searchquery+"\uf8ff").addValueEventListener(valuelistener2);
My database is like this:
"g12" : {
"Books" : {
"-Mi_He4vHXOuKHNL7yeU" : {
"title" : "Technical Sciences P1"
},
"-Mi_He50tUPTN9XDiVow" : {
"title" : "Life Sciences"
},
"-Mi_He51dhQfl3RAjysQ" : {
"title" : "Technical Sciences P2"
}}
While the code works, it only returns the first value that matches the query and doesn't fetch the rest of the data even though it matches.
If I put a "T" as my search query, I just get the first title "Technical Sciences P1 " and don't get the other one with P2
(Sorry for the vague and common question title, it's just I've been looking for a solution for so long)

While the codes works, it only returns the first value that matches the query
That's the expected behavior since Firebase Realtime Database does not support native indexing or search for text fields in database properties.
When you are using the following query:
fdb.orderByChild("title").startAt(searchquery).endAt(searchquery+"\uf8ff")
It means that you are trying to get all elements that start with searchquery. For example, if you have a title called "Don Quixote" and you search for "Don", your query will return the correct results. However, searching for "Quix" will yield no results.
You might consider downloading the entire node to search for fields client-side but this solution isn't practical at all. To enable full-text search of your Firebase Realtime Database data, I recommend you to use a third-party search service like Algolia or Elasticsearch.
If you consider at some point in time to try using Cloud Firestore, please see the following example:
Is it possible to use Algolia query in FirestoreRecyclerOptions?
To see how it works with Cloud Firestore but in the same way, you can use it with Firebase Realtime Database.

Using two different structures in one Firebase database

I'm currently working on a project where I need two different types of non-related datastructures in my Java Android app. One being users, the other being types of food.
Users are set up like this:
users
userid
name
age
gender
weight
height
But, I also need one that looks like this, which must be searchable:
foods
name
carbohydrate
fat
protein
Is it possible to user the same database (preferrable Firebase, as I'm already using that), or do I need to add another database to the app I'm making?
Edit: I ended up exporting the JSON, rewriting it according to the good answers given here, and then importing it again. It works flawlessly. Thanks for your answers!
{
"foods" : {
"name" : {
"carbohydrates" : "5",
"fats" : "5",
"proteins" : "5"
}
},
"users" : {
"FjtMNTcDrOP2wcaPAa0E0Cc1jRz2" : {
"activity" : "Moderate Exercise (3–5 days/week)",
"age" : "40",
"gender" : "Male",
"height" : "180",
"name" : "Flex",
"weight" : "86"
}
}
}

Yes, it is possible to model multiple entity types (such as your users and foods) in the Firebase Realtime Database. While it doesn't have the concept of a table, it's a hierarchy of JSON values and you can model any structure you want in that.
For example, you could express you data model with this JSON:
{
"users": {
"userid": {
"name": "value",
"age": 42,
"gender: "value",
"weight": 190,
"height": 172
}
},
"foods": {
"name": {
"carbohydrate": 42
"fat": 11,
"protein": 8
}
}
}
In relational terms, the above model defines two "tables": users and foods. In Android code you can define separate references to each of these with:
DatabaseReference rootReference = FirebaseDatabase.getInstance().getReference();
DatabaseReference usersReference = rootReference.getChild("users");
DatabaseReference foodsReference = rootReference.getChild("foods");

Yes, you can use the same Firebase Realtime Database to store that data.
The RTDB can be simplified down to being just a JSON tree. So for your desired implementation, you would have two keys at the root of your database (such as "users" and "foods").
{
"users": {
"userid1": {
"name": "somestring",
"age": "somenum",
"gender": "somestring",
"height": "somenum",
"weight": "somenum",
...
},
...
},
"foods": {
"food1": {
"name": "somename",
"carbs": "somenum",
"fat": "somepercent",
"protein": "somepercent",
...
},
...
}
}
You can also add or remove more root keys as you wish and your project takes shape.
However,
As #Tamir Abutbul suggests in their answer, I would use Cloud Firestore for this project over the RTDB.
The reason for this is that based on your data, you are likely going to need to filter results by a number of different values at a time in the future. Cloud Firestore supports these types of queries natively (docs) whereas you'd need to write a custom solution for the RTDB.
Getting Started with Cloud Firestore

You can use Firebase with Cloud Firestore.
Create "users" collections with your wanted data structure and another collection called "foods" with its own data structure.
The next step is just to decide when to use any one of those collections(according to your app logic).

Fetching referenced mongodb documents in another collection using Morphia

I've been trying to wrap around my head around this...
I have the following referenced documents in a users and groups collection.
group documents
{
"_id" : ObjectId("52eabc9914cc8d6cc1e6f723"),
"className" : "org.xxxxxx.sms.core.domain.Group",
"name" : "CE Play group",
"description" : "CE Play group",
"creationdate" : ISODate("2014-01-30T20:56:57.848Z"),
"user" : DBRef("users", ObjectId("52ea69c714ccd207329b2476"))
}
{
"_id" : ObjectId("52ea69c714ccd207329b2477"),
"className" : "org.xxxxxx.sms.core.domain.Group",
"name" : "Default",
"description" : "Default sms group",
"creationdate" : ISODate("2014-01-30T15:03:35.916Z"),
"user" : DBRef("users", ObjectId("52ea69c714ccd207329b2476"))
}
users document
{
"_id" : ObjectId("52ea69c714ccd207329b2476"),
"className" : "org.xxxxxx.core.domain.User",
"username" : "jomski2009",
"firstname" : "Jome",
"lastname" : "Akpoduado",
"email" : "jomea#example.com",
"active" : true,
"usergroups" : [
DBRef("groups", ObjectId("52ea69c714ccd207329b2477")),
DBRef("groups", ObjectId("52eabc9914cc8d6cc1e6f723"))
]
}
I have a Morphia Datastore singleton object which has been set up in a class to retrieve a user group and perform some manipulations on. Say I wanted to fetch the group named "Default" with a supplied username "jomski2009" , how would I achieve this in Morphia without fetching the usergroups as a list and iterating over the list of groups just to find the group I want?
Thanks.

DBRef is used in Mongo as a client side concept. MongoDB does not do joins so the purpose of DBRef is to hand over to the client a location from which to fetch the required object. This is handled by various client libraries in different ways.
If it is feasible for your application to do so, you might want to take a look at using Embedded Annotation instead of the Reference type. Or at the very least to include a list of usernames in your Group objects in addition to the object references. This will allow you to filter these in queries.
Also it is worthwhile looking at moving any unique identifiers like "username" to the _id field of the document as long as it is always unique. Primary key lookups are always faster.

Elasticsearch multiple fields autosuggestion

I want to implement autosuggestion functionality using elastic search. I can use nGram filters to match partial words on multiple fields and its working fine as expected. Output of the search returns full document with multiple fields as required. Now my problem is, how do I give autosuggestion to the user based on the matching field. e.g. I have got 5 fields:
{userId:'rakesh',firstName:'Rakesh','lastName':'Goyal','mobileNo':'123-123-1234','alternativeMobileNo':'123-123-1235'}
{userId:'goyal',firstName:'Goyal','lastName':'Rakshit','mobileNo':'123-123-1236','alternativeMobileNo':'123-123-1237'}
In the above example if user types 123, I want to return 123-123-1234, 123-123-1235, 123-123-1236, 123-123-1237 (4 auto suggestions).
Similarly if user types Rak, I want to return Rakesh, Rakshit (2 auto suggestions).
How do I know match exists in mobileNo and alternativeMobileNo field for first example and return results accordingly?
How do I know match exists in firstName and lastName field for second example and return results accordingly?

How do I give autosuggestion to the user based on the matching field?
When user types 123, store it in a Java variable, prepare a query like below inserting that variable into and send a request to ElasticSearch.
{
"query" : {
"query_string" : {
"query" : "*123*"
}
}
}
The above query will manage to check it in both fields mobileNo and alternativeMobileNo.
Similarly, if user types Rak, the query will be similar to the previous one,
{
"query" : {
"query_string" : {
"query" : "*Rak*"
}
}
}
And I think you want to use highlighter api to answer your last how questions, which allows to highlight search results on one or more fields.
A screenshot of highlight example in es :

Elasticsearch for logging - need architectural advice

I am trying to come up with an optimized architecture to store event logging messages on Elasticsearch.
Here are my specs/needs:
Messages are read-only; once entered, they are only queried for reporting.
No free text search. User will use only filters for reporting.
Must be able to do timestamp range queries.
Mainly need to filter by agent and customer interactions (in addition to other fields).
customers and agents belong to the same location.
So the most frequently executed query will be: get all LogItems given client_id, customer_id, and timestamp range.
Here is what a LogItem looks like:
"_source": {
"agent_id" : 14,
"location_id" : 2,
"customer_id" : 5289,
"timestamp" : 1320366520000, //Java Long millis since epoch
"event_type" : 7,
"screen_id" : 12
}
I need help indexing my data.
I have been reading what is an elasticsearch index? and using elasticsearch to serve events for customers to get an idea of a good indexing architecture, but I need assistance from the pros.
So here are my questions:
The article suggests creating "One index per day". How would I do range queries with that architecture? (eg: is it possible to query on index range?)
Currently I'm using one big index. If I create one index per location_id, how do I use shards for further organization of my records?
Given the specs above, is there a better architecture you can suggest?
What fields should I filter with vs query with?
EDIT: Here's a sample query run from my app:
{
"query" : {
"bool" : {
"must" : [ {
"term" : {
"agent_id" : 6
}
}, {
"range" : {
"timestamp" : {
"from" : 1380610800000,
"to" : 1381301940000,
"include_lower" : true,
"include_upper" : true
}
}
}, {
"terms" : {
"event_type" : [ 4, 7, 11 ]
}
} ]
}
},
"filter" : {
"term" : {
"customer_id" : 56241
}
}
}

You can definitely search on multiple indices. You can use wildcards or a comma-separated list of indices for instance, but keep in mind that index names are strings, not dates.
Shards are not for organizing your data but to distribute it and eventually scale out. How you do that is driven by your data and what you do with it. Have a look at this talk: http://vimeo.com/44716955 .
Regarding your question about filters VS queries, have a look at this other question.

Take a good look at logstash (and kibana). They are all about solving this problem. If you decide to roll your own architecture for this, you might copy some of their design.