Is it possible to query a firestore collection to get all document that starts with a specific string?
I have gone through the documentation but do not find any suitable query for this.
You can but it's tricky. You need to search for documents greater than or equal to the string you want and less than a successor key.
For example, to find documents containing a field 'foo' staring with 'bar' you would query:
db.collection(c)
.where('foo', '>=', 'bar')
.where('foo', '<', 'bas');
This is actually a technique we use in the client implementation for scanning collections of documents matching a path. Our successor key computation is called by a scanner which is looking for all keys starting with the current user id.
same as answered by Gil Gilbert.
Just an enhancement and some sample code.
use String.fromCharCode and String.charCodeAt
var strSearch = "start with text here";
var strlength = strSearch.length;
var strFrontCode = strSearch.slice(0, strlength-1);
var strEndCode = strSearch.slice(strlength-1, strSearch.length);
var startcode = strSearch;
var endcode= strFrontCode + String.fromCharCode(strEndCode.charCodeAt(0) + 1);
then filter code like below.
db.collection(c)
.where('foo', '>=', startcode)
.where('foo', '<', endcode);
Works on any Language and any Unicode.
Warning: all search criteria in firestore is CASE SENSITIVE.
Extending the previous answers with a shorter version:
const text = 'start with text here';
const end = text.replace(/.$/, c => String.fromCharCode(c.charCodeAt(0) + 1));
query
.where('stringField', '>=', text)
.where('stringField', '<', end);
IRL example
async function search(startsWith = '') {
let query = firestore.collection(COLLECTION.CLIENTS);
if (startsWith) {
const end = startsWith.replace(
/.$/, c => String.fromCharCode(c.charCodeAt(0) + 1),
);
query = query
.where('firstName', '>=', startsWith)
.where('firstName', '<', end);
}
const result = await query
.orderBy('firstName')
.get();
return result;
}
If you got here looking for a Dart/Flutter version
Credit to the java answer by Kyo
final strFrontCode = term.substring(0, term.length - 1);
final strEndCode = term.characters.last;
final limit =
strFrontCode + String.fromCharCode(strEndCode.codeUnitAt(0) + 1);
final snap = await FirebaseFirestore.instance
.collection('someCollection')
.where('someField', isGreaterThanOrEqualTo: term)
.where('someField', isLessThan: limit)
.get();
I found this, which works perfectly for startsWith
const q = query(
collection(firebaseApp.db, 'capturedPhotos'),
where('name', '>=', name),
where('name', '<=', name + '\uf8ff')
)
The above are correct! Just wanted to give an updated answer!
var end = s[s.length-1]
val newEnding = ++end
var newString = s
newString.dropLast(1)
newString += newEnding
query
.whereGreaterThanOrEqualTo(key, s)
.whereLessThan(key, newString)
.get()
Related
I am using hibernate-generic-dao for a searching function. Since I only need to show one of the records if they have same value on a column field. But I am not sure how to achieve this by the search / filter functions.
package com.googlecode.genericdao.search;
PersonContact domain object:
...
#Column(name = "group_key", length = 20)
public String getGroupKey() {
return groupKey;
}
#Formula(value = "(SELECT status from person_contact m " +
" WHERE m.case = case AND m.movement_id = movement_id )")
public String getActiveRecord() {
return activeRecord;
}
...
Search search = new Search();
search.addFilterNotNull("groupKey"); //groupKey is the field I want to use "group by / unqiue" with it
search.addFilterEqual("type","C");
search.addFilterCustom("{activeRecord} != 'I' ");
search.setMaxResults(limit);//for paging
search.setFirstResult(startIdx);
SearchResult<PersonContact> resultObj = PersonContactDAO.searchAndCount(search);
You should probably ask this question by opening an issue in the repository for that project here: https://github.com/vincentruan/hibernate-generic-dao
It seems though as if the project is abandoned, so unless you feel like digging into the details, you should probably try to get away from it.
I am trying to call a select query from my Java Program but i am getting Invalid Numbererror.
I am trying to call the query something like this
code
String[] l_arr = new String[]{"CUSTOMERCUMULATIVELIMIT"};
l_rs = JDBCEngine.executeQuery(MessageFormat.format(
" SELECT LIMITS, P.PACKAGE_ID FROM MSTGLOBALLIMITSPACKAGE P ,MSTCUSTOMERPROFILE C WHERE P.PACKAGE_ID = {0} AND C.ID_CUST = ? AND C.ID_ENTITY = ? AND C.TYPEUSER = ? AND C.ID_ENTITY = P.ID_ENTITY AND P.PACKAGE_ID NOT LIKE ?",l_arr );
I am getting query something like this:
Result
SELECT LIMITS, P.PACKAGE_ID
FROM MSTGLOBALLIMITSPACKAGE P, MSTCUSTOMERPROFILE C
WHERE P.PACKAGE_ID = CUSTOMERCUMULATIVELIMIT
AND C.ID_CUST = '00000144'
AND C.ID_ENTITY = 'B001'
AND C.TYPEUSER = 'ECU'
AND C.ID_ENTITY = P.ID_ENTITY
AND P.PACKAGE_ID NOT LIKE '*%'
I am getting the error Invalid number in the line P.PACKAGE_ID = {0}. I gone through and come to know that if i use to_char i can avoid this error. But I am not sure how to use to_char in P.PACKAGE_ID = {0}.
Also PACKAGE_ID is of type Varchar2 and CUSTOMERCUMULATIVELIMIT is a column in MSTCUSTOMERPROFILE of type Number.
Any help or suggestion will be appreciated. Thanks.
just add to_char to your sql, change {0} to to_char({0})
l_rs = JDBCEngine.executeQuery(MessageFormat.format( " SELECT LIMITS, P.PACKAGE_ID FROM MSTGLOBALLIMITSPACKAGE P ,MSTCUSTOMERPROFILE C WHERE P.PACKAGE_ID = to_char({0}) AND C.ID_CUST = ? AND C.ID_ENTITY = ? AND C.TYPEUSER = ? AND C.ID_ENTITY = P.ID_ENTITY AND P.PACKAGE_ID NOT LIKE ?",l_arr );
You are working with this line :
WHERE P.PACKAGE_ID = CUSTOMERCUMULATIVELIMIT
As there are no single quotes, SQL consider CUSTOMERCUMULATIVELIMIT as a number, not a string.
Adding quotes should solve this.
l_rs = JDBCEngine.executeQuery(MessageFormat.format(
" SELECT LIMITS, P.PACKAGE_ID FROM MSTGLOBALLIMITSPACKAGE P ,MSTCUSTOMERPROFILE C WHERE P.PACKAGE_ID = '{0}' AND C.ID_CUST = ? AND C.ID_ENTITY = ? AND C.TYPEUSER = ? AND C.ID_ENTITY = P.ID_ENTITY AND P.PACKAGE_ID NOT LIKE ?",l_arr );
I have spring-data-mogodb application on java or kotlin, and need create text search request to mongodb by spring template.
In mongo shell it look like that:
db.stores.find(
{ $text: { $search: "java coffee shop" } },
{ score: { $meta: "textScore" } }
).sort( { score: { $meta: "textScore" } } )
I already tried to do something but it is not exactly what i need:
#override fun getSearchedFiles(searchQuery: String, pageNumber: Long, pageSize: Long, direction: Sort.Direction, sortColumn: String): MutableList<SystemFile> {
val matching = TextCriteria.forDefaultLanguage().matching(searchQuery)
val match = MatchOperation(matching)
val sort = SortOperation(Sort(direction, sortColumn))
val skip = SkipOperation((pageNumber * pageSize))
val limit = LimitOperation(pageSize)
val aggregation = Aggregation
.newAggregation(match, skip, limit)
.withOptions(Aggregation.newAggregationOptions().allowDiskUse(true).build())
val mappedResults = template.aggregate(aggregation, "files", SystemFile::class.java).mappedResults
return mappedResults
}
May be someone already working with text searching on mongodb with java, please share your knowledge with us )
Setup Text indexes
First you need to set up text indexes on the fields on which you want to perform your text query.
If you are using Spring data mongo to insert your documents in your database, you can use #TextIndexed annotation and indexes will be built while inserting your document.
#Document
class MyObject{
#TextIndexed(weight=3) String title;
#TextIndexed String description;
}
If your document are already inserted in your database, you need to build your text indexes manually
TextIndexDefinition textIndex = new TextIndexDefinitionBuilder()
.onField("title", 3)
.onField("description")
.build();
After the build and config of your mongoTemplate you can pass your text indexes/
template.indexOps(MyObject.class).ensureIndex(textIndex);
Building your text query
List<MyObject> getSearchedFiles(String textQuery){
TextQuery textQuery = TextQuery.queryText(new TextCriteria().matchingAny(textQuery)).sortByScore();
List<MyObject> result = mongoTemplate.find(textQuery, MyObject.class, "myCollection");
return result
}
I'm currently working on a fetaure that will allow the system to search public services receipts by the combination of 6 parameters which can be null meaning that receipts shouldn't be filtered by this parameter: accountNumber, amountRangeMin, amountRangeMax, dateRangeMin, dateRangeMax, publicServiceId. However making a method for each combination of the parameters is not an option, I'm thinking that there must be a better way, at first my approach was as following:
On my Service I have this method:
public Map<String,Object> findPublicServiceReceiptsByParams(Integer accountNumber, BigDecimal amountRangeMin,
BigDecimal amountRangeMax, LocalDate dateRangeMin, LocalDate dateRangeMax, Integer publicServiceId) {
Map<String,Object> publicServiceReceipts = new HashMap<String,Object>();
String accountNumberFilter = !(accountNumber==null) ? accountNumber.toString() : "AccountNumberTableName";
String amountRangeMinFilter = !(amountRangeMin==null) ? amountRangeMin.toString() : "table.AmountColumnName";
String amountRangeMaxFilter = !(amountRangeMax==null) ? amountRangeMax.toString() : "table.AmountColumnName";
String dateRangeMinFilter = !(dateRangeMin==null) ? dateRangeMin.toString() : "Table.ReceiptCreationDateColumn";
String dateRangeMaxFilter = !(dateRangeMax==null) ? dateRangeMax.toString() : "Table.ReceiptCreationDateColumn";
String publicServiceIdFilter = !(publicServiceId==null) ? publicServiceId.toString() : "table.publicServiceIdColumn";
publicServiceReceipts = publicServiceReceiptRepository.findPublicServiceReceiptsByParams(accountNumberFilter,
amountRangeMinFilter, amountRangeMaxFilter, dateRangeMinFilter, dateRangeMaxFilter,
publicServiceIdFilter);
return publicServiceReceipts;
}
And then in my repository I had:
final static String FIND_PUBLIC_SERVICES_BY_ARGS = "Select (Insert whatever logic should go in here to select columns from receipts the where clause is the one that matters)"
+ " WHERE ACT.ACT_AccountNumber=:accountNumberFilter\n"
+ " AND PSE.PSE_Id=:publicServiceIdFilter\n"
+ " AND PSR.PSR_CreateDate BETWEEN :dateRangeMinFilter AND :dateRangeMaxFilter\n"
+ " AND PSR.PSR_Amount BETWEEN :amountRangeMinFilter AND :amountRangeMaxFilter\n"
+ " order by PSR.PSR_CreateDate desc";
#Query(nativeQuery = true, value = FIND_PUBLIC_SERVICES_BY_ARGS)
Map<String, Object> findPublicServiceReceiptsByParams(#Param("accountNumberFilter") String accountNumberFilter,
#Param("amountRangeMinFilter") String amountRangeMinFilter,
#Param("amountRangeMaxFilter") String amountRangeMaxFilter,
#Param("dateRangeMinFilter") String dateRangeMinFilter,
#Param("dateRangeMaxFilter") String dateRangeMaxFilter,
#Param("publicServiceIdFilter") String publicServiceIdFilter);
}
My reasoning was that if a parameter was null meant that whoever consumed the Web Service is not interested in that paramater so if that happens I set that variable as the Column Name so that it wouldn't affect in the WHERE clause and in theory make it simpler, but what I found was that It would send the names as Strings so it wouldn't be recognized as an sql statement which was the flaw in my thinking and as I said there must be another way other than writing each method for each combination, I appreciate any help :).
You should use the Criteria API, which was designed for creating dynamic queries. Named queries aren't really meant to be used in this case.
With it you can do something like this:
#PersistenceContext
EntityManager em;
List<YourEntity> method(String argument) {
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<YourEntity> cq = cb.createQuery(YourEntity.class);
Root<YourEntity> root = cq.from(YourEntity.class);
List<Predicate> predicates = new ArrayList<>();
if (argument == null) {
predicates.add(cb.equal(root.get("yourAttribute"), argument);
}
// rest of your logic goes here
cq.where(predicates.toArray(new Predicate[]{}));
return em.createQuery(cq).getResultList();
}
I found a way to fix this, I did it like this (I'm going to show only the native Query since it's the inly thing that i changed):
DECLARE #actNum varchar(50),#crdNum varchar(50),#pseId varchar(50),#dateMin varchar(50),#dateMax varchar(50),#amountMin varchar(50),#amountMax varchar(50)
SET #actNum = :actNum
SET #crdNum = :crdNum
SET #pseId = :pseId
SET #dateMin = :dateMin
SET #dateMax = :dateMax
SET #amountMin = :amountMin
SET #amountMax = :amountMax
--Whatever Select with joins statement
WHERE ACT.ACT_AccountNumber = CASE WHEN #actNum = 'N/A'
THEN ACT.ACT_AccountNumber
ELSE #actNum END
AND CRD_CardNumber = CASE WHEN #crdNum = 'N/A'
THEN CRD_CardNumber
ELSE #crdNum END
AND PSE.PSE_Id= CASE WHEN #pseId = 'N/A'
THEN PSE.PSE_Id
ELSE #pseId END
AND PSR.PSR_CreateDate >= CASE WHEN #dateMin = 'N/A'
THEN PSR.PSR_CreateDate
ELSE #dateMin END
AND PSR.PSR_CreateDate <= CASE WHEN #dateMax = 'N/A'
THEN PSR.PSR_CreateDate
ELSE #dateMax END
AND PSR.PSR_Amount BETWEEN CASE WHEN #amountMin = 'N/A'
THEN PSR.PSR_Amount
ELSE #amountMin END
AND CASE WHEN #amountMax = 'N/A'
THEN PSR.PSR_Amount
ELSE #amountMax END
ORDER BY PSR.PSR_CreateDate DESC
The backend will send the parameters as either "N/A" (if it shouldn't be used to filter data) or the actual value, this worked fine for me!
i am trying to fetch all the documents from a database without knowing the exact url's . I got one query
DocumentPage documents =docMgr.read();
while (documents.hasNext()) {
DocumentRecord document = documents.next();
System.out.println(document.getUri());
}
But i do not have specific urls , i want all the documents
The first step is to enable your uris lexicon on the database.
You could eval some XQuery and run cts:uris() (or server-side JS and run cts.uris()):
ServerEvaluationCall call = client.newServerEval()
.xquery("cts:uris()");
for ( EvalResult result : call.eval() ) {
String uri = result.getString();
System.out.println(uri);
}
Two drawbacks are: (1) you'd need a user with privileges and (2) there is no pagination.
If you have a small number of documents, you don't need pagination. But for a large number of documents pagination is recommended. Here's some code using the search API and pagination:
// do the next eight lines just once
String options =
"<options xmlns='http://marklogic.com/appservices/search'>" +
" <values name='uris'>" +
" <uri/>" +
" </values>" +
"</options>";
QueryOptionsManager optionsMgr = client.newServerConfigManager().newQueryOptionsManager();
optionsMgr.writeOptions("uriOptions", new StringHandle(options));
// run the following each time you need to list all uris
QueryManager queryMgr = client.newQueryManager();
long pageLength = 10000;
queryMgr.setPageLength(pageLength);
ValuesDefinition query = queryMgr.newValuesDefinition("uris", "uriOptions");
// the following "and" query just matches all documents
query.setQueryDefinition(new StructuredQueryBuilder().and());
int start = 1;
boolean hasMore = true;
Transaction transaction = client.openTransaction();
try {
while ( hasMore ) {
CountedDistinctValue[] uriValues =
queryMgr.values(query, new ValuesHandle(), start, transaction).getValues();
for (CountedDistinctValue uriValue : uriValues) {
String uri = uriValue.get("string", String.class);
//System.out.println(uri);
}
start += uriValues.length;
// this is the last page if uriValues is smaller than pageLength
hasMore = uriValues.length == pageLength;
}
} finally {
transaction.commit();
}
The transaction is only necessary if you need a guaranteed "snapshot" list isolated from adds/deletes happening concurrently with this process. Since it adds some overhead, feel free to remove it if you don't need such exactness.
find out the page length and in the queryMgr you can specify the starting point to access. Keep on increasing the starting point and loop through all the URL. I was able to fetch all URI. This could be not so good approach but works.
List<String> uriList = new ArrayList<>();
QueryManager queryMgr = client.newQueryManager();
StructuredQueryBuilder qb = new StructuredQueryBuilder();
StructuredQueryDefinition querydef = qb.and(qb.collection("xxxx"), qb.collection("whatever"), qb.collection("whatever"));//outputs 241152
SearchHandle results = queryMgr.search(querydef, new SearchHandle(), 10);
long pageLength = results.getPageLength();
long totalResults = results.getTotalResults();
System.out.println("Total Reuslts: " + totalResults);
long timesToLoop = totalResults / pageLength;
for (int i = 0; i < timesToLoop; i = (int) (i + pageLength)) {
System.out.println("Printing Results from: " + (i) + " to: " + (i + pageLength));
results = queryMgr.search(querydef, new SearchHandle(), i);
MatchDocumentSummary[] summaries = results.getMatchResults();//10 results because page length is 10
for (MatchDocumentSummary summary : summaries) {
// System.out.println("Extracted friom URI-> " + summary.getUri());
uriList.add(summary.getUri());
}
if (i >= 1000) {//number of URI to store/retreive. plus 10
break;
}
}
uriList= uriList.stream().distinct().collect(Collectors.toList());
return uriList;