Develop Reference

Lucene LongPoint Range search doesn't work

I am using Lucene 8.2.0 in Java 11.
I am trying to index a Long value so that I can filter by it using a range query, for example like so: +my_range_field:[1 TO 200]. However, any variant of that, even my_range_field:[* TO *], returns 0 results in this minimal example. As soon as I remove the + from it to make it an OR, I get 2 results.
So I am thinking I must make a mistake in how I index it, but I can't make out what it might be.
From the LongPoint JavaDoc:
An indexed long field for fast range filters. If you also need to store the value, you should add a separate StoredField instance.
Finding all documents within an N-dimensional shape or range at search time is efficient. Multiple values for the same field in one document is allowed.
This is my minimal example:
public static void main(String[] args) {
Directory index = new RAMDirectory();
StandardAnalyzer analyzer = new StandardAnalyzer();
try {
IndexWriter indexWriter = new IndexWriter(index, new IndexWriterConfig(analyzer));
Document document1= new Document();
Document document2= new Document();
document1.add(new LongPoint("my_range_field", 10));
document1.add(new StoredField("my_range_field", 10));
document2.add(new LongPoint("my_range_field", 100));
document2.add(new StoredField("my_range_field", 100));
document1.add(new TextField("my_text_field", "test content 1", Field.Store.YES));
document2.add(new TextField("my_text_field", "test content 2", Field.Store.YES));
indexWriter.deleteAll();
indexWriter.commit();
indexWriter.addDocument(document1);
indexWriter.addDocument(document2);
indexWriter.commit();
indexWriter.close();
QueryParser parser = new QueryParser("text", analyzer);
IndexSearcher indexSearcher = new IndexSearcher(DirectoryReader.open(index));
String luceneQuery = "+my_text_field:test* +my_range_field:[1 TO 200]";
Query query = parser.parse(luceneQuery);
System.out.println(indexSearcher.search(query, 10).totalHits.value);
} catch (IOException e) {
} catch (ParseException e) {
}
}

You need to first use StandardQueryParser, then provide the parser with a PointsConfig map, essentially hinting which fields are to be treated as Points. You'll now get 2 results.
// Change this line to the following
StandardQueryParser parser = new StandardQueryParser(analyzer);
IndexSearcher indexSearcher = new IndexSearcher(DirectoryReader.open(dir));
/* Added code */
PointsConfig longConfig = new PointsConfig(new DecimalFormat(), Long.class);
Map<String, PointsConfig> pointsConfigMap = new HashMap<>();
pointsConfigMap.put("my_range_field", longConfig);
parser.setPointsConfigMap(pointsConfigMap);
/* End of added code */
String luceneQuery = "+my_text_field:test* +my_range_field:[1 TO 200]";
// Change the query to the following
Query query = parser.parse(luceneQuery, "text");

I found the solution to my problem.
I was under the impression that the query parser could just parse any query string correctly. That doesn't seem to be the case.
Using
Query rangeQuery = LongPoint.newRangeQuery("my_range_field", 1L, 11L);
Query searchQuery = new WildcardQuery(new Term("my_text_field", "test*"));
Query build = new BooleanQuery.Builder()
.add(searchQuery, BooleanClause.Occur.MUST)
.add(rangeQuery, BooleanClause.Occur.MUST)
.build();
returned the correct result.

Mongodb java aggregate match with objectid type

I am trying to aggregate values based on group by, match and sort. However, my matching field type is ObjectId. I have an input parameter which is a type of ObjectId(ObjectId settingId), however, below code does not return anything.
Can anyone find the problem in my code?
AggregateIterable < Document > iterable = thermalComfortCollection.aggregate(Arrays.asList(
new Document("$group", new Document("_id", "$Timestamp").append("ThermalComfortList", new Document("$push", "$ThermalComfort"))),
new Document("$match", new Document("settingID", settingId)),
new Document("$sort", new Document("_id", 1))));

You doing a group on the first stage, print that result to check if there's a "settingID" field on the top level.
From you $group stage it seems like the output will be:
[
{
_id : value of $Timestamp,
ThermalComfortList : []},
...
]
When posisble do the $match stage before the $group stage. $match is then able to use the (i hope available) index on settingID

You can use the BasicDBObject as follows:
DBObject match = new BasicDBObject("$match", new BasicDBObject("settingID", new ObjectId("")));
DBObject group = new BasicDBObject("$group", new BasicDBObject("_id", "$Timestamp").append("ThermalComfortList", new BasicDBObject("$push", "$ThermalComfort")));
DBObject group = new BasicDBObject("$sort", new BasicDBObject("_id", 1));
List<DBObject> aggregateList = new ArrayList<DBObject>();
aggregateList.add(match11);
aggregateList.add(group11);
aggregateList.add(group11);
AggregationOutput result = collection.aggregate(aggregateList);

Lucene Prefix on title/content is extremely slow

Here's what I did, and I have only 150 files with 2KB average file size.
BooleanQuery.Builder builder = new BooleanQuery.Builder();
// filename : prefix search
BooleanQuery.Builder filenameBuilder = new BooleanQuery.Builder();
for (String s : queryStr.trim().split("\\s+")) {
PrefixQuery filenameQuery1 = new PrefixQuery(new Term(Indexer.FIELD_NAME_FILENAME, s));
filenameBuilder.add(filenameQuery1, BooleanClause.Occur.SHOULD);
}
// content : prefix search
BooleanQuery.Builder contentBuilder = new BooleanQuery.Builder();
for (String s : queryStr.trim().split("\\s+")) {
PrefixQuery contentQuery1 = new PrefixQuery(new Term(Indexer.FIELD_NAME_CONTENT, s));
contentBuilder.add(contentQuery1, BooleanClause.Occur.MUST);
}
// Set filename and content with different weights.
Query filenameQuery = filenameBuilder.build();
Query contentQuery = contentBuilder.build();
filenameQuery.setBoost(4.0f);
contentQuery.setBoost(1.0f);
// Merge filename and content queries : filename OR content
builder.add(filenameQuery, BooleanClause.Occur.SHOULD);
builder.add(contentQuery, BooleanClause.Occur.SHOULD);
Query query = builder.build();
Note : If I search filename only, it seems a lot faster but filename + content turn out 10+ seconds!
Also, I've tried to change PrefixQuery on content to TermQuery (Which I mean to do exact search), but it still can't make any improvement.

How to use MultiFieldQueryParser from Lucene?

I am using Version.Lucene_29. Using the normal string query method i could do the following:
Directory directory = new FSDirectory(...);
//Start Lucene retrieval.
IndexSearcher iSearch = new IndexSearcher(directory, true);
Analyzer analyzer = new WhitespaceAnalyzer();
QueryParser parser = new QueryParser(Version.LUCENE_29, "content", analyzer);
String str = 'filename:testfile.txt AND filetext:"Singapore food"'
Query query = parser.parse(str);
ScoreDoc[] hits = iSearch.search(query, 1000).scoreDocs;
How do i fire a query using MultiFieldQueryParser in Lucene similar to the string query method?
MultiFieldQueryParser multiParser = new MultiFieldQueryParser(
Version.LUCENE_29, new String[] {"content", "ne"}, analyzer);
str = ???
Query = ????
ScoreDoc[] hits = iSearch.search(query, 1000).scoreDocs;

MultiFieldQueryParser allows you to search for a "WORD" in more then one Fileds with same Analyzer.
e.g.
Query query = MultiFieldQueryParser.parse("development",
new String[]{"title", "subject"},
new SimpleAnalyzer());
it will look for word development in Field : "title" and Field : "subject"

MultiFieldQueryParser is-a QueryParser, MultiFieldQueryParser creates the two Queries into a BooleanClause in this case. So it also supports filename:testfile.txt AND filetext:"Singapore food".

How to specify two Fields in Lucene QueryParser?

I read How to incorporate multiple fields in QueryParser? but i didn't get it.
At the moment i have a very strange construction like:
parser = New QueryParser("bodytext", analyzer)
parser2 = New QueryParser("title", analyzer)
query = parser.Parse(strSuchbegriff)
query2 = parser.Parse(strSuchbegriff)
What can i do for something like:
parser = New QuerParser ("bodytext" , "title",analyzer)
query =parser.Parse(strSuchbegriff)
so the Parser looks for the searching word in the field "bodytext" an in the field "title".

There are 3 ways to do this.
The first way is to construct a query manually, this is what QueryParser is doing internally. This is the most powerful way to do it, and means that you don't have to parse the user input if you want to prevent access to some of the more exotic features of QueryParser:
IndexReader reader = IndexReader.Open("<lucene dir>");
Searcher searcher = new IndexSearcher(reader);
BooleanQuery booleanQuery = new BooleanQuery();
Query query1 = new TermQuery(new Term("bodytext", "<text>"));
Query query2 = new TermQuery(new Term("title", "<text>"));
booleanQuery.add(query1, BooleanClause.Occur.SHOULD);
booleanQuery.add(query2, BooleanClause.Occur.SHOULD);
// Use BooleanClause.Occur.MUST instead of BooleanClause.Occur.SHOULD
// for AND queries
Hits hits = searcher.Search(booleanQuery);
The second way is to use MultiFieldQueryParser, this behaves like QueryParser, allowing access to all the power that it has, except that it will search over multiple fields.
IndexReader reader = IndexReader.Open("<lucene dir>");
Searcher searcher = new IndexSearcher(reader);
Analyzer analyzer = new StandardAnalyzer();
MultiFieldQueryParser queryParser = new MultiFieldQueryParser(
new string[] {"bodytext", "title"},
analyzer);
Hits hits = searcher.Search(queryParser.parse("<text>"));
The final way is to use the special syntax of QueryParser see here.
IndexReader reader = IndexReader.Open("<lucene dir>");
Searcher searcher = new IndexSearcher(reader);
Analyzer analyzer = new StandardAnalyzer();
QueryParser queryParser = new QueryParser("<default field>", analyzer);
// <default field> is the field that QueryParser will search if you don't
// prefix it with a field.
string special = "bodytext:" + text + " OR title:" + text;
Hits hits = searcher.Search(queryParser.parse(special));
Your other option is to create new field when you index your content called bodytextandtitle, into which you can place the contents of both bodytext and title, then you only have to search one field.

We can not use
BooleanQuery booleanQuery = new BooleanQuery();
We have to use builder
BooleanQuery.Builder finalQuery = new BooleanQuery.Builder();
then we can use finalQuery.build(); to get query
more generic way to do this is
private static TopDocs search(Map filters, IndexSearcher searcher) throws Exception {
StandardAnalyzer analyzer = new StandardAnalyzer();
BooleanQuery.Builder finalQuery = new BooleanQuery.Builder();
for(String attribute : filters.keySet()) {
QueryParser queryParser = new QueryParser(attribute, analyzer);
Query query = queryParser.parse(filters.get(attribute));
finalQuery.add(query, Occur.MUST);
}
TopDocs hits = searcher.search(finalQuery.build(),10);
return hits;
}