Specify dynamic sort order with StructuredQueryBuilder - java

When using MarkLogic 7 with the Java Client API, I am currently in the process of moving my query definitions from a StringQueryDefinition to a StructuredQueryDefinition allowing me to construct and manipulate the query programmatically.
With the string query I was able to use the sort operator with sort:{my-sort-order} successfully which in turn referred to names of predefined orders as specified in the query options (https://docs.marklogic.com/guide/search-dev/query-options#id_30002), but cannot find the API docs a related method allowing me to specify the sort order with the structured query builder.
What is the recommended way on how to specify the sort order when using a StructuredQueryDefinition?
UPDATE
Based on Erik's proposal, this is how the code snippet currently looks like, but it doesn't solve the problem, since the operator-state has to go as child on the query element and not as child on the search element:
RawStructuredQueryDefinition queryDef = qb.build(qb.and(qb.term(..), qb.rangeConstraint(...)));
String sorting = "<operator-state><operator-name>sort</operator-name><state-name>" + orderBy + "</state-name></operator-state>";
String combi = "<search xmlns='http://marklogic.com/appservices/search'>" + queryDef.toString() + sorting + "</search>";
RawCombinedQueryDefinition combinedQueryDef = queryManager.newRawCombinedQueryDefinition(new StringHandle(combi), OPTIONS);
// DOES NOT WORK, but will lead to MarkLogicIOException "Could not construct search results: parser error"
// Possible solution is to modify the queryDef DOM your own

We used the <options> tag of the <search> element sent to the server.
This requires an ugly string concatenation, but it does not require anything server-side other than an index on the sort property.
For the format of the XML, see this link or the search.xsd :
http://docs.marklogic.com/guide/rest-dev/appendixb#id_33716
The idea is to generate an XML like this one :
<search xmlns='http://marklogic.com/appservices/search'>
<query xmlns="http://marklogic.com/appservices/search">
<collection-query>
<uri>a_collection</uri>
</collection-query>
</query>
<options>
<sort-order type="xs:dateTime" direction="descending">
<json-property>a_field</json-property>
</sort-order>
</options>
</search>
We did it like this :
First, build your StructuredQueryDefinition
StructuredQueryDefinition queryDef = sb.collection("my_collection);
Build the <options> element seen above
String xmlSortNode =
" <options>" +
" <sort-order type=\"xs:dateTime\" direction=\"descending\">" +
" <json-property>a_field</json-property>" +
" </sort-order>" +
" </options>";
Build the full search element
String searchXml="<search xmlns='http://marklogic.com/appservices/search'>"
+ queryDef.serialize()
+ xmlSortNode
+ "</search>";
Execute the query
queryManager.newRawCombinedQueryDefinition(new StringHandle(searchXml),"all");

At present, StructuredQueryBuilder only builds criteria. You would have to refer to persisted query options with the sort order or send a combined search with both the criteria and options.

We’re tracking this on GitHub. Please feel free to add information there.

This is the solution I ended up with, basically I use the programmatic way on building a query with StructuredQueryBuilder and then inject the operator-state, referring to a pre-defined sort order in my options (OPTIONS_ALL):
org.jdom2.Document doc = new SAXBuilder().build(new StringReader(queryDef.serialize()));
if (!StringUtils.isEmpty(orderBy)) {
Element operatorState = new Element("operator-state", NAMESPACE_SEARCH);
operatorState.addContent(new Element("operator-name", NAMESPACE_SEARCH).setText("sort"));
operatorState.addContent(new Element("state-name", NAMESPACE_SEARCH).setText(orderBy));
doc.getRootElement().addContent(operatorState);
}
RawStructuredQueryDefinition rawQueryDef =
queryManager.newRawStructuredQueryDefinition(new JDOMHandle(doc), OPTIONS_ALL);
// ~~
SearchHandle resultsHandle = new SearchHandle();
queryManager.search(rawQueryDef, resultsHandle, start);
With
public static final Namespace NAMESPACE_SEARCH = Namespace.getNamespace("http://marklogic.com/appservices/search");

Sort order is considered an operator. Take a look at operator-state and you'll find an example of sorting in a structured query.

Related

How to enable `relevance-trace` using MarkLogic Java API?

I'm implementing quite a complex search using MarkLogic Java API. I would like to enable relevance-trace (Relavance trace) to see how my results are scored. Unfortunately, I don't know how to enable it in Java API. I have tried something like:
DatabaseClient client = initClient();
var qmo = client.newServerConfigManager().newQueryOptionsManager();
var searchOptions = "<search:options xmlns=\"http://marklogic.com/appservices/search\">\n"
+ " <search-option>relevance-trace</search-option>\n"
+ " </search:options>";
qmo.writeOptions("searchOptions", new StringHandle(searchOptions).withFormat(Format.XML));
QueryManager qm = client.newQueryManager();
StructuredQueryBuilder qb = qm.newStructuredQueryBuilder("searchOptions");
// query definition
qm.search(query, new SearchHandle())
Unfortunately it ends up with following error:
"Local message: /config/query write failed: Internal Server Error. Server Message: XDMP-DOCNONSBIND:
xdmp:get-request-body(\"xml\") -- No namespace binding for prefix search at line 1 . See the
MarkLogic server error log for further detail."
My question is how to use search options in MarkLogic API, especially I'm interested in relevance-trace and simple-score
Update 1
As suggested by #Jamess Kerr I have change my options to
var searchOptions = "<options xmlns=\"http://marklogic.com/appservices/search\">\n"
+ " <search-option>relevance-trace</search-option>\n"
+ " </options>";
but unfortunately, it still doesn't work. After that change I get error:
Local message: /config/query write failed: Internal Server Error. Server Message: XDMP-UPDATEFUNCTIONFROMQUERY: xdmp:apply(function() as item()*) -- Cannot apply an update function from a query . See the MarkLogic server error log for further detail.
Your search options XML uses the search: namespace prefix but you don't define that prefix. Since you are setting the default namespace, just drop the search: prefix from the search:options open and close tags.
The original Java Query contains both syntactical and semantic issues:
First of all, it is an invalid MarkLogic XQuery in the sense that it has only query option(s) portion. Bypassing the namespace binding prefix is another wrong end of the stick.
To tweak your original query, please replace a search text in between the search:qtext tag ( the pink line ) and run the query.
Result:
Matched and Listing 2 documents:
Matched 1 locations in /medals/coin_1333113127296.xml with 94720 score:
73. …pulsating maple leaf coin another world-first, the [Royal Canadian Mint]is proud to launch a numismatic breakthrough from its ambitious and creative R&D team...
Matched 1 locations in /medals/coin_1333078361643.xml with 94720 score:
71. ...the [Royal Canadian Mint]and Royal Australian Mint have put an end to the dispute relating to the circulation coin colouring process...
Without a semantic criterion, to put it into context, your original query will be an equivalent of removing the search:qtext and performing a fuzzy search.
Note:
If you use serialised term search or search constraints instead of text search, you should get higher score results.
MarkLogic Java API operates in unfiltered mode by default, while the cts:search operates in filtered mode by default. Just be mindful of how you construct the query and the expected score in Java API.
And it is really intended for bulk data write/extract/transformation. qconsole is, in my opinion, more befitting to tune specific query and gather search score, relevance and computation details.

Apache Lucene createWeight() for wildcard query

I'm using Apache Lucene 6.6.0 and I'm trying to extract terms from the search query. Current version of code looks like this:
Query parsedQuery = new AnalyzingQueryParser("", analyzer).parse(query);
Weight weight = parsedQuery.createWeight(searcher, false);
Set<Term> terms = new HashSet<>();
weight.extractTerms(terms);
It works pretty much fine, but recently I noticed that it doesn't support queries with wildcards (i.e. * sign). If the query contains wildcard(s), then I get an exception:
java.lang.UnsupportedOperationException: Query
id:123*456 does not implement createWeight at
org.apache.lucene.search.Query.createWeight(Query.java:66) at
org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:751)
at
org.apache.lucene.search.BooleanWeight.(BooleanWeight.java:60)
at
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:225)
So is there a way to use createWeight() with wildcarded queries? Or maybe there's another way to extract search terms from query without createWeight()?
Long story short, it is necessary to rewrite the query, for example, as follows:
final AnalyzingQueryParser analyzingQueryParser = new AnalyzingQueryParser("", analyzer);
// TODO: The rewrite method can be overridden.
// analyzingQueryParser.setMultiTermRewriteMethod(MultiTermQuery.CONSTANT_SCORE_BOOLEAN_REWRITE);
Query parsedQuery = analyzingQueryParser.parse(query);
// Here parsedQuery is an instance of the org.apache.lucene.search.WildcardQuery class.
parsedQuery = parsedQuery.rewrite(reader);
// Here parsedQuery is an instance of the org.apache.lucene.search.MultiTermQueryConstantScoreWrapper class.
final Weight weight = parsedQuery.createWeight(searcher, false);
final Set<Term> terms = new HashSet<>();
weight.extractTerms(terms);
Please refer to the thread:
Nabble: Lucene - Java Users - How to get the terms matching a WildCardQuery in Lucene 6.2?
Mail archive: How to get the terms matching a WildCardQuery in Lucene 6.2?
for further details.
It seems the mentioned Stack Overflow question is this one: How to get matches from a wildcard Query in Lucene 6.2.

Using API calls to retrieve commit ID that is generated when you revert a Gerrit change

I've been trying to retrieve the generated commit ID from a reverted Gerrit change using https://github.com/uwolfer/gerrit-rest-java-client but haven't been able to find a way to do so.
One of the ways I've been trying to get this ID is trying to get access to the related changes list.
The REST API documentation shows that you can use a query to retrieve this list.
How can I retrieve this list using API calls?
Is there another way to retrieve this commit ID?
I want to use this to track reverts and be able to analyze possible impacts this revert has on the project.
Found a way to solve this issue. What I did was adding a "&o=MESSAGES" tag to the query to retrieve the full change history list where the revert message gives you the target commit ID.
I then transfer the Collection<> that is returned into a list so I can easily access all the messages.
Collection<ChangeMessageInfo> messageColl = gerritClient.changes().query("commit:<commitID>&o=MESSAGES").get().get(0).messages;
final List<ChangeMessageInfo> messageList = new ArrayList<>(messageColl);
The revert message is usually the last entry of the change history log.
List of tags that can be used in a similar manner can be found here. You need to scroll down a bit to find the tags.
UPDATE:
Found an even more effecient way of finding the reverted commits.
With the code below you are able to retrieve the body message below the subject on Gerrit which in turn enables the possibility to query the commit ID that is presented in that field.
List<String> revertedCommits = new ArrayList<>();
revertedCommits.add(<commitID>);
String revertedCommit = "unknown";
Map<String, RevisionInfo> revisionInfo = gerritClient.changes().query("commit:" + revertedCommits.get(revertedCommits.size() - 1) + "&o=CURRENT_REVISION&o=COMMIT_FOOTERS").get().get(0).revisions;
Pattern p = Pattern.compile(Pattern.quote("This reverts commit ") + "(.*?)" + Pattern.quote("."));
Matcher m = p.matcher(revisionInfo.values().iterator().next().commitWithFooters);
while (m.find()) {
revertedCommit = m.group(1);
}
This can then be iterated through to find all the reverts connected to the first commit.
Note that I use the "&o=CURRENT_REVISION" and "&o=COMMIT_FOOTERS" tags in the query to access this information. Without these tags, you only get an empty array.

MongoDB Text Index using Java Driver

Using the MongoDB Java API, I have not been able to successfully locate a full example using text search. The code I am using is this:
DBCollection coll;
String searchString = "Test String";
coll.createIndex(new BasicDBObject ("blogcomments", "text"));
DBObject q = start("blogcomments").text(searchString).get();
The name of my collection that I am performing the search on is blogcomments. creatIndex() is the replacement method for the deprecated method ensureIndex(). I have seen examples for how to use the createIndex(), but not how to execute actual searches with the Java API. Is this the correct way to go about doing this?
That's not quite right. Queries that use indexes of type "text" can not specify a field name at query time. Instead, the field names to include in the index are specified at index creation time. See the documentation for examples. Your query will look like this:
DBObject q = QueryBuilder.start().text(searchString).get();

Problem with JDOQL to obtain results with a "contains" request

I am using Google App Engine for a project and I need to do some queries on the database. I use the JDOQL to ask the database. In my case I want to obtain the university that contains the substring "array". I think my query has a mistake because it returns the name of universities in the alphabetical order and not the ones containing the substring.
Query query = pm.newQuery("SELECT FROM " + University.class.getName() + " WHERE name.contains("+array+") ORDER BY name RANGE 0, 5");
Could someone tell me what's wrong in my query?
Thank you for your help!
EDIT
I have a list of universities store and I have a suggestbox where we can request a university by his name. And I want to autocomplete the requested name.
App engine does not support full-text searches, you should star issue 217. However, A partial workaround is possible. And in your case I think it is a good fit.
First thing, adjust your model such that there is a lower (or upper case) version of the name as well -- I will assume it is called lname. Unless you want your queries to be case-sensitive.
Then you query like this:
Query query = pm.newQuery(University.class);
query.setFilter("lname >= startNameParam");
query.setFilter("lname < stopNameParam");
query.setOrdering("lname asc");
query.declareParameters("String startNameParam");
query.declareParameters("String stopNameParam");
query.setRange(0, 5);
List<University> results = (List<University>) query.execute(search_value, search_value + "z");
The correct way to do this is like this -
Query query = pm.newQuery(University.class,":p.contains(name)");
query.setOrdering("name asc");
query.setRange(0, 5);
List univs = q.execute(Arrays.asList(array));
(note- In this case the :p is an implicit param name you can replace with any name)

Categories

Resources