Develop Reference

Mappings field empty in elasticsearch

I am trying to create elasticsearch index using Java API ,
if we see below index mappings filed is coming as empty .
# curl -XGET localhost:9200/initest2
{"initest2":{"aliases":{},**"mappings":{}**,"settings":{"index":{"keyspace":"keyspace_test","number_of_shards":"5","provided_name":"initest2","max_result_window":"10000000","creation_date":"1671025346350","analysis":{"normalizer":{"testnormalizer":{"filter":["asciifolding"],"type":"custom","char_filter":[]}}},"number_of_replicas":"0","uuid":"BK9oTTKETZeWqiuR_tGd8w","version":{"created":"6080499"}}}}}
JAVA code :
String indexQuery = "";
StringBuilder indexString = new StringBuilder("");
String test = "{\"settings\": {\"max_result_window\":10000000,\"keyspace\":\""+keyspaceName+'\u0022' +",\"analysis\": "
+ "{\"normalizer\":{\"testnormalizer\": {\"type\": \"custom\",\"char_filter\": [],\"filter\": [\"asciifolding\"]}}}},"
+ "\"mappings\":{\""+tableName+"\":{\"discover\" : \".*\",\"properties\":{";
indexString.append(test);
for (String i : indexColumns)
{
indexString.append('\u0022'+i+'\u0022' +":" +"{ \"type\": \"keyword\",\"normalizer\": \"testnormalizer\"}" + ",");
}
indexString.delete(indexString.length()-1, indexString.length()).append("}}}}");
indexQuery = indexString.toString();
logger.info("index query is " + indexQuery);
CreateIndexRequest request = new CreateIndexRequest(tableName);
request.source(indexQuery, XContentType.JSON);
CreateIndexResponse createIndexResponse= restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);
while printing the query before calling restHighLevel client :
index query is
{"settings": {"max_result_window":10000000,"keyspace":"ndl","analysis": {"normalizer":{"testnormalizer": {"type": "custom","char_filter": [],"filter": ["asciifolding"]}}}},"mappings":{"initest2":{"discover" : ".*","properties":{"name":{ "type": "keyword","normalizer": "testnormalizer"}}}}}
Please assist if any method to be updated in java code / any properties to be updated in elassandra
Current versrion of elasticsearch : 6.8.4
Note : tried using request.settings() & request.mappings() but it didnt help

How can I access Schema from the QueryResponse while calling getQueryResults method from my Java application?

I am using google.cloud.bigquery library to execute and create query using bigquery.query() method. I want to fetch the Schema details from the response but whenever the query returns no result, I am getting EmptyTableResult instead of the response which should have Schema and Fields listed inside it. I used another approach which creates job and then using the query job, I am calling bigquery.getQueryResults which should return QueryResponse object. Below is the code snippet.
QueryJobConfiguration queryConfig =
QueryJobConfiguration.newBuilder(queryString)
.setDefaultDataset(bqDatasetId).setUseLegacySql(false)
.setFlattenResults(true).build();
JobId jobId = JobId.of(UUID.randomUUID().toString());
Job queryJob = bigQuery.create(JobInfo.newBuilder(queryConfig).setJobId(jobId).build());
// Wait for the query to complete.
queryJob = queryJob.waitFor();
// Check for errors
if (queryJob == null) {
throw new RuntimeException("Job no longer exists");
} else if (queryJob.getStatus().getError() != null) {
throw new RuntimeException(queryJob.getStatus().getError().toString());
}
// Get the results.
QueryResponse response = bigQuery.getQueryResults(queryJob.getJobId());
System.out.println(response);
Here, in sysout statement, I am getting the proper response, but whenever I am trying to use response.getSchema(), it is giving me compilation error saying getSchema() is not visible. Can anyone help me with this? Is this approach correct or there is any other approach which can do the same thing?

You need to call getQueryResults() on the Job object instead. This will give you back a TableResult object. You can then call getSchema() to get the schema of the query/table/job. So, putting it all together:
QueryJobConfiguration queryConfig =
QueryJobConfiguration.newBuilder(
"SELECT "
+ "CONCAT('https://stackoverflow.com/questions/', CAST(id as STRING)) as url, "
+ "view_count "
+ "FROM `bigquery-public-data.stackoverflow.posts_questions` "
+ "WHERE tags like '%google-bigquery%' "
+ "ORDER BY favorite_count DESC LIMIT 10")
.setUseLegacySql(false)
.build();
JobId jobId = JobId.of(UUID.randomUUID().toString());
Job queryJob = BIGQUERY.create(JobInfo.newBuilder(queryConfig).setJobId(jobId).build());
queryJob = queryJob.waitFor();
TableResult result = queryJob.getQueryResults(); //<--you need this
Schema schema = result.getSchema(); //<--..and this
System.out.println(schema);
Which yields:
Connected to the target VM, address: '127.0.0.1:64695', transport: 'socket'
Schema{fields=[Field{name=url, type=STRING, mode=NULLABLE, description=null}, Field{name=view_count, type=INTEGER, mode=NULLABLE, description=null}]}

After searching a lot, I came to a conclusion that it is better to use reflections to invoke the invisible QueryRespons.getSchema() method and it worked like a charm. Though, reflection is not an ideal solution for this but it has resolved my problem.

public static JSONArray convert(TableResult resultSet) throws Exception {
JSONArray jsonArray = new JSONArray();
Schema schema = resultSet.getSchema();
FieldList MyFields = schema.getFields();
Field MyField2 = null;
for (FieldValueList row : resultSet.iterateAll()) {
JSONObject obj = new JSONObject();
for (int i = 0; i < MyFields.size(); i++) {
MyField2 = MyFields.get(i);
obj.put(MyField2.getName().toLowerCase(), row.get(i).getValue());
}
jsonArray.put(obj);
}
return jsonArray;
}

Another approach is to use the solution in this ticket Querying Column Headers in GBQ
This is running a query and not using the api

How to create a query with query_string using Elasticsearch Java Api

Currently my query request body looks like
{
"query": {
"query_string": {
"default_field": "file",
"query": "Email OR #gmail.com #yahoo.com"
}
},
"highlight": {
"fields": {
"file": {
}
}
}
}
My java code looks like
String queryString = "{"
+ "\"query_string\": "
+ "{"
+ "\"default_field\":"
+ " \"file\","
+ " \"query\": \"Email OR #gmail.com #yahoo.com\""
+ "}"
+ "}";
with following API calls
SearchRequestBuilder searchRequestBuilder = client.prepareSearch()
.setIndices("resume")
.setTypes("docs").setQuery(queryString).addHighlightedField("file");
SearchResponse response = searchRequestBuilder.execute().actionGet();
I would prefer a more api based approach for "queryString" part.I am not able to find any api which handles "query_string" part of the request. There are apis for match_all,match, term so on and so forth but not for query_string
Any help would be really appreciated

QueryBuilders is the factory for creating any query including query_string. From documentation:
import static org.elasticsearch.index.query.QueryBuilders.*;
QueryBuilder qb = queryStringQuery("+kimchy -elasticsearch");
Your query would be built as follows:
QueryBuilder qb = queryStringQuery("Email OR #gmail.com #yahoo.com").defaultField("file");
And the full example would be:
SearchRequestBuilder searchRequestBuilder = client.prepareSearch()
.setIndices("resume")
.setTypes("docs").setQuery(qb).addHighlightedField("file");
SearchResponse response = searchRequestBuilder.execute().actionGet();

Convert JSON to hash map with Jackson- Java

Before this is marked as a duplicate please read the question (I did look at similar ones). Thank you.
For simplicity, assume I have JSON like this:
{
"clients" : [
{
"name" : "client 1",
"id" : 1
},
{
"name" : "client 2",
"id" : 2
}
],
"other" : {
"something" : ""
}
...
}
So I want to create a hash map of only the clients and their fields. The basic question is how would I go about doing this using Jackson methods for a single JSON array like clients? I've tried to look online but all of the examples that I have seen either don't use Jackson or only are for a single JSON object like so:
HashMap<String, String>[] values = new ObjectMapper().readValue(jsonString, new TypeReference<HashMap<String, String>[]>() {});
I've also seen Gson examples and I know I can do some string parsing magic:
jsonSting = jsonString.substring(jsonString.indexOf("["), (jsonString.indexOf("]")+1))
to get it in a format that I can use, but I want to try it with Jackson to avoid importing another library. Any ideas?
Rephrasing the question:
So if I only had a list of clients like so:
jsonString = [{"name" : "client 1","id" : 1},{"name" : "client 2","id" : 2}]
then I could just do:
HashMap[] values = new ObjectMapper().readValue(jsonString, new TypeReference[]>() {});
to get what I want. I am basically asking if there is a way using Jackson methods to get the jsonString above from the large JSON section on top. I know I can easily do it with this example with string parsing but there will be more complex situations in the future and string parsing is not really considered best practice

You can extract a part of the JSON tree using the Jackson tree model API and then convert it to an array of maps.
Here is an example:
public class JacksonReadPart {
public static final String JSON = "{\n" +
" \"clients\" : [\n" +
" {\n" +
" \"name\" : \"client 1\",\n" +
" \"id\" : 1\n" +
" },\n" +
" {\n" +
" \"name\" : \"client 2\",\n" +
" \"id\" : 2\n" +
" }\n" +
"],\n" +
" \"other\" : {\n" +
" \"something\" : \"\"\n" +
" }\n" +
"\n" +
"}";
public static void main(String[] args) throws IOException {
ObjectMapper mapper = new ObjectMapper();
JsonNode node = mapper.readTree(JSON).path("clients");
// non type safe
Map<String, Object>[] clients = mapper.treeToValue(node, Map[].class);
System.out.println(Arrays.toString(clients));
// type safe
JsonParser parser = mapper.treeAsTokens(node);
clients = parser.readValueAs(new TypeReference<Map<String, Object>[]>() {});
System.out.println(Arrays.toString(clients));
}
}
Output:
[{name=client 1, id=1}, {name=client 2, id=2}]
[{name=client 1, id=1}, {name=client 2, id=2}]

Retrieving a Document from a QueryResult with OpenCMIS

I'm using the Apache Chemistry OpenCMIS java library. Given a QueryResult (e.g. I found a document or a bunch of documents by searching on metadata properties), is this a reasonable way to retrieve the Document object itself? Or is there a more efficient way?
ItemIterable<QueryResult> results = session.query("SELECT * FROM cmis:document WHERE cmis:name LIKE 'test%'", false);
for(QueryResult hit: results) {
Document document = (Document) session.getObject(session.createObjectId((String) hit.getPropertyValueById("cmis:objectId")));
}

Try the Session.queryObjects() method.

For me this works perfectly
String myType = "my:documentType";
// get the query name of cmis:objectId
ObjectType type = session.getTypeDefinition(myType);
PropertyDefinition<?> objectIdPropDef = type.getPropertyDefinitions().get(PropertyIds.OBJECT_ID);
String objectIdQueryName = objectIdPropDef.getQueryName();
String queryString = "SELECT " + objectIdQueryName + " FROM " + type.getQueryName();
// execute query
ItemIterable<QueryResult> results = session.query(queryString, false);
for (QueryResult qResult : results) {
String objectId = qResult.getPropertyValueByQueryName(objectIdQueryName);
Document doc = (Document) session.getObject(session.createObjectId(objectId));
}
Found here: https://chemistry.apache.org/java/examples/example-process-query-results.html