Develop Reference

How to get error message in JSON SCHEMA VALIDATION using EVERIT in java?

I am trying to validate a json schema against the json input.
I am using
org.everit.json.schema-1.0.0.jar
My json input
{
"id": 200,
"name": "Green Learner",
"cost": 0
}
My JSON SCHEMA .
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Youtube Channel",
"description": "Youtube Channel for software development training",
"type": "object",
"properties": {
"id": {
"description": "The unique identifier for a product",
"type": "integer"
},
"name": {
"description": "Name of the the channle",
"type": "string"
},
"cost": {
"type": "number",
"minimum": 100,
"maximum":10000
}
},
"required": ["id", "name", "cost"]
}
JAVA Code for validation.
import org.everit.json.schema.loader.SchemaLoader;
import org.json.JSONObject;
import org.json.JSONTokener;
import org.everit.json.schema.Schema;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
/**
*
* #author amitkumar
*/
public class JsonValidate {
public static void main(String[] args) throws FileNotFoundException {
File schemaFile = new File("schema.json");
JSONTokener schemaData = new JSONTokener(new FileInputStream(schemaFile));
JSONObject jsonSchema = new JSONObject(schemaData);
//json data
File jsonData = new File("product_invalid.json");
JSONTokener jsonDataFile = new JSONTokener(new FileInputStream(jsonData));
JSONObject jsonObject = new JSONObject(jsonDataFile);
Schema schemaValidator = SchemaLoader.load(jsonSchema);
schemaValidator.validate(jsonObject);
System.out.println(jsonObject.getInt("cost"));
}
}
When i run the code with org.everit.json.schema-1.0.0.jar, i get following error message .
Exception in thread "main"
org.everit.json.schema.ValidationException:v0.0 is not higher or equal
to 100
This is the warning message i get when i use
json-schema-validator-1.0.42.jar comes with com.networknt it clearly mention me object name which got error.
$.Cost: 0.0 is not higher or equal to 100
i want to do the same with org.everit.json.schema-1.0.0.jar, which object in my json input got the error .It does not show me the object name .

If the library does not support your requirement, you could log the missing information by yourself. Something like
try {
schemaValidator.validate(jsonObject);
} catch(ValidationException e) {
LOG.error("Validation error in Object: '{}'", yourObjectName, e)
}
By the way, there are more newer versions than 1.0.0 of org.everit.json.schema available: https://search.maven.org/artifact/org.everit.json/org.everit.json.schema
Edit add additional information:
The ValidationException has methods like getPointerToViolation() which might get you the information you need. See JavaDoc of the current version: https://erosb.github.io/everit-json-schema/javadoc/1.12.2/org/everit/json/schema/ValidationException.html#getPointerToViolation()
Be aware, that version 1.0.0 does not have this methods (JavaDoc of v1.0.0)!

How to implement Post API JSON with Spring?

I'm having difficulty implementing a JSON to send as a POST call in Spring.
Which is the fastest and most effective way to turn this json into a java object or a map and make the call?
below is an example of a json to send:
{
"apiVersion": "apps/v1",
"kind": "Deployment",
"metadata": {
"name": "edge-ws"
},
"spec": {
"selector": {
"matchLabels": {
"run": "edge-ws"
}
},
"replicas": 1,
"template": {
"metadata": {
"labels": {
"run": "edge-ws"
}
},
"spec": {
"containers": [
{
"name": "edge-ws",
"image": "server-tim:latest",
"imagePullPolicy": "Never",
"ports": [
{
"containerPort": 80
}
]
}
]
}
}
}
}
this and the second body that has a value (nodeport) that must be taken from a field entered by the user front end side.(page created in html)
{
"apiVersion": "v1",
"kind": "Service",
"metadata": {
"name": "edge-ws",
"labels": {
"run": "edge-ws"
}
},
"spec": {
"type": "NodePort",
"ports": [
{
"port": 8080,
"targetPort": 80,
"nodePort": 30200,
"protocol": "TCP",
"name": "http"
}
],
"selector": {
"run": "edge-ws"
}
}
}
Both files must be sent with a single click on a button on the front end side.the first call with the first body starts and if everything is ok the second body starts
What should the class that maps objects look like? What should the controller look like instead?
They also gave me an address to call that can only be used on the machine, how can I test this call locally?
Thanks in advance!

You can use google's Gson library to convert the JsonString to Object and then use below code:
Gson gson = new Gson();
Object requestObject = gson.fromJson(jsonString, Object.class);
ResponseObject responseObject = restTemplate.postForObject(url, requestObject, ResponseObject.class);

Map ElasticSearch LowLevelRestClient's Response to Bean

I am using Elastic LowLevelRestClient to interact with my elastic instance, when I query the elastic using my search query it returns the response which is wrapped as an HttpEntity.
As per documentation of Elastic Reading Responses EntityUtils class of Apache provides a way to convert this HttpEntity into String which gives me below response. I just want to map this response to an appropriate Object.
My Code Snippet:
Request request = new Request("GET", "/neeraj_party/_search");
request.setJsonEntity(searchQuery);
Response response = lowLevelClient.performRequest(request);
String responseBody = EntityUtils.toString(response.getEntity());
ResponseBody looks like this
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 32.986195,
"hits": [
{
"_index": "neeraj_party",
"_type": "party",
"_id": "28588489",
"_score": 32.986195,
"_source": {
"name": "MUST HK LTD",
"city_nm": "郑州",
"#timestamp": "2019-03-23T18:28:07.305Z",
"type": "json",
"legal_nm": "MUST HK Ltd",
"gr_id": "28588489",
"path": "/ssd/sdds",
"address": "郑州",
"state_province_cd": "180",
"country_iso2_cd": "CN",
"host": "neeraj.com",
"postal_cd": "450000",
"#version": "1"
}
}
]
}
}
My Question is simple
Do ElasticSearch provide any such bean which can represent this response, or Should I create my own CustomBean.

You could use SearchResponse Object to achieve this.
If you use the search(SearchRequest) method, It gives you back a SearchResponse object (including aggs).
Or you also could make the SearchResponse from that String using this method.
public static SearchResponse getSearchResponseFromJson(String jsonResponse){
try {
NamedXContentRegistry registry = new
NamedXContentRegistry(DashboardCuke.getDefaultNamedXContents());
XContentParser parser =
JsonXContent.jsonXContent.createParser(registry, jsonResponse);
return SearchResponse.fromXContent(parser);
}catch (IOException e) {
System.out.println("exception " + e);
}catch (Exception e){
System.out.println("exception " + e);
}
return new SearchResponse();
}
I got this information from here: ElasticSearch Forum

Best practice to search ingest-attachment from documents (2k+ documents with ingest-attachment)

Am fetching the indexed documents from elastic search using Java API. But am getting Null as a response from elastic search when Index having more number of document like (2k+).
If index doesnt have more documents less than 500 something, the below Java API code is working properly.
More number of documents in Index, creating issue. ( Is that something like performance issue while fetching ?)
I used ingest-attachment processor plugin for attachment, i attached PDF in my documents.
But if i search with the same query using kibana with curl script am getting response, and am able to see the results in Kibana
Please find my java code below
private final static String ATTACHMENT = "document_attachment";
private final static String TYPE = "doc";
public static void main(String args[])
{
RestHighLevelClient restHighLevelClient = null;
try {
restHighLevelClient = new RestHighLevelClient(RestClient.builder(new HttpHost("localhost", 9200, "http"),
new HttpHost("localhost", 9201, "http")));
} catch (Exception e) {
System.out.println(e.getMessage());
}
SearchRequest contentSearchRequest = new SearchRequest(ATTACHMENT);
SearchSourceBuilder contentSearchSourceBuilder = new SearchSourceBuilder();
contentSearchRequest.types(TYPE);
QueryStringQueryBuilder attachmentQB = new QueryStringQueryBuilder("Activa");
attachmentQB.defaultField("attachment.content");
contentSearchSourceBuilder.query(attachmentQB);
contentSearchSourceBuilder.size(50);
contentSearchRequest.source(contentSearchSourceBuilder);
SearchResponse contentSearchResponse = null;
try {
contentSearchResponse = restHighLevelClient.search(contentSearchRequest); // returning null response
} catch (IOException e) {
e.getLocalizedMessage();
}
System.out.println("Request --->"+contentSearchRequest.toString());
System.out.println("Response --->"+contentSearchResponse.toString());
SearchHit[] contentSearchHits = contentSearchResponse.getHits().getHits();
long contenttotalHits=contentSearchResponse.getHits().totalHits;
System.out.println("condition Total Hits --->"+contenttotalHits);
Please find my script that am using in kibana., am getting response for the below script.
GET document_attachment/_search?pretty
{
"query" :{
"match": {"attachment.content": "Activa"}
}
}
Please find the below search request from Java API
SearchRequest{searchType=QUERY_THEN_FETCH, indices=[document_attachment], indicesOptions=IndicesOptions[id=38, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false], types=[doc], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=128, source={"size":50,"query":{"match":{"attachment.content":{"query":"Activa","operator":"OR","prefix_length":0,"max_expansions":50,"fuzzy_transpositions":true,"lenient":false,"zero_terms_query":"NONE","auto_generate_synonyms_phrase_query":true,"boost":1.0}}}}}
Please find my mapping details
{
"document_attachment": {
"mappings": {
"doc": {
"properties": {
"app_language": {
"type": "text"
},
"attachment": {
"properties": {
"author": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"content": {
"type": "text",
"analyzer": "custom_analyzer"
},
"content_length": {
"type": "long"
},
"content_type": {
"type": "text"
},
"date": {
"type": "date"
},
"language": {
"type": "text"
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"catalog_description": {
"type": "text"
},
"fileContent": {
"type": "text"
}
}
}
}
}
}
}
Please find my settings details
PUT _ingest/pipeline/document_attachment
{
"description" : "Extract attachment information",
"processors" : [
{
"attachment" : {
"field" : "fileContent"
}
}
]
}
Am getting this error only when am trying to search based on attachment.content , If i search with some other field am able to get results.
Am using ElasticSearch 6.2.3 version
Please find the error below.
org.apache.http.ContentTooLongException: entity content is too long [105539255] for the configured buffer limit [104857600]
at org.elasticsearch.client.HeapBufferedAsyncResponseConsumer.onEntityEnclosed(HeapBufferedAsyncResponseConsumer.java:76)
at org.apache.http.nio.protocol.AbstractAsyncResponseConsumer.responseReceived(AbstractAsyncResponseConsumer.java:131)
at org.apache.http.impl.nio.client.MainClientExec.responseReceived(MainClientExec.java:315)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseReceived(DefaultClientExchangeHandlerImpl.java:147)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.responseReceived(HttpAsyncRequestExecutor.java:303)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:255)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "main" java.lang.NullPointerException
at com.es.utility.DocumentSearch.main(DocumentSearch.java:88)

ElasticSearch hits have no fields

I am having a problem where when I run a search on elastic using the java api I get back results... but when I try and extract values from the results there are no fields.
ElasticSearch v5.3.1
Elastic API: org.elasticsearch.client:transport v5.3.0
My code:
SearchRequestBuilder srb = client.prepareSearch("documents").setTypes("documents").setSearchType(SearchType.QUERY_THEN_FETCH).setQuery(qb).setFrom(0).setSize((10)).setExplain(false);
srb.addDocValueField("title.raw");
SearchResponse response = srb.get();
response.getHits().forEach(new Consumer<SearchHit>() {
#Override
public void accept(SearchHit hit) {
System.out.println(hit);
Map<String, SearchHitField> fields = hit.getFields();
Object obj = fields.get("title.raw").getValue();
}
});
When the forEach runs the obj is always coming back null. Fields has an item in it with the key of title.raw and it has a SearchHitField.

Fields is only used when you are trying to obtain stored fields. By default you should obtain fields using the source. With the following code samples I'll try to explain it.
PUT documents
{
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
},
"mappings": {
"document": {
"properties": {
"title": {
"type": "text",
"store": true
},
"description": {
"type": "text"
}
}
}
}
}
PUT documents/document/_bulk
{"index": {}}
{"title": "help me", "description": "description of help me"}
{"index": {}}
{"title": "help me two", "description": "another description of help me"}
GET documents/_search
{
"query": {
"match": {
"title": "help"
}
},
"stored_fields": ["title"],
"_source": {
"includes": ["description"]
}
}
Next the response, notice the difference between the stored field Title and the normal field description.
{
"_index": "documents",
"_type": "document",
"_id": "AVuciB12YLj8D0X3N5We",
"_score": 0.14638957,
"_source": {
"description": "another description of help me"
},
"fields": {
"title": [
"help me two"
]
}
}