Using spring configuration to read data from json into spring bean it works for simple type but i am getting the above error when try to read an array
This is my config
#Bean
Container container() throws IOException {
ObjectMapper objectMapper = new ObjectMapper();
return objectMapper.readValue(getClass().getClassLoader().getResource("book.json"), Container.class);
}
I have decided to put in a container as suggested so its ok to have my config look like this ? As i will be using these data to create Some spring controllers.
POJO CLASS
public class Container {
String requestedUrl;
List<Item> items;}
This is the json data ,
{
"requestedUrl": "https://www.googleapis.com/books/v1/volumes?q=java&maxResults=40",
"items": [
{
"kind": "books#volume",
"id": "7tkN1CYzn2cC",
"etag": "pfjjxSpetIM",
"selfLink": "https://www.googleapis.com/books/v1/volumes/7tkN1CYzn2cC",
"volumeInfo": {
"title": "A Hypervista of the Java Landscape",
"publisher": "InfoStrategist.com",
"industryIdentifiers": [
{
"type": "ISBN_13",
"identifier": "9781592432172"
},
{
"type": "ISBN_10",
"identifier": "1592432174"
}
],
"readingModes": {
"text": true,
"image": true
},
"printType": "BOOK",
"maturityRating": "NOT_MATURE",
"allowAnonLogging": false,
"contentVersion": "1.0.1.0.preview.3",
"imageLinks": {
"smallThumbnail": "http://books.google.com/books/content?id=7tkN1CYzn2cC&printsec=frontcover&img=1&zoom=5&edge=curl&source=gbs_api",
"thumbnail": "http://books.google.com/books/content?id=7tkN1CYzn2cC&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api"
},
"language": "en",
"previewLink": "http://books.google.pl/books?id=7tkN1CYzn2cC&pg=PP1&dq=java&hl=&cd=1&source=gbs_api",
"infoLink": "http://books.google.pl/books?id=7tkN1CYzn2cC&dq=java&hl=&source=gbs_api",
"canonicalVolumeLink": "https://books.google.com/books/about/A_Hypervista_of_the_Java_Landscape.html?hl=&id=7tkN1CYzn2cC"
},
"saleInfo": {
"country": "PL",
"saleability": "NOT_FOR_SALE",
"isEbook": false
}, //Shortened for brevity
How can edit the config to get it work
You are passing the json object which is bounded by {} but trying to convert it to array. With this json you should convert to another class like
public class Container {
private String requestedUrl;
private List<Item> items;
// getters/setters
}
Related
I have a complex index with a ngram analyzer. I want to be able to create a new index through the Java API. I am currently using Kotlin for this but using the same framework. I have created the schema for this index as so:
{
"settings": {
"index": {
"max_ngram_diff": 20,
"search.idle.after": "10m"
},
"analysis": {
"analyzer": {
"ngram3_analyzer": {
"tokenizer": "ngram3_tokenizer",
"filter": [
"lowercase"
]
}
},
"tokenizer": {
"ngram3_tokenizer": {
"type": "ngram",
"min_gram": 3,
"max_gram": 20
}
}
}
},
"mappings": {
"dynamic": "strict",
"_doc": {
"properties": {
"name": {
"type": "keyword",
"fields": {
"partial": {
"type": "text",
"analyzer": "ngram3_analyzer",
"search_analyzer": "keyword"
},
"text": {
"type": "text"
}
}
},
"location": {
"type": "geo_shape",
"ignore_malformed": true
},
"type": {
"type": "keyword"
},
"sort": {
"type": "integer"
}
}
}
}
}
This json schema works when manually passing it via a rest client PUT call.
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "new_index_created"
}
Passing the same schema via elastic java API using the following koltin function:
private fun createIndex(index: String, schema: String) {
val createIndexRequest = CreateIndexRequest(index).mapping(schema, XContentType.JSON)
getClient().indices().create(createIndexRequest, RequestOptions.DEFAULT)
}
I get this response:
Elasticsearch exception [type=mapper_parsing_exception, reason=Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters: [settings : {index={max_ngram_diff=20, search.idle.after=10m}, analysis={analyzer={ngram3_analyzer={filter=[lowercase], tokenizer=ngram3_tokenizer}}, tokenizer={ngram3_tokenizer={min_gram=3, type=ngram, max_gram=20}}}}] [mappings : {_doc={properties={name={type=keyword, fields={text={type=text}, partial={search_analyzer=keyword, analyzer=ngram3_analyzer, type=text}}}, location={ignore_malformed=true, type=geo_shape}, sort={type=integer}, type={type=keyword}}}, dynamic=strict}]]
any help on this issue would be great :)
The error you get is because you're passing both mappings and settings into the mapping(...) call.
You can either call mapping() with only the mappings section and setting() with the settings section, or you can call source() like this:
val createIndexRequest = CreateIndexRequest(index).source(schema, XContentType.JSON)
^
|
change this
I am using ES 7.2.1 to store large amount of location based data and querying for near-by locations.
For location coordinates, I am using GeoPoint fields from my java codebase.
ES: 7.2.1
Spring Data Elasticsearch: 4.0.0.DATAES-690-SNAPSHOT
MVN org.elasticsearch: 7.2.1
Template:
curl -X PUT "localhost:9200/_template/store_locator_template?pretty"
{
"order": 1,
"index_patterns": [
"store_locator_*"
],
"settings": {
},
"mappings": {
"properties": {
"esId": {
"type": "keyword"
},
"geoPoint": {
"type": "geo_point"
},
"storeName": {
"type": "keyword"
}
}
}
}
When trying to insert data via bulkIndex(), I am getting this error:
org.springframework.data.elasticsearch.ElasticsearchException:
Bulk indexing has failures. Use ElasticsearchException.getFailedDocuments()
for detailed messages [{QObQeXEBqxAg6uMFyeNZ=ElasticsearchException
[Elasticsearch exception
[type=illegal_argument_exception, reason=
mapper [geoPoint] of different type,
current_type [geo_point], merged_type [ObjectMapper]]]}]
Entity:
#Getter
#Setter
#ToString
#EqualsAndHashCode(of = "esId", callSuper = false)
#NoArgsConstructor
#Document(indexName = "store_locator_index", replicas = 0, createIndex = false)
public class EsEntity {
#Id
#Field(type = FieldType.Text)
private String esId;
#GeoPointField
private GeoPoint geoPoint;
#Field(type = FieldType.Text)
private String storeName;
}
UPDATE:
If I use the below code, it works fine. it puts the mapping as required and spring data es does no complain!
//clazz -> entity class with #Document annotation
boolean indexCreated = false;
if (!elasticsearchOperations.indexExists(clazz)) {
indexCreated = elasticsearchOperations.createIndex(clazz);
}
if (indexCreated) {
elasticsearchOperations.refresh(clazz);
elasticsearchOperations.putMapping(clazz); --> Does the MAGIC
}
... And the mapping generated from the above code is:
{
"esentity":{ ---> Why is this here??
"properties":{
"esId":{
"type":"keyword",
"index":true
},
"geoPoint":{
"type":"geo_point"
}
}
}
}
It is adding a type, by the name of my entity class, to the mapping!
====================
Also.....
Everything seems to be working for:
ES: 6.4.3
Spring Data Elasticsearch: 3.1.X
I am able to (via template) insert document with GeoPoint field.
The index is generated automatically when doc is inserted via code.
The same set of code works fine with no error!!!!
Here's my template:
curl -X PUT "localhost:9200/_template/store_locator_template?pretty"
{
"order": 1,
"index_patterns": [
"store_locator_*"
],
"settings": {
},
"mappings": {
"store_locator_index": {
"properties": {
"esId": {
"type": "keyword"
},
"geoPoint": {
"type": "geo_point"
},
"storeName": {
"type": "keyword"
}
}
}
}
}
Here's the mapping:
{
"mapping": {
"properties": {
"esId": {
"type": "keyword"
},
"geoPoint": {
"type": "geo_point"
}
}
}
}
There are some things that don't match in the code you show:
In the first template you show, you define the storeName to be of type keyword, but on the entity you have it as type text.
A field annotated with #Id is always a type keyword, the #Field annotation defining it as type text is ignored.
I used the following versions: ES 7.3.0 (don't have 7.2.1 on my machine), Spring Data 4.0 current master, client libs set to 7.3.0.
When I don't have the template defined, but create the index with the code you showed:
boolean indexCreated = false;
Class<EsEntity> clazz = EsEntity.class;
if (!elasticsearchOperations.indexExists(clazz)) {
indexCreated = elasticsearchOperations.createIndex(clazz);
}
if (indexCreated) {
elasticsearchOperations.refresh(clazz);
elasticsearchOperations.putMapping(clazz);
}
I get the following index:
{
"store_locator_index": {
"aliases": {},
"mappings": {
"properties": {
"esId": {
"type": "keyword"
},
"geoPoint": {
"type": "geo_point"
},
"storeName": {
"type": "text"
}
}
},
"settings": {
"index": {
"refresh_interval": "1s",
"number_of_shards": "1",
"provided_name": "store_locator_index",
"creation_date": "1587073075464",
"store": {
"type": "fs"
},
"number_of_replicas": "0",
"uuid": "72aZqWDtS7KLDMwdkgVtag",
"version": {
"created": "7030099"
}
}
}
}
}
The mapping looks like it should, there is no type info in the mapping (this was written by the Spring Data Elasticsearch 3.2 version when using ES 6, but i not used anymore)
When I add the template you showed and then do a bulk insert with the following code:
EsEntity es1 = new EsEntity();
es1.setEsId("1");
es1.setGeoPoint(new GeoPoint(12, 34));
es1.setStoreName("s1");
IndexQuery query1 = new IndexQueryBuilder().withId("1").withObject(es1).build();
EsEntity es2 = new EsEntity();
es2.setEsId("2");
es2.setGeoPoint(new GeoPoint(56, 78));
es2.setStoreName("s2");
IndexQuery query2 = new IndexQueryBuilder().withId("2").withObject(es2).build();
elasticsearchOperations.bulkIndex(Arrays.asList(query1, query2), IndexCoordinates.of("store_locator_index"));
then the following index is created (note that store_name is type keywordnow, coming from the template):
{
"store_locator_index": {
"aliases": {},
"mappings": {
"properties": {
"_class": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"esId": {
"type": "keyword"
},
"geoPoint": {
"type": "geo_point"
},
"storeName": {
"type": "keyword"
}
}
},
"settings": {
"index": {
"creation_date": "1587073540386",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "LqzXMC5uRmKmImIzblFBOQ",
"version": {
"created": "7030099"
},
"provided_name": "store_locator_index"
}
}
}
}
and the two documents are inserted as they should:
{
"took": 22,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "store_locator_index",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"_class": "com.sothawo.springdataelastictest.EsEntity",
"esId": "1",
"geoPoint": {
"lat": 12.0,
"lon": 34.0
},
"storeName": "s1"
}
},
{
"_index": "store_locator_index",
"_type": "_doc",
"_id": "2",
"_score": 1.0,
"_source": {
"_class": "com.sothawo.springdataelastictest.EsEntity",
"esId": "2",
"geoPoint": {
"lat": 56.0,
"lon": 78.0
},
"storeName": "s2"
}
}
]
}
}
So I cannot find an error in the code, but you should check the templates and existing indices if there are conflicting entries.
I want my client to be able to request metadata about a #RequestMapping, like the #Size and #Length values of each field. Does Spring have a way of doing this?
Closest thing might be Spring HATEOAS Affordances for HAL forms.
Spring HATEOAS Affordances
Responses produce output which includes all the metadata about the request model along with validation rules and method.
{
"_embedded": {
"employees": [...]
},
"_links": {
"self": {
"href": "http://localhost:8080/employees"
}
},
"_templates": {
"default": {
"title": null,
"method": "post",
"contentType": "",
"properties":[
{
"name": "firstName",
"required": true
},
{
"name": "id",
"required": true
},
{
"name": "lastName",
"required": true
},
{
"name": "role",
"required": true
}
]
}
}
}
I have nested Json file, I would like to read specific objects from Json file using request methods. Mostly to make exacly endpoints I need. I'm trying even to read everything but it doesn't work.
So I need help with this, I mean how to read whole Json file, after that I think I can handle to make endpoints like I need :
/api/category/{categoryName}/books
I arleady tried something like this :
#GetMapping(produces = MediaType.APPLICATION_JSON_VALUE)
public #ResponseBody
Object getBeers() {
ClassPathResource resource = new ClassPathResource("static/books.json");
try {
ObjectMapper mapper = new ObjectMapper();
return mapper.readValue(resource.getInputStream(), Object.class);
} catch (IOException e) {
e.printStackTrace();
}
return null; //also tried with path here
}
I have results like this:
(java.io.FileNotFoundException: class path resource [static/books.json] cannot be opened because it does not exist)
I tried write path in all other ways. Not working.
So I'm guessing I'm not doin' it right
Here is whole json if someone needs it:
https://pastebin.com/yruFS5SM
Below is part of my Json, after that it's repeating with similar objects.
{
"requestedUrl": "https://www.googleapis.com/books/v1/volumes?q=java&maxResults=40",
"items": [
{
"kind": "books#volume",
"id": "7tkN1CYzn2cC",
"etag": "pfjjxSpetIM",
"selfLink": "https://www.googleapis.com/books/v1/volumes/7tkN1CYzn2cC",
"volumeInfo": {
"title": "A Hypervista of the Java Landscape",
"publisher": "InfoStrategist.com",
"industryIdentifiers": [
{
"type": "ISBN_13",
"identifier": "9781592432172"
},
{
"type": "ISBN_10",
"identifier": "1592432174"
}
],
"readingModes": {
"text": true,
"image": true
},
"printType": "BOOK",
"maturityRating": "NOT_MATURE",
"allowAnonLogging": false,
"contentVersion": "1.0.1.0.preview.3",
"imageLinks": {
"smallThumbnail": "http://books.google.com/books/content?id=7tkN1CYzn2cC&printsec=frontcover&img=1&zoom=5&edge=curl&source=gbs_api",
"thumbnail": "http://books.google.com/books/content?id=7tkN1CYzn2cC&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api"
},
"language": "en",
"previewLink": "http://books.google.pl/books?id=7tkN1CYzn2cC&pg=PP1&dq=java&hl=&cd=1&source=gbs_api",
"infoLink": "http://books.google.pl/books?id=7tkN1CYzn2cC&dq=java&hl=&source=gbs_api",
"canonicalVolumeLink": "https://books.google.com/books/about/A_Hypervista_of_the_Java_Landscape.html?hl=&id=7tkN1CYzn2cC"
},
"saleInfo": {
"country": "PL",
"saleability": "NOT_FOR_SALE",
"isEbook": false
},
"accessInfo": {
"country": "PL",
"viewability": "PARTIAL",
"embeddable": true,
"publicDomain": false,
"textToSpeechPermission": "ALLOWED",
"epub": {
"isAvailable": true,
"acsTokenLink": "http://books.google.pl/books/download/A_Hypervista_of_the_Java_Landscape-sample-epub.acsm?id=7tkN1CYzn2cC&format=epub&output=acs4_fulfillment_token&dl_type=sample&source=gbs_api"
},
"pdf": {
"isAvailable": true,
"acsTokenLink": "http://books.google.pl/books/download/A_Hypervista_of_the_Java_Landscape-sample-pdf.acsm?id=7tkN1CYzn2cC&format=pdf&output=acs4_fulfillment_token&dl_type=sample&source=gbs_api"
},
"webReaderLink": "http://play.google.com/books/reader?id=7tkN1CYzn2cC&hl=&printsec=frontcover&source=gbs_api",
"accessViewStatus": "SAMPLE",
"quoteSharingAllowed": false
}
}
{
"icon": "service",
"title": "A",
"type": "service",
"children": [
{
"icon": "sharedlibraries",
"title": "sharedlibraries",
"type": "sharedlibraries",
"children": [
{
"icon": "war",
"title": "abc.war ( ui-shared-lib )",
"path": "abc/common/Services/1.2.0/lib/comp.war",
"reference": >"abc/common/templates/applications/11.1.2.3.jar/config/config.xml",
"type": "sharedlibrary",
"version": "11.1.2.0#11.1.2.0",
"children": [
{
"icon": "jar",
"title": "comp1.jar",
"path": >"abc/common/SharedServices/1.2.0/lib/comp.war/WEB-INF/lib/comp.jar",
"reference": >"abc/common/Services/1.2.0/lib/comp.war/WEB-INF/lib",
"type": "jar",
"thirdpartyjar": "true"
}
]
},
:
:
:
}
I would need to retrieve, attribute "path", of all nodes with name "children", whose "thirdpartyjar" atribute= true. Is this possible using jackson?
Update: I tried following:
public void parse(File file) throws JsonProcessingException,
IOException {
ObjectMapper objectMapper = new ObjectMapper();
//JsonNode rootNode = objectMapper.readTree(file);
Model model = objectMapper.readValue(file, Model.class);
listThirdPartyJars(model);
while (true) {
Model children = model.getChildren();
if (!(children == null)) {
listThirdPartyJars(children);
model = children;
} else {
break;
}
}
}
public void listThirdPartyJars(Model model) {
boolean thirdPartyJars = model.isThirdPartyJar();
if (thirdPartyJars == true)
System.out.println(model.getPath());
}
But, came across following exception:
com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of com.manager.Model out of START_ARRAY token
at [Source: D:\my_json.json; line: 4, column: 22] (through reference chain: com.manager.Model["children"])
Considering the JSON you posted in your question (with a few fixes to make it valid):
{
"icon": "service",
"title": "A",
"type": "service",
"children": [
{
"icon": "sharedlibraries",
"title": "sharedlibraries",
"type": "sharedlibraries",
"children": [
{
"icon": "war",
"title": "abc.war ( ui-shared-lib )",
"path": "abc/common/Services/1.2.0/lib/comp.war",
"reference": "abc/common/templates/applications/11.1.2.3.jar/config/config.xml",
"type": "sharedlibrary",
"version": "11.1.2.0#11.1.2.0",
"children": [
{
"icon": "jar",
"title": "comp1.jar",
"path": "abc/common/SharedServices/1.2.0/lib/comp.war/WEB-INF/lib/comp.jar",
"reference": "abc/common/Services/1.2.0/lib/comp.war/WEB-INF/lib",
"type": "jar",
"thirdpartyjar": "true"
}
]
}
]
}
]
}
Using Jayway JsonPath and a query such as $..children[?(#.thirdpartyjar=='true')].path will do the trick and will give you the following result:
[
"abc/common/SharedServices/1.2.0/lib/comp.war/WEB-INF/lib/comp.jar"
]
You can test it here.
Using Jayway JsonPath
To use Jayway JsonPath, add the following dependency to your project:
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>2.3.0</version>
</dependency>
And then read the JSON document:
String json = "...";
List<String> paths = JsonPath.read(json, "$..children[?(#.thirdpartyjar=='true')].path");
Alternatively to Jayway JsonPath, you can map your JSON to Java classes (or simply use the Jackson Tree Model), parse the JSON using Jackson, iterate recursively over the tree and extract the data you need.
Yes, it's possible. You would need to have a model which references itself as an attribute, somewhat like :
public class Model {
String icon;
String title;
String path;
String reference;
String type;
String version;
boolean thirdPartyJar;
Model children;
...getters, setters and overriden methods
}
And you can map your Json string to this object using ObjectMapper to further access the children with attribute thirdPartyJar as true.