Altering dynamic mapping Elasticsearch 5.3 - java

Many of the string fields in my application need to be mapped dynamically in elasticsearch 5.3. All new fields that end in id or ids should be mapped and indexed automatically by elastic as such:
"_my_propertyId":
{
"type": "keyword"
}
I defined a dynamic template for the index/type like this
"mappings": {
"my_type": {
"dynamic_templates": [
{
"id_as_keywords": {
"match": "*id|*Id|*Ids",
"match_mapping_type": "string",
"mapping": {
"type": "keyword"
}
}
}
]
Yet, elastic still creates the properties like this:
"_someIds": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
I'm not sure what i'm doing wrong or why this is the default mapping for dynamic string fields now. However, I need to be able dynamically map all properties that end in id or ids as keywords, without ignore_above and fully indexed so I can search for them using the searchAPI. Ideas? Why is this the default string mapping now (I understand the introduction of keyword/text, but still)?
Update
Found a good article on these default settings:
Strings

You can use match_pattern parameter to have more control on match parameter. Find the updated dynamic template below:
"dynamic_templates": [
{
"id_as_keywords": {
"match_mapping_type": "string",
"match_pattern": "regex",
"match": ".*(id|Id|Ids)",
"mapping": {
"type": "keyword"
}
}
}
]
You can read more about match_pattern here.

Related

Prevent duplicate class generation (__1) with jsonschema2pojo

I have some json schema that I try to convert to pojo classes using jsonschema2pojo.
Unfortunately, I get some duplicated classes generated with an additional __1 postfix on the classname.
You can test this at https://www.jsonschema2pojo.org/.
Add this example and hit Preview:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"something": {
"type": "object",
"properties": {}
},
"other": {
"type": "object",
"properties": {
"physical": {
"$ref": "#/properties/something"
}
}
}
}
}
I get the classes Something and Something__1. They have the same code (except the class name).
I found other questions, where someone commented that one can change some ObjectRule and RuleFactory, but I don't want to patch the library.
Is there something with my schema or is this a bug?
jung,
The below code might be working for you, I am facing the same issue for a week and walked through the lots of docs and references available.
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"other": {
"type": "object",
"properties": {
"physical": {
"existingJavaType":"com.example.Something" // package name with class name
}
}
},
"something": {
"type": "object"
}
}
}
Refer this

OpenAPI Custom Generator - How do I prevent "AllOf" Class Generation in OpenApi

I'm building a custom generator to generate TypeScript/Angular models for an angular application. For a starting point, I copied the code from https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/TypeScriptAngularClientCodegen.java
I'm trying to figure out how to prevent the generator from generating "AllOf" and from extending models from "AllOf" models.
The generator already generates a model with everything it needs without needing to extend from *AllOf.
I've been able to modify the .java file to prevent it from importing classes that with with "AllOf", but I can't find any documentation or examples to restrict extending from classes that end with "AllOf"
Am I missing something? It seems like there should be a way to tell the generator to just not import or create "AllOf" classes.
// what I get:
import { ValidatePassword } from './validate-password.model';
export interface ChangePassword extends ChangePasswordAllOf, ValidatePassword {
...
}
// what I want
import { ValidatePasswordModel } from './validate-password.model';
export interface ChangePassword extends ValidatePassword {
...
}
Here's my modelGeneric.mustache template:
export interface {{classname}}{{#allOf}}{{#-first}} extends {{/-first}}{{{.}}}{{^-last}}, {{/-last}}{{/allOf}} { {{>modelGenericAdditionalProperties}}
{{#vars}}
{{#description}}
/**
* {{{.}}}
*/
{{/description}}
{{#isReadOnly}}readonly {{/isReadOnly}}{{{name}}}{{^required}}?{{/required}}: {{#isEnum}}{{{datatypeWithEnum}}}{{/isEnum}}{{^isEnum}}{{{dataType}}}{{/isEnum}};
{{/vars}}
}{{>modelGenericEnums}}
Here's the relevant schema sample:
...
"ChangePassword": {
"allOf": [
{
"$ref": "#/components/schemas/ValidatePassword"
},
{
"type": "object",
"additionalProperties": false,
"required": [
"OldPassword"
],
"properties": {
"OldPassword": {
"title": "Current password",
"type": "string",
"minLength": 1
}
}
}
]
},
...
According to this: https://github.com/OpenAPITools/openapi-generator/issues/3100
Could you try:
"ChangePassword": {
"type": "object",
"additionalProperties": false,
"required": [
"OldPassword"
],
"properties": {
"OldPassword": {
"title": "Current password",
"type": "string",
"minLength": 1
}
},
"allOf": [{
"$ref": "#/components/schemas/ValidatePassword"
}]
}

How to pass formData for POST request in swagger.json?

In my play framework application, I have registered APIs in route file as:
POST /api/rmt-create-request controllers.Api.CreateRMTRequestForm
On action of controller, I am using following code to access formData submitted with form submit as :
public Result CreateRMTRequestForm()
{
Map<String, String[]> params = request().body().asMultipartFormData().asFormUrlEncoded();
Its working fine as API when I submit the form with forntend application.
I am trying to create APIs documentation with swagger.ui in which within swagger.json file I have written following JSON data.
"paths": {"/api/rmt-create-request": {
"post": {
"tags": [
"RMT APIs"
],
"description" : "Return newly created request data",
"operationId": "create-new-rmt-request",
"consumes": ["application/x-www-form-urlencoded"],
"parameters": [
{
"name": "rootNodeName",
"in": "formData",
"description": "Root node class name for item",
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/rmt-request-data"
}
}
}
},
"default": {
"$ref": "#/components/responses/default"
}
}
}
},
While inspecting RequestHeader data, its not showing content-Type property with value 'multipart/form-data' as well as formData are not attached, which makes controller to throw null exception.
Can anyone help whats missing in swagger.json file ?
You are mixing OpenAPI 2.0 and 3.0 syntax.
In OpenAPI 3.0, request body (including form data) is defined using the requestBody keyword instead of in: formData parameters.
Also, OAS3 does not use consumes. The media types consumed by the operation are specified inside the requestBody.
"paths": {
"/api/rmt-create-request": {
"post": {
"tags": [
"RMT APIs"
],
"description": "Return newly created request data",
"operationId": "create-new-rmt-request",
"requestBody": {
"content": {
"multipart/form-data": { // or "application/x-www-form-urlencoded" - depending on what you need
"schema": {
"type": "object",
"properties": {
"rootNodeName": {
"type": "string",
"description": "Root node class name for item"
}
}
}
}
}
}
}
}
}
More information: Describing Request Body

How to access Object inside a record while using script with ScriptedMetricAggregationBuilder elasticsearch?

"recordOne": {
"properties": {
"id": {
"type": "integer"
},
"recordtwo": {
"properties": {
"propertyone": {
"type": "integer"
},
"propertytwo":{
"type":"date"
}
}
}
},
"_parent": {
"type": "parentrecord"
}
}
Here i am trying to access recordtwo properties like "doc.recordtwo.propertyone"
i am getting this exception :
'Elasticsearch exception [type=illegal_argument_exception, reason=No
field found for [recordtwo] in mapping with types [recordOne]]'
Though i am able to access id. i.e.
"doc.id"
I tried below and it works.
"doc['recordtwo.propertyone']"

Spring-Data-Elasticsearch settings: Spring can't find config file?

With Spring-Data-Elasticsearch, I am trying to use analyzers and mappings defined in elasticsearch_config.json.
This JSON file is in /src/main/resources folder.
My JAVA model looks like:
#Document(indexName = "test", type="Tweet")
#Setting(settingPath = "/elasticsearch_config.json")
public class Tweet {
#Id
private String idStr;
/** other fields, getters and setters are omitted **/
}
elasticsearch_config.json contains both settings and mappings:
{
"settings": { /* some filters */},
"mappings": { /* some types' mappings*/ }
}
I tried with curl, I get no problem indexing/searching.
My problem:
I want to use #Setting to configure my mappings (not curl), but #Setting annotation doesn't seem to work.
With #Setting, when I use curl to check my mapping, I don't get all the mappings I defined in elasticsearch_config.json :
curl -X GET "localhost:9200/test/_mapping?pretty=true"
My guess is that Spring can't find the elasticsearch_config.json properly.
However I have already checked that myproject/src/main/resources is in my project's build path...
I appreciate any help, thank you!
Notes:
I have found this solution that may work, but:
1. I don't understand where nodeBuilder comes from
2. I may be wrong but, isn't there a solution only using #Setting?
UPDATES: My mappings
I separeted analyzers from mappings. mappings.json looks like this:
{
"mappings": {
"Tweet": {
"_id": {
"path":"idStr",
"properties": {
"idStr": {
"type":"string",
"index":"not_analyzed",
"store":true
}
}
},
"numeric_detection" : true,
"dynamic_templates": [
{
"id_fields": {
"match":"userId*",
"match_mapping_type": "string",
"mapping": {
"type":"string",
"index":"no",
"store":true
}
}
},
{
"name_fields": {
"match":"*Name",
"match_mapping_type": "string",
"mapping": {
"type":"string",
"index":"no",
"store":true
}
}
}
],
"properties": {
"text": {
"type": "string",
"index":"analyzed",
"analyzer":"custom_analyzer",
"store": true
},
"createdAt": {
"type": "date",
"format": "yyyy-MM-dd",
"index":"analyzed",
"store": true
},
"testVersion": {
"type": "integer",
"index":"not_analyzed",
"store":false
}
}
}
}
}
Finally found out why it was not working!!
Like Val said, I decomposed my elasticsearch_config.json file into settings.json and mappings.json.
My project/src/main/ressources architecture:
- mappings
+ mappings.json
- settings
+ settings.json
And
#Document(indexName = "test", type="SentimentTweet")
#Setting(settingPath = "/settings/settings.json")
#Mapping(mappingPath = "/mappings/mappings.json")
However, in mappings.json, I should omit the field mappings and DIRECTLY put the content of the mapping.
INSTEAD OF:
{
"mappings": {
"Tweet": {
/* MAPPINGS CONFIGURATION ARE OMITTED */
}
}
}
Only writes in mappings.json:
{
"Tweet": {
/* MAPPINGS CONFIGURATION ARE OMITTED */
}
}
The same should be done for settings.json
The #Setting annotation should point to a file containing only the settings part. If you also want to specify your custom mapping, you need to use the #Mapping annotation and give it the path to your mapping file. It goes like this:
#Document(indexName = "test", type="Tweet")
#Setting(settingPath = "/settings/settings.json")
#Mapping(mappingPath = "/mappings/mappings.json")
public class Tweet {
#Id
private String idStr;
/** other fields, getters and setters are omitted **/
}
Then you need to store settings.json in myproject/src/main/resources/settings/ and mappings.json in myproject/src/main/resources/mappings/.
That should work.

Categories

Resources