I define a json shema like this
{
"type": "object",
"properties": {
"firstName": {
"type": "string",
},
"lastName": {
"type": "string",
},
"age": {
"type": "integer"
}
}
}
and here is my json
{
"firstName": "John",
"lastName": "Doe",
"age": 21,
"abcd": "how to get this field",
"efg": "and this field"
}
I want to get the extra field which is not defined in json schema, just like "abcd" and "efg".
Output like this: ["abcd","efg"]
If you want to add a random property (like abcd) instead of generic( firstName, lastname, age), it would be better if you have a map object in your JSON schema, where you can fill in any key, value pairs ( like abcd - "how to get this field").
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"age": {
"type": "integer"
},
"attributes": {"$ref": "#/definitions/StringMap"}
},
"required": [
"firstName",
"lastName",
"age"
],
"definitions": {
"StringMap": {
"type": "object",
"additionalProperties": {"type": "string"}
}
}
}
I think you are asking how the JSON schema is changed to reject the input if extra properties are added by the user. In that case, you can set additionalProperties to false.
{
"type": "object",
"properties": {
"firstName": {
"type": "string",
},
"lastName": {
"type": "string",
},
"age": {
"type": "integer"
}
},
"additionalProperties": false
}
I was confused with your original question in the GitHub.
https://github.com/networknt/json-schema-validator/issues/322
Related
I have a streaming app where it listens to some data and then transforming the data by pushing the data into a new topic. I use avro schema for both to read/write my data into topics. The problem is when i consume the data from the final destination by using the command in the below. However, my data is a little complex with some array and json inside of it and i suspect that my avro schemas might not be correct for my purpose. There is no error or anything, I can see all my data on my final topic but the "Pets" field are duplicated for some reason and i can't understand why. In fact, i only add one new field (job_id) to my existing data in the avro schema, i don't make big changes on it when i transform it.
./bin/kafka-console-consumer --topic my_topic \
--bootstrap-server localhost:9092 \
Here's the json data i have
{
"Person":{
"id":"104440",
"Name":"William",
"LastName":"Dorsey",
"archived":false,
"Timezone":"America/Los_Angeles",
"brandCompanyName":"Twitter",
"brandID":"cf545a7b",
"creatorID":"1234",
"currency":"USD",
"dateCreated":"2020-09-07T02:56:22Z",
"dateModified":"2020-09-07T02:57:24Z",
"disabled":false,
"endDate":"2020-11-29T19:51:00-08:00",
"startDate":"2020-08-31T20:55:00-07:00",
"totalBudget":0
},
"Pets":[
{
"Name":"Pawny",
"Id":"4214",
"budget":"0",
"adoptionDate":"2020-09-07T02:56:22Z",
"year":"2",
"type":"Golden",
"gender":"male"
}
],
"CreationTime":"1604036638"
}
my avro schema
{
"name": "MyClass",
"type": "record",
"namespace": "com.acme.avro",
"fields": [
{
"name": "Person",
"type": {
"name": "Person",
"type": "record",
"fields": [
{
"name": "id",
"type": "string"
},
{
"name": "Name",
"type": "string"
},
{
"name": "LastName",
"type": "string"
},
{
"name": "archived",
"type": "boolean"
},
{
"name": "Timezone",
"type": "string"
},
{
"name": "brandCompanyName",
"type": "string"
},
{
"name": "brandID",
"type": "string"
},
{
"name": "creatorID",
"type": "string"
},
{
"name": "currency",
"type": "string"
},
{
"name": "dateCreated",
"type": "int",
"logicalType": "date"
},
{
"name": "dateModified",
"type": "int",
"logicalType": "date"
},
{
"name": "disabled",
"type": "boolean"
},
{
"name": "endDate",
"type": "int",
"logicalType": "date"
},
{
"name": "startDate",
"type": "int",
"logicalType": "date"
},
{
"name": "totalBudget",
"type": "int"
}
]
}
},
{
"name": "Pets",
"type": {
"type": "array",
"items": {
"name": "Pets_record",
"type": "record",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "Id",
"type": "string"
},
{
"name": "budget",
"type": "string"
},
{
"name": "adoptionDate",
"type": "int",
"logicalType": "date"
},
{
"name": "year",
"type": "string"
},
{
"name": "type",
"type": "string"
},
{
"name": "gender",
"type": "string"
}
]
}
}
},
{
"name": "CreationTime",
"type": "string"
},
{
"name":"jobID",
"type":"string"
}
]
}
the output in my topic when i consume the topic - the pets field are duplicated for some reason? I can't figure out why
{
"id":"104440",
"Name":"William",
"LastName:"Dorsey",
"archived":false,
"Timezone":"America/Los_Angeles",
"brandCompanyName":"Twitter",
"brandID":"cf545a7b",
"creatorID":"1234",
"currency":"USD",
"dateCreated":"2020-09-07T02:56:22Z",
"dateModified":"2020-09-07T02:57:24Z",
"disabled":false,
"endDate":"2020-11-29T19:51:00-08:00",
"startDate":"2020-08-31T20:55:00-07:00",
"totalBudget":0,
"Pets":[
{
"Name":"Pawny",
"Id":"4214",
"budget":"0",
"adoptionDate":2020-09-07T02:56:22Z",
"year":"2",
"type":"Golden",
"gender":"male"
}
],
"CreationTime":1604036638,
"jobID":12512,
"pets":[
{
"Name":"Pawny",
"Id":"4214",
"budget":"0",
"adoptionDate":2020-09-07T02:56:22Z",
"year":"2",
"type":"Golden",
"gender":"male"
}
]
}
It's because i was using Uppercase name in my field names... Wandering in endless loops for 24 hours, i was finally able to figure out this if anyone ran into same issue. Please read here and use lowercase name for your fieldnames. When i changed my field name to "pet". The duplicates are gone
I'm trying to write a JSON schema for my JSON object and I'm not able to follow the error.
I want my JSON object to be stored in the following manner in Java:
public class Category {
private Map<String, List<String>> categoryMapping;
}
Sample JSON:
{
"categoryMapping": {
"categoryA": ["a","b","c"],
"categoryB": ["x","y","z"],
"categoryC": ["x","y","z"]
}
}
However if I write the schema in the following way:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "$id$",
"description": "list_of_values-1",
"required": [
"categoryMapping"
],
"properties": {
"categoryMapping": {
"$id": "#/properties/categoryMapping",
"type": "object",
"title": "The categoryMapping Schema",
"properties": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
}
I get the following error: The property '#/properties/categoryMapping/properties/type' of type String did not match the following type: object in schema http://json-schema.org/draft-04/schema#
But if I specify the types of categories it works:
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "$id$",
"description": "list_of_values-1",
"properties": {
"categoryMapping": {
"$id": "#/properties/categoryMapping",
"type": "object",
"title": "The Categorymapping Schema",
"required": [
"categoryA",
"categoryB",
"categoryC"
],
"properties": {
"categoryA": {
"$id": "#/properties/categoryMapping/properties/categoryA",
"type": "array",
"title": "The Categorya Schema",
"items": {
"$id": "#/properties/categoryMapping/properties/categoryA/items",
"type": "string",
"title": "The Items Schema",
"default": "",
"examples": [
"a",
"b",
"c"
],
"pattern": "^(.*)$"
}
},
"categoryB": {
"$id": "#/properties/categoryMapping/properties/categoryB",
"type": "array",
"title": "The Categoryb Schema",
"items": {
"$id": "#/properties/categoryMapping/properties/categoryB/items",
"type": "string",
"title": "The Items Schema",
"default": "",
"examples": [
"x",
"y",
"z"
],
"pattern": "^(.*)$"
}
},
"categoryC": {
"$id": "#/properties/categoryMapping/properties/categoryC",
"type": "array",
"title": "The Categoryc Schema",
"items": {
"$id": "#/properties/categoryMapping/properties/categoryC/items",
"type": "string",
"title": "The Items Schema",
"default": "",
"examples": [
"x",
"y",
"z"
],
"pattern": "^(.*)$"
}
}
}
}
}
}
Is there a way to write the schema without explicitly specifying a list of all category types?
So your sample JSON is actually a type with three properties, which is why the schema you have generated requires you to define each property explicitly, even though they are effectively of the same type.
If you were willing to modify your sample json a little bit, however:
{
"categoryMapping": [
{
"name": "categoryA",
"map": ["a","b","c"]
},
{
"name": "categoryB",
"map": ["x","y","z"]
},
{
"name": "categoryC",
"map": ["x","y","z"]
}
]
}
Then you could validate it with the following schema:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"categoryMapping": {
"type": "array",
"items": {
"type": "object",
"required": [
"name",
"map"
],
"properties": {
"name": {
"type": "string"
},
"map": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
}
}
}
Because you can specify the minimum and maximum number of items allowed in an array, you could restrict the number of categories to 3 and the number of "maps" to 3 also if you wanted to.
I want convert json document into json schema. I googled it but not got the exact idea according to my requirement.
here is JSON
{
"empId":1001,
"firstName":"jonh",
"lastName":"Springer",
"title": "Engineer",
"address": {
"city": "Mumbai",
"street": "FadkeStreet",
"zipCode":"420125",
"privatePhoneNo":{
"privateMobile": "2564875421",
"privateLandLine":"251201546"
}
},
"salary": 150000,
"department":{
"departmentId": 10521,
"departmentName": "IT",
"companyPhoneNo":{
"cMobile": "8655340546",
"cLandLine": "10251215465"
},
"location":{
"name": "mulund",
"locationId": 14500
}
}
}
I want to generate like this
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Employee",
"properties": {
"empId": {
"type": "integer"
},
"firstName":{
"type":"string"
},
"lastName": {
"type": "string"
},
"title": {
"type": "string"
},
"address": {
"type": "object",
"properties": {
"city": {
"type": "string"
},
"street": {
"type": "string"
},
"zipCode": {
"type": "string"
},
"privatePhoneNo": {
"type": "object",
"properties": {
"privateMobile": {
"type": "string"
},
"privateLandLine": {
"type": "string"
}
}
}
}
},
"salary": {
"type": "number"
},
"department": {
"type": "object",
"properties": {
"departmentId": {
"type": "integer"
},
"departmentName": {
"type": "string"
},
"companyPhoneNo": {
"type": "object",
"properties": {
"cMobile": {
"type": "string"
},
"cLandLine": {
"type": "string"
}
}
},
"location": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"locationId": {
"type": "integer"
}
}
}
}
}
}
}
Is there any library is doing like this or what is another way?
https://github.com/perenecabuto/json_schema_generator
http://jsonschema.net/#/
I'm think this maybe will help
It's been a while since this was asked but I was having the same issue. So far the best solution I have come across is this library:
https://github.com/saasquatch/json-schema-inferrer
I found this from the json-schema doc itself. It has links to implementations for other languages as well:
https://json-schema.org/implementations.html#from-data
I want to validate list of string (ccEmailAddresses) in my pojo. I'm using Jsonschema2pojo to create Java pojo from json.
Json -
{
"$schema": "http://json-schema.org/draft-03/schema",
"title": "Email Recipient",
"description": "Schema for an email recipient document.",
"type": "object",
"additionalProperties": false,
"required": true,
"properties": {
"firstName": {
"type": "string",
"maxLength": 30
},
"lastName": {
"type": "string",
"minLength": 1,
"maxLength": 50
},
"emailAddress": {
"type": "string",
"pattern": "|^[\\w-']+(?:[\\.\\+&=][\\w-']+)*#(?:[\\w-]+\\.)+[a-zA-Z]{2,10}$",
"maxLength": 255,
"required": true
},
"ccEmailAddresses": {
"type": "array",
"items": {
"type" : "string",
"pattern": "|^[\\w-']+(?:[\\.\\+&=][\\w-']+)*#(?:[\\w-]+\\.)+[a-zA-Z]{2,10}$"
}
}
}
}
As given above, pattern does not validate ccEmailAddresses. However it does validate emailAddress. So for single element it just works fine but not for list.
Further I tried putting pattern as -
{
"$schema": "http://json-schema.org/draft-03/schema",
"title": "Email Recipient",
"description": "Schema for an email recipient document.",
"type": "object",
"additionalProperties": false,
"required": true,
"properties": {
"firstName": {
"type": "string",
"maxLength": 30
},
"lastName": {
"type": "string",
"minLength": 1,
"maxLength": 50
},
"emailAddress": {
"type": "string",
"pattern": "|^[\\w-']+(?:[\\.\\+&=][\\w-']+)*#(?:[\\w-]+\\.)+[a-zA-Z]{2,10}$",
"maxLength": 255,
"required": true
},
"ccEmailAddresses": {
"type": "array",
"pattern": "|^[\\w-']+(?:[\\.\\+&=][\\w-']+)*#(?:[\\w-]+\\.)+[a-zA-Z]{2,10}$",
"items": {
"type" : "string"
}
}
}
}
Even this doesn't work. I would not prefer moving out from Jsonschema2pojo because of this.
I couldn't find any documentation on this but this must be very common use case.
Any help would be highly appreciated.
Thanks,
I want convert json document into json schema. I googled it but not got the exact idea according to my requirement.
here is JSON
{
"empId":1001,
"firstName":"jonh",
"lastName":"Springer",
"title": "Engineer",
"address": {
"city": "Mumbai",
"street": "FadkeStreet",
"zipCode":"420125",
"privatePhoneNo":{
"privateMobile": "2564875421",
"privateLandLine":"251201546"
}
},
"salary": 150000,
"department":{
"departmentId": 10521,
"departmentName": "IT",
"companyPhoneNo":{
"cMobile": "8655340546",
"cLandLine": "10251215465"
},
"location":{
"name": "mulund",
"locationId": 14500
}
}
}
I want to generate like this
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "Employee",
"properties": {
"empId": {
"type": "integer"
},
"firstName":{
"type":"string"
},
"lastName": {
"type": "string"
},
"title": {
"type": "string"
},
"address": {
"type": "object",
"properties": {
"city": {
"type": "string"
},
"street": {
"type": "string"
},
"zipCode": {
"type": "string"
},
"privatePhoneNo": {
"type": "object",
"properties": {
"privateMobile": {
"type": "string"
},
"privateLandLine": {
"type": "string"
}
}
}
}
},
"salary": {
"type": "number"
},
"department": {
"type": "object",
"properties": {
"departmentId": {
"type": "integer"
},
"departmentName": {
"type": "string"
},
"companyPhoneNo": {
"type": "object",
"properties": {
"cMobile": {
"type": "string"
},
"cLandLine": {
"type": "string"
}
}
},
"location": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"locationId": {
"type": "integer"
}
}
}
}
}
}
}
Is there any library is doing like this or what is another way?
https://github.com/perenecabuto/json_schema_generator
http://jsonschema.net/#/
I'm think this maybe will help
It's been a while since this was asked but I was having the same issue. So far the best solution I have come across is this library:
https://github.com/saasquatch/json-schema-inferrer
I found this from the json-schema doc itself. It has links to implementations for other languages as well:
https://json-schema.org/implementations.html#from-data