I have the following json data object:
{
"name": "John",
"favorite_number": 5,
"favorite_color" : "green"
}
The JSON schema for this object looks like this:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Person",
"description": "some description",
"type": "object",
"properties": {
"name": {
"description": "name",
"type": "string"
},
"favorite_number": {
"type": "number",
},
"favorite_color": {
"type": "string",
},
},
"required": ["name", "favorite_number","favorite_color"]
}
I'm able to use this JSON schema, to validate whether my data object conforms to it:
public static boolean isJsonValid(String schemaText, String jsonText) throws ProcessingException, IOException
{
final JsonSchema schemaNode = getSchemaNode(schemaText);
final JsonNode jsonNode = getJsonNode(jsonText);
return isJsonValid(schemaNode, jsonNode);
}
In my java application, I'm receiving a corresponding AVRO schema for this object from a REST API call, and that schema looks like this:
{
"namespace": "example.avro",
"type": "record",
"name": "Person",
"fields": [
{"name": "name", "type": "string"},
{"name": "favorite_number", "type": ["int", "null"]},
{"name": "favorite_color", "type": ["string", "null"]}
]
}
Is there a commonly acceptable way of converting such AVRO schema into a JSON schema?
Download: avro-tools-1.7.4.jar (or latest version from repository)
Run: java -jar avro-tools-1.7.4.jar tojson avro-filename.avro>output-filename.json
This will create output-filename.json file with all the data. If output-filename.json already exists it will override it.
Related
I have a streaming app where it listens to some data and then transforming the data by pushing the data into a new topic. I use avro schema for both to read/write my data into topics. The problem is when i consume the data from the final destination by using the command in the below. However, my data is a little complex with some array and json inside of it and i suspect that my avro schemas might not be correct for my purpose. There is no error or anything, I can see all my data on my final topic but the "Pets" field are duplicated for some reason and i can't understand why. In fact, i only add one new field (job_id) to my existing data in the avro schema, i don't make big changes on it when i transform it.
./bin/kafka-console-consumer --topic my_topic \
--bootstrap-server localhost:9092 \
Here's the json data i have
{
"Person":{
"id":"104440",
"Name":"William",
"LastName":"Dorsey",
"archived":false,
"Timezone":"America/Los_Angeles",
"brandCompanyName":"Twitter",
"brandID":"cf545a7b",
"creatorID":"1234",
"currency":"USD",
"dateCreated":"2020-09-07T02:56:22Z",
"dateModified":"2020-09-07T02:57:24Z",
"disabled":false,
"endDate":"2020-11-29T19:51:00-08:00",
"startDate":"2020-08-31T20:55:00-07:00",
"totalBudget":0
},
"Pets":[
{
"Name":"Pawny",
"Id":"4214",
"budget":"0",
"adoptionDate":"2020-09-07T02:56:22Z",
"year":"2",
"type":"Golden",
"gender":"male"
}
],
"CreationTime":"1604036638"
}
my avro schema
{
"name": "MyClass",
"type": "record",
"namespace": "com.acme.avro",
"fields": [
{
"name": "Person",
"type": {
"name": "Person",
"type": "record",
"fields": [
{
"name": "id",
"type": "string"
},
{
"name": "Name",
"type": "string"
},
{
"name": "LastName",
"type": "string"
},
{
"name": "archived",
"type": "boolean"
},
{
"name": "Timezone",
"type": "string"
},
{
"name": "brandCompanyName",
"type": "string"
},
{
"name": "brandID",
"type": "string"
},
{
"name": "creatorID",
"type": "string"
},
{
"name": "currency",
"type": "string"
},
{
"name": "dateCreated",
"type": "int",
"logicalType": "date"
},
{
"name": "dateModified",
"type": "int",
"logicalType": "date"
},
{
"name": "disabled",
"type": "boolean"
},
{
"name": "endDate",
"type": "int",
"logicalType": "date"
},
{
"name": "startDate",
"type": "int",
"logicalType": "date"
},
{
"name": "totalBudget",
"type": "int"
}
]
}
},
{
"name": "Pets",
"type": {
"type": "array",
"items": {
"name": "Pets_record",
"type": "record",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "Id",
"type": "string"
},
{
"name": "budget",
"type": "string"
},
{
"name": "adoptionDate",
"type": "int",
"logicalType": "date"
},
{
"name": "year",
"type": "string"
},
{
"name": "type",
"type": "string"
},
{
"name": "gender",
"type": "string"
}
]
}
}
},
{
"name": "CreationTime",
"type": "string"
},
{
"name":"jobID",
"type":"string"
}
]
}
the output in my topic when i consume the topic - the pets field are duplicated for some reason? I can't figure out why
{
"id":"104440",
"Name":"William",
"LastName:"Dorsey",
"archived":false,
"Timezone":"America/Los_Angeles",
"brandCompanyName":"Twitter",
"brandID":"cf545a7b",
"creatorID":"1234",
"currency":"USD",
"dateCreated":"2020-09-07T02:56:22Z",
"dateModified":"2020-09-07T02:57:24Z",
"disabled":false,
"endDate":"2020-11-29T19:51:00-08:00",
"startDate":"2020-08-31T20:55:00-07:00",
"totalBudget":0,
"Pets":[
{
"Name":"Pawny",
"Id":"4214",
"budget":"0",
"adoptionDate":2020-09-07T02:56:22Z",
"year":"2",
"type":"Golden",
"gender":"male"
}
],
"CreationTime":1604036638,
"jobID":12512,
"pets":[
{
"Name":"Pawny",
"Id":"4214",
"budget":"0",
"adoptionDate":2020-09-07T02:56:22Z",
"year":"2",
"type":"Golden",
"gender":"male"
}
]
}
It's because i was using Uppercase name in my field names... Wandering in endless loops for 24 hours, i was finally able to figure out this if anyone ran into same issue. Please read here and use lowercase name for your fieldnames. When i changed my field name to "pet". The duplicates are gone
Is there any Java package which I can use to convert a JSON string to JSON schema? I have looked up online. I found libraries in Python, Ruby and NodeJS, but not in Java
All the Java libraries generate JSON schema from a POJO
I think that you can try this library on github it does exactly what you want from a JSON, You only need to build it and use json-string-schema-generator
String json = "{\"sectors\": [{\"times\":[{\"intensity\":30," +
"\"start\":{\"hour\":8,\"minute\":30},\"end\":{\"hour\":17,\"minute\":0}}," +
"{\"intensity\":10,\"start\":{\"hour\":17,\"minute\":5},\"end\":{\"hour\":23,\"minute\":55}}]," +
"\"id\":\"dbea21eb-57b5-44c9-a953-f61816fd5876\"}]}";
String result = JsonSchemaGenerator.outputAsString("Schedule", "this is a test", json);
/* sample output
{
"title": "Schedule",
"description": "this is a test",
"type": "object",
"properties": {
"sectors": {
"type": "array",
"items": {
"properties": {
"times": {
"type": "array",
"items": {
"properties": {
"intensity": {
"type": "number"
},
"start": {
"type": "object",
"properties": {
"hour": {
"type": "number"
},
"minute": {
"type": "number"
}
}
},
"end": {
"type": "object",
"properties": {
"hour": {
"type": "number"
},
"minute": {
"type": "number"
}
}
}
}
}
},
"id": {
"type": "string"
}
}
}
}
}
}
*/
// To generate JSON schema into a file
JsonSchemaGenerator.outputAsFile("Schedule", "this is a test", json, "output-schema.json");
// To generate POJO(s)
JsonSchemaGenerator.outputAsPOJO("Schedule", "this is a test", json, "com.example", "generated-sources");
}
I am trying to validate json with json schema, problem is i have created different json schema files for complex object. I need to include in to main schema using ref tag. and trying to validate my json using everit lib. schema got loaded but when i trying to validate my sample json it is not validating inner schema object.
InnerObject.json
{
"$id": "http://example.com/example.json",
"type": "object",
"definitions": {},
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"typeCode": {
"$id": "/properties/typeCode",
"type": "string"
},
"description": {
"$id": "/properties/description",
"type": "string"
},
"expDate": {
"$id": "/properties/expDate",
"type": "string"
},
"issuingCountry": {
"$id": "/properties/issuingCountry",
"type": "string"
}
},
"required": [
"typeCode",
"description",
"expDate",
"issuingCountry"
]
}
OuterObject.JSON
{
"$id": "http://example.com/example.json",
"type": "object",
"definitions": {
},
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"firstName": {
"$id": "/properties/firstName",
"type": "string"
},
"lastName": {
"$id": "/properties/lastName",
"type": "string"
},
"innerObject": {
"$id": "#item",
"type": "object",
"$ref": "file://com/mypack/innerObject.json"
}
},
"required": [
"firstName",
"lastName",
"innerObject"
]
}
Both files are inside my project src/main/resource
I have json validator class which is use to test my schema.
import org.everit.json.schema.Schema;
import org.everit.json.schema.loader.SchemaLoader;
import org.json.JSONObject;
import org.json.JSONTokener;
public class JSONSchemaValidator {
public static void main(String[] args) {
JSONObject jsonSchema = new JSONObject(new JSONTokener(JSONSchemaValidator.class
.getResourceAsStream("/com/schema/outer.schema.json")));
JSONObject outerJson = new JSONObject();
JSONObject innerJson = new JSONObject();
innerJson.put("expDate", "expDate");
innerJson.put("typeCode", "typeCode");
outerJson.put("firstName", "firstName");
outerJson.put("lastName", "lastName");
outerJson.put("innerObject", innerJson);
Schema schema = SchemaLoader.load(jsonSchema);
System.out.println(schema.getDescription());
schema.validate(outerJson);
System.out.println("done");
}
}
Its validating 1st level using schema but not for inner level. can any one suggest what is did wrong, so that its not validating the json.
Sample JSON which i am trying to validate is :
{"firstName":"firstName","lastName":"lastName","innerObject":{"expDate":"expDate","typeCode":"typeCode"}}
It should thrown an error as "typeCode" "description","expDate",issuingCountry" 4 fields are mandetory and in input i am passing only two. so for remaining two it should thrown an error.
Just provide inner json file in outer json with ref tag.
for example :
{
"$id": "http://example.com/example.json",
"type": "object",
"definitions": {
},
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"firstName": {
"$id": "/properties/firstName",
"type": "string"
},
"lastName": {
"$id": "/properties/lastName",
"type": "string"
},
"innerObject": {
"$ref": "innerObject.json"
}
},
"required": [
"firstName",
"lastName",
"innerObject"
]
}
In Java code you need to set resolutionScope, you need to provide path of your inner json.
String path="file:" + Paths.get("").toUri().getPath() + "PATH_OF_YOUR_JSONSchema";
System.out.println("older"+ path);
if (path.contains(" ")) {
path=path.replaceAll(" ", "%20");
}
SchemaLoader sl=SchemaLoader.builder().schemaJson(jsonSchema).resolutionScope(path ).build();
Schema schema=sl.load().build();
schema.validate(yourJsonObject);
I have a json response like this(Response is getting in the com.jayway.restassured.response.Response format).
[{
gameIdGlobal: 1947634,
season: 2017,
views: [{
name: "Recap",
displayOrder: 1,
groups: [{
type: "static",
displayOrder: 1
}
],
localizedName: {
ENG: "Recap",
ESP: "Resumen"
}
}
]
}
]
From this I need to validate the json schema of views object only.No need to validate the entire json . for that
I have created a json schema for views object only schema1.
schema1.json
{
"type": "array",
"items": {
"id": "view.json",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"displayOrder": {
"type": "integer",
"minimum": 1
},
"groups": {
"type": "array"
},
"localizedName": {
"type": "object",
"properties": {
"ENG": {
"type": "string",
"description": "the name of the view in english"
},
"ESP": {
"type": "string",
"description": "the name of the view in spanish"
}
}
}
}
}
}
How can i perform the schema validation of particular json object(views object from josn response)
code
Response response = RestAssured.given().when().get(getURL);
ValidatableResponse valResponse = response.then().contentType(ContentType.JSON);
valResponse.body(schema1.json("schema1.json")).assertThat();
You can specify that Additional Properties are allowed on the object that holds the array as its property. Here is the schema for the entire response json object:
{
"$schema": "http://json-schema.org/draft-06/schema#",
"type": "array",
"items": {
"type": "object",
"required": ["views"],
"additionalProperties": true,
"properties": {
"views": {
"type": "array",
"items": {
"id": "view.json",
...
}
}
}
Hi I'm using the JSON Schema evaluator in version 2.2.6 to validate my server responses. These responses can contain single objects of type A, B or C, but also composite objects, e.g., D containing an array of A objects. To reuse the schema definitions of each object, I started to describe all entities in the same file as described here. Now my problem is, that I have to reference one of those single objects when validating the response.
Here is my (not) SWE.
JSON schema file:
{
"id":"#root",
"properties": {
"objecta": {
"type": "object",
"id":"#objecta",
"properties": {
"attribute1": {"type": "integer"},
"attribute2": {"type": "null"},
},
"required": ["attribute1", "attribute2"]
},
"objectb": {
"type": "object",
"id":"#objectb",
"properties": {
"attribute1": {"type": "integer"},
"attribute2": {
"type": "array",
"items": {
"$ref": "#/objecta"
}
}
}
},
"required": ["attribute1", "attribute2"]
},
}
}
Now I want to validate a server response containing object B. For this, I tried the following:
public class SchemeValidator {
public static void main(String[] args) {
String jsonData = pseudoCodeFileLoad("exampleresponse/objectb.txt");
final File jsonSchemaFile = new File("resources/jsonschemes/completescheme.json");
final URI uri = jsonSchemaFile.toURI();
ProcessingReport report = null;
try {
JsonSchemaFactory factory = JsonSchemaFactory.byDefault();
final JsonSchema schema = factory.getJsonSchema(uri.toString() + "#objectb");
JsonNode data = JsonLoader.fromString(jsonData);
report = schema.validate(data);
} catch (JsonParseException jpex) {
// ... handle parsing errors etc.
}
}
}
The problem is that the scheme is not loaded correctly. I either get no error (even for invalid responses) or I get fatal: illegalJsonRef as the scheme seems to be empty. How can I use the schema of object b in my Java code? Thank you!!
It looks like your $ref is incorrect. It needs to be a relative reference from the base of the JSON Schema file (see here).
So your JSON schema would become:
{
"id":"#root",
"properties": {
"objecta": {
"type": "object",
"id":"#objecta",
"properties": {
"attribute1": {"type": "integer"},
"attribute2": {"type": "null"},
},
"required": ["attribute1", "attribute2"]
},
"objectb": {
"type": "object",
"id":"#objectb",
"properties": {
"attribute1": {"type": "integer"},
"attribute2": {
"type": "array",
"items": {
"$ref": "#/properties/objecta"
}
}
}
},
"required": ["attribute1", "attribute2"]
},
}
}
I've added '/properties' to your $ref. It's operating like XPath to the definition of the object in the schema file.