What is the correct way to create avro schema for object with array of strings?
I am trying to create avro schema to object that have array of strings according to official documenation? but I get error.
https://avro.apache.org/docs/1.8.1/spec.html
[ERROR] Failed to execute goal org.apache.avro:avro-maven-plugin:1.8.2:schema (default) on project email: Execution default of goal org.apache.avro:avro-maven-plugin:1.8.2:schema failed: "array" is not a defined name. The type of the "parameters" field must be a defined name or a {"type": ...} expression. -> [Help 1]
Why my schema is inccorect?
[
{
"type": "record",
"namespace": "com.example",
"name": "Topic",
"fields": [
{ "name": "subject", "type": "string" },
{ "name": "parameters", "type": "array", "items": "string" }
]
}
]
Think this should work:
{
"name":"parameters",
"type": {
"type": "array",
"items": "string"
}
}
Related
I'm very new to Using AVRO Schema and I have a use case where I need to validate AVRO schema data with regular expressions. I mean the field value can allow Numbers OR only allow Alphabets OR AlphaNumeric values OR fixed min-max length etc. I tried like below but it's not working. if any idea please assist me for the same .
AVRO schema is :
{
"type": "record",
"name": "Employee",
"namespace": "com.test.avro",
"fields": [
{
"name": "empId",
"type": "string",
"pattern": "[1-9]"
},
{
"name": "empName",
"type": "string",
"pattern": "[a-zA-Z]"
},
{
"name": "createdDate",
"type": "string",
"pattern": "^[1-9]$"
},
{
"name": "mobile",
"type": "string",
"pattern": "^[1-9]*$"
}
]
}
Thanks in Advance.
I am using Avro Schema to validate my schema is:
{
"name": "Example",
"type": "record",
"fields": [
{
"name": "custId",
"type": "string"
},
{
"name": "danger",
"type": {
"type": "enum",
"symbols": [0,1,2,3],
"name": "danger"
}}
]
}
i have a requirement where i want field "danger" to have values 0,1,2,3 . If something else is given schema should not validate it. I know enum type are used for that but it allow only string.
How can i achieve this.
Can I have a complex type in an Avro's map value?
Example:
{
"type": "record",
"default": null,
"name": "Student",
"namespace": "com.schema",
"fields": [
{
"name": "name",
"type": "string"
},
{
"name": "age",
"type": "int"
},
{
"name": "grades",
"default": null,
"type": ["null", {"type": "map", "values": ["null","string", "int"]}]
}
]
}
{"type" : "map", "values" : "Student"}
I tried to insert the record creation directly on the values. It's compile and creates the class, but when I try to send a message, I have an exception: org.apache.avro.UnresolvedUnionException: Not in union
Trying to serialize Java Objects represented as a Map. This requires me to build an Avro Schema first. I am using Avro SchemaBuilder for schema generation. Have already succeeded in schema generation for Map's objects. However can't deal with schema building for overall Map.
Final schema should look like the following (created this one manually):
{
"type": "record",
"name": "MapObject",
"namespace": "test",
"fields": [
{
"name": "CacheMap",
"type": {
"type": "map",
"values": [
{
"type": "record",
"name": "User",
"namespace": "test",
"fields": [
{
"name": "id",
"type": "long"
},
{
"name": "status",
"type": "string"
}
]
},
{
"type": "record",
"name": "UserKey",
"namespace": "test",
"fields": [
{
"name": "key",
"type": "long"
},
{
"name": "keyPrint",
"type": "string"
}
]
}
]
}
}
]
}
How can I pass to different records for key and value of a map in SchemaBuilder?
SchemaBuilder.map().values() does not allow to pass more than one Schema as a parameter for .value().
Apache Avro Unions can be of help here.
SchemaBuilder.builder()
.map()
.values(SchemaBuilder.unionOf().type(firstSchema).and().type(secondSchema).endUnion());
where the firstSchema and the secondSchema are separate schemas already defined for required User and UserKey 'records' (considering the schema from question)
I use com.github.fge.jsonschema.main.JsonSchema to validate json.
This is json schema:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Team data",
"description": "Validation schema",
"type": "object",
"additionalProperties": false,
"required": [
],
"properties": {
"name": {
"type": "string",
"minLength": 2,
"maxLength": 255,
"description": "Name"
}
}
}
And this is json to validate against schema:
{"name" : "name"}
This is valid when I use online validator to check, but in test I get an error:
Caused by: com.github.fge.jsonschema.core.exceptions.InvalidSchemaException: fatal: invalid JSON Schema, cannot continue
Syntax errors:
[ {
"level" : "error",
"message" : "array must have at least one element",
"domain" : "syntax",
"schema" : {
"loadingURI" : "#",
"pointer" : ""
},
"keyword" : "required"
} ]
level: "fatal"
at com.github.fge.jsonschema.processors.validation.InstanceValidator.process(InstanceValidator.java:114) ~[json-schema-validator-2.2.10.jar:?]
at com.github.fge.jsonschema.processors.validation.ValidationProcessor.process(ValidationProcessor.java:56) ~[json-schema-validator-2.2.10.jar:?]
at com.github.fge.jsonschema.processors.validation.ValidationProcessor.process(ValidationProcessor.java:34) ~[json-schema-validator-2.2.10.jar:?]
at com.github.fge.jsonschema.core.processing.ProcessingResult.of(ProcessingResult.java:79) ~[json-schema-core-1.2.10.jar:?]
at com.github.fge.jsonschema.main.JsonSchemaImpl.doValidate(JsonSchemaImpl.java:77) ~[json-schema-validator-2.2.10.jar:?]
at com.github.fge.jsonschema.main.JsonSchemaImpl.validate(JsonSchemaImpl.java:100) ~[json-schema-validator-2.2.10.jar:?]
at com.github.fge.jsonschema.main.JsonSchemaImpl.validate(JsonSchemaImpl.java:110) ~[json-schema-validator-2.2.10.jar:?]...
I can't see where the error is.
Well, the error seems to be in your scheme instead of in the file to validate. The required property is of the array type, but the array has no element.
The validation succeeds when you either remove the property required, or provide at least one string element indicating which properties should be required:
"required": [
"name"
]
Since Draft 4 of the JSON Schema documentation, the array must have at least one element.