Type mismatch in key from map - java

I'm getting this error:
java.lang.Exception: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, received org.apache.hadoop.io.LongWritable
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, received org.apache.hadoop.io.LongWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1074)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:715)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:125)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
from this code:
public class MyAvroApplication {
private static class MyReducer extends Reducer<Email, Text, Text, Text> {
private Text from = new Text();
private Text subject = new Text();
public void reduce(Email key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
from.set(key.getFrom().toString());
subject.set(key.getSubject().toString());
context.write(from, subject);
}
}
private static class MyAvroMapper extends Mapper<Text, Email, Email, Text> {
protected void map(Email email, NullWritable value, Context context) throws IOException, InterruptedException {
context.write(email,
new Text(email.getSubject().toString()));
}
}
public static void main(String[] args) throws Exception {
UserGroupInformation ugi = UserGroupInformation.createRemoteUser("hdfs");
ugi.doAs(new PrivilegedExceptionAction<Void>() {
public Void run() throws Exception {
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://127.0.0.1:8020/user/rich");
conf.set("hadoop.job.ugi", "hdfs");
Job job = Job.getInstance(conf, "mytest");
AvroJob.setInputKeySchema(job, Email.getClassSchema());
AvroJob.setOutputKeySchema(job, Email.getClassSchema());
job.setJarByClass(MyApplication.class);
job.setMapperClass(MyAvroMapper.class);
job.setCombinerClass(MyReducer.class);
job.setReducerClass(MyReducer.class);
// job.setInputFormatClass(AvroKeyInputFormat.class);
// job.setOutputFormatClass(AvroKeyOutputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
FileInputFormat.addInputPath(job, new Path("/user/rich/in/"));
FileOutputFormat.setOutputPath(job, new Path("/user/rich/out/"));
boolean result = job.waitForCompletion(true);
System.out.println(result);
return null;
}
});
}
}
I'm really struggling to understand how to match up the various key-value parameters on the way in and out everywhere. My intention is to read some Avro format files in as an Email and then run some simple calculations, just counting the number of emails would be a good start.
Edit:
A similar error occurs with this Mapper:
private static class MyAvroMapper extends Mapper<Text, AvroKey<Email>, AvroKey<Email>, Text> {
protected void map(Text key, AvroKey<Email> email, Context context) throws IOException, InterruptedException {
context.write(email,
new Text(email.datum().getSubject().toString()));
}
}
And the error:
java.lang.Exception: java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text
at com.test.myapp.MyAvroApplication$MyAvroMapper.map(MyAvroApplication.java:1)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
The Email class generated from the schema:
#SuppressWarnings("all")
#org.apache.avro.specific.AvroGenerated
public class Email extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"Email\",\"namespace\":\"com.test.myapp.avro\",\"fields\":[{\"name\":\"message_id\",\"type\":[\"null\",\"string\"],\"doc\":\"\"},{\"name\":\"date\",\"type\":[\"long\",\"null\"]},{\"name\":\"from\",\"type\":{\"type\":\"record\",\"name\":\"EmailAddress\",\"fields\":[{\"name\":\"name\",\"type\":[\"null\",\"string\"],\"doc\":\"\"},{\"name\":\"address\",\"type\":[\"null\",\"string\"],\"doc\":\"\"}]}},{\"name\":\"subject\",\"type\":[\"string\",\"null\"]},{\"name\":\"body\",\"type\":[\"string\",\"null\"]},{\"name\":\"tos\",\"type\":[\"null\",{\"type\":\"array\",\"items\":[\"null\",\"EmailAddress\"]}],\"doc\":\"\"},{\"name\":\"ccs\",\"type\":[\"null\",{\"type\":\"array\",\"items\":[\"null\",\"EmailAddress\"]}],\"doc\":\"\"},{\"name\":\"bccs\",\"type\":[\"null\",{\"type\":\"array\",\"items\":[\"null\",\"EmailAddress\"]}],\"doc\":\"\"}]}");
public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }
/** */
#Deprecated public java.lang.CharSequence message_id;
#Deprecated public java.lang.Long date;
#Deprecated public com.test.myapp.avro.EmailAddress from;
#Deprecated public java.lang.CharSequence subject;
#Deprecated public java.lang.CharSequence body;
/** */
#Deprecated public java.util.List<com.test.myapp.avro.EmailAddress> tos;
/** */
#Deprecated public java.util.List<com.test.myapp.avro.EmailAddress> ccs;
/** */
#Deprecated public java.util.List<com.test.myapp.avro.EmailAddress> bccs;
/**
* Default constructor. Note that this does not initialize fields
* to their default values from the schema. If that is desired then
* one should use <code>newBuilder()</code>.
*/
public Email() {}
/**
* All-args constructor.
*/
public Email(java.lang.CharSequence message_id, java.lang.Long date, com.test.myapp.avro.EmailAddress from, java.lang.CharSequence subject, java.lang.CharSequence body, java.util.List<com.test.myapp.avro.EmailAddress> tos, java.util.List<com.test.myapp.avro.EmailAddress> ccs, java.util.List<com.test.myapp.avro.EmailAddress> bccs) {
this.message_id = message_id;
this.date = date;
this.from = from;
this.subject = subject;
this.body = body;
this.tos = tos;
this.ccs = ccs;
this.bccs = bccs;
}
public org.apache.avro.Schema getSchema() { return SCHEMA$; }
// Used by DatumWriter. Applications should not call.
public java.lang.Object get(int field$) {
switch (field$) {
case 0: return message_id;
case 1: return date;
case 2: return from;
case 3: return subject;
case 4: return body;
case 5: return tos;
case 6: return ccs;
case 7: return bccs;
default: throw new org.apache.avro.AvroRuntimeException("Bad index");
}
}
// Used by DatumReader. Applications should not call.
#SuppressWarnings(value="unchecked")
public void put(int field$, java.lang.Object value$) {
switch (field$) {
case 0: message_id = (java.lang.CharSequence)value$; break;
case 1: date = (java.lang.Long)value$; break;
case 2: from = (com.test.myapp.avro.EmailAddress)value$; break;
case 3: subject = (java.lang.CharSequence)value$; break;
case 4: body = (java.lang.CharSequence)value$; break;
case 5: tos = (java.util.List<com.test.myapp.avro.EmailAddress>)value$; break;
case 6: ccs = (java.util.List<com.test.myapp.avro.EmailAddress>)value$; break;
case 7: bccs = (java.util.List<com.test.myapp.avro.EmailAddress>)value$; break;
default: throw new org.apache.avro.AvroRuntimeException("Bad index");
}
}
/**
* Gets the value of the 'message_id' field.
* */
public java.lang.CharSequence getMessageId() {
return message_id;
}
/**
* Sets the value of the 'message_id' field.
* * #param value the value to set.
*/
public void setMessageId(java.lang.CharSequence value) {
this.message_id = value;
}
/**
* Gets the value of the 'date' field.
*/
public java.lang.Long getDate() {
return date;
}
/**
* Sets the value of the 'date' field.
* #param value the value to set.
*/
public void setDate(java.lang.Long value) {
this.date = value;
}
/**
* Gets the value of the 'from' field.
*/
public com.test.myapp.avro.EmailAddress getFrom() {
return from;
}
/**
* Sets the value of the 'from' field.
* #param value the value to set.
*/
public void setFrom(com.test.myapp.avro.EmailAddress value) {
this.from = value;
}
/**
* Gets the value of the 'subject' field.
*/
public java.lang.CharSequence getSubject() {
return subject;
}
/**
* Sets the value of the 'subject' field.
* #param value the value to set.
*/
public void setSubject(java.lang.CharSequence value) {
this.subject = value;
}
/**
* Gets the value of the 'body' field.
*/
public java.lang.CharSequence getBody() {
return body;
}
/**
* Sets the value of the 'body' field.
* #param value the value to set.
*/
public void setBody(java.lang.CharSequence value) {
this.body = value;
}
/**
* Gets the value of the 'tos' field.
* */
public java.util.List<com.test.myapp.avro.EmailAddress> getTos() {
return tos;
}
/**
* Sets the value of the 'tos' field.
* * #param value the value to set.
*/
public void setTos(java.util.List<com.test.myapp.avro.EmailAddress> value) {
this.tos = value;
}
/**
* Gets the value of the 'ccs' field.
* */
public java.util.List<com.test.myapp.avro.EmailAddress> getCcs() {
return ccs;
}
/**
* Sets the value of the 'ccs' field.
* * #param value the value to set.
*/
public void setCcs(java.util.List<com.test.myapp.avro.EmailAddress> value) {
this.ccs = value;
}
/**
* Gets the value of the 'bccs' field.
* */
public java.util.List<com.test.myapp.avro.EmailAddress> getBccs() {
return bccs;
}
/**
* Sets the value of the 'bccs' field.
* * #param value the value to set.
*/
public void setBccs(java.util.List<com.test.myapp.avro.EmailAddress> value) {
this.bccs = value;
}
/** Creates a new Email RecordBuilder */
public static com.test.myapp.avro.Email.Builder newBuilder() {
return new com.test.myapp.avro.Email.Builder();
}
/** Creates a new Email RecordBuilder by copying an existing Builder */
public static com.test.myapp.avro.Email.Builder newBuilder(com.test.myapp.avro.Email.Builder other) {
return new com.test.myapp.avro.Email.Builder(other);
}
/** Creates a new Email RecordBuilder by copying an existing Email instance */
public static com.test.myapp.avro.Email.Builder newBuilder(com.test.myapp.avro.Email other) {
return new com.test.myapp.avro.Email.Builder(other);
}
/**
* RecordBuilder for Email instances.
*/
public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase<Email>
implements org.apache.avro.data.RecordBuilder<Email> {
private java.lang.CharSequence message_id;
private java.lang.Long date;
private com.test.myapp.avro.EmailAddress from;
private java.lang.CharSequence subject;
private java.lang.CharSequence body;
private java.util.List<com.test.myapp.avro.EmailAddress> tos;
private java.util.List<com.test.myapp.avro.EmailAddress> ccs;
private java.util.List<com.test.myapp.avro.EmailAddress> bccs;
/** Creates a new Builder */
private Builder() {
super(com.test.myapp.avro.Email.SCHEMA$);
}
/** Creates a Builder by copying an existing Builder */
private Builder(com.test.myapp.avro.Email.Builder other) {
super(other);
if (isValidValue(fields()[0], other.message_id)) {
this.message_id = data().deepCopy(fields()[0].schema(), other.message_id);
fieldSetFlags()[0] = true;
}
if (isValidValue(fields()[1], other.date)) {
this.date = data().deepCopy(fields()[1].schema(), other.date);
fieldSetFlags()[1] = true;
}
if (isValidValue(fields()[2], other.from)) {
this.from = data().deepCopy(fields()[2].schema(), other.from);
fieldSetFlags()[2] = true;
}
if (isValidValue(fields()[3], other.subject)) {
this.subject = data().deepCopy(fields()[3].schema(), other.subject);
fieldSetFlags()[3] = true;
}
if (isValidValue(fields()[4], other.body)) {
this.body = data().deepCopy(fields()[4].schema(), other.body);
fieldSetFlags()[4] = true;
}
if (isValidValue(fields()[5], other.tos)) {
this.tos = data().deepCopy(fields()[5].schema(), other.tos);
fieldSetFlags()[5] = true;
}
if (isValidValue(fields()[6], other.ccs)) {
this.ccs = data().deepCopy(fields()[6].schema(), other.ccs);
fieldSetFlags()[6] = true;
}
if (isValidValue(fields()[7], other.bccs)) {
this.bccs = data().deepCopy(fields()[7].schema(), other.bccs);
fieldSetFlags()[7] = true;
}
}
/** Creates a Builder by copying an existing Email instance */
private Builder(com.test.myapp.avro.Email other) {
super(com.test.myapp.avro.Email.SCHEMA$);
if (isValidValue(fields()[0], other.message_id)) {
this.message_id = data().deepCopy(fields()[0].schema(), other.message_id);
fieldSetFlags()[0] = true;
}
if (isValidValue(fields()[1], other.date)) {
this.date = data().deepCopy(fields()[1].schema(), other.date);
fieldSetFlags()[1] = true;
}
if (isValidValue(fields()[2], other.from)) {
this.from = data().deepCopy(fields()[2].schema(), other.from);
fieldSetFlags()[2] = true;
}
if (isValidValue(fields()[3], other.subject)) {
this.subject = data().deepCopy(fields()[3].schema(), other.subject);
fieldSetFlags()[3] = true;
}
if (isValidValue(fields()[4], other.body)) {
this.body = data().deepCopy(fields()[4].schema(), other.body);
fieldSetFlags()[4] = true;
}
if (isValidValue(fields()[5], other.tos)) {
this.tos = data().deepCopy(fields()[5].schema(), other.tos);
fieldSetFlags()[5] = true;
}
if (isValidValue(fields()[6], other.ccs)) {
this.ccs = data().deepCopy(fields()[6].schema(), other.ccs);
fieldSetFlags()[6] = true;
}
if (isValidValue(fields()[7], other.bccs)) {
this.bccs = data().deepCopy(fields()[7].schema(), other.bccs);
fieldSetFlags()[7] = true;
}
}
/** Gets the value of the 'message_id' field */
public java.lang.CharSequence getMessageId() {
return message_id;
}
/** Sets the value of the 'message_id' field */
public com.test.myapp.avro.Email.Builder setMessageId(java.lang.CharSequence value) {
validate(fields()[0], value);
this.message_id = value;
fieldSetFlags()[0] = true;
return this;
}
/** Checks whether the 'message_id' field has been set */
public boolean hasMessageId() {
return fieldSetFlags()[0];
}
/** Clears the value of the 'message_id' field */
public com.test.myapp.avro.Email.Builder clearMessageId() {
message_id = null;
fieldSetFlags()[0] = false;
return this;
}
/** Gets the value of the 'date' field */
public java.lang.Long getDate() {
return date;
}
/** Sets the value of the 'date' field */
public com.test.myapp.avro.Email.Builder setDate(java.lang.Long value) {
validate(fields()[1], value);
this.date = value;
fieldSetFlags()[1] = true;
return this;
}
/** Checks whether the 'date' field has been set */
public boolean hasDate() {
return fieldSetFlags()[1];
}
/** Clears the value of the 'date' field */
public com.test.myapp.avro.Email.Builder clearDate() {
date = null;
fieldSetFlags()[1] = false;
return this;
}
/** Gets the value of the 'from' field */
public com.test.myapp.avro.EmailAddress getFrom() {
return from;
}
/** Sets the value of the 'from' field */
public com.test.myapp.avro.Email.Builder setFrom(com.test.myapp.avro.EmailAddress value) {
validate(fields()[2], value);
this.from = value;
fieldSetFlags()[2] = true;
return this;
}
/** Checks whether the 'from' field has been set */
public boolean hasFrom() {
return fieldSetFlags()[2];
}
/** Clears the value of the 'from' field */
public com.test.myapp.avro.Email.Builder clearFrom() {
from = null;
fieldSetFlags()[2] = false;
return this;
}
/** Gets the value of the 'subject' field */
public java.lang.CharSequence getSubject() {
return subject;
}
/** Sets the value of the 'subject' field */
public com.test.myapp.avro.Email.Builder setSubject(java.lang.CharSequence value) {
validate(fields()[3], value);
this.subject = value;
fieldSetFlags()[3] = true;
return this;
}
/** Checks whether the 'subject' field has been set */
public boolean hasSubject() {
return fieldSetFlags()[3];
}
/** Clears the value of the 'subject' field */
public com.test.myapp.avro.Email.Builder clearSubject() {
subject = null;
fieldSetFlags()[3] = false;
return this;
}
/** Gets the value of the 'body' field */
public java.lang.CharSequence getBody() {
return body;
}
/** Sets the value of the 'body' field */
public com.test.myapp.avro.Email.Builder setBody(java.lang.CharSequence value) {
validate(fields()[4], value);
this.body = value;
fieldSetFlags()[4] = true;
return this;
}
/** Checks whether the 'body' field has been set */
public boolean hasBody() {
return fieldSetFlags()[4];
}
/** Clears the value of the 'body' field */
public com.test.myapp.avro.Email.Builder clearBody() {
body = null;
fieldSetFlags()[4] = false;
return this;
}
/** Gets the value of the 'tos' field */
public java.util.List<com.test.myapp.avro.EmailAddress> getTos() {
return tos;
}
/** Sets the value of the 'tos' field */
public com.test.myapp.avro.Email.Builder setTos(java.util.List<com.test.myapp.avro.EmailAddress> value) {
validate(fields()[5], value);
this.tos = value;
fieldSetFlags()[5] = true;
return this;
}
/** Checks whether the 'tos' field has been set */
public boolean hasTos() {
return fieldSetFlags()[5];
}
/** Clears the value of the 'tos' field */
public com.test.myapp.avro.Email.Builder clearTos() {
tos = null;
fieldSetFlags()[5] = false;
return this;
}
/** Gets the value of the 'ccs' field */
public java.util.List<com.test.myapp.avro.EmailAddress> getCcs() {
return ccs;
}
/** Sets the value of the 'ccs' field */
public com.test.myapp.avro.Email.Builder setCcs(java.util.List<com.test.myapp.avro.EmailAddress> value) {
validate(fields()[6], value);
this.ccs = value;
fieldSetFlags()[6] = true;
return this;
}
/** Checks whether the 'ccs' field has been set */
public boolean hasCcs() {
return fieldSetFlags()[6];
}
/** Clears the value of the 'ccs' field */
public com.test.myapp.avro.Email.Builder clearCcs() {
ccs = null;
fieldSetFlags()[6] = false;
return this;
}
/** Gets the value of the 'bccs' field */
public java.util.List<com.test.myapp.avro.EmailAddress> getBccs() {
return bccs;
}
/** Sets the value of the 'bccs' field */
public com.test.myapp.avro.Email.Builder setBccs(java.util.List<com.test.myapp.avro.EmailAddress> value) {
validate(fields()[7], value);
this.bccs = value;
fieldSetFlags()[7] = true;
return this;
}
/** Checks whether the 'bccs' field has been set */
public boolean hasBccs() {
return fieldSetFlags()[7];
}
/** Clears the value of the 'bccs' field */
public com.test.myapp.avro.Email.Builder clearBccs() {
bccs = null;
fieldSetFlags()[7] = false;
return this;
}
#Override
public Email build() {
try {
Email record = new Email();
record.message_id = fieldSetFlags()[0] ? this.message_id : (java.lang.CharSequence) defaultValue(fields()[0]);
record.date = fieldSetFlags()[1] ? this.date : (java.lang.Long) defaultValue(fields()[1]);
record.from = fieldSetFlags()[2] ? this.from : (com.test.myapp.avro.EmailAddress) defaultValue(fields()[2]);
record.subject = fieldSetFlags()[3] ? this.subject : (java.lang.CharSequence) defaultValue(fields()[3]);
record.body = fieldSetFlags()[4] ? this.body : (java.lang.CharSequence) defaultValue(fields()[4]);
record.tos = fieldSetFlags()[5] ? this.tos : (java.util.List<com.test.myapp.avro.EmailAddress>) defaultValue(fields()[5]);
record.ccs = fieldSetFlags()[6] ? this.ccs : (java.util.List<com.test.myapp.avro.EmailAddress>) defaultValue(fields()[6]);
record.bccs = fieldSetFlags()[7] ? this.bccs : (java.util.List<com.test.myapp.avro.EmailAddress>) defaultValue(fields()[7]);
return record;
} catch (Exception e) {
throw new org.apache.avro.AvroRuntimeException(e);
}
}
}
}

I think there's a problem in the mapper. When you declare:
private static class MyAvroMapper extends Mapper<Text, Email, Email, Text> {
you're telling hadoop that you're going to expect the couple (Text, Email) as the types of the key and the value of the input and the couple (Email, Text) as the type for key and values of what the mapper will emit (note that are the four types declared as generics for the class.
But when you write a signature like this:
protected void map(Email email, NullWritable value, Context context) throws IOException, InterruptedException {
you're telling hadoop that the key and the value type for the input are Email and NullWritable, hence the exception.

Related

Avro Schema alias usage via Java file

I tried below avro schema :
{
"type": "record",
"name": "Tests",
"fields": [
{
"name": "name",
"alias": "first_name",
"type": "string",
"default": ""
}
]
}
when I "compile schema" this using avro tools I get the corresponding java file.
/**
* Autogenerated by Avro
*
* DO NOT EDIT DIRECTLY
*/
import org.apache.avro.generic.GenericArray;
import org.apache.avro.specific.SpecificData;
import org.apache.avro.util.Utf8;
import org.apache.avro.message.BinaryMessageEncoder;
import org.apache.avro.message.BinaryMessageDecoder;
import org.apache.avro.message.SchemaStore;
#org.apache.avro.specific.AvroGenerated
public class Tests extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
private static final long serialVersionUID = 8259235364501890516L;
public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"Tests\",\"fields\":[{\"name\":\"name\",\"type\":\"string\",\"default\":\"\",\"alias\":\"first_name\"}]}");
public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }
private static SpecificData MODEL$ = new SpecificData();
private static final BinaryMessageEncoder<Tests> ENCODER =
new BinaryMessageEncoder<Tests>(MODEL$, SCHEMA$);
private static final BinaryMessageDecoder<Tests> DECODER =
new BinaryMessageDecoder<Tests>(MODEL$, SCHEMA$);
/**
* Return the BinaryMessageEncoder instance used by this class.
* #return the message encoder used by this class
*/
public static BinaryMessageEncoder<Tests> getEncoder() {
return ENCODER;
}
/**
* Return the BinaryMessageDecoder instance used by this class.
* #return the message decoder used by this class
*/
public static BinaryMessageDecoder<Tests> getDecoder() {
return DECODER;
}
/**
* Create a new BinaryMessageDecoder instance for this class that uses the specified {#link SchemaStore}.
* #param resolver a {#link SchemaStore} used to find schemas by fingerprint
* #return a BinaryMessageDecoder instance for this class backed by the given SchemaStore
*/
public static BinaryMessageDecoder<Tests> createDecoder(SchemaStore resolver) {
return new BinaryMessageDecoder<Tests>(MODEL$, SCHEMA$, resolver);
}
/**
* Serializes this Tests to a ByteBuffer.
* #return a buffer holding the serialized data for this instance
* #throws java.io.IOException if this instance could not be serialized
*/
public java.nio.ByteBuffer toByteBuffer() throws java.io.IOException {
return ENCODER.encode(this);
}
/**
* Deserializes a Tests from a ByteBuffer.
* #param b a byte buffer holding serialized data for an instance of this class
* #return a Tests instance decoded from the given buffer
* #throws java.io.IOException if the given bytes could not be deserialized into an instance of this class
*/
public static Tests fromByteBuffer(
java.nio.ByteBuffer b) throws java.io.IOException {
return DECODER.decode(b);
}
private java.lang.CharSequence name;
/**
* Default constructor. Note that this does not initialize fields
* to their default values from the schema. If that is desired then
* one should use <code>newBuilder()</code>.
*/
public Tests() {}
/**
* All-args constructor.
* #param name The new value for name
*/
public Tests(java.lang.CharSequence name) {
this.name = name;
}
public org.apache.avro.specific.SpecificData getSpecificData() { return MODEL$; }
public org.apache.avro.Schema getSchema() { return SCHEMA$; }
// Used by DatumWriter. Applications should not call.
public java.lang.Object get(int field$) {
switch (field$) {
case 0: return name;
default: throw new IndexOutOfBoundsException("Invalid index: " + field$);
}
}
// Used by DatumReader. Applications should not call.
#SuppressWarnings(value="unchecked")
public void put(int field$, java.lang.Object value$) {
switch (field$) {
case 0: name = (java.lang.CharSequence)value$; break;
default: throw new IndexOutOfBoundsException("Invalid index: " + field$);
}
}
/**
* Gets the value of the 'name' field.
* #return The value of the 'name' field.
*/
public java.lang.CharSequence getName() {
return name;
}
/**
* Sets the value of the 'name' field.
* #param value the value to set.
*/
public void setName(java.lang.CharSequence value) {
this.name = value;
}
/**
* Creates a new Tests RecordBuilder.
* #return A new Tests RecordBuilder
*/
public static Tests.Builder newBuilder() {
return new Tests.Builder();
}
/**
* Creates a new Tests RecordBuilder by copying an existing Builder.
* #param other The existing builder to copy.
* #return A new Tests RecordBuilder
*/
public static Tests.Builder newBuilder(Tests.Builder other) {
if (other == null) {
return new Tests.Builder();
} else {
return new Tests.Builder(other);
}
}
/**
* Creates a new Tests RecordBuilder by copying an existing Tests instance.
* #param other The existing instance to copy.
* #return A new Tests RecordBuilder
*/
public static Tests.Builder newBuilder(Tests other) {
if (other == null) {
return new Tests.Builder();
} else {
return new Tests.Builder(other);
}
}
/**
* RecordBuilder for Tests instances.
*/
#org.apache.avro.specific.AvroGenerated
public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase<Tests>
implements org.apache.avro.data.RecordBuilder<Tests> {
private java.lang.CharSequence name;
/** Creates a new Builder */
private Builder() {
super(SCHEMA$);
}
/**
* Creates a Builder by copying an existing Builder.
* #param other The existing Builder to copy.
*/
private Builder(Tests.Builder other) {
super(other);
if (isValidValue(fields()[0], other.name)) {
this.name = data().deepCopy(fields()[0].schema(), other.name);
fieldSetFlags()[0] = other.fieldSetFlags()[0];
}
}
/**
* Creates a Builder by copying an existing Tests instance
* #param other The existing instance to copy.
*/
private Builder(Tests other) {
super(SCHEMA$);
if (isValidValue(fields()[0], other.name)) {
this.name = data().deepCopy(fields()[0].schema(), other.name);
fieldSetFlags()[0] = true;
}
}
/**
* Gets the value of the 'name' field.
* #return The value.
*/
public java.lang.CharSequence getName() {
return name;
}
/**
* Sets the value of the 'name' field.
* #param value The value of 'name'.
* #return This builder.
*/
public Tests.Builder setName(java.lang.CharSequence value) {
validate(fields()[0], value);
this.name = value;
fieldSetFlags()[0] = true;
return this;
}
/**
* Checks whether the 'name' field has been set.
* #return True if the 'name' field has been set, false otherwise.
*/
public boolean hasName() {
return fieldSetFlags()[0];
}
/**
* Clears the value of the 'name' field.
* #return This builder.
*/
public Tests.Builder clearName() {
name = null;
fieldSetFlags()[0] = false;
return this;
}
#Override
#SuppressWarnings("unchecked")
public Tests build() {
try {
Tests record = new Tests();
record.name = fieldSetFlags()[0] ? this.name : (java.lang.CharSequence) defaultValue(fields()[0]);
return record;
} catch (org.apache.avro.AvroMissingFieldException e) {
throw e;
} catch (java.lang.Exception e) {
throw new org.apache.avro.AvroRuntimeException(e);
}
}
}
#SuppressWarnings("unchecked")
private static final org.apache.avro.io.DatumWriter<Tests>
WRITER$ = (org.apache.avro.io.DatumWriter<Tests>)MODEL$.createDatumWriter(SCHEMA$);
#Override public void writeExternal(java.io.ObjectOutput out)
throws java.io.IOException {
WRITER$.write(this, SpecificData.getEncoder(out));
}
#SuppressWarnings("unchecked")
private static final org.apache.avro.io.DatumReader<Tests>
READER$ = (org.apache.avro.io.DatumReader<Tests>)MODEL$.createDatumReader(SCHEMA$);
#Override public void readExternal(java.io.ObjectInput in)
throws java.io.IOException {
READER$.read(this, SpecificData.getDecoder(in));
}
#Override protected boolean hasCustomCoders() { return true; }
#Override public void customEncode(org.apache.avro.io.Encoder out)
throws java.io.IOException
{
out.writeString(this.name);
}
#Override public void customDecode(org.apache.avro.io.ResolvingDecoder in)
throws java.io.IOException
{
org.apache.avro.Schema.Field[] fieldOrder = in.readFieldOrderIfDiff();
if (fieldOrder == null) {
this.name = in.readString(this.name instanceof Utf8 ? (Utf8)this.name : null);
} else {
for (int i = 0; i < 1; i++) {
switch (fieldOrder[i].pos()) {
case 0:
this.name = in.readString(this.name instanceof Utf8 ? (Utf8)this.name : null);
break;
default:
throw new java.io.IOException("Corrupt ResolvingDecoder.");
}
}
}
}
}
I am wondering where is the getter/setter for alias? How can i use it? I understand if I have avro record then I can use the alias but how to I generate Avro record using Java file using that alias?
aliases is responsible for providing an alternate name to a field. Documentation from Apache Avro.
If you update alias to aliases, this will work:
{
"type": "record",
"name": "Tests",
"fields": [
{
"name": "first_name",
"aliases": ["former_name_of_field"],
"type": "string",
"default": ""
}
]
}
This is an optional field to support schema evolution.
/**
* Autogenerated by Avro
*
* DO NOT EDIT DIRECTLY
*/
import org.apache.avro.generic.GenericArray;
import org.apache.avro.specific.SpecificData;
import org.apache.avro.util.Utf8;
import org.apache.avro.message.BinaryMessageEncoder;
import org.apache.avro.message.BinaryMessageDecoder;
import org.apache.avro.message.SchemaStore;
#org.apache.avro.specific.AvroGenerated
public class Tests extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
private static final long serialVersionUID = -2440753220882419647L;
public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"Tests\",\"fields\":[{\"name\":\"first_name\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"},\"default\":\"\",\"aliases\":[\"former_name_of_field\"]}]}");
public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }
private static SpecificData MODEL$ = new SpecificData();
private static final BinaryMessageEncoder<Tests> ENCODER =
new BinaryMessageEncoder<Tests>(MODEL$, SCHEMA$);
private static final BinaryMessageDecoder<Tests> DECODER =
new BinaryMessageDecoder<Tests>(MODEL$, SCHEMA$);
/**
* Return the BinaryMessageEncoder instance used by this class.
* #return the message encoder used by this class
*/
public static BinaryMessageEncoder<Tests> getEncoder() {
return ENCODER;
}
/**
* Return the BinaryMessageDecoder instance used by this class.
* #return the message decoder used by this class
*/
public static BinaryMessageDecoder<Tests> getDecoder() {
return DECODER;
}
/**
* Create a new BinaryMessageDecoder instance for this class that uses the specified {#link SchemaStore}.
* #param resolver a {#link SchemaStore} used to find schemas by fingerprint
* #return a BinaryMessageDecoder instance for this class backed by the given SchemaStore
*/
public static BinaryMessageDecoder<Tests> createDecoder(SchemaStore resolver) {
return new BinaryMessageDecoder<Tests>(MODEL$, SCHEMA$, resolver);
}
/**
* Serializes this Tests to a ByteBuffer.
* #return a buffer holding the serialized data for this instance
* #throws java.io.IOException if this instance could not be serialized
*/
public java.nio.ByteBuffer toByteBuffer() throws java.io.IOException {
return ENCODER.encode(this);
}
/**
* Deserializes a Tests from a ByteBuffer.
* #param b a byte buffer holding serialized data for an instance of this class
* #return a Tests instance decoded from the given buffer
* #throws java.io.IOException if the given bytes could not be deserialized into an instance of this class
*/
public static Tests fromByteBuffer(
java.nio.ByteBuffer b) throws java.io.IOException {
return DECODER.decode(b);
}
#Deprecated public java.lang.String first_name;
/**
* Default constructor. Note that this does not initialize fields
* to their default values from the schema. If that is desired then
* one should use <code>newBuilder()</code>.
*/
public Tests() {}
/**
* All-args constructor.
* #param first_name The new value for first_name
*/
public Tests(java.lang.String first_name) {
this.first_name = first_name;
}
public org.apache.avro.specific.SpecificData getSpecificData() { return MODEL$; }
public org.apache.avro.Schema getSchema() { return SCHEMA$; }
// Used by DatumWriter. Applications should not call.
public java.lang.Object get(int field$) {
switch (field$) {
case 0: return first_name;
default: throw new IndexOutOfBoundsException("Invalid index: " + field$);
}
}
// Used by DatumReader. Applications should not call.
#SuppressWarnings(value="unchecked")
public void put(int field$, java.lang.Object value$) {
switch (field$) {
case 0: first_name = value$ != null ? value$.toString() : null; break;
default: throw new IndexOutOfBoundsException("Invalid index: " + field$);
}
}
/**
* Gets the value of the 'first_name' field.
* #return The value of the 'first_name' field.
*/
public java.lang.String getFirstName() {
return first_name;
}
/**
* Sets the value of the 'first_name' field.
* #param value the value to set.
*/
public void setFirstName(java.lang.String value) {
this.first_name = value;
}
/**
* Creates a new Tests RecordBuilder.
* #return A new Tests RecordBuilder
*/
public static Tests.Builder newBuilder() {
return new Tests.Builder();
}
/**
* Creates a new Tests RecordBuilder by copying an existing Builder.
* #param other The existing builder to copy.
* #return A new Tests RecordBuilder
*/
public static Tests.Builder newBuilder(Tests.Builder other) {
if (other == null) {
return new Tests.Builder();
} else {
return new Tests.Builder(other);
}
}
/**
* Creates a new Tests RecordBuilder by copying an existing Tests instance.
* #param other The existing instance to copy.
* #return A new Tests RecordBuilder
*/
public static Tests.Builder newBuilder(Tests other) {
if (other == null) {
return new Tests.Builder();
} else {
return new Tests.Builder(other);
}
}
/**
* RecordBuilder for Tests instances.
*/
#org.apache.avro.specific.AvroGenerated
public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase<Tests>
implements org.apache.avro.data.RecordBuilder<Tests> {
private java.lang.String first_name;
/** Creates a new Builder */
private Builder() {
super(SCHEMA$);
}
/**
* Creates a Builder by copying an existing Builder.
* #param other The existing Builder to copy.
*/
private Builder(Tests.Builder other) {
super(other);
if (isValidValue(fields()[0], other.first_name)) {
this.first_name = data().deepCopy(fields()[0].schema(), other.first_name);
fieldSetFlags()[0] = other.fieldSetFlags()[0];
}
}
/**
* Creates a Builder by copying an existing Tests instance
* #param other The existing instance to copy.
*/
private Builder(Tests other) {
super(SCHEMA$);
if (isValidValue(fields()[0], other.first_name)) {
this.first_name = data().deepCopy(fields()[0].schema(), other.first_name);
fieldSetFlags()[0] = true;
}
}
/**
* Gets the value of the 'first_name' field.
* #return The value.
*/
public java.lang.String getFirstName() {
return first_name;
}
/**
* Sets the value of the 'first_name' field.
* #param value The value of 'first_name'.
* #return This builder.
*/
public Tests.Builder setFirstName(java.lang.String value) {
validate(fields()[0], value);
this.first_name = value;
fieldSetFlags()[0] = true;
return this;
}
/**
* Checks whether the 'first_name' field has been set.
* #return True if the 'first_name' field has been set, false otherwise.
*/
public boolean hasFirstName() {
return fieldSetFlags()[0];
}
/**
* Clears the value of the 'first_name' field.
* #return This builder.
*/
public Tests.Builder clearFirstName() {
first_name = null;
fieldSetFlags()[0] = false;
return this;
}
#Override
#SuppressWarnings("unchecked")
public Tests build() {
try {
Tests record = new Tests();
record.first_name = fieldSetFlags()[0] ? this.first_name : (java.lang.String) defaultValue(fields()[0]);
return record;
} catch (org.apache.avro.AvroMissingFieldException e) {
throw e;
} catch (java.lang.Exception e) {
throw new org.apache.avro.AvroRuntimeException(e);
}
}
}
#SuppressWarnings("unchecked")
private static final org.apache.avro.io.DatumWriter<Tests>
WRITER$ = (org.apache.avro.io.DatumWriter<Tests>)MODEL$.createDatumWriter(SCHEMA$);
#Override public void writeExternal(java.io.ObjectOutput out)
throws java.io.IOException {
WRITER$.write(this, SpecificData.getEncoder(out));
}
#SuppressWarnings("unchecked")
private static final org.apache.avro.io.DatumReader<Tests>
READER$ = (org.apache.avro.io.DatumReader<Tests>)MODEL$.createDatumReader(SCHEMA$);
#Override public void readExternal(java.io.ObjectInput in)
throws java.io.IOException {
READER$.read(this, SpecificData.getDecoder(in));
}
#Override protected boolean hasCustomCoders() { return true; }
#Override public void customEncode(org.apache.avro.io.Encoder out)
throws java.io.IOException
{
out.writeString(this.first_name);
}
#Override public void customDecode(org.apache.avro.io.ResolvingDecoder in)
throws java.io.IOException
{
org.apache.avro.Schema.Field[] fieldOrder = in.readFieldOrderIfDiff();
if (fieldOrder == null) {
this.first_name = in.readString();
} else {
for (int i = 0; i < 1; i++) {
switch (fieldOrder[i].pos()) {
case 0:
this.first_name = in.readString();
break;
default:
throw new java.io.IOException("Corrupt ResolvingDecoder.");
}
}
}
}
}
I do not believe it is possible to set aliases via Java code.

Writing avro messages from kafka to hdfs without third party plugins or libraries

I have generated an Apache Avro class which is used to produce and consume the Avro messages by the client application as well as the server side code.
I have been searching on the internet for achieving a couple of use cases.
writing the Avro messages as binary data on the HDFS.
Converting the byte array binary data by using the Avro generated
class to get the messages as a string and write them to HDFS.
Are there any other ways of achieving the above use cases without any third party libraries like Confluent or Twitter Bijection API.
Btw, I would like to write all the Avro messages to a single HDFS file.
Avro Generated class:
/**
* Autogenerated by Avro
*
* DO NOT EDIT DIRECTLY
*/
package com.avrotohdfs.classes.avro;
import org.apache.avro.message.BinaryMessageDecoder;
import org.apache.avro.message.BinaryMessageEncoder;
import org.apache.avro.message.SchemaStore;
import org.apache.avro.specific.SpecificData;
import org.apache.avro.specific.SpecificRecordBase;
#SuppressWarnings("all")
#org.apache.avro.specific.AvroGenerated
public class AvroSyslogMessage extends SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
private static final long serialVersionUID = -793689732516755717L;
public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"AvroSyslogMessage\",\"namespace\":\"com.cisco.sso.ssodata.avro\",\"fields\":[{\"name\":\"partyID\",\"type\":\"string\"},{\"name\":\"partyName\",\"type\":[\"string\",\"null\"]},{\"name\":\"applianceID\",\"type\":\"string\"},{\"name\":\"message\",\"type\":\"string\"},{\"name\":\"inventoryName\",\"type\":[\"string\",\"null\"]},{\"name\":\"senttime\",\"type\":[\"string\",\"null\"]}]}");
public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }
private static SpecificData MODEL$ = new SpecificData();
private static final BinaryMessageEncoder<AvroSyslogMessage> ENCODER =
new BinaryMessageEncoder<AvroSyslogMessage>(MODEL$, SCHEMA$);
private static final BinaryMessageDecoder<AvroSyslogMessage> DECODER =
new BinaryMessageDecoder<AvroSyslogMessage>(MODEL$, SCHEMA$);
/**
* Return the BinaryMessageDecoder instance used by this class.
*/
public static BinaryMessageDecoder<AvroSyslogMessage> getDecoder() {
return DECODER;
}
/**
* Create a new BinaryMessageDecoder instance for this class that uses the specified {#link SchemaStore}.
* #param resolver a {#link SchemaStore} used to find schemas by fingerprint
*/
public static BinaryMessageDecoder<AvroSyslogMessage> createDecoder(SchemaStore resolver) {
return new BinaryMessageDecoder<AvroSyslogMessage>(MODEL$, SCHEMA$, resolver);
}
/** Serializes this AvroSyslogMessage to a ByteBuffer. */
public java.nio.ByteBuffer toByteBuffer() throws java.io.IOException {
return ENCODER.encode(this);
}
/** Deserializes a AvroSyslogMessage from a ByteBuffer. */
public static AvroSyslogMessage fromByteBuffer(
java.nio.ByteBuffer b) throws java.io.IOException {
return DECODER.decode(b);
}
#Deprecated public java.lang.CharSequence partyID;
#Deprecated public java.lang.CharSequence partyName;
#Deprecated public java.lang.CharSequence applianceID;
#Deprecated public java.lang.CharSequence message;
#Deprecated public java.lang.CharSequence inventoryName;
#Deprecated public java.lang.CharSequence senttime;
/**
* Default constructor. Note that this does not initialize fields
* to their default values from the schema. If that is desired then
* one should use <code>newBuilder()</code>.
*/
public AvroSyslogMessage() {}
/**
* All-args constructor.
* #param partyID The new value for partyID
* #param partyName The new value for partyName
* #param applianceID The new value for applianceID
* #param message The new value for message
* #param inventoryName The new value for inventoryName
* #param senttime The new value for senttime
*/
public AvroSyslogMessage(java.lang.CharSequence partyID, java.lang.CharSequence partyName, java.lang.CharSequence applianceID, java.lang.CharSequence message, java.lang.CharSequence inventoryName, java.lang.CharSequence senttime) {
this.partyID = partyID;
this.partyName = partyName;
this.applianceID = applianceID;
this.message = message;
this.inventoryName = inventoryName;
this.senttime = senttime;
}
public org.apache.avro.Schema getSchema() { return SCHEMA$; }
// Used by DatumWriter. Applications should not call.
public java.lang.Object get(int field$) {
switch (field$) {
case 0: return partyID;
case 1: return partyName;
case 2: return applianceID;
case 3: return message;
case 4: return inventoryName;
case 5: return senttime;
default: throw new org.apache.avro.AvroRuntimeException("Bad index");
}
}
// Used by DatumReader. Applications should not call.
#SuppressWarnings(value="unchecked")
public void put(int field$, java.lang.Object value$) {
switch (field$) {
case 0: partyID = (java.lang.CharSequence)value$; break;
case 1: partyName = (java.lang.CharSequence)value$; break;
case 2: applianceID = (java.lang.CharSequence)value$; break;
case 3: message = (java.lang.CharSequence)value$; break;
case 4: inventoryName = (java.lang.CharSequence)value$; break;
case 5: senttime = (java.lang.CharSequence)value$; break;
default: throw new org.apache.avro.AvroRuntimeException("Bad index");
}
}
/**
* Gets the value of the 'partyID' field.
* #return The value of the 'partyID' field.
*/
public java.lang.CharSequence getPartyID() {
return partyID;
}
/**
* Sets the value of the 'partyID' field.
* #param value the value to set.
*/
public void setPartyID(java.lang.CharSequence value) {
this.partyID = value;
}
/**
* Gets the value of the 'partyName' field.
* #return The value of the 'partyName' field.
*/
public java.lang.CharSequence getPartyName() {
return partyName;
}
/**
* Sets the value of the 'partyName' field.
* #param value the value to set.
*/
public void setPartyName(java.lang.CharSequence value) {
this.partyName = value;
}
/**
* Gets the value of the 'applianceID' field.
* #return The value of the 'applianceID' field.
*/
public java.lang.CharSequence getApplianceID() {
return applianceID;
}
/**
* Sets the value of the 'applianceID' field.
* #param value the value to set.
*/
public void setApplianceID(java.lang.CharSequence value) {
this.applianceID = value;
}
/**
* Gets the value of the 'message' field.
* #return The value of the 'message' field.
*/
public java.lang.CharSequence getMessage() {
return message;
}
/**
* Sets the value of the 'message' field.
* #param value the value to set.
*/
public void setMessage(java.lang.CharSequence value) {
this.message = value;
}
/**
* Gets the value of the 'inventoryName' field.
* #return The value of the 'inventoryName' field.
*/
public java.lang.CharSequence getInventoryName() {
return inventoryName;
}
/**
* Sets the value of the 'inventoryName' field.
* #param value the value to set.
*/
public void setInventoryName(java.lang.CharSequence value) {
this.inventoryName = value;
}
/**
* Gets the value of the 'senttime' field.
* #return The value of the 'senttime' field.
*/
public java.lang.CharSequence getSenttime() {
return senttime;
}
/**
* Sets the value of the 'senttime' field.
* #param value the value to set.
*/
public void setSenttime(java.lang.CharSequence value) {
this.senttime = value;
}
/**
* Creates a new AvroSyslogMessage RecordBuilder.
* #return A new AvroSyslogMessage RecordBuilder
*/
public static com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder newBuilder() {
return new com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder();
}
/**
* Creates a new AvroSyslogMessage RecordBuilder by copying an existing Builder.
* #param other The existing builder to copy.
* #return A new AvroSyslogMessage RecordBuilder
*/
public static com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder newBuilder(com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder other) {
return new com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder(other);
}
/**
* Creates a new AvroSyslogMessage RecordBuilder by copying an existing AvroSyslogMessage instance.
* #param other The existing instance to copy.
* #return A new AvroSyslogMessage RecordBuilder
*/
public static com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder newBuilder(com.cisco.sso.ssodata.avro.AvroSyslogMessage other) {
return new com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder(other);
}
/**
* RecordBuilder for AvroSyslogMessage instances.
*/
public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase<AvroSyslogMessage>
implements org.apache.avro.data.RecordBuilder<AvroSyslogMessage> {
private java.lang.CharSequence partyID;
private java.lang.CharSequence partyName;
private java.lang.CharSequence applianceID;
private java.lang.CharSequence message;
private java.lang.CharSequence inventoryName;
private java.lang.CharSequence senttime;
/** Creates a new Builder */
private Builder() {
super(SCHEMA$);
}
/**
* Creates a Builder by copying an existing Builder.
* #param other The existing Builder to copy.
*/
private Builder(com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder other) {
super(other);
if (isValidValue(fields()[0], other.partyID)) {
this.partyID = data().deepCopy(fields()[0].schema(), other.partyID);
fieldSetFlags()[0] = true;
}
if (isValidValue(fields()[1], other.partyName)) {
this.partyName = data().deepCopy(fields()[1].schema(), other.partyName);
fieldSetFlags()[1] = true;
}
if (isValidValue(fields()[2], other.applianceID)) {
this.applianceID = data().deepCopy(fields()[2].schema(), other.applianceID);
fieldSetFlags()[2] = true;
}
if (isValidValue(fields()[3], other.message)) {
this.message = data().deepCopy(fields()[3].schema(), other.message);
fieldSetFlags()[3] = true;
}
if (isValidValue(fields()[4], other.inventoryName)) {
this.inventoryName = data().deepCopy(fields()[4].schema(), other.inventoryName);
fieldSetFlags()[4] = true;
}
if (isValidValue(fields()[5], other.senttime)) {
this.senttime = data().deepCopy(fields()[5].schema(), other.senttime);
fieldSetFlags()[5] = true;
}
}
/**
* Creates a Builder by copying an existing AvroSyslogMessage instance
* #param other The existing instance to copy.
*/
private Builder(com.cisco.sso.ssodata.avro.AvroSyslogMessage other) {
super(SCHEMA$);
if (isValidValue(fields()[0], other.partyID)) {
this.partyID = data().deepCopy(fields()[0].schema(), other.partyID);
fieldSetFlags()[0] = true;
}
if (isValidValue(fields()[1], other.partyName)) {
this.partyName = data().deepCopy(fields()[1].schema(), other.partyName);
fieldSetFlags()[1] = true;
}
if (isValidValue(fields()[2], other.applianceID)) {
this.applianceID = data().deepCopy(fields()[2].schema(), other.applianceID);
fieldSetFlags()[2] = true;
}
if (isValidValue(fields()[3], other.message)) {
this.message = data().deepCopy(fields()[3].schema(), other.message);
fieldSetFlags()[3] = true;
}
if (isValidValue(fields()[4], other.inventoryName)) {
this.inventoryName = data().deepCopy(fields()[4].schema(), other.inventoryName);
fieldSetFlags()[4] = true;
}
if (isValidValue(fields()[5], other.senttime)) {
this.senttime = data().deepCopy(fields()[5].schema(), other.senttime);
fieldSetFlags()[5] = true;
}
}
/**
* Gets the value of the 'partyID' field.
* #return The value.
*/
public java.lang.CharSequence getPartyID() {
return partyID;
}
/**
* Sets the value of the 'partyID' field.
* #param value The value of 'partyID'.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder setPartyID(java.lang.CharSequence value) {
validate(fields()[0], value);
this.partyID = value;
fieldSetFlags()[0] = true;
return this;
}
/**
* Checks whether the 'partyID' field has been set.
* #return True if the 'partyID' field has been set, false otherwise.
*/
public boolean hasPartyID() {
return fieldSetFlags()[0];
}
/**
* Clears the value of the 'partyID' field.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder clearPartyID() {
partyID = null;
fieldSetFlags()[0] = false;
return this;
}
/**
* Gets the value of the 'partyName' field.
* #return The value.
*/
public java.lang.CharSequence getPartyName() {
return partyName;
}
/**
* Sets the value of the 'partyName' field.
* #param value The value of 'partyName'.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder setPartyName(java.lang.CharSequence value) {
validate(fields()[1], value);
this.partyName = value;
fieldSetFlags()[1] = true;
return this;
}
/**
* Checks whether the 'partyName' field has been set.
* #return True if the 'partyName' field has been set, false otherwise.
*/
public boolean hasPartyName() {
return fieldSetFlags()[1];
}
/**
* Clears the value of the 'partyName' field.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder clearPartyName() {
partyName = null;
fieldSetFlags()[1] = false;
return this;
}
/**
* Gets the value of the 'applianceID' field.
* #return The value.
*/
public java.lang.CharSequence getApplianceID() {
return applianceID;
}
/**
* Sets the value of the 'applianceID' field.
* #param value The value of 'applianceID'.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder setApplianceID(java.lang.CharSequence value) {
validate(fields()[2], value);
this.applianceID = value;
fieldSetFlags()[2] = true;
return this;
}
/**
* Checks whether the 'applianceID' field has been set.
* #return True if the 'applianceID' field has been set, false otherwise.
*/
public boolean hasApplianceID() {
return fieldSetFlags()[2];
}
/**
* Clears the value of the 'applianceID' field.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder clearApplianceID() {
applianceID = null;
fieldSetFlags()[2] = false;
return this;
}
/**
* Gets the value of the 'message' field.
* #return The value.
*/
public java.lang.CharSequence getMessage() {
return message;
}
/**
* Sets the value of the 'message' field.
* #param value The value of 'message'.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder setMessage(java.lang.CharSequence value) {
validate(fields()[3], value);
this.message = value;
fieldSetFlags()[3] = true;
return this;
}
/**
* Checks whether the 'message' field has been set.
* #return True if the 'message' field has been set, false otherwise.
*/
public boolean hasMessage() {
return fieldSetFlags()[3];
}
/**
* Clears the value of the 'message' field.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder clearMessage() {
message = null;
fieldSetFlags()[3] = false;
return this;
}
/**
* Gets the value of the 'inventoryName' field.
* #return The value.
*/
public java.lang.CharSequence getInventoryName() {
return inventoryName;
}
/**
* Sets the value of the 'inventoryName' field.
* #param value The value of 'inventoryName'.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder setInventoryName(java.lang.CharSequence value) {
validate(fields()[4], value);
this.inventoryName = value;
fieldSetFlags()[4] = true;
return this;
}
/**
* Checks whether the 'inventoryName' field has been set.
* #return True if the 'inventoryName' field has been set, false otherwise.
*/
public boolean hasInventoryName() {
return fieldSetFlags()[4];
}
/**
* Clears the value of the 'inventoryName' field.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder clearInventoryName() {
inventoryName = null;
fieldSetFlags()[4] = false;
return this;
}
/**
* Gets the value of the 'senttime' field.
* #return The value.
*/
public java.lang.CharSequence getSenttime() {
return senttime;
}
/**
* Sets the value of the 'senttime' field.
* #param value The value of 'senttime'.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder setSenttime(java.lang.CharSequence value) {
validate(fields()[5], value);
this.senttime = value;
fieldSetFlags()[5] = true;
return this;
}
/**
* Checks whether the 'senttime' field has been set.
* #return True if the 'senttime' field has been set, false otherwise.
*/
public boolean hasSenttime() {
return fieldSetFlags()[5];
}
/**
* Clears the value of the 'senttime' field.
* #return This builder.
*/
public com.cisco.sso.ssodata.avro.AvroSyslogMessage.Builder clearSenttime() {
senttime = null;
fieldSetFlags()[5] = false;
return this;
}
#Override
#SuppressWarnings("unchecked")
public AvroSyslogMessage build() {
try {
AvroSyslogMessage record = new AvroSyslogMessage();
record.partyID = fieldSetFlags()[0] ? this.partyID : (java.lang.CharSequence) defaultValue(fields()[0]);
record.partyName = fieldSetFlags()[1] ? this.partyName : (java.lang.CharSequence) defaultValue(fields()[1]);
record.applianceID = fieldSetFlags()[2] ? this.applianceID : (java.lang.CharSequence) defaultValue(fields()[2]);
record.message = fieldSetFlags()[3] ? this.message : (java.lang.CharSequence) defaultValue(fields()[3]);
record.inventoryName = fieldSetFlags()[4] ? this.inventoryName : (java.lang.CharSequence) defaultValue(fields()[4]);
record.senttime = fieldSetFlags()[5] ? this.senttime : (java.lang.CharSequence) defaultValue(fields()[5]);
return record;
} catch (java.lang.Exception e) {
throw new org.apache.avro.AvroRuntimeException(e);
}
}
}
#SuppressWarnings("unchecked")
private static final org.apache.avro.io.DatumWriter<AvroSyslogMessage>
WRITER$ = (org.apache.avro.io.DatumWriter<AvroSyslogMessage>)MODEL$.createDatumWriter(SCHEMA$);
#Override public void writeExternal(java.io.ObjectOutput out)
throws java.io.IOException {
WRITER$.write(this, SpecificData.getEncoder(out));
}
#SuppressWarnings("unchecked")
private static final org.apache.avro.io.DatumReader<AvroSyslogMessage>
READER$ = (org.apache.avro.io.DatumReader<AvroSyslogMessage>)MODEL$.createDatumReader(SCHEMA$);
#Override public void readExternal(java.io.ObjectInput in)
throws java.io.IOException {
READER$.read(this, SpecificData.getDecoder(in));
}
}
Code to write the Avro messages as a byte array to HDFS:
#Autowired
private PropertyConfig config;
FSDataOutputStream out = null;
public void consume() throws IOException {
String topic = config.getDedupServiceConsumerTopic();
String consGroup = config.getDedupServiceConsGroup();
KafkaConsumer<String, AvroSyslogMessage> consumer = new GenericConsumer<String, AvroSyslogMessage>()
.initialize(topic, consGroup, STREAMSERDE.STRINGDESER, STREAMSERDE.AVRODESER);
logger.debug("Dedupe Kafka Consumer Initialized......");
try {
while (true) {
ConsumerRecords<String, AvroSyslogMessage> records = consumer.poll(100);
for (ConsumerRecord<String, AvroSyslogMessage> record : records) {
logger.debug("record.offset() = " + record.offset() + " : record.key() = " + record.key());
AvroSyslogMessage avroMessage = record.value();
logger.info("avro Message = " + avroMessage);
Configuration config = new Configuration();
FileSystem fs = FileSystem.get(config);
String s = fs.getHomeDirectory() + "/syslog";
Path path = new Path(s);
DistributedFileSystem dfs = new DistributedFileSystem();
if (!dfs.exists(path)) {
dfs.createNewFile(path);
out = fs.create(path);
}
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput ooput = new ObjectOutputStream(bos);
ooput.writeObject(avroMessage);
logger.info("Writing avro message to hdfs");
out.write(bos.toByteArray());
}
}
} catch (Exception e) {
logger.error("Error occured while processing message", e);
} finally {
logger.debug("debupe kafka consume is closing");
consumer.close();
out.close();
}
}
My question is to understand if this is the right approach to write Avro messages as byte arrays to the HDFS.

Get a list of the elements under a certain table column

This table is produced by my code below. (The table's data is rubbish)
Question:
How can I get a list of the items under a certain column in the order they appear?
I want to do something like table.GetListFromColumn("carmake") so I would have ["Aston Martin" , "Porche 911", "ferrari"].
Or beanItemContainer.GetListFromColumn("carmake")?
#Override
public void init(VaadinRequest request) {
BrakeResponseTime time1 = new BrakeResponseTime(0.2f);
BrakeResponseTime time2 = new BrakeResponseTime(0.8f);
BrakeResponseTime time3 = new BrakeResponseTime(0.5f);
Collection<SportsCar>c = new ArrayList<SportsCar>();
c.add(new SportsCar("ferrari", 180.0, "Tom", time1));
c.add(new SportsCar("Aston Martin", 165.0, "Harry", time2));
c.add(new SportsCar("Porche 911", 145.0, "Dick", time3));
MyBeanItemContainer container = new MyBeanItemContainer(SportsCar.class,c);
Table table = new Table();
t.setContainerDataSource(container);
this.setContent(table);
}
.
public class BrakeResponseTime implements Comparable<BrakeResponseTime> {
float time;
/**
* Create a Demo.BrakeResponseTime.
*/
public BrakeResponseTime(float time) {
this.time = time;
}
/**
* Get the time.
*
* #return the time.
*/
public float getTime() {
return time;
}
/**
* Set the time.
*
* #param time
* the time.
*/
public void setTime(float time) {
this.time = time;
}
#Override
public String toString() {
return time +"/%";
}
/*
* (non-Javadoc)
*
* #see java.lang.Comparable#compareTo(java.lang.Object)
*/
#Override
public int compareTo(BrakeResponseTime o) {
if (null == o) {
return 1;
}
if(time==o.getTime()){
return 0;
}
return (time < o.getTime() ? -1 : 1);
}
}
.
public class MyBeanItemContainer extends BeanItemContainer<SportsCar> {
/**
* Create a Demo.MyBeanItemContainer.
*/
public MyBeanItemContainer(Class<SportsCar> type, Collection<SportsCar> collection) {
super(type, collection);
}
private int compareBrakeResponseTimeWithNullCheck(BrakeResponseTime o1, BrakeResponseTime o2) {
if (null == o1 && (null == o2)) {
return 0;
} else if (null == o1 && (null != o2)) {
return -1;
}else if(null != o1 && (null == o2)){
return 1;
} else {
return o1.compareTo(o2);
}
}
#Override
public void sort(final Object[] propertyId, final boolean[] ascending) {
super.sort(propertyId, ascending);
System.out.println(Arrays.toString(propertyId));
final boolean sortAscending = ascending[0];
final Object sortContainerPropertyId = propertyId[0];
List<SportsCar> list = super.getAllItemIds();
Collections.sort(list, new Comparator<SportsCar>() {
#Override
public int compare(final SportsCar o1, final SportsCar o2) {
int result = 0;
if ("responseTime".equals(sortContainerPropertyId)) {
result = compareBrakeResponseTimeWithNullCheck(o1.getResponseTime(), o2.getResponseTime());
}
if (!sortAscending) {
result *= -1;
}
return result;
}
});
}
}
public class SportsCar {
String carMake;
double topspeed;
String driver;
BrakeResponseTime responseTime;
public SportsCar(String carMake, double topspeed, String driver, BrakeResponseTime responseTime) {
this.carMake = carMake;
this.topspeed = topspeed;
this.driver = driver;
this.responseTime = responseTime;
}
/**
* Get the carMake.
*
* #return the carMake.
*/
public String getcarMake() {
return carMake;
}
/**
* Set the carMake.
*
* #param carMake
* the carMake.
*/
public void setcarMake(String carMake) {
this.carMake = carMake;
}
/**
* Get the topspeed.
*
* #return the topspeed.
*/
public double getTopspeed() {
return topspeed;
}
/**
* Set the topspeed.
*
* #param topspeed
* the topspeed.
*/
public void setTopspeed(double topspeed) {
this.topspeed = topspeed;
}
/**
* Get the driver.
*
* #return the driver.
*/
public String getDriver() {
return driver;
}
/**
* Set the driver.
*
* #param driver
* the driver.
*/
public void setDriver(String driver) {
this.driver = driver;
}
/**
* Get the responseTime.
* #return the responseTime.
*/
public BrakeResponseTime getResponseTime() {
return responseTime;
}
/**
* Set the responseTime.
* #param responseTime the responseTime.
*/
public void setResponseTime(BrakeResponseTime responseTime) {
this.responseTime = responseTime;
}
}
I hadn't realised I could access the bean container's properties in this way.
container.getIdByIndex(0).getCarMake()
This is what I was looking for!

get enum name from enum value [duplicate]

This question already has answers here:
Convert from enum ordinal to enum type
(15 answers)
Closed 7 years ago.
I've read a lot about how obtain the corresponding name of an enum from its value using java, but no example seems to work for me! What is wrong?
public class Extensions {
public enum RelationActiveEnum
{
Invited(0),
Active(1),
Suspended(2);
private final int value;
private RelationActiveEnum(final int value) {
this.value = value;
}
}
}
and in another class I use:
int dbValue = supp.ACTIVE;
Extensions.RelationActiveEnum enumValue(dbValue);
String stringName = enumValue.toString(); //Visible
// OR
int dbValuee = supp.ACTIVE;
String stringValue = Enum.GetName(typeof(RelationActiveEnum), dbValue);
I should work, right? but it doesn't!!!! it tells me that dbValue cannote be cast to RelationActiveEnum...
Say we have:
public enum MyEnum {
Test1, Test2, Test3
}
To get the name of a enum variable use name():
MyEnum e = MyEnum.Test1;
String name = e.name(); // Returns "Test1"
To get the enum from a (string) name, use valueOf():
String name = "Test1";
MyEnum e = Enum.valueOf(MyEnum.class, name);
If you require integer values to match enum fields, extend the enum class:
public enum MyEnum {
Test1(1), Test2(2), Test3(3);
public final int value;
MyEnum(final int value) {
this.value = value;
}
}
Now you can use:
MyEnum e = MyEnum.Test1;
int value = e.value; // = 1
And lookup the enum using the integer value:
MyEnum getValue(int value) {
for(MyEnum e: MyEnum.values()) {
if(e.value == value) {
return e;
}
}
return null;// not found
}
Since your 'value' also happens to match with ordinals you could just do:
public enum RelationActiveEnum {
Invited,
Active,
Suspended;
private final int value;
private RelationActiveEnum() {
this.value = ordinal();
}
}
And getting a enum from the value:
int value = 1;
RelationActiveEnum enumInstance = RelationActiveEnum.values()[value];
I guess an static method would be a good place to put this:
public enum RelationActiveEnum {
public static RelationActiveEnum fromValue(int value)
throws IllegalArgumentException {
try {
return RelationActiveEnum.values()[value]
} catch(ArrayIndexOutOfBoundsException e) {
throw new IllegalArgumentException("Unknown enum value :"+ value);
}
}
}
Obviously this all falls apart if your 'value' isn't the same value as the enum ordinal.
You could create a lookup method. Not the most efficient (depending on the enum's size) but it works.
public static String getNameByCode(int code){
for(RelationActiveEnum e : RelationActiveEnum.values()){
if(code == e.value) return e.name();
}
return null;
}
And call it like this:
RelationActiveEnum.getNameByCode(3);
What you can do is
RelationActiveEnum ae = Enum.valueOf(RelationActiveEnum.class,
RelationActiveEnum.ACTIVE.name();
or
RelationActiveEnum ae = RelationActiveEnum.valueOf(
RelationActiveEnum.ACTIVE.name();
or
// not recommended as the ordinal might not match the value
RelationActiveEnum ae = RelationActiveEnum.values()[
RelationActiveEnum.ACTIVE.value];
By if you want to lookup by a field of an enum you need to construct a collection such as a List, an array or a Map.
public enum RelationActiveEnum {
Invited(0),
Active(1),
Suspended(2);
private final int code;
private RelationActiveEnum(final int code) {
this.code = code;
}
private static final Map<Integer, RelationActiveEnum> BY_CODE_MAP = new LinkedHashMap<>();
static {
for (RelationActiveEnum rae : RelationActiveEnum.values()) {
BY_CODE_MAP.put(rae.code, rae);
}
}
public static RelationActiveEnum forCode(int code) {
return BY_CODE_MAP.get(code);
}
}
allows you to write
String name = RelationActiveEnum.forCode(RelationActiveEnum.ACTIVE.code).name();
In my case value was not an integer but a String.
getNameByCode method can be added to the enum to get name of a String value-
enum CODE {
SUCCESS("SCS"), DELETE("DEL");
private String status;
/**
* #return the status
*/
public String getStatus() {
return status;
}
/**
* #param status
* the status to set
*/
public void setStatus(String status) {
this.status = status;
}
private CODE(String status) {
this.status = status;
}
public static String getNameByCode(String code) {
for (int i = 0; i < CODE.values().length; i++) {
if (code.equals(CODE.values()[i].status))
return CODE.values()[i].name();
}
return null;
}
If you want something more efficient in runtime condition, you can have a map that contains every possible choice of the enum by their value. But it'll be juste slower at initialisation of the JVM.
import java.util.HashMap;
import java.util.Map;
/**
* Example of enum with a getter that need a value in parameter, and that return the Choice/Instance
* of the enum which has the same value.
* The value of each choice can be random.
*/
public enum MyEnum {
/** a random choice */
Choice1(4),
/** a nother one */
Choice2(2),
/** another one again */
Choice3(9);
/** a map that contains every choices of the enum ordered by their value. */
private static final Map<Integer, MyEnum> MY_MAP = new HashMap<Integer, MyEnum>();
static {
// populating the map
for (MyEnum myEnum : values()) {
MY_MAP.put(myEnum.getValue(), myEnum);
}
}
/** the value of the choice */
private int value;
/**
* constructor
* #param value the value
*/
private MyEnum(int value) {
this.value = value;
}
/**
* getter of the value
* #return int
*/
public int getValue() {
return value;
}
/**
* Return one of the choice of the enum by its value.
* May return null if there is no choice for this value.
* #param value value
* #return MyEnum
*/
public static MyEnum getByValue(int value) {
return MY_MAP.get(value);
}
/**
* {#inheritDoc}
* #see java.lang.Enum#toString()
*/
public String toString() {
return name() + "=" + value;
}
/**
* Exemple of how to use this class.
* #param args args
*/
public static void main(String[] args) {
MyEnum enum1 = MyEnum.Choice1;
System.out.println("enum1==>" + String.valueOf(enum1));
MyEnum enum2GotByValue = MyEnum.getByValue(enum1.getValue());
System.out.println("enum2GotByValue==>" + String.valueOf(enum2GotByValue));
MyEnum enum3Unknown = MyEnum.getByValue(4);
System.out.println("enum3Unknown==>" + String.valueOf(enum3Unknown));
}
}
This is my take on it:
public enum LoginState {
LOGGED_IN(1), LOGGED_OUT(0), IN_TRANSACTION(-1);
private int code;
LoginState(int code) {
this.code = code;
}
public int getCode() {
return code;
}
public static LoginState getLoginStateFromCode(int code){
for(LoginState e : LoginState.values()){
if(code == e.code) return e;
}
return LoginState.LOGGED_OUT; //or null
}
};
And I have used it with System Preferences in Android like so:
LoginState getLoginState(int i) {
return LoginState.getLoginStateFromCode(
prefs().getInt(SPK_IS_LOGIN, LoginState.LOGGED_OUT.getCode())
);
}
public static void setLoginState(LoginState newLoginState) {
editor().putInt(SPK_IS_LOGIN, newLoginState.getCode());
editor().commit();
}
where pref and editor are SharedPreferences and a SharedPreferences.Editor

Getting data from JSON

I'm trying to get the values out of this JSON string but I'm having a hard time achieving this.
{"DebugLogId":"1750550","RequestId":"17505503","Result":
{"Code":"","DebugLogId":"1750550","Message":""},
"Suggestions":[
{"Ranking":"1","Score":"60","Title":"This is a test message 1"},
{"Ranking":"2","Score":"60","Title":"This is a test message 2"}
]}
What way would be easiest to access the data in 'Suggestions'? I'm using the GSON module. Ideally I would like to put it all in a HashMap.
Thanks for any help and/or suggestions!
Thanks for any help!
Hope this helps:
App.java:
package sg.java.play_sof_json_6596072;
import com.google.gson.Gson;
public class App {
public static void main(String[] args) {
Gson gson = new Gson();
String jsonString = "{\"DebugLogId\":\"1750550\",\"RequestId\":\"17505503\",\"Result\":{\"Code\":\"\",\"DebugLogId\":\"1750550\",\"Message\":\"\"},\"Suggestions\":[{\"Ranking\":\"1\",\"Score\":\"60\",\"Title\":\"This is a test message 1\"},{\"Ranking\":\"2\",\"Score\":\"60\",\"Title\":\"This is a test message 2\"}]}";
Debug obj = (Debug) gson.fromJson(jsonString, Debug.class);
System.out.println(obj.getSuggestionList().get(1).getTitle());
}
}
Debug.java:
package sg.java.play_sof_json_6596072;
import java.util.List;
import com.google.gson.annotations.SerializedName;
public class Debug {
#SerializedName("DebugLogId")
private String debugLogId;
#SerializedName("RequestId")
private String requestId;
#SerializedName("Result")
private Result result;
#SerializedName("Suggestions")
private List<Suggestion> suggestionList;
/**
* #return the debugLogId
*/
public final String getDebugLogId() {
return this.debugLogId;
}
/**
* #param debugLogId the debugLogId to set
*/
public final void setDebugLogId(String debugLogId) {
this.debugLogId = debugLogId;
}
/**
* #return the requestId
*/
public final String getRequestId() {
return this.requestId;
}
/**
* #param requestId the requestId to set
*/
public final void setRequestId(String requestId) {
this.requestId = requestId;
}
/**
* #return the result
*/
public final Result getResult() {
return this.result;
}
/**
* #param result the result to set
*/
public final void setResult(Result result) {
this.result = result;
}
/**
* #return the suggestionList
*/
public final List<Suggestion> getSuggestionList() {
return this.suggestionList;
}
/**
* #param suggestionList the suggestionList to set
*/
public final void setSuggestionList(List<Suggestion> suggestionList) {
this.suggestionList = suggestionList;
}
}
Result.java:
package sg.java.play_sof_json_6596072;
import com.google.gson.annotations.SerializedName;
public class Result {
#SerializedName("Code")
private String code;
#SerializedName("DebugLogId")
private String debugLogId;
#SerializedName("Message")
private String messahe;
/**
* #return the code
*/
public final String getCode() {
return this.code;
}
/**
* #param code the code to set
*/
public final void setCode(String code) {
this.code = code;
}
/**
* #return the debugLogId
*/
public final String getDebugLogId() {
return this.debugLogId;
}
/**
* #param debugLogId the debugLogId to set
*/
public final void setDebugLogId(String debugLogId) {
this.debugLogId = debugLogId;
}
/**
* #return the messahe
*/
public final String getMessahe() {
return this.messahe;
}
/**
* #param messahe the messahe to set
*/
public final void setMessahe(String messahe) {
this.messahe = messahe;
}
}
Suggestion.java:
package sg.java.play_sof_json_6596072;
import com.google.gson.annotations.SerializedName;
public class Suggestion {
#SerializedName("Ranking")
private String ranking;
#SerializedName("Score")
private String score;
#SerializedName("Title")
private String title;
/**
* #return the ranking
*/
public final String getRanking() {
return this.ranking;
}
/**
* #param ranking the ranking to set
*/
public final void setRanking(String ranking) {
this.ranking = ranking;
}
/**
* #return the score
*/
public final String getScore() {
return this.score;
}
/**
* #param score the score to set
*/
public final void setScore(String score) {
this.score = score;
}
/**
* #return the title
*/
public final String getTitle() {
return this.title;
}
/**
* #param title the title to set
*/
public final void setTitle(String title) {
this.title = title;
}
}
I'm recommend you to use flexjson library http://flexjson.sourceforge.net/
IMHO, it more simple and usable library. I has used GSON first time, but then switched all my projects to flexjson instead of GSON.
Using the standard json classes in android:
JSONObject o = new JSONObject("your string");
JSONArray a = o.getJSONArray("Suggestions");
int i = 0;
while ( i < a.length())
{
o = a.getJSONObject(i);
//do something with o, like o.getString("Title") ...
++i;
}
Ideally I would like to put it all in a HashMap.
If you can switch libraries, Jackson can achieve that with just one line of code.
Map map = new ObjectMapper().readValue(json, Map.class);
This would deserialize any JSON object into a HashMap, composed of just Java SE components. I haven't yet seen another Java-to/from-JSON library that can do that.
The same can be accomplished with Gson, but it requires a few more lines of code. Here's one such solution.
JsonElement je = new JsonParser().parse(json);
JsonObject jo = je.getAsJsonObject();
Map<String, Object> map = createMapFromJsonObject(jo);
// ...
static Map<String, Object> createMapFromJsonObject(
JsonObject jo)
{
Map<String, Object> map = new HashMap<String, Object>();
for (Entry<String, JsonElement> entry : jo.entrySet())
{
String key = entry.getKey();
JsonElement value = entry.getValue();
map.put(key, getValueFromJsonElement(value));
}
return map;
}
static Object getValueFromJsonElement(JsonElement je)
{
if (je.isJsonObject())
{
return createMapFromJsonObject(je.getAsJsonObject());
}
else if (je.isJsonArray())
{
JsonArray array = je.getAsJsonArray();
List<Object> list = new ArrayList<Object>(array.size());
for (JsonElement element : array)
{
list.add(getValueFromJsonElement(element));
}
return list;
}
else if (je.isJsonNull())
{
return null;
}
else // must be primitive
{
JsonPrimitive p = je.getAsJsonPrimitive();
if (p.isBoolean()) return p.getAsBoolean();
if (p.isString()) return p.getAsString();
// else p is number, but don't know what kind
String s = p.getAsString();
try
{
return new BigInteger(s);
}
catch (NumberFormatException e)
{
// must be a decimal
return new BigDecimal(s);
}
}
}
(I copied this code from my blog post at http://programmerbruce.blogspot.com/2011/06/gson-v-jackson.html.)

Categories

Resources