Develop Reference

Java Jolt with quarkus error java.lang.Boolean; no valid constructor

I am using:
Java 17
Quarkus 2.12.0.Final
Jolt 0.1.7
graalvm-ce-22.0.0
Native image
mvn clean package -DskipTests -Pnative -Dquarkus.native.container-build=true
Using this transformer:
{
"operation": "default",
"spec": {
"user": {
"enabled": true
}
}
}
When a run the application using non-native image, it works!!
When a run the application the application with native image, occurs this error:
at com.bazaarvoice.jolt.common.DeepCopy.simpleDeepCopy(DeepCopy.java:53)
at com.bazaarvoice.jolt.defaultr.MapKey.applyLiteralKeyToContainer(MapKey.java:57)
at com.bazaarvoice.jolt.defaultr.MapKey.applyChild(MapKey.java:44)
at com.bazaarvoice.jolt.defaultr.Key.applyChildren(Key.java:162)
at com.bazaarvoice.jolt.defaultr.MapKey.applyLiteralKeyToContainer(MapKey.java:67)
at com.bazaarvoice.jolt.defaultr.MapKey.applyChild(MapKey.java:44)
at com.bazaarvoice.jolt.defaultr.Key.applyChildren(Key.java:162)
at com.bazaarvoice.jolt.Defaultr.transform(Defaultr.java:242)
at com.bazaarvoice.jolt.Chainr$ContextualTransformAdapter.transform(Chainr.java:113)
at com.bazaarvoice.jolt.Chainr.doTransform(Chainr.java:240)
at com.bazaarvoice.jolt.Chainr.transform(Chainr.java:180)
at com.mypackage.myclass.Play.run(Play.java:20)
at io.quarkus.amazon.lambda.runtime.AmazonLambdaRecorder$1.processRequest(AmazonLambdaRecorder.java:170)
at io.quarkus.amazon.lambda.runtime.AbstractLambdaPollLoop$1.run(AbstractLambdaPollLoop.java:130)
at java.lang.Thread.run(Thread.java:833)
at com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:704)
at com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:202)
caused by: java.io.InvalidClassException: java.lang.Boolean; no valid constructor
at java.io.ObjectStreamClass$ExceptionInfo.newInvalidClassException(ObjectStreamClass.java:174)
at java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:921)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1744)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:514)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:472)
at com.bazaarvoice.jolt.common.DeepCopy.simpleDeepCopy(DeepCopy.java:50)
... 26 more
As java.lang.Boolean has no default constructor, I tried registerForReflection, but not works too.
EDIT
I have found I workaround:
{
"operation": "default",
"spec": {
"user": {
"enabled": "true"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"user": {
"enabled": "=toBoolean"
}
}
}
EDIT 2
This also happen with other values like Integer, Long, because they do not have default constructor.
Those errors happen because how the copy is made:
Look here, but I am Newbie in native image, so I don't know why this kind of copy does not work.

How to use java runtime 11 in EMR cluster AWS

I'm creating a cluter in EMR aws and when spark runs my application I'm getting error below:
Exception in thread "main" java.lang.UnsupportedClassVersionError:
com/example/demodriver/MyClassFromJAR has been compiled by a more recent version of the Java Runtime (class file version 55.0),
this version of the Java Runtime only recognizes class file versions up to 52.0
I'm using releaseLabel emr-6.5.0 on clusters and My driver jar is built in java11
How run java11 application in EMR? Or this error is about anything else?

In the most recent versions of EMR, java 11 is installed. To enable it, you can provide the following configuration.
[
{
"Classification": "spark-env",
"Configurations": [
{
"Classification": "export",
"Properties": {
"JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
}
}
]
},
{
"Classification": "spark-defaults",
"Properties": {
"spark.executorEnv.JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
}
}
]
This does not seem to be documented.
The defaultJavaOptions and extraJavaOptions might contain incompatible options for java 11 which you still might need to adapt/update.

Here is the full configuration including the necessary JVM options:
[
{
"Classification": "spark-env",
"Configurations": [
{
"Classification": "export",
"Properties": {
"JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
}
}
]
},
{
"Classification": "spark-defaults",
"Properties": {
"spark.executorEnv.JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64",
"spark.driver.defaultJavaOptions": "-XX:OnOutOfMemoryError='kill -9 %p' -XX:MaxHeapFreeRatio=70",
"spark.executor.defaultJavaOptions": "-verbose:gc -Xlog:gc*::time -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:OnOutOfMemoryError='kill -9 %p' -XX:MaxHeapFreeRatio=70 -XX:+IgnoreUnrecognizedVMOptions"
}
}
]

For running Spark (3.3.0) on EMR 6 (6.9.0) I had to provide the following (note the additional "hadoop-env" and the empty "Properties" elements):
[
{
"Classification": "hadoop-env",
"Configurations": [
{
"Classification": "export",
"Configurations": [],
"Properties": {
"JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
}
}
],
"Properties": {}
},
{
"Classification": "spark-env",
"Configurations": [
{
"Classification": "export",
"Configurations": [],
"Properties": {
"JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
}
}
],
"Properties": {}
},
{
"Classification": "spark-defaults",
"Properties": {
"spark.executorEnv.JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
}
}
]

Recording sales amount in datadog metrics

I'm trying to record my website sales $ amount in datadog. However I'm getting way more than the actual value.
I'm using java-dogstatsd client and spring. My application is running on 3 hosts. I recorded all metrics (using sendWebOrder method) but no luck.
#EnableConfigurationProperties({DataDogProperties.class})
#Component
public class DDMetrics {
#Autowired
DataDogProperties dataDogProperties;
#Autowired
private NonBlockingStatsDClient statsd;
private Map<TopicPartition,Long> lags = new HashMap<>();
#Bean
private NonBlockingStatsDClient initClient() {
NonBlockingStatsDClient metricsClient = new NonBlockingStatsDClient(
dataDogProperties.getServiceName(),
dataDogProperties.getHostname(),
dataDogProperties.getPort();
return metricsClient;
}
public void sendWebOrder(WebOrder webOrder) {
List<String> tags = new ArrayList<>();
tags.add("transactionType:" + webOrder.getTransactionType());
tags.add("dataSourceType:" + webOrder.getDataSourceType()));
statsd.count("amount_count", webOrder.getAmount(), String.join(",", tags));
statsd.recordDistributionValue("amount_dist", webOrder.getAmount(), String.join(",", tags));
statsd.recordHistogramValue("amount_hist", webOrder.getAmount(), String.join(",", tags));
statsd.recordGaugeValue("amount_gauge", webOrder.getAmount(), String.join(",", tags));
statsd.incrementCounter("weborder", String.join(",", tags));
}
I'm trying to generate a datadog toplist by transactiontype. I'm not getting the correct amount in any of the metrics (tried mainly count, gauge and histogram.sum). Here is my datadog config:
{
"viz": "toplist",
"requests": [
{
"q": "top(sum:projecta.webtransactions.amount_histogram.sum{$TransactionType} by {transactiontype}, 10, 'sum', 'desc')",
"type": "area",
"style": {
"palette": "dog_classic",
"type": "solid",
"width": "normal"
},
"aggregator": "sum",
"conditional_formats": []
}
],
"autoscale": true
}
What am I missing? Is this the correct way to record money value? Do I've to do any rollup in config?
Any help is appreciated.

json file I/O with pretty print format using gson in java?

I already know how gson works and also know how to enable pretty
print.
I want to use gson and not simplejson.
The problem I had is that I wasn't able to create a file consisting of a List of Employee objects.
I've seen every other java threads in stackoverflow, mkyong, google's github and many others sites and I still wasn't able to accomplish this simple thing.
I already know how to read a file with this specific format but I wasn't able to write it.
The problem is I was not able to combine all these things together in a program.
A List of objects in gson with pretty print enabled must have the proper indentation, and every object must be separated with a comma and those objects must be wrapped between [ ] .
The problem explained with code
:
public class Employee implements Serializable {
private String lastName;
private String address;
private int id;
private String name;
}
I want to create a json file with the exact following content
[
{
"id":1,
"name": "John",
"lastName": "Doe",
"address": "NY City"
},
{
"id":2,
"name": "John",
"lastName": "Doe",
"address": "Canada"
},
{
"id":3,
"name": "John",
"lastName": "Doe",
"address": "Las Vegas"
},
]
I managed to create and write a json file of Person objects (as gson json Person objects), and read it, but again only as Person objects, where every line is an independent object, not a part of a List or Array of Person objects, like this
{"id":1,"name": "John","last": "Doe","address": "NY City"}
{"id":2,"name": "John","last": "Doe","address": "Canada"}
{"id":3,"name": "John","last": "Doe","address": "Las Vegas"}
but that's not what I want my final program to do.
I was also able to hard code a file with the following info and format and successfully obtain the Person objects
[
{
"id":1,
"name": "John",
"lastName": "Doe",
"address": "NY City"
},
{
"id":2,
"name": "John",
"lastName": "Doe",
"address": "Canada"
},
{
"id":3,
"name": "John",
"lastName": "Doe",
"address": "Las Vegas"
},
]
but I don't know how to create and write this json file with a java program
as an array of Person objects, where every Person object is a part of this
list or array with pretty print format, as the one I hard coded and
am able to read.
How can I do that in an elegant way?
EDIT---
Thanks a lot to #Shyam!
This is my final code, hope it helps someone.
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.reflect.TypeToken;
public class TestFileOfGsonWriter {
Gson gson ;
String filePath ;
BufferedReader bufferToReader ;
public TestFileOfGsonWriter()
{
this.filePath =
"C:\\FileOfGsonSingleListOfEmployees\\employees.json" ;
}
public List<Employee> createEmployees()
{
Employee arya = new Employee("Stark", "#81, 2nd main, Winterfell", 2, "Arya");
Employee jon = new Employee("Snow", "#81, 2nd main, Winterfell", 1, "Jon");
Employee sansa = new Employee("Stark", "#81, 2nd main, Winterfell", 3, "Sansa");
List<Employee> employees = new ArrayList<>();
employees.add(jon);
employees.add(arya);
employees.add(sansa);
return employees ;
}
public void jsonWriter(List<Employee> employees, String filePath)
{
this.gson = new GsonBuilder().setPrettyPrinting().create();
try(FileWriter writer = new FileWriter(filePath))
{
gson.toJson(employees,writer);
writer.close();
}
catch(IOException e)
{
e.printStackTrace();
}
}
public void showEmployeeObjects()
{
try {
List<Employee> employees = this.getAllEmployees();
employees.forEach(e -> Employee.showEmployeeDetails(e));
} catch (IOException e) {
e.printStackTrace();
}
}
public ArrayList<Employee> getAllEmployees() throws IOException
{
FileReader reader = new FileReader(this.filePath);
this.bufferToReader = new BufferedReader(reader) ;
ArrayList <Employee> employees = this.gson.fromJson(getJson(),
new TypeToken<ArrayList<Employee>>(){}.getType());
return employees ;
}
private String getJson() throws IOException
{
StringBuilder serializedEmployee = new StringBuilder();
String line ;
while ( (line = this.bufferToReader.readLine()) != null )
{
serializedEmployee.append(line);
}
this.bufferToReader.close();
return serializedEmployee.toString();
}
public static void main(String[] args) {
TestFileOfGsonWriter testWriting = new TestFileOfGsonWriter() ;
List<Employee> employees = testWriting.createEmployees();
testWriting.jsonWriter(employees, testWriting.filePath);
testWriting.showEmployeeObjects();
}
}
I modified my Employee class so it would match with his dummy objects which were better I think, this is how it looks now.
import java.io.Serializable;
public class Employee implements Serializable {
private static final long serialVersionUID = 1L;
String name ;
String address;
String lastName ;
int id ;
public static void showEmployeeDetails(Employee e)
{
System.out.println();
System.out.println("Employee's name: "+e.name);
System.out.println("Employee's last name"+e.lastName);
System.out.println("Employee's address: "+e.address);
System.out.println("Employee's ID: "+e.id);
}
public Employee(String myName, String myAddress, int myId, String myLastName)
{
this.name = myName ;
this.address = myAddress;
this.lastName = myLastName;
this.id = myId ;
}
}
So, the json file the program creates looks exactly how I wanted:
[
{
"name": "Snow",
"address": "#81, 2nd main, Winterfell",
"lastName": "Jon",
"id": 1
},
{
"name": "Stark",
"address": "#81, 2nd main, Winterfell",
"lastName": "Arya",
"id": 2
},
{
"name": "Stark",
"address": "#81, 2nd main, Winterfell",
"lastName": "Sansa",
"id": 3
}
]
and finally, this is the output:
Employee's name: Snow
Employee's last nameJon
Employee's address: #81, 2nd main, Winterfell
Employee's ID: 1
Employee's name: Stark
Employee's last nameArya
Employee's address: #81, 2nd main, Winterfell
Employee's ID: 2
Employee's name: Stark
Employee's last nameSansa
Employee's address: #81, 2nd main, Winterfell
Employee's ID: 3

As you are new, I'll quickly walk you through the process of writing a List of Employee objects to a JSON file with pretty printing:
Step 1: Create a method that takes in a List and a String filePath:
public void jsonWriter(List<Employee> employees, String filePath)
Step 2: Build a Gson Object with pretty printing enabled:
Gson gson = new GsonBuilder().setPrettyPrinting().create();
Step 3: Write your List<Employee> to a JSON file in the given filePath using a FileWriter:
try(FileWriter writer = new FileWriter(filePath)) {
gson.toJson(employees, writer);
writer.close();
} catch (IOException e) {
e.printStackTrace();
}
Finally the entire method should look something like this:
public void jsonWriter(List<Employee> employees, String filePath) {
Gson gson = new GsonBuilder().setPrettyPrinting().create();
try(FileWriter writer = new FileWriter(filePath)) {
gson.toJson(employees, writer);
writer.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Step 4: Now, build your Employee objects, add them to a List and call this method with the appropriate filePath
Employee arya = new Employee("Stark", "#81, 2nd main, Winterfell", 2, "Arya");
Employee jon = new Employee("Snow", "#81, 2nd main, Winterfell", 1, "Jon");
Employee sansa = new Employee("Stark", "#81, 2nd main, Winterfell", 3, "Sansa");
List<Employee> employees = new ArrayList<>();
employees.add(jon);
employees.add(arya);
employees.add(sansa);
jsonWriter(employees, "C:/downloads/employees.json");
After running this code, the contents of JSON file will look something like this:
[
{
"lastName": "Snow",
"address": "#81, 2nd main, Winterfell",
"id": 1,
"name": "Jon"
},
{
"lastName": "Stark",
"address": "#81, 2nd main, Winterfell",
"id": 2,
"name": "Arya"
},
{
"lastName": "Stark",
"address": "#81, 2nd main, Winterfell",
"id": 3,
"name": "Sansa"
}
]
I hope this will help you in your learning process.
Note: I've used some random Employee names and details. You can replace it with your required details.

Firstly store those objects into a list of objects. Then add this line of code for pretty print.
Gson gson = new GsonBuilder().setPrettyPrinting().create();
System.out.println(gson.toJson (list));

How can I deserialize JSON to Java Object using GSON when the JSON using dates as property names?

I have a JSON response that I want to deserialize using GSON. The JSON structure uses date strings as property names. How would one go about deserializing such.
{
2015-04-23: [{
seqNum: 1,
distance: 13,
start: "123 Main St"
end: "225 Broadway"
},
{seqNum: 2
distance: 21,
start: "225 Broadway"
end: "12 West St"
}
]
}

In my answer I considered you can have a JSON response like that:
{
"2015-04-23": [
{
"seqNum": 1,
"distance": 13,
"start": "123 Main St",
"end": "225 Broadway"
},
{
"seqNum": 2,
"distance": 21,
"start": "225 Broadway",
"end": "12 West St"
}
],
"2015-04-24": [
{
"seqNum": 1,
"distance": 13,
"start": "123 Main St",
"end": "225 Broadway"
},
{
"seqNum": 2,
"distance": 21,
"start": "225 Broadway",
"end": "12 West St"
}
]
}
So the first step would be to create the appropriate classes.
class MyObject {
private List<DateMapping> dateMappings;
public MyObject(List<DateMapping> dateMappings) {
this.dateMappings = dateMappings;
}
...
}
class DateMapping {
private Date date;
private List<Sequence> sequences;
public DateMapping(Date date, List<Sequence> sequences) {
this.date = date;
this.sequences = sequences;
}
...
}
class Sequence {
private int seqNum;
private int distance;
private String start;
private String end;
...
}
Now how can you parse the keys as they are dynamic? The answer is to use a custom deserializer:
class MyObjectAdapter implements JsonDeserializer<MyObject> {
private static final SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
private static final Type listSequenceType = new TypeToken<List<Sequence>>(){}.getType();
public MyObject deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
List<DateMapping> dateMappings = new ArrayList<>();
for(Map.Entry<String, JsonElement> entry : json.getAsJsonObject().entrySet()) {
try {
dateMappings.add(new DateMapping(dateFormat.parse(entry.getKey()), context.deserialize(entry.getValue(), listSequenceType)));
} catch (ParseException e) {
e.printStackTrace();
throw new RuntimeException("Can't parse the date");
}
}
return new MyObject(dateMappings);
}
}
From there you can see that I iterate over each entry of the top-level JsonObject from which you create a new DateMapping for each entry. Then you just have to register the adapter in the Gson parser and you're done.
Gson gson = new GsonBuilder().registerTypeAdapter(MyObject.class, new MyObjectAdapter()).create();
MyObject myObject = gson.fromJson(new FileReader(new File("myJson")), MyObject.class);
Running on the sample above produces the output:
MyObject =>
DateMapping Thu Apr 23 00:00:00 CEST 2015, sequences=[Sequence{seqNum=1, distance=13, start='123 Main St', end='225 Broadway'}, Sequence{seqNum=2, distance=21, start='225 Broadway', end='12 West St'}]
DateMapping Fri Apr 24 00:00:00 CEST 2015, sequences=[Sequence{seqNum=1, distance=13, start='123 Main St', end='225 Broadway'}, Sequence{seqNum=2, distance=21, start='225 Broadway', end='12 West St'}]
If you have only a single entry, you could skip the creation of the class MyObject, and get the first entry of the JsonObject from which you create a single DateMapping instance.
Hope it helps! :)