Rename columns in spark using #JsonProperty while creating Datasets - java

Is there way to rename the column names in dataset using Jackson annotations while creating a Dataset?
My encoder class is as follows:
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;
import scala.Serializable;
import javax.persistence.Table;
#Builder
#Data
#AllArgsConstructor
#EqualsAndHashCode
#Table(name = "sample_table")
public class SampleRecord implements Serializable {
#JsonProperty("sample_id")
private Long sampleId;
#JsonProperty("sample_name")
private String name;
#JsonProperty("sample_desc")
private String description;
}
My aim is to rename the columns according to the #JsonProperty, so that I can re-use the same class and json functionality.
Please find related versions of modules:
- Spark : 2.4.0 (with scala 2.11)
- jackson-module-scala_2.11 : 2.9.6
Let me know if you need more information. Help appreciated.

public class SampleRecord implements Serializable {
private Long sampleId;
private String name;
private String description;
#JsonProperty("sample_id")
public void setSampleId(Long sampleId) {
this.sampleId = sampleId;
}
#JsonProperty("sample_name")
public void setName(String name) {
this.name = name;
}
#JsonProperty("sample_desc")
public void setDescription(String description) {
this.description = description;
}
}

Interesting idea. The way I would do it:
Ingest your data in a dataframe.
Write a utility method that takes the dataframe and class name (here SampleRecord).
Use introspection to read the annotations (you could eventually add some if you need to define specific properties).
Rename the columns with withColumnRenamed() on the dataframe.
Return the modified dataframe.
hih

Related

How to load a combobox with vaadin fusion

Hi I'm using the vaadin starter in order to learn more about vaadin.
I just started a new project (Java+Typescript)
I am having issues to solve an issue.
I have a Users and Rol Entity, being Rol an attribute of User, the thing is when I am setting the views created with vaading start I am trying to set up a combo box to load Roles to be used to create a new user but nothing work so far.
In the tutorial in vaading web page they solve this in a way that is way different to the arch and files created by the vaadin start so I thought that maybe would be another way to do it.
My entities
User
package com.example.application.data.entity;
import com.vaadin.fusion.Nonnull;
import com.example.application.data.AbstractEntity;
import javax.persistence.Entity;
import javax.persistence.ManyToOne;
#Entity
public class Users extends AbstractEntity {
#ManyToOne
#Nonnull
private Rol rol;
public Rol getRol() {
return rol;
}
public void setRol(Rol rol) {
this.rol = rol;
}
}
Rol
package com.example.application.data.entity;
import com.vaadin.fusion.Nonnull;
import com.example.application.data.AbstractEntity;
import javax.persistence.Entity;
#Entity
public class Rol extends AbstractEntity{
#Nonnull
private String name;
#Nonnull
private String description;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
}
What should I do to load this with all roles in order to select one in my users-view.ts
<vaadin-combo-box label="Rol" id="rol" ${field(this.binder.model.rol)} item-label-path="name"></vaadin-combo-box>
Right now I'getting this
How the combobox shows
Thanks in advance guys.
My Solution was:
added tis lines to my typescrypt class
#state()
private roles: Rol[] = [];
#state from 'lit/decorators.js'
then in the connectedCallback function added this line
this.roles = await RolesEndpoint.listAll();
listAll() is a method that I created on my endpint class.
like this:
#Nonnull
public List<#Nonnull Rol> listAll() {
return service.listAll();
}
and in my service class
public List<Rol> listAll() {
return repository.findAll();
}
Now you can call the data in your combobox element
<vaadin-combo-box .items=${this.roles} label="Rol" id="rol" ${field(this.binder.model.rol)} item-label-path="name" item-value-path="id"></vaadin-combo-box>
I hope this can be helpful.

There is an empty array in the http response body if I just return an array of Java Instances in Spring Boot

I return an array of Java Instances in my Spring-Boot-Get-Started project.
package com.wepay.business.resource;
import com.wepay.business.model.Good;
import com.wepay.business.repo.GoodRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.util.ArrayList;
import java.util.List;
#CrossOrigin(origins = {"http://localhost:3000", "http://localhost:9000", "http://localhost:8083"})
#RestController
#RequestMapping("/api")
public class GoodResource {
#Autowired
GoodRepository repository;
#GetMapping("/getGood")
public List<Good> getAllGoods() {
List<Good> goods = new ArrayList<>();
repository.findAll().forEach(goods::add);
return goods;
}
}
package com.wepay.business.repo;
import com.wepay.business.model.Good;
import org.springframework.data.repository.CrudRepository;
public interface GoodRepository extends CrudRepository<Good, Long> {
}
package com.wepay.business.model;
import javax.persistence.*;
#Entity
#Table(name = "good")
public class Good {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#Column(name = "name")
private String name;
#Column(name = "price")
private double price;
#Column(name = "img")
private String img;
#Column(name = "info")
private String info;
#Column(name = "amount")
private int amount;
#Column(name = "address")
private String address;
#Column(name = "soldAmount")
private String soldAmount;
#Column(name = "sellerId")
private String sellerId;
public Good(){
}
public Good(String name, Double price, String info, int amount) {
this.name = name;
this.price = price;
this.info = info;
this.amount = amount;
}
public Good(Long id, String goodName, Double unitPrice, String goodInfo, int amount) {
this(goodName, unitPrice, goodInfo, amount);
this.id = id;
}
public void setId(Long id) {
this.id = id;
}
}
The value of goods is an array of Java Instacnes
But there is only an empty array in the http response body.
I guess that I should return an array of JSON objects rather than Java Instances.
Do I need to convert the Java Instances to JSON objects? If so, is there any framework to help us to do this job?
I have been blocked by this issue since last week. Thanks in advance.
The problem resides in the fact that your Good class has no getters (atleast by what I see in your post). Add the getters and this should work.
I think you can use JpaRepository<T, ID> instead of CrudRepository<T, ID> so in this case there's no need to instantiate another List<Good>, because the repository.findAll() already returns List<Good> inside the JpaRepository, although by the way you're doing, it should also work normally.
Do I need to convert the Java Instances to JSON objects? If so, is there any framework to help us to do this job?
No. Spring already do it for you by using Jackson's serializer.
try return repository.findAll();
If there is no specific reason to use CrudRepository you can change it to JpaRepository
By doing this you can avoid conversion of Iterator to List and use like this.
public interface GoodRepository extends JpaRepository<Good, Long> {
}
// Controller
#GetMapping("/getGood")
public List<Good> getAllGoods() {
return repository.findAll();
}
Also, Make sure Getter Setter is in place for each persistable field.

Neo4j OGM & Multiple similar classes

I'm using Neo4j ogm to map many (over 20) similar classes into neo4j db which are different in just
relationship name, name and direction.
Each class implements the "Classification" interface with just one method which is the same
in every class (consist on adding relation into collection of node)
Example node:
#NodeEntity
public class ExampleClass implements Classification {
#GraphId
private Long id;
private String name;
#Relationship(type = "EXAMPLE_1", direction = "OUTGOING")
private Set<Species> classification = new HashSet<>();
public ExampleClass (){}
public ExampleClass (String name) {
this.name = name;
}
public Set<Species> getClassification(){
return classification;
}
#Override
public void specifiedAs(Species species){
classification.add(species);
}
and analogously:
#NodeEntity
public class ExampleClass2 implements Classification {
#GraphId
private Long id;
private String name;
#Relationship(type = "EXAMPLE_2", direction = "OUTGOING")
private Set<Species> classification = new HashSet<>();
public ExampleClass2 (){}
public ExampleClass2 (String name) {
this.name = name;
}
public Set<Species> getClassification(){
return classification;
}
#Override
public void specifiedAs(Species species){
classification.add(species);
}
}
I'm looking for possibility to reduce count of those similar classes
and create... maybe one generic class in which I can define label,property name and realtionship type also.
I prefer still using spring-data and Neo4j OGM.
You could improve this by introducing an super class containing all the common properties, and just have the specific relationships in your ExampleClassXXX classes.
Note that the relationship types cannot be dynamic, so you cannot have just a generic class by itself.

Duplicate Values while using ObjectMapper of Jackson

My Bean class is as below. When the mapping happens, the JSON object contains duplicate values.
Response:
{"Id":"00PJ0000003mOgMMAU","Name":"web.xml","name":"web.xml","id":"00PJ0000003mOgMMAU"}
Why the values are getting duplicated?
import org.codehaus.jackson.annotate.JsonIgnoreProperties;
import org.codehaus.jackson.annotate.JsonProperty;
#JsonIgnoreProperties(ignoreUnknown = true)
public class AttachmentsMapper
{
#JsonProperty(value = "Id")
private String Id;
#JsonProperty(value = "Name")
private String Name;
public String getId() {
return Id;
}
public void setId(String Id) {
this.Id = Id;
}
public String getName() {
return Name;
}
public void setName(String Name) {
this.Name = Name;
}
}
It doesn't print duplicate the same field twice it prints 2 different fields that it finds. Jackson sees you want to print "name" because you have a getter called getName() and "Name" because you have annotated the Name field as #JsonProperty with a different key. It sees different fields because "name" != "Name". Two solutions :
Move the annotation to the getter. The field is ignored by default because it's private E.g.
#JsonProperty(value = "Name")
public String getName() {
return Name;
}
Use a more recent version of Jackson as you seem to be using 1.8 from com.codehaus. Use 1.9 from there or even better use the latest from com.fasterxml. I tried your code as it is with 1.9 and it worked without moving the annotation.
In Jackson 2 try to disable Jackson visibility for all the sources (getters, setters, fields, etc.) and then just enable the visibility for the object fields:
import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.ObjectMapper;
...
ObjectMapper mapper = new ObjectMapper();
mapper.setVisibility(PropertyAccessor.ALL, Visibility.NONE);
mapper.setVisibility(PropertyAccessor.FIELD, Visibility.ANY);

Object properties lowercased when reading as JSONObject

I have the following problem. I'm reading a list of records from my MySQL database with Hibernate template, and then I need to modify the structure so I'm JSONObject and JSONArray (using I guess the official library : http://www.json.org/java/). If I'm using the List as a server response, records fields are properly named (thanks to #JsonProperty annotation used). But if I'm trying to create a JSONObject out of this List element, I'm getting all my fields starting with small letter, which breaks my UI.
This is my 'Task' model used :
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.Table;
import org.codehaus.jackson.annotate.JsonAutoDetect;
import org.codehaus.jackson.annotate.JsonProperty;
import org.codehaus.jackson.annotate.JsonIgnoreProperties;
#JsonAutoDetect
#JsonIgnoreProperties(ignoreUnknown = true)
#Entity
#Table(name="tasks")
public class Task {
#Id
#GeneratedValue
#Column(name="Id")
private int Id;
#Column(name="Name", nullable=false)
private String Name;
#JsonProperty("Id")
public int getId() {
return Id;
}
#JsonProperty("Id")
public void setId(int id) {
this.Id = id;
}
#JsonProperty("Name")
public String getName() {
return Name;
}
#JsonProperty("Name")
public void setName(String name) {
this.Name = name;
}
}
and here's the code used for getting records from the DB (stripped of all the unnecessary parts):
public List<Task> getEvents() {
DetachedCriteria criteria = DetachedCriteria.forClass(Task.class);
return hibernateTemplate.findByCriteria(criteria);
}
private static JSONArray read() throws JSONException{
List<Task> list = getEvents();
Iterator<Task> listIterator = list.iterator();
JSONArray ret = new JSONArray();
String parentId;
while(listIterator.hasNext()){
Task task = listIterator.next();
JSONObject taskJSON = new JSONObject(task);
ret.put(taskJSON);
}
}
As you can see in my server response, all fields names start with small letter :
{"id":18,"name":"Release"}
Any ideas how to override this ?
Your class is overannotated, and breaks Java code conventions.
The minimum required is as follows. Everything else you've added is done by default.
#Entity
#Table(name="tasks")
public class Task {
#Id
#GeneratedValue
#Column(name="Id")
#JsonProperty("Id")
private int id;
#Column(name="Name", nullable=false)
#JsonProperty("Name")
private String name;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
To serialise your class all you should need is the ObjectMapper class
String json = new ObjectMapper().writeValueAsString(getEvents());
The output of which should look like:
[{"Id":18,"Name":"Build"}, {"Id":19,"Name":"Release"}]
I would discourage using capitalised property names if possible as it goes against general code conventions.
The JSON.org API is intended for very simple serialization/deserialization, it can't do what your looking for. Having said that, the majority of your annotations are actually from Jackson, which can do what your trying to accomplish.
You already have the POJOs properly annotated for Jackson, so return a JSON string conforming to them, serialize using an ObjectMapper:
final List<Task> list = getEvents();
final ObjectMapper mapper = new ObjectMapper();
final String json = mapper.writeValueAsString(list);

Categories

Resources