We have a big table with a lot of columns. After we moved to MySQL Cluster, the table cannot be created because of:
ERROR 1118 (42000): Row size too large. The maximum row size for the used table type, not counting BLOBs, is 14000. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs
As an example:
#Entity #Table (name = "appconfigs", schema = "myproject")
public class AppConfig implements Serializable
{
#Id #Column (name = "id", nullable = false)
#GeneratedValue (strategy = GenerationType.IDENTITY)
private int id;
#OneToOne #JoinColumn (name = "app_id")
private App app;
#Column(name = "param_a")
private ParamA parama;
#Column(name = "param_b")
private ParamB paramb;
}
It's a table for storing configuration parameters. I was thinking that we can combine some columns into one and store it as JSON object and convert it to some Java object.
For example:
#Entity #Table (name = "appconfigs", schema = "myproject")
public class AppConfig implements Serializable
{
#Id #Column (name = "id", nullable = false)
#GeneratedValue (strategy = GenerationType.IDENTITY)
private int id;
#OneToOne #JoinColumn (name = "app_id")
private App app;
#Column(name = "params")
//How to specify that this should be mapped to JSON object?
private Params params;
}
Where we have defined:
public class Params implements Serializable
{
private ParamA parama;
private ParamB paramb;
}
By using this we can combine all columns into one and create our table. Or we can split the whole table into several tables. Personally I prefer the first solution.
Anyway my question is how to map the Params column which is text and contains JSON string of a Java object?
You can use a JPA converter to map your Entity to the database.
Just add an annotation similar to this one to your params field:
#Convert(converter = JpaConverterJson.class)
and then create the class in a similar way (this converts a generic Object, you may want to specialize it):
#Converter(autoApply = true)
public class JpaConverterJson implements AttributeConverter<Object, String> {
private final static ObjectMapper objectMapper = new ObjectMapper();
#Override
public String convertToDatabaseColumn(Object meta) {
try {
return objectMapper.writeValueAsString(meta);
} catch (JsonProcessingException ex) {
return null;
// or throw an error
}
}
#Override
public Object convertToEntityAttribute(String dbData) {
try {
return objectMapper.readValue(dbData, Object.class);
} catch (IOException ex) {
// logger.error("Unexpected IOEx decoding json from database: " + dbData);
return null;
}
}
}
That's it: you can use this class to serialize any object to json in the table.
The JPA AttributeConverter is way too limited to map JSON object types, especially if you want to save them as JSON binary.
You don’t have to create a custom Hibernate Type to get JSON support, All you need to do is use the Hibernate Types OSS project.
For instance, if you're using Hibernate 5.2 or newer versions, then you need to add the following dependency in your Maven pom.xml configuration file:
<dependency>
<groupId>com.vladmihalcea</groupId>
<artifactId>hibernate-types-52</artifactId>
<version>${hibernate-types.version}</version>
</dependency>
Now, you need to declare the new type either at the entity attribute level or, even better, at the class level in a base class using #MappedSuperclass:
#TypeDef(name = "json", typeClass = JsonType.class)
And the entity mapping will look like this:
#Type(type = "json")
#Column(columnDefinition = "json")
private Location location;
If you're using Hibernate 5.2 or later, then the JSON type is registered automatically by MySQL57Dialect.
Otherwise, you need to register it yourself:
public class MySQLJsonDialect extends MySQL55Dialect {
public MySQLJsonDialect() {
super();
this.registerColumnType(Types.JAVA_OBJECT, "json");
}
}
And, set the hibernate.dialect Hibernate property to use the fully-qualified class name of the MySQLJsonDialect class you have just created.
If you need to map json type property to json format when responding to the client (e.g. rest API response), add #JsonRawValue as the following:
#Column(name = "params", columnDefinition = "json")
#JsonRawValue
private String params;
This might not do the DTO mapping for server-side use, but the client will get the property properly formatted as json.
It is simple
#Column(name = "json_input", columnDefinition = "json")
private String field;
and in mysql database your column 'json_input' json type
There is a workaround for those don't want write too much code.
Frontend -> Encode your JSON Object to string base64 in POST method, decode it to json in GET method
In POST Method
data.components = btoa(JSON.stringify(data.components));
In GET
data.components = JSON.parse(atob(data.components))
Backend -> In your JPA code, change the column to String or BLOB, no need Convert.
#Column(name = "components", columnDefinition = "json")
private String components;
In this newer version of spring boot and MySQL below code is enough
#Column( columnDefinition = "json" )
private String string;
I was facing quotes issue so I commented below line in my project
#spring.jpa.properties.hibernate.globally_quoted_identifiers=true
I had a similar problem, and solved it by using #Externalizer annotation and Jackson to serialize/deserialize data (#Externalizer is OpenJPA-specific annotation, so you have to check with your JPA implementation similar possibility).
#Persistent
#Column(name = "params")
#Externalizer("toJSON")
private Params params;
Params class implementation:
public class Params {
private static final ObjectMapper mapper = new ObjectMapper();
private Map<String, Object> map;
public Params () {
this.map = new HashMap<String, Object>();
}
public Params (Params another) {
this.map = new HashMap<String, Object>();
this.map.putAll(anotherHolder.map);
}
public Params(String string) {
try {
TypeReference<Map<String, Object>> typeRef = new TypeReference<Map<String, Object>>() {
};
if (string == null) {
this.map = new HashMap<String, Object>();
} else {
this.map = mapper.readValue(string, typeRef);
}
} catch (IOException e) {
throw new PersistenceException(e);
}
}
public String toJSON() throws PersistenceException {
try {
return mapper.writeValueAsString(this.map);
} catch (IOException e) {
throw new PersistenceException(e);
}
}
public boolean containsKey(String key) {
return this.map.containsKey(key);
}
// Hash map methods
public Object get(String key) {
return this.map.get(key);
}
public Object put(String key, Object value) {
return this.map.put(key, value);
}
public void remove(String key) {
this.map.remove(key);
}
public Object size() {
return map.size();
}
}
HTH
If you are using JPA version 2.1 or higher you can go with this case.
Link Persist Json Object
public class HashMapConverter implements AttributeConverter<Map<String, Object>, String> {
#Override
public String convertToDatabaseColumn(Map<String, Object> customerInfo) {
String customerInfoJson = null;
try {
customerInfoJson = objectMapper.writeValueAsString(customerInfo);
} catch (final JsonProcessingException e) {
logger.error("JSON writing error", e);
}
return customerInfoJson;
}
#Override
public Map<String, Object> convertToEntityAttribute(String customerInfoJSON) {
Map<String, Object> customerInfo = null;
try {
customerInfo = objectMapper.readValue(customerInfoJSON,
new TypeReference<HashMap<String, Object>>() {});
} catch (final IOException e) {
logger.error("JSON reading error", e);
}
return customerInfo;
}
}
A standard JSON object would represent those attributes as a HashMap:
#Convert(converter = HashMapConverter.class)
private Map<String, Object> entityAttributes;
Related
I have multiple objects in my array using . If I then send this to my Spring Boot backend with axios and output the FormData beforehand, I get the following image. That fits. In the backend, however, I need this list of objects as an entity. In this case, of type List. Do I do that?
Frontend code:
let data = new FormData();
...
data.append("zugeordnet", JSON.stringify(personNamen));
await axios.post("/neuerEintrag", data,...)
React:
Backend:
#PostMapping("/neuerEintrag")
public String neuerEintrag(HttpServletRequest req,#RequestParam("zugeordnet") List<?> zugeordnet,..) {
List<User> userListe = (List<User>) zugeordnet;
for(User inListe : userListe) //ERROR here
{
System.out.println("USER :" + inListe);
}
...
}
java.lang.ClassCastException: class java.lang.String cannot be cast to class com.home.calendar.User.User
UPDATE
For completeness, here is the user entity and the complete method for a new entry.
#PostMapping("/neuerEintrag")
public String neuerEintrag(HttpServletRequest req, #RequestParam("beschreibung") String beschreibung,
#RequestParam("datum") Date datum, #RequestBody List<User> zugeordnet,
#RequestBody List<Freunde> kontaktAuswahl, #RequestParam("neuAlt") String neuAlt,
#RequestParam("kalenderId") int kalenderId) { }
The User Entity:
#Entity
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private int id;
private String name;
#JsonIgnoreProperties("user")
#OneToMany(cascade = CascadeType.ALL, orphanRemoval = true, mappedBy = "user")
private List<Kalender> kalenderEinträge;
public User() {
super();
// TODO Auto-generated constructor stub
}
public User(String name, List<Kalender> kalenderEinträge) {
super();
this.name = name;
this.kalenderEinträge = kalenderEinträge;
}
public List<Kalender> getKalenderEinträge() {
return kalenderEinträge;
}
[getter/setter]
Spring can't parse an unknown object.
To get it work, I suggest a new class for the "request".
#Data // lombok - this generates getter/setters/equals/hashcode for you
public class NeuerEintragRequest {
private List<User> zugeordnet;
private String beschreibung;
private int kalendarId;
// and your others fields
}
The controller can now use very type-safe objects.
#PostMapping("/neuerEintrag")
public String neuerEintrag(#RequestBody NeuerEintragRequest request) {
for(User user : request.getUserlist()) {
// a logging framework is a lot better. Try to use log4j or slf4j.
log.info("USER: {}", user);
}
...
}
Typescript
Let axios handle the typing and serializing. See this tutorial: https://masteringjs.io/tutorials/axios/post-json
To post all the needed data, you can create a new object.
// no formdata - just send the object
const data = { zugeordnet: personNamen, kalendarId: 123, beschreibung: 'abc' };
await axios.post("/neuerEintrag", data);
You can also create a interface in typescript, but this is going to much for a stackoverflow-answer. Try to learn more about spring and typescript.
Based on question & comments ,
your front end call data.append("zugeordnet", JSON.stringify(personNamen)); is converting your object to List<String> instead of List<User>.
So you can transform this List<String> to List<User> in your postMapping:
#PostMapping("/neuerEintrag")
public String neuerEintrag(HttpServletRequest req,#RequestParam("zugeordnet") List<?> zugeordnet,..) {
ObjectMapper mapper=new ObjectMapper();
for(String str:zugeordnet){
System.out.println(mapper.readValue(str, User.class));
}
...
}
I'm trying to persist java object into jsonb column into Postgres database using eclipselink, but I am stuck at the moment on writing an appropriate converter. Can anyone help me or give me a good example of persisting jsonb types in java with eclipselink?
Here is model code:
#Entity
#Table(name = TABLE_NAME)
public class SystemEventModel implements Serializable {
public static final String TABLE_NAME = "system_event";
#Id
#Column(name = "id")
private Long id;
#Lob
#Column(name = "event_data", columnDefinition = "jsonb")
private JsonObject eventData;
public SystemEventModel(JsonObject eventData) {
this.eventData = eventData;
}
}
Currently, I just get the error "ERROR: column "event_data" is of type jsonb but the expression is of type bytea" when I try to persist this object. I'm aware why it is but I don't know how to write a converter since I don't understand in what type should I convert the JsonObject type for it to get passed like jsonb.
Here is what I mean in this converter sample:
#Converter(autoApply = true)
public class AppAttributeTypeAttributeConverter implements AttributeConverter<JsonObject, ???> {
#Override
public ??? convertToDatabaseColumn(JsonObject eventData) {
return ???;
}
#Override
public JsonObject convertToEntityAttribute(??? eventData) {
return ???;
}
}
And here is snippet for persisting:
String details = "{ \"pressed\": \"yes\", \"isDefault\": \"true\", \"type\": \"BUTTON\"}";
JsonReader jsonReader = Json.createReader(new StringReader(details));
JsonObject jsonObject = jsonReader.readObject();
jsonReader.close();
SystemEventModel eventModel = new SystemEventModel(jsonObject);
em.persist(eventModel);
Thanks in advance!
Here is what you can do that may work:
Change
#Lob
#Column(name = "event_data", columnDefinition = "jsonb")
private JsonObject eventData;
to
#Lob
#Type(type = "jsonb")
#Column(name = "event_data", columnDefinition = "jsonb")
private String eventData;
public SystemEventModel(String eventData) {
this.eventData = eventData;
}
Then persist by using this:
String details = "{ \"pressed\": \"yes\", \"isDefault\": \"true\", \"type\": \"BUTTON\"}";
SystemEventModel eventModal = new SystemEventModel(details); // or you can use setter method.
em.persist(eventModel);
I hope it helps!!
Refer How to map a String JPA property to a JSON column using Hibernate
Thanks...
EDIT:-
#Override
public Object convertToDatabaseColumn(JsonObject eventData) {
try {
PGobject out = new PGobject();
out.setType("json");
out.setValue(eventData.toString());
return out;
} catch (Exception e) {
throw new IllegalArgumentException("Unable to serialize to json field ", e);
}
}
#Override
public JsonObject convertToEntityAttribute(Object eventData) {
try {
if (eventData instanceof PGobject && ((PGobject) eventData).getType().equals("json")) {
return mapper.reader(new TypeReference<JsonObject>() {
}).readValue(((PGobject) eventData).getValue());
}
return Json.createObjectBuilder().build();
} catch (IOException e) {
throw new IllegalArgumentException("Unable to deserialize to json field ", e);
}
}
This is the Solution I found for JPA not specifically for hibernate or eclipselink.
My question is kind of similar to Prevent GSON from serializing JSON string but the solution there uses GSON library and I am restricted to using Jackson (fasterxml).
I have an entity class as follows:
package com.dawson.model;
import com.dawson.model.audit.BaseLongEntity;
import lombok.extern.log4j.Log4j;
import javax.persistence.*;
#Table(name = "queue", schema = "dawson")
#Entity
#Log4j
public class Queue extends BaseLongEntity {
protected String requestType;
protected String body;
protected Queue() {
}
public Queue(String requestType, String body) {
this.requestType = requestType;
this.body = body;
}
#Column(name = "request_type")
public String getRequestType() {
return requestType;
}
public void setRequestType(String requestType) {
this.requestType = requestType;
}
#Column(name = "body")
#Lob
public String getBody() {
return body;
}
public void setBody(String body) {
this.body = body;
}
}
I want to populate the body field with the json string representation of a map and then send this as part of the ResponseEntity. Something as follows:
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
mapper.writer().withDefaultPrettyPrinter();
HashMap<String, String> map = new HashMap<>(5);
map.put("inquiry", "How Can I solve the problem with Jackson double serialization of strings?");
map.put("phone", "+12345677890");
Queue queue = null;
try {
queue = new Queue("General Inquiry", mapper.writeValueAsString(map));
} catch (JsonProcessingException e) {
e.printStackTrace();
}
String test = mapper.writeValueAsString(map)
System.out.println(test);
Expected Output: "{"requestType": "General Inquiry","body": "{"inquiry":"How Can I solve the problem with Jackson double serialization of strings?","phone":"+12345677890"}"}"
Actual Output:"{"requestType": "General Inquiry","body": "{\"inquiry\":\"How Can I solve the problem with Jackson double serialization of strings?\",\"phone\":\"+12345677890\"}"}"
I am using
Jackson Core v2.8.2
I tried playing with
#JsonIgnore
and
#JsonProperty
tags but that doesn't help because my field is already serialized from the map when writing to the Entity.
Add the #JsonRawValue annotation to the body property. This makes Jackson treat the contents of the property as a literal JSON value, that should not be processed.
Be aware that Jackson doesn't do any validation of the field's contents, which makes it dangerously easy to produce invalid JSON.
I have a JPA entity object with following structure:
#Table(name="item_info")
class Item(){
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
private Long id;
#Column(name="item_name")
private String itemName;
#Column(name="product_sku")
private String productSku;
#Column(name="item_json")
private String itemJsonString;
#Transient
private ItemJson itemJson;
//Getters and setters
}
The itemJsonString field contains a json string value such as '{"key1":"value1","key2":"value2"}'
And the itemJson field contains the corresponding object which maps to the json string.
I get this entity object from database as follows:
Item item = itemRepository.findOne(1L); // Returns item with id 1
Now, the itemJson field is null since it is a transient field. And I have to set it manually using Jackson's ObjectMapper as follows:
itemJson = objectMapper.readValue(item.getItemJsonString(), ItemJson.class);
How can I make it such that when I do itemRepository.findOne(), it returns an Item object with the itemJson field mapped to the json String automatically?
Your best bet would be to implement a javax.persistence.Converter. It would look something like:
import javax.persistence.AttributeConverter;
import javax.persistence.Converter;
#Converter(autoApply = true)
public class LocalDateAttributeConverter implements AttributeConverter<ItemJson, String> {
#Override
public String convertToDatabaseColumn(ItemJson entityValue) {
if( entityValue == null )
return null;
ObjectMapper mapper = new ObjectMapper();
return mapper.writeValueAsString(entityValue);
}
#Override
public ItemJson convertToEntityAttribute(String databaseValue) {
if( databaseValue == null )
return null;
ObjectMapper mapper = new ObjectMapper();
return mapper.readValue(databaseValue, ItemJson.class);
}
}
I've used this with WildFly and didn't have to do anything except have it be in the war file I was deploying.
Here is the full working version of AttributeConverter + JPA + Kotlin.
Entity Class
In my case, database was mysql (8.x), which supports JSON as the underlying data type for column definition, and we can apply a custom converter using #Convert annotation.
#Entity
data class Monitor (
#Id
val id: Long? = null,
#Column(columnDefinition = "JSON")
#Convert(converter = AlertConverter::class)
var alerts: List<Alert> = emptyList(),
var active: Boolean = false
)
Converter Definition
Attribute converter needs to specify the conversion mechanism from data to db and reverse. We are using Jackson to convert a java object into String format and vice versa.
#Converter(autoApply = true)
class AlertConverter : AttributeConverter<List<Alert>, String> {
private val objectMapper = ObjectMapper()
override fun convertToDatabaseColumn(data: List<Alert>?): String {
return if (data != null && !data.isEmpty())
objectMapper.writeValueAsString(data)
else ""
}
override fun convertToEntityAttribute(dbData: String?): List<Alert> {
if (StringUtils.isEmpty(dbData)) {
return emptyList()
}
return objectMapper.readValue(dbData, object : TypeReference<List<Alert>>() {})
}
}
You could postLoad callback for manipulating entity after it's loaded. So try something like this inside your entity class
#PostLoad
public void afterLoad() {
ObjectMapper mapper = new ObjectMapper();
itemJson = mapper.readValue(item.getItemJsonString(), ItemJson.class);
}
I am trying to parse a jcelulas object that has a few many to one relationships. I can't seem to parse them correctly. Any help is appreciated.
Model:
#Entity
#Table(name = "jcelulas", catalog = "7jogos")
public class Jcelulas implements java.io.Serializable {
private Integer id;
private Jconcorrentes jconcorrentes;
private Jgrelhas jgrelhas;
private Jcodigos jcodigos;
private Jpremios jpremios;
private boolean checked;
private Date dataChecked;
// getters and setters
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "ConcorrentesId")
public Jconcorrentes getJconcorrentes() {
return this.jconcorrentes;
}
public void setJconcorrentes(Jconcorrentes jconcorrentes) {
this.jconcorrentes = jconcorrentes;
}
}
Controller:
#RequestMapping(value = "/jtabuleiros/play/commit",
method = RequestMethod.POST,
headers = {"Content-type=application/json"})
#ResponseBody
public JsonResponse playcelula(#ModelAttribute DataJson celula,#RequestBody String json) {
System.out.println(celula.toString());
System.out.println(json);
ObjectMapper mapper = new ObjectMapper();
try {
// read from file, convert it to user class
Jcelulas user = mapper.readValue(json, Jcelulas.class);
// display to console
System.out.println(user);
} catch (JsonGenerationException e) {
e.printStackTrace();
} catch (JsonMappingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return new JsonResponse("OK","");
}
Request:
{
"id":1,
"jconcorrentes":1,
"jgrelhas":1,
"jcodigos":1
}
How should I parse jconcorrentes? I tried as an int and got the following error:
org.codehaus.jackson.map.JsonMappingException: Can not instantiate value of type [simple type, class com.setelog.spring.model.Jconcorrentes] from JSON integral number; no single-int-arg constructor/factory method (through reference chain: com.setelog.spring.model.Jcelulas["jconcorrentes"])
Jconcorrentes:
#Entity
#Table(name = "jconcorrentes", catalog = "7jogos")
public class Jconcorrentes implements java.io.Serializable {
private Integer id;
....
private Date dataRegisto;
private Set<Jcelulas> jcelulases = new HashSet<Jcelulas>(0);
}
PS: These models were generated with hibernate from the mysql database
The problem is that jconcorrentes is being serialized as a number, not JSON Object. So Jackson does not know how to construct a Jconcorrentes out of value 1.