Develop Reference

Insert embeded document without reading whole document - spring, mongo

I have this factory collection :
#Document(collection = "factory")
public class Factory
{
Private List<Product> products;
}
which embeds the Product as products.
When I have to add a product to an existing factory :
#Autowired
private FactoryRepository factoryRepository;
public void addProduct(Long id, Product product) {
Factory f = factoryRepository.findById(id);
f.addProduct(product);
factoryRepository.save(f);
}
However, the issue is that product is a large object which contains a set of heavy attributes and the factory can have 2000 products.
So, the retrieved factory causes large memory consumption although it is not required in this phase. Is there a way to append a new product object directly into the factory document without reading the whole object?
EDIT:
As for the comments, I tried :
public void addProduct(Long id, Product product) {
Document find = new Document("_id",id);
Document listItem = new Document("products",product);
Document push = new Document("$push", listItem);
collection.updateOne(find,push);
}
This gives error :
org.bson.codecs.configuration.CodecConfigurationException:
Can't find a codec for class product
So I modified to convert it to a string before push :
public void addProduct(Long id, Product product) {
Document find = new Document("_id",id);
ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
Document listItem = new Document("products",ow.writeValueAsString(product));
Document push = new Document("$push", listItem);
collection.updateOne(find,push);
}
This pushed object correctly but when reading :
org.springframework.core.convert.ConverterNotFoundException:
No converter found capable of converting from type [java.lang.String] to type [Product]
Still, I got nowhere here. Any ideas on fixing this issue?

You should use MongoTemplate to update product with push to add to existing products. Something like
package com.example.demo;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.ApplicationRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.Update;
import java.util.List;
#SpringBootApplication
public class So62173077Application {
public static void main(String[] args) {
SpringApplication.run(So62173077Application.class, args);
}
#Autowired
private MongoTemplate mongoTemplate;
#Document(collection = "factory")
public class Factory
{
private Long id;
private List<Product> products;
}
public Long createFactory() {
Factory factory = new Factory();
factory.id = 1L;
return mongoTemplate.insert(factory).id;
}
public void addProduct(Long id) {
Query query = new Query();
query.addCriteria(Criteria.where("id").is(id));
Update update = new Update();
Product product = new Product();
product.name = "stackoverflow";
update.push("products", product);
mongoTemplate.updateFirst(query, update, Factory.class);
}
private class Product {
private String name;
}
#Bean
public ApplicationRunner runner() {
return args -> {
//Long id = createFactory();
addProduct(1L);
};
}
}

MongoDB large array may be discouraged (fetching, atomic operations, complex querying).
Depending on your needs but I would suggest you to deal with Product in a brand new "product" collection. Your current issue will be solved as well.
Regards.

How to not write Option.None with jackson objectMapper (and read it)?

I use jackson ObjectMapper to serialize and deserialize some data of mine, which have fields of javaslang Option type. I use JavaslangModule (and Jdk8Module). And when it write the json, Option.None value fields are written as null.
To reduce the json size and provide some simple backward compatibility when later adding new fields, what I want is that:
fields with Option.None value are simply not written,
missing json fields that correspond to data model of Option type, be set to Option.None upon reading
=> Is that possible, and how?
Note:
I think that not-writing/removing null json fields would solve (1). Is it possible? And then, would reading it works (i.e. if model field with Option value is missing in the json, set it None?

Luckily there is a much simpler solution.
1) In your ObjectMapper configuration, set serialization inclusion to only include non absent field:
#Bean
public ObjectMapper objectMapper() {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.registerModules(vavr());
objectMapper.setSerializationInclusion(NON_ABSENT);
return objectMapper;
}
2) Set the default value of your optional fields to Option.none:
#Data
#AllArgsConstructor
#NoArgsConstructor
public class Foo {
private Option<String> bar = Option.none(); // If the JSON field is null or not present, the field will be initialized with none
}
That's it!
And the even better news is that it works for all Iterables, not just for Option. In particular it also works for Vavr List type!

I found a solution that works with immuatble (lombok #Value) models:
add a filter on all Object using mixIn that doesn't write Option.None (see "the solution" below)
my existing ObjectMapper (with JavaslangModule) is already setting None to Option field when the corresponding json entry is missing
The code
import static org.assertj.core.api.Assertions.assertThat;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonFilter;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectWriter;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.ser.PropertyWriter;
import com.fasterxml.jackson.databind.ser.impl.SimpleBeanPropertyFilter;
import com.fasterxml.jackson.databind.ser.impl.SimpleFilterProvider;
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module;
import javaslang.control.Option;
import javaslang.jackson.datatype.JavaslangModule;
import lombok.AllArgsConstructor;
import lombok.Value;
import org.junit.Test;
import java.io.IOException;
import java.lang.reflect.Field;
public class JsonModelAndSerialization {
// Write to Json
// =============
private static ObjectMapper objectMapper = new ObjectMapper()
.registerModule(new Jdk8Module())
.registerModule(new JavaslangModule())
// not required but provide forward compatibility on new field
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
static String write(Object data) throws JsonProcessingException {
SimpleBeanPropertyFilter filter = new NoneOptionPropertyFilter();
objectMapper.addMixIn(Object.class, NoneOptionFilter.class);
final SimpleFilterProvider filters = new SimpleFilterProvider().setDefaultFilter(filter);
ObjectWriter writer = objectMapper.writer(filters);
return writer.writeValueAsString(data);
}
// Filter classes
// ==============
#JsonFilter("Filter None")
private static class NoneOptionFilter {}
private static class NoneOptionPropertyFilter extends SimpleBeanPropertyFilter {
#Override
public void serializeAsField(
Object pojo, JsonGenerator jgen,
SerializerProvider provider, PropertyWriter writer) throws Exception{
Field field = pojo.getClass().getDeclaredField(writer.getName());
if(field.getType().equals(Option.class)){
field.setAccessible(true);
Option<?> value = (Option<?>) field.get(pojo);
if(value.isEmpty()) return;
}
super.serializeAsField(pojo, jgen, provider, writer);
}
}
// Usage example
// =============
// **important note**
// For #Value deserialization, a lombok config file should be added
// in the source folder of the model class definition
// with content:
// lombok.anyConstructor.addConstructorProperties = true
#Value
#AllArgsConstructor(onConstructor_={#JsonCreator})
public static class StringInt {
private int intValue;
private Option<String> stringValue;
}
#Value
#AllArgsConstructor(onConstructor_={#JsonCreator})
public static class StringIntPair {
private StringInt item1;
private StringInt item2;
}
#Test
public void readWriteMyClass() throws IOException {
StringIntPair myClass = new StringIntPair(
new StringInt(6 * 9, Option.some("foo")),
new StringInt( 42, Option.none()));
String json = write(myClass);
// {"item1":{"intValue":54,"stringValue":"foo"},"item2":{"intValue":42}}
StringIntPair myClass2 = objectMapper.readValue(json, StringIntPair.class);
assertThat(myClass2).isEqualTo(myClass);
}
}
The advantages:
reduce size of json when having Option.None (thus adding Option fields in the model doesn't cost size when not used)
it provides backward reading compatibility when later adding field with Option type in the model (which will default to None)
The disadvantage:
It is not possible to differentiate correct data with None field value and incorrect data where the field has erroneously been forgotten. I think this is quite acceptable.

Duplicate values in the output of ObjectMapper.writeValueAsString

I am using Jackson ObjectMapper to (de)serialize a class with polymorphic nested class. The deserialization of JSON to the class is working fine but when I try to serialize the class to JSON using writeValueAsString function I observe duplicate values in the output
public class Movie {
private String movieName;
#JsonTypeInfo(use=Id.NAME,include=As.EXTERNAL_PROPERTY,property="movieName")
#JsonSubTypes({#JsonSubTypes.Type(value = StarWarsParams.class, name = "starwars")})
private MovieParams movieParams;
/* Getters and setters follow */
}
/* Empty class */
public class MovieParams {
}
public class StarWarsParams extends MovieParams{
private String characterName;
#JsonTypeInfo(use=Id.NAME,include=As.EXTERNAL_PROPERTY,property="characterName")
#JsonSubTypes({#JsonSubTypes.Type(value = SithParameters.class, name = "Darth Vader")})
private CharacterParams characterParams;
/* Getters and setters follow */
}
/* Empty class */
public class CharacterParams {
}
public class SithParameters extends CharacterParams {
private boolean canShootLightning;
}
The code snippet where the conversion is done as follows:
Movie movie = new Movie();
movie.setMovieName("starwars");
StarWarsParams starWarsParams = new StarWarsParams();
starWarsParams.setCharacterName("Darth Vader");
SithParameters sithParameters = new SithParameters();
sithParameters.setCanShootLightning(false);
starWarsParams.setCharacterParams(sithParameters);
movie.setMovieParams(starWarsParams);
ObjectMapper mapper = new ObjectMapper();
String jsonStringSample = mapper.writeValueAsString(movie);
System.out.println(jsonStringSample);
The output, in which movieName and characterName have duplicates are as follows:
{"movieName":"starwars","movieParams":{"characterName":"Darth Vader","characterParams":{"canShootLightning":false},"characterName":"Darth Vader"},"movieName":"starwars"}

This problem appears with older versions of Jackson e.g. 1.9.2 but not the latest ones from com.fasterxml. Jackson identifies 2 fields one from the #JsonTypeInfo annotation and one from the getter. Two solutions :
Use a more recent version of Jackson from com.fasterxml
Move the #JsonTypeInfo annotation over the getter instead of over the field e.g.
#JsonTypeInfo(use = Id.NAME, include = As.EXTERNAL_PROPERTY, property = "characterName")
public String getCharacterName() {
return characterName;
}

Customized JSON Object using Serialization is Very Simple.
I have wrote a class in my project to get Serialized JSONObject. i am giving u a Idea to how to Implement this in Project.
Application (POJO Class)
import java.io.Serializable;
import java.util.List;
import org.webservice.business.serializer.ApplicationSerializer;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
#JsonSerialize(using=ApplicationSerializer.class)
public class Application implements Serializable {
private static final long serialVersionUID = 1L;
private double amount;
private String businessType;
private String currency;
private int duration;
}
Now ApplicationSerializer class that contains the Customization using Serialization Logic................
package org.webservice.business.serializer;
import java.io.IOException;
import org.webservice.business.dto.Application;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
public class ApplicationSerializer extends JsonSerializer<Application> {
#Override
public void serialize(Application prm_objObjectToSerialize, JsonGenerator prm_objJsonGenerator, SerializerProvider prm_objSerializerProvider) throws IOException, JsonProcessingException {
if (null == prm_objObjectToSerialize) {
} else {
try {
prm_objJsonGenerator.writeStartObject();
prm_objJsonGenerator.writeNumberField("amount", prm_objObjectToSerialize.getAmount());
prm_objJsonGenerator.writeNumberField("duration", prm_objObjectToSerialize.getDuration());
prm_objJsonGenerator.writeStringField("businesstype", prm_objObjectToSerialize.getBusinessType());
prm_objJsonGenerator.writeStringField("currency", prm_objObjectToSerialize.getCurrency());
} catch (Exception v_exException) {
v_exException.printStackTrace()
} finally {
prm_objJsonGenerator.writeEndObject();
}
}
}

Array Serialize Gives Qualified Class name using Jackson

I am using JSON Serialization. Here is my code.
I need to change the qualified class name using Annotation. I don't have to use Map or another class. Name should be picked from Annotation.
package com.test;
import com.fasterxml.jackson.annotation.JsonRootName;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.annotation.JsonTypeName;
#JsonTypeName("Product")
#JsonRootName("Product")
#JsonTypeInfo(use=JsonTypeInfo.Id.NAME, include=JsonTypeInfo.As.PROPERTY, property="type")
public class ProductDTO {
private String name;
private String description;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
}
Test Class:-
package com.test;
import java.io.IOException;
import java.util.ArrayList;
import org.junit.Test;
import com.fasterxml.jackson.core.JsonGenerationException;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class ProductDTOTestCase {
#Test
public void testPersistAndFindById() throws JsonGenerationException, JsonMappingException, IOException {
ObjectMapper mapper = new ObjectMapper();
mapper.enableDefaultTyping();
//mapper.enableDefaultTyping(ObjectMapper.DefaultTyping.OBJECT_AND_NON_CONCRETE, JsonTypeInfo.As.WRAPPER_ARRAY);
ProductDTO productDTO = new ProductDTO();
productDTO.setDescription("Product 4 - Test");
ArrayList<ProductDTO> arrayList = new ArrayList<ProductDTO>();
arrayList.add(productDTO);
// Do not change this line
String writeValueAsString = mapper.writeValueAsString(arrayList);
System.out.println(writeValueAsString);
// /Assert.assertTrue(writeValueAsString.contentEquals("Entity"));
}
}
It gives me:-
[["com.test.ProductDTO",{"name":null,"description":"Product 4 - Test"}]]
But I want
[["Product",{"name":null,"description":"Product 4 - Test"}]]

Problem is that you serialize basically a generic ArrayList (the Class of your ArrayList, since no generic info is available).
As a consequence, our basic List.class has simply no #JsonTypeInfo annotation or whatsoever about the items contained, no type information is available, that's why you get the simple name of your class.
Solutions:
Use custom sub-class like "class MyList extends ArrayList { }" --> it will NOT suffer from this kind of type erasure (generic type info is hidden, but retained) and will work as expected, OR
Use ObjectWriter and specify full generic type in your serialization: mapper.writerForType(listType).writeValueAsString() (listType you can construct using TypeFactory or using TypeReference)

Spring - How to use BeanPropertyRowMapper without matching column names

I work on an application that has been converted from pure JDBC to Spring template with row mapper. The issue that I have is that the column in database doesn't match the property names which prevent me from using BeanPropertyRowMapper easily.
I saw some posts about using aliases in queries. This would work but it makes it impossible to do a SELECT *
Isn't there any annotation that can be used with BeanPropertyRowMapper as #Column from JPA?

I saw Some posts about using aliases in queries
This is actually an approach suggested in JavaDocs:
To facilitate mapping between columns and fields that don't have matching names, try using column aliases in the SQL statement like "select fname as first_name from customer".
From: BeanPropertyRowMapper.
impossible to do a SELECT *
Please do not use SELECT *. This makes you vulnerable to any database schema change, including completely backward compatible ones like adding or rearranging columns.
Isn't there any annotation that can be used with BeanPropertyRowMapper as #Column from JPA?
Yes, it is called jpa, hibernate and maybe ibatis. Seriously, either use aliases or implement your own RowMapper, Spring is not a full-featured orm.

You can override the BeanPropertyRowMapper.underscoreName, and get the name of the Column annotation to mapping the field with #Column(name = "EXAMPLE_KEY") in the PropertyDescriptor(getter/setter binding).
#Slf4j
public class ColumnRowMapper<T> extends BeanPropertyRowMapper<T> {
private ColumnRowMapper(final Class<T> mappedClass)
{
super(mappedClass);
}
#Override
protected String underscoreName(final String name)
{
final Column annotation;
final String columnName;
Field declaredField = null;
try
{
declaredField = getMappedClass().getDeclaredField(name);
}
catch (NoSuchFieldException | SecurityException e)
{
log.warn("Ups, field «{}» not found in «{}».", name, getMappedClass());
}
if (declaredField == null || (annotation = declaredField.getAnnotation(Column.class)) == null
|| StringUtils.isEmpty(columnName = annotation.name()))
{
return super.underscoreName(name);
}
return StringUtils.lowerCase(columnName);
}
/**
* New instance.
*
* #param <T> the generic type
* #param mappedClass the mapped class
* #return the bean property row mapper
*/
public static <T> BeanPropertyRowMapper<T> newInstance(final Class<T> mappedClass)
{
return new ColumnRowMapper<>(mappedClass);
}
}

A version of above mapper but with early initiation of mapping index, since reflection is way too slow:
import java.lang.reflect.Field;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import javax.persistence.Column;
import lombok.NonNull;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang.StringUtils;
import org.springframework.jdbc.core.BeanPropertyRowMapper;
#Slf4j
public class ColumnRowMapper<T> extends BeanPropertyRowMapper<T> {
private Map<String, String> columnIndex;
private ColumnRowMapper(final Class<T> mappedClass)
{
super(mappedClass);
}
#Override
protected void initialize(Class<T> mappedClass) {
columnIndex = new ConcurrentHashMap<>();
for (Field f: mappedClass.getDeclaredFields()) {
String fieldName = f.getName();
Column annotation = f.getAnnotation(Column.class);
if (annotation == null) {
continue;
}
String columnName = annotation.name();
if (StringUtils.isEmpty(columnName)) {
continue;
}
columnIndex.put(fieldName, StringUtils.lowerCase(columnName));
}
super.initialize(mappedClass);
}
#Override
protected #NonNull String underscoreName(final #NonNull String name)
{
if (columnIndex.containsKey(name)) {
return columnIndex.get(name);
}
return super.underscoreName(name);
}
/**
* New instance.
*
* #param <T> the generic type
* #param mappedClass the mapped class
* #return the bean property row mapper
*/
public static <T> BeanPropertyRowMapper<T> newInstance(final Class<T> mappedClass)
{
return new ColumnRowMapper<>(mappedClass);
}
}