I have a Spring Boot app which is using MySQL db.
At the moment I'm trying to do the following:
- deserialize instances from the *.csv files;
- inject them into the MySQL db.
For the simple instances there are no issues. But in case if I have an object with ManyToMany or OneToMany relations, deserialization is not working correctly. Currently I'm using Jackson dependency for *.csv deserialization:
CsvMapper csvMapper = new CsvMapper();
csvMapper.disable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY);
CsvSchema csvSchema = csvMapper.schemaFor(type).withHeader().withColumnSeparator(';').withLineSeparator("\n");
MappingIterator<Object> mappingIterator = csvMapper.readerFor(type).with(csvSchema).readValues(csv);
List<Object> objects = new ArrayList<>();
while (mappingIterator.hasNext()) {
objects.add(mappingIterator.next());
}
Example of the instance with many to many: (Idea is that one app can have different versions)
public class Application {
private Long id;
private String name;
#OneToMany(mappedBy = "application")
private Set<Version> versions = new HashSet<>();
}
For insertion into the DB I'm using Spring Boot entities that are #Autowired.
My first question is - what should I put as an input into the CSV file to deserialize it correctly ? Because if I have :
id;name;
1;testName;
(skipping versions), I'm having a trouble. The same even if I try to put some values into the version. So I don't know how to provide correctly the input for Jackson CSV deserialization in case of SET + later, how can I persist this entity ? Should I first put all the versions into the DB and then try to put applications?
Any thoughts? Thanks in advance!
Use ApacheCommons to parse csv.
final byte[] sourceCsv;
String csvString = new String(sourceCsv);
CSVFormat csvFormat = CSVFormat.DEFAULT;
List<CSVRecord> csvRecord = csvFormat.parse(new
StringReader(csvString)).getRecords();
It will help you to deserialize and to store in database.
Related
im using fasterxml jackson to serialize an entity in a java project, the problem im having is the following one, when the entity is converted to JSON I get something like this:
{ "#id":1
"father":{
"#id":2,
"name":"robert smith"
"country":{"#id":3, "countryId":10, "countryName":"USA"}
},
"mother":{
"#id":4,
"name":"mary dune"
"country":{#ref:3}
}
}
Father and Mother has a reference to the same Country object, but when is serialized mother only have the reference to #id:3. I know that this is done to avoid circular refernce and that kind of stuff, but in this case I need that with the country entity it doesnt use reference and just put the whole entity.
Is there some kind of annotation to achieve that? like putting an annotation in the country entity? I just want this with country and no other entity, I want the other ones to respect the ref system.
The result im looking for is something like:
{"#id":1
"father":{
"#id":2,
"name":"robert smith"
"country":{"#id":3, "countryId":10, "countryName":"USA"}
},
"mother":{
"#id":4,
"name":"mary dune"
"country":{"#id":3, "countryId":10, "countryName":"USA"}
}
}
This is the objectMapper im using:
public ObjectMapperContextResolver() {
JavaTimeModule javaTimeModule = new JavaTimeModule();
javaTimeModule.addSerializer(
LocalDateTime.class,
new LocalDateTimeSerializer(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm")));
javaTimeModule.addSerializer(
LocalDate.class,
new LocalDateSerializer(DateTimeFormatter.ofPattern("yyyy-MM-dd")));
javaTimeModule.addDeserializer(LocalDateTime.class,
new LocalDateTimeDeserializer(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm")));
javaTimeModule.addDeserializer(LocalDate.class,
new LocalDateDeserializer(DateTimeFormatter.ofPattern("yyyy-MM-dd")));
defaultMapper.registerModule(new Jdk8Module())
.registerModule(javaTimeModule)
.registerModule(new BlankStringsAsNullModule())
.setSerializationInclusion(JsonInclude.Include.NON_NULL)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.configure(DeserializationFeature.ACCEPT_EMPTY_STRING_AS_NULL_OBJECT, true)
.configure(SerializationFeature.USE_EQUALITY_FOR_OBJECT_ID, true);
}
I am trying to find how to use mongo Atlas search indexes, from java application, which is using spring-data-mongodb to query the data, can anyone share an example for it
what i found was as code as below, but that is used for MongoDB Text search, though it is working, but not sure whether it is using Atlas search defined index.
TextQuery textQuery = TextQuery.queryText(new TextCriteria().matchingAny(text)).sortByScore();
textQuery.fields().include("cast").include("title").include("id");
List<Movies> movies = mongoOperations
.find(textQuery, Movies.class);
I want smaple java code using spring-data-mongodb for below query:
[
{
$search: {
index: 'cast-fullplot',
text: {
query: 'sandeep',
path: {
'wildcard': '*'
}
}
}
}
]
It will be helpful if anyone can explain how MongoDB Text Search is different from Mongo Atlas Search and correct way of using Atalas Search with the help of java spring-data-mongodb.
How to code below with spring-data-mongodb:
Arrays.asList(new Document("$search",
new Document("index", "cast-fullplot")
.append("text",
new Document("query", "sandeep")
.append("path",
new Document("wildcard", "*")))),
new Document())
Yes, spring-data-mongo supports the aggregation pipeline, which you'll use to execute your query.
You need to define a document list, with the steps defined in your query, in the correct order. Atlas Search must be the first step in the pipeline, as it stands. You can translate your query to the aggregation pipeline using the Mongo Atlas interface, they have an option to export the pipeline array in the language of your choosing. Then, you just need to execute the query and map the list of responses to your entity class.
You can see an example below:
public class SearchRepositoryImpl implements SearchRepositoryCustom {
private final MongoClient mongoClient;
public SearchRepositoryImpl(MongoClient mongoClient) {
this.mongoClient = mongoClient;
}
#Override
public List<SearchEntity> searchByFilter(String text) {
// You can add codec configuration in your database object. This might be needed to map
// your object to the mongodb data
MongoDatabase database = mongoClient.getDatabase("aggregation");
MongoCollection<Document> collection = database.getCollection("restaurants");
List<Document> pipeline = List.of(new Document("$search", new Document("index", "default2")
.append("text", new Document("query", "Many people").append("path", new Document("wildcard", "*")))));
List<SearchEntity> searchEntityList = new ArrayList<>();
collection.aggregate(pipeline, SearchEntity.class).forEach(searchEntityList::add);
return searchEntityList;
}
}
I'm working with the MarkLogic POJO Databinding Interface at the moment. I'm able to write POJOs to MarkLogic. Now I want to search those POJOs and retrieve the search results. I'm following the instructions from: https://docs.marklogic.com/guide/java/binding#id_89573 However, the search results don't seem to return the correct objects. I'm getting a JSONMappingException. Here's the code:
HashMap<String, MatchedPropertyInfo> matchedProperties = new HashMap<String, MatchedPropertyInfo>();
PropertyMatches PM = new PropertyMatches(123,"uri/prefix/location2", "uri/prefix", 1234,0,"/aKey","/aLocation",true,matchedProperties);
MatchedPropertyInfo MPI1 = new MatchedPropertyInfo("matched/property/uri1", "matched/property/key1", "matched/property/location1", true,"ValueMatch1", 12, 1*1.0/3, true);
MatchedPropertyInfo MPI2 = new MatchedPropertyInfo("matched/property/uri2", "matched/property/key2", "matched/property/location2", true,"ValueMatch2", 14, 1.0/2.0, true);
PM.getMatchedProperties().put("matched/property/prefix/location1", MPI1);
PM.getMatchedProperties().put("matched/property/prefix/location2", MPI2);
PojoRepository myClassRepo = client.newPojoRepository(PropertyMatches.class, Long.class);
myClassRepo.write(PM);
PojoQueryBuilder qb = myClassRepo.getQueryBuilder();
PojoPage<PropertyMatches> matches = myClassRepo.search(qb.value("uri", "uri/prefix/location2"),1);
if (matches.hasContent()) {
while (matches.hasNext()) {
PropertyMatches aPM = matches.next();
System.out.println(" " + aPM.getURI());
}
} else {
System.out.println(" No matches");
}
The PropertyMatches (PM) object is succesfully written to the MarkLogic database. This class contains a member: private String URI which is initiated with "uri/prefix/location2". The matches.hasContent() returns true in the example above. However, I'm getting an error on PropertyMatches aPM = matches.next();
Searching POJOs in MarkLogic and read them into your Java program requires the POJOs to have an empty constructor. In this case PropertyMatches should have public PropertyMatches(){} and MatchedPropertyInfo should have public MatchedPropertyInfo(){}
Thanks #sjoerd999 for posting the answer you found. Just to add some documentation references, this topic is discussed here: http://docs.marklogic.com/guide/java/binding#id_54408 and here: https://docs.marklogic.com/javadoc/client/com/marklogic/client/pojo/PojoRepository.html.
Also worth noting is you can have multiple parameters in the consructor, you just have to do it the Jackson way. Here are examples of two ways (with annotations and without): https://manosnikolaidis.wordpress.com/2015/08/25/jackson-without-annotations/
I'd recommend using annotations as that's built-in with Jackson. But if you want to do it without annotations, here's the code:
ObjectMapper mapper = new ObjectMapper();
// Avoid having to annotate the Person class
// Requires Java 8, pass -parameters to javac
// and jackson-module-parameter-names as a dependency
mapper.registerModule(new ParameterNamesModule());
// make private fields of Person visible to Jackson
mapper.setVisibility(FIELD, ANY);
If you want to do this with PojoRepository you'll have to use the unsupported getObjectMapper method to get the ObjectMapper and call registerModule and setVisibility on that:
ObjectMapper objectMapper = ((PojoRepositoryImpl) myClassRepo).getObjectMapper();
I want to create JSON Schema manually using GSON but i dont find any JsonSchema element support in GSON. I dont want to convert a pojo to schema but want to create schema programatically . Is there any way in GSON ? May be something like following.
**1 JsonSchema schema = new JsonSchema();
2 schema.Type = JsonSchemaType.Object;
3 schema.Properties = new Dictionary<string, JsonSchema>
4{
5 { "name", new JsonSchema { Type = JsonSchemaType.String } },
6 {
7 "hobbies", new JsonSchema
8 {
9 Type = JsonSchemaType.Array,
10 Items = new List<JsonSchema> { new JsonSchema { Type = JsonSchemaType.String } }
11 }
12 },
13};**
You may consider using everit-org/json-schema for programmatically creating JSON Schemas. Although it is not properly documented, its builder classes form a fluent API which lets you do it. Example:
Schema schema = ObjectSchema.builder()
.addPropertySchema("name", StringSchema.builder().build())
.addPropertySchema("hobbies", ArraySchema.builder()
.allItemSchema(StringSchema.builder().build())
.build())
.build();
It is slightly different syntax than what you described, but it can be good for the same purpose.
(disclaimer: I'm the author of everit-org/json-schema)
I tried to build a schema as suggested above, see Everit schema builder includes unset properties as null
I have a problem with update query using Spring Data MongoDB. I retrieve some object's _id as BigInteger value. Then I want to make following query:
Query query = new Query(Criteria.where("_id").is(id));
Update update = new Update();
update.set("version",version);
mongoOperations.updateFirst(query, update, Audit.class);
Query part fails to match any documents since id value passed to is() somehow must be converted to ObjectId. I can't find any documentation on this kind of conversion. Will appreciate any help.
p.s.: SpringData Mongodb version 1.2
You can convert it also manually:
ObjectId convertedId = new ObjectId(bigInteger.toString(16));
Query query = new Query(Criteria.where("_id").is(convertedId));
You probably want to write a custom Spring converter BigInteger => ObjectId and ObjectId => BigInteger.
See the doc part here:
http://static.springsource.org/spring-data/data-document/docs/current/reference/html/#d0e2670
------UPDATE------
It seems that this kind of converter already exists in the Spring-Data-MongoDB library:
http://static.springsource.org/spring-data/data-document/docs/1.0.0.M1/api/org/springframework/data/document/mongodb/SimpleMongoConverter.ObjectIdToBigIntegerConverter.html
So you just have to specify it in your Spring configuration.
Alternatively you can add an 'id' field to your collection classes or potentially a base class and annotate it with org.springframework.data.annotation.Id, as below:
import org.springframework.data.annotation.Id;
public abstract class BaseDocument {
#Id
protected long id;
This will allow you to perform the queries of the form:
public boolean doesDocumentExist(Class clazz, long documentId) {
Query queryCriteria = new Query(Criteria.where("id").is(documentId));
return mongoTemplate.count(queryCriteria, clazz) == 1;
}
Annotating your own id field with '#Id' will store your id as the mongo objectId, therefore saving you from doing the conversion yourself.
//get the converter from the mongoTemplate
MappingMongoConverter converter = (MappingMongoConverter)mongoTemplate.getConverter();
//get the conversion service from the mongo converter
ConversionService conversionService = converter.getConversionService();
//iterate the status list and get the each id to add the arraylist
for(Status status: statusList){
ObjectId objectIdVal = conversionService.convert(status.getId(), ObjectId.class);
**//here status.getId() returns the BigInteger**
statusArrayList.add(objectIdVal);
}
//get the users list whose status is active and cancel
query.addCriteria(new Criteria().where("status.$id").in(statusArrayList));
List<User> usersList = mongoTemplate.find(query, User.class);
You can convert a BigIngeter to ObjectId using the hex representation of the BigInteger. However, an ObjectId is supposed to be exactly 24 characters long, and parsing a shorter string will fail in Java. Thus it's better to ensure that the hex representation is 0-padded appropriately:
String hexString24 = StringUtils.leftPad(bigInteger.toString(16), 24, "0");
ObjectId convertedId = new ObjectId(hexString24);
Query query = new Query(Criteria.where("_id").is(convertedId));