Develop Reference

Springboot MongoDB delete an object from array of objects

I'm trying to delete an object from an array of objects in mongodb collection. I've followed multiple answers from SO but I couldn't able to make it work.
customer_id and address_pin will be passed from querystring. I want to remove an object from address_india where address_pin matches with the value passed via querystring.
MongoDB collection object
{
"_id":{
"$oid":"61fa7de3b3de485b30403c56"
},
"customer_id":"c7e690cf-c8d9-4a7b-b146-ade742958452",
"customer_addresses":{
"address_india":[
{
"address_id":"ccc428bd-3e4d-49e5-8569-ebacb181ad1e",
"address_pin":"MD0BuCQUxT", // I've to remove this object
"address_type":"sales"
},
{
"address_id":"ccc428bd-3e4d-49e5-8569-ebacb181asdf",
"address_pin":"MD0BuCQUXy",
"address_type":"marketing"
}
]
},
"create_timestamp":{
"$numberLong":"1643806179346"
},
"modified_timestamp":{
"$numberLong":"1643806179346"
}
}
Controller
return new ResponseEntity<CustomerAddressResponse>(
this.customerAddressService.deleteCustomerAddressById(customerId),
HandleResponseHeader.getResponseHeaders(UUID), HttpStatus.OK);
ServiceImpl
#Override
public CustomerAddressResponse deleteCustomerAddressById(String customerId)
throws Exception {
try {
if (isValidUUID(customerId)) {
Date now = new Date();
Long dateStartTime = now.getTime();
Stopwatch stopWatch = Stopwatch.createStarted();
CustomerAddress customerAddress = customerAddressDao
.findByCustomerId(customerId);
String pin = httpServletRequest.getParameter("address_pin")
.trim();
Query query = new Query();
query.addCriteria(Criteria.where("customer_id")
.is(customerId)
.and("customer_addresses.address_india")
.elemMatch(Criteria.where("address_pin").is(pin)));
Update update = new Update();
update.pull("customer_addresses.address_india", new Query().addCriteria(Criteria.where("address_pin").is(pin)));
FindAndModifyOptions options = FindAndModifyOptions.options();
options.returnNew(true);
MongoTemplate template = null;
template.findAndModify(query, update, options, CustomerAddress.class);
}
}
}
I'm getting java.lang.NullPointerException at template.findAndModify(query, update, options, CustomerAddress.class);
How do I delete an object from array of objects in mongodb from springboot?

This is how you do from mongo js shell for single document , it may give you some idea on how to do from springboot:
db.collection.update(
{
"customer_id": "c7e690cf-c8d9-4a7b-b146-ade742958452"},
{
$pull: {
"customer_addresses.address_india": {
"address_pin": "MD0BuCQUxT"
}
}
})
playground

Is there a possibility to store the single event information at a time in a JSONObject/JsonNode using the Jackson JsonParser

I am trying to read the events from a large JSON file one-by-one using the Jackson JsonParser. I would like to store each event temporarily in an Object something like JsonObject or any other object which I later want to use for some further processing.
I was previously reading the JSON events one-by-one and storing them into my own custom context: Old Post for JACKSON JsonParser Context which is working fine. However, rather than context, I would like to store them into jsonObject or some other object one by one.
Following is my sample JSON file:
{
"#context":"https://context.org/context.jsonld",
"isA":"SchoolManagement",
"format":"application/ld+json",
"schemaVersion":"2.0",
"creationDate":"2021-04-21T10:10:09+00:00",
"body":{
"members":[
{
"isA":"student",
"name":"ABCS",
"class":10,
"coaching":[
"XSJSJ",
"IIIRIRI"
],
"dob":"1995-04-21T10:10:09+00:00"
},
{
"isA":"teacher",
"name":"ABCS",
"department":"computer science",
"school":{
"name":"ABCD School"
},
"dob":"1995-04-21T10:10:09+00:00"
},
{
"isA":"boardMember",
"name":"ABCS",
"board":"schoolboard",
"dob":"1995-04-21T10:10:09+00:00"
}
]
}
}
At a time I would like to store only one member such as student or teacher in my JsonObject.
Following is the code I have so far:
What's the best way to store each event in an Object which I can later use for some processing.
Then again clear that object and use it for the next event?
public class Main {
private JSONObject eventInfo;
private final String[] eventTypes = new String[] { "student", "teacher", "boardMember" };
public static void main(String[] args) throws JsonParseException, JsonMappingException, IOException, JAXBException, URISyntaxException {
// Get the JSON Factory and parser Object
JsonFactory jsonFactory = new JsonFactory();
JsonParser jsonParser = jsonFactory.createParser(new File(Main.class.getClassLoader().getResource("inputJson.json").toURI()));
JsonToken current = jsonParser.nextToken();
// Check the first element is Object
if (current != JsonToken.START_OBJECT) {
throw new IllegalStateException("Expected content to be an array");
}
// Loop until the start of the EPCIS EventList array
while (jsonParser.nextToken() != JsonToken.START_ARRAY) {
System.out.println(jsonParser.getCurrentToken() + " --- " + jsonParser.getCurrentName());
}
// Goto the next token
jsonParser.nextToken();
// Call the method to loop until the end of the events file
eventTraverser(jsonParser);
}
// Method which will traverse through the eventList and read event one-by-one
private static void eventTraverser(JsonParser jsonParser) throws IOException {
// Loop until the end of the EPCIS events file
while (jsonParser.nextToken() != JsonToken.END_OBJECT) {
//Is there a possibility to store the complete object directly in an JSON Object or I need to again go through every token to see if is array and handle it accordingly as mentioned in my previous POST.
}
}
}

After trying some things I was able to get it working. I am posting the whole code as it can be useful to someone in the future cause I know how frustrating it is to find the proper working code sample:
public class Main
{
public void xmlConverter (InputStream jsonStream) throws IOException,JAXBException, XMLStreamException
{
// jsonStream is the input JSOn which is normally passed by reading the JSON file
// Get the JSON Factory and parser Object
final JsonFactory jsonFactory = new JsonFactory ();
final JsonParser jsonParser = jsonFactory.createParser (jsonStream);
final ObjectMapper objectMapper = new ObjectMapper ();
//To read the duplicate keys if there are any key duplicate json
final SimpleModule module = new SimpleModule ();
module.addDeserializer (JsonNode.class, new JsonNodeDupeFieldHandlingDeserializer ());
objectMapper.registerModule (module);
jsonParser.setCodec (objectMapper);
// Check the first element is Object if not then invalid JSON throw error
if (jsonParser.nextToken () != JsonToken.START_OBJECT)
{
throw new IllegalStateException ("Expected content to be an array");
}
while (!jsonParser.getText ().equals ("members"))
{
//Skipping the elements till members key
// if you want you can do some process here
// I am skipping for now
}
// Goto the next token
jsonParser.nextToken ();
while (jsonParser.nextToken () != JsonToken.END_ARRAY)
{
final JsonNode jsonNode = jsonParser.readValueAsTree ();
//Check if the JsonNode is valid if not then exit the process
if (jsonNode == null || jsonNode.isNull ())
{
System.out.println ("End Of File");
break;
}
// Get the eventType
final String eventType = jsonNode.get ("isA").asText ();
// Based on eventType call different type of class
switch (eventType)
{
case "student":
final Student studentInfo =
objectMapper.treeToValue (jsonNode, Student.class);
//I was calling the JAXB Method as I was doing the JSON to XML Conversion
xmlCreator (studentInfo, Student.class);
break;
case "teacher":
final Teacher teacherInfo =
objectMapper.treeToValue (jsonNode, Teacher.class);
xmlCreator (teacherInfo, Teacher.class);
break;
}
}
}
//Method to create the XML using the JAXB
private void xmlCreator (Object eventInfo,
Class eventType) throws JAXBException
{
private final StringWriter sw = new StringWriter ();
// Create JAXB Context object
JAXBContext context = JAXBContext.newInstance (eventType);
// Create Marshaller object from JAXBContext
Marshaller marshaller = context.createMarshaller ();
// Print formatted XML
marshaller.setProperty (Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
// Do not add the <xml> version tag
marshaller.setProperty (Marshaller.JAXB_FRAGMENT, Boolean.TRUE);
// XmlSupportExtension is an interface that every class such as Student Teacher implements
// xmlSupport is a method in XmlSupportExtension which has been implemented in all classes
// Create the XML based on type of incoming event type and store in SW
marshaller.marshal (((XmlSupportExtension) eventInfo).xmlSupport (),
sw);
// Add each event within the List
eventsList.add (sw.toString ());
// Clear the StringWritter for next event
sw.getBuffer ().setLength (0);
}
}
This is the class that overrides the JACKSON class.
This can be used if your Json has duplicate JSON keys. Follow this post for the complete explnation if you need. If you dont need then skip this part and remove the part of the code module from the above class:
Jackson #JsonAnySetter ignores values of duplicate key when used with Jackson ObjectMapper treeToValue method
#JsonDeserialize(using = JsonNodeDupeFieldHandlingDeserializer.class)
public class JsonNodeDupeFieldHandlingDeserializer extends JsonNodeDeserializer {
#Override
protected void _handleDuplicateField(JsonParser p, DeserializationContext ctxt, JsonNodeFactory nodeFactory, String fieldName,
ObjectNode objectNode, JsonNode oldValue, JsonNode newValue) {
ArrayNode asArrayValue = null;
if (oldValue.isArray()) {
asArrayValue = (ArrayNode) oldValue;
} else {
asArrayValue = nodeFactory.arrayNode();
asArrayValue.add(oldValue);
}
asArrayValue.add(newValue);
objectNode.set(fieldName, asArrayValue);
}
}

I am iterating the Mongodb Collection with Java for loop which is creating heap Memory issue

#RequestMapping(value = "/id/", method = RequestMethod.GET)
public String getClaimsDetailById(#RequestParam(value = "userId") String userId,
#RequestParam(value = "id") String id) throws JsonProcessingException {
MongoDatabase database = this.mongoClient.getDatabase(this.database);
MongoCollection<Document> collection = database.getCollection(this.collection);
List claim = new ArrayList();
Document document = new Document("_id", new ObjectId(id));
List<Document> claims = collection.find(document).into(claim);
List<Document> claimsUpdatedList = new ArrayList<>();
for (Document doc : claims) {
if (null != doc.get("Common")) {
Document common = (Document) doc.get("Common");
if (null != common.get("EffectiveDate")) {
Date date = (Date) common.get("EffectiveDate");
common.put("EffectiveDate",convertDate(date));
}
if (null != common.get("ExpirationDate")) {
Date date = (Date) common.get("ExpirationDate");
common.put("ExpirationDate",convertDate(date));
}
doc.put("Common",common);
claimsUpdatedList.add(doc);
}
}
JsonWriterSettings writerSettings = JsonWriterSettings.builder().outputMode(JsonMode.SHELL).indent(true).build();
return claimsUpdatedList.get(0).toJson(writerSettings);
}

The problem is that you're using into(), which loads all the data at once into the List, therefore possibly overloading the heap if the data is very large.
Instead, you should be using iterator(), which uses the database cursor to process the data one-by-one, without loading them into application memory. This concept is not restricted to Mongo, it is a common mechanism with all databases (which is why the very concept of cursors exists).
For example:
...
FindIterable<Document> documentCursor = collection.find(document);
for (Document doc : documentCursor) {
...
}

super csv nested bean

I have a csv
id,name,description,price,date,name,address
1,SuperCsv,Write csv file,1234.56,28/03/2016,amar,jp nagar
I want to read it and store it to json file.
I have created two bean course(id,name,description,price,date) and person(name,address)
on reading by bean reader i'm not able to set the person address.
The (beautified) output is
Course [id=1,
name=SuperCsv,
description=Write csv file,
price=1234.56,
date=Mon Mar 28 00:00:00 IST 2016,
person=[
Person [name=amar, address=null],
Person [name=null, address=jpnagar]
]
]
I want the adress to set with name
My code:
public static void readCsv(String csvFileName) throws IOException {
ICsvBeanReader beanReader = null;
try {
beanReader = new CsvBeanReader(new FileReader(csvFileName), CsvPreference.STANDARD_PREFERENCE);
// the header elements are used to map the values to the bean (names must match)
final String[] header = beanReader.getHeader(true);
final CellProcessor[] processors = getProcessors();
final String[] fieldMapping = new String[header.length];
for (int i = 0; i < header.length; i++) {
if (i < 5) {
// normal mappings
fieldMapping[i] = header[i];
} else {
// attribute mappings
fieldMapping[i] = "addAttribute";
}}
ObjectMapper mapper=new ObjectMapper();
Course course;
List<Course> courseList=new ArrayList<Course>();
while ((course = beanReader.read(Course.class, fieldMapping, processors)) != null) {
// process course
System.out.println(course);
courseList.add(course);
}
private static CellProcessor[] getProcessors(){
final CellProcessor parsePerson = new CellProcessorAdaptor() {
public Object execute(Object value, CsvContext context) {
return new Person((String) value,null);
}
};
final CellProcessor parsePersonAddress = new CellProcessorAdaptor() {
public Object execute(Object value, CsvContext context) {
return new Person(null,(String) value);
}
};
return new CellProcessor[] {
new ParseInt(),
new NotNull(),
new Optional(),
new ParseDouble(),
new ParseDate("dd/MM/yyyy"),
new Optional(parsePerson),
new Optional(parsePersonAddress)
};

SuperCSV is the first parser I have seen that lets you create an object within an object.
for what you are wanting you can try Apache Commons CSV or openCSV (CSVToBean) to map but to do this you need to have the setters of the inner class (setName, setAddress) in the outer class so the CSVToBean to pick it up. That may or may not work.
What I normally tell people is to have a plain POJO that has all the fields in the csv - a data transfer object. Let the parser create that then use a utility/builder class convert the plain POJO into the nested POJO you want.

converting Document objects in MongoDB 3 to POJOS

I'm saving an object with a java.util.Date field into a MongoDB 3.2 instance.
ObjectMapper mapper = new ObjectMapper();
String json = mapper.writeValueAsString(myObject);
collection.insertOne(Document.parse(json));
the String contains:
"captured": 1454549266735
then I read it from the MongoDB instance:
final Document document = collection.find(eq("key", value)).first();
final String json = document.toJson();
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
xx = mapper.readValue(json, MyClass.class);
the deserialization fails:
java.lang.RuntimeException: com.fasterxml.jackson.databind.JsonMappingException:
Can not deserialize instance of java.util.Date out of START_OBJECT token
I see that the json string created by "document.toJson()" contains:
"captured": {
"$numberLong": "1454550216318"
}
instead of what was there originally ("captured": 1454549266735)
MongoDB docs say they started using "MongoDB Extended Json". I tried both Jackson 1 and 2 to parse it - no luck.
what is the easiest way to convert those Document objects provided by MongoDB 3 to Java POJOs? maybe I can skip toJson() step altogether?
I tried mongojack - that one does not support MongoDB3.
Looked at couple other POJO mappers listed on MongoDB docs page - they all require putting their custom annotations to Java classes.

You should define and use custom JsonWriterSettings to fine-tune JSON generation:
JsonWriterSettings settings = JsonWriterSettings.builder()
.int64Converter((value, writer) -> writer.writeNumber(value.toString()))
.build();
String json = new Document("a", 12).append("b", 14L).toJson(settings);
Will produce:
{ "a" : 12, "b" : 14 }
If you will not use custom settings then document will produce extended json:
{ "a" : 12, "b" : { "$numberLong" : "14" } }

This looks like Mongo Java driver bug, where Document.toJson profuces non-standard JSON even if JsonMode.STRICT is used. This problem is described in the following bug https://jira.mongodb.org/browse/JAVA-2173 for which I encourage you to vote.
A workaround is to use com.mongodb.util.JSON.serialize(document).

I save a tag with my mongo document that specifies the original type of the object stored. I then use Gson to parse it with the name of that type. First, to create the stored Document
private static Gson gson = new Gson();
public static Document ConvertToDocument(Object rd) {
if (rd instanceof Document)
return (Document)rd;
String json = gson.toJson(rd);
Document doc = Document.parse(json);
doc.append(TYPE_FIELD, rd.getClass().getName());
return doc;
}
then to read the document back into the Java,
public static Object ConvertFromDocument(Document doc) throws CAAException {
String clazzName = doc.getString(TYPE_FIELD);
if (clazzName == null)
throw new RuntimeException("Document was not stored in the DB or got stored without becing created by itemToStoredDocument()");
Class<?> clazz;
try {
clazz = (Class<?>) Class.forName(clazzName);
} catch (ClassNotFoundException e) {
throw new CAAException("Could not load class " + clazzName, e);
}
json = com.mongodb.util.JSON.serialize(doc);
return gson.fromJson(json, clazz);
}
Thanks to Aleksey for pointing out JSON.serialize().

It looks like you are using Date object inside "myObject". In that case, you should use a DateSerializer that implements JsonSerializer<LocalDate>, JsonDeserializer<LocalDate> and then register it with GsonBuilder. Sample code follows:
public class My_DateSerializer implements JsonSerializer<LocalDate>,
JsonDeserializer<LocalDate> {
#Override
public LocalDate deserialize(JsonElement json, Type typeOfT,
JsonDeserializationContext context) throws JsonParseException {
final String dateAsString = json.getAsString();
final DateTimeFormatter dtf = DateTimeFormat.forPattern(DATE_FORMAT);
if (dateAsString.length() == 0)
{
return null;
}
else
{
return dtf.parseLocalDate(dateAsString);
}
}
#Override
public JsonElement serialize(LocalDate src, Type typeOfSrc,
JsonSerializationContext context) {
String retVal;
final DateTimeFormatter dtf = DateTimeFormat.forPattern(DATE_FORMAT);
if (src == null)
{
retVal = "";
}
else
{
retVal = dtf.print(src);
}
return new JsonPrimitive(retVal);
}
}
Now register it with GsonBuilder:
final GsonBuilder builder = new GsonBuilder()
.registerTypeAdapter(LocalDate.class, new My_DateSerializer());
final Gson gson = builder.create();