Is it currently possible to retrieve a subset of a json object stored in Dynamo? For example, I have an attribute named record of JSON type which is an array of JSON objects:
records:
[
{"K1": "V1" },
{"K2": "V2" },
{"K3": "V3" },
{"K4": "V4" }
]
I store them in the JSON format in Dynamo. I wanted to know if I can only retrieve key value pairs 1 to 2 and not the 3rd and 4th one? I am unsure if I can provide a specific filter expression to do this operation.
If it is possible, I would love to hear the methodology as to how it can be done?
Thanks!
Firstly, there is no JSON data type in DynamoDB. If you mean the data is stored as DynamoDB data type MAP, then the below solution should work for you.
In short, the filter expression should be something like below:-
FilterExpression : 'records.K1 = :recordsK1Value and records.K2 = :recordsK2Value'
If you need to have only "records.K1" and "records.K2" in output, you can use project expression for that.
ProjectionExpression : 'records.K1, records.K2'
Full code:-
public List<String> queryMoviesAndFilterByMapAttribute() {
List<String> moviesJsonList = new ArrayList<>();
DynamoDB dynamoDB = new DynamoDB(dynamoDBClient);
Table table = dynamoDB.getTable("Movies");
QuerySpec querySpec = new QuerySpec();
querySpec.withKeyConditionExpression("yearkey = :yearval and title = :titleval")
//.withProjectionExpression("records.K1, records.K2")
.withFilterExpression("records.K1 = :recordsK1Value and records.K2 = :recordsK2Value").withValueMap(
new ValueMap().withNumber(":yearval", 1991).withString(":titleval", "Movie with map attribute")
.withString(":recordsK1Value", "V1").withString(":recordsK2Value", "V2"));
IteratorSupport<Item, QueryOutcome> iterator = table.query(querySpec).iterator();
while (iterator.hasNext()) {
Item movieItem = iterator.next();
System.out.println("Movie data ====================>" + movieItem.toJSONPretty());
moviesJsonList.add(movieItem.toJSON());
}
return moviesJsonList;
}
Sample output with all fields (i.e. without project expression):-
Movie data ====================>{
"yearkey" : 1991,
"records" : {
"K1" : "V1",
"K2" : "V2",
"K3" : "V3",
"K4" : "V4"
},
"title" : "Movie with map attribute"
}
Sample output after un-commenting the project expression:-
Please note that other fields such as yearkey, title, K3 and K4 are not present in the output.
Movie data ====================>{
"records" : {
"K1" : "V1",
"K2" : "V2"
}
}
Related
I have the following JSON file:
{
"meta" : {
"stock" : "AWS",
"date modified" : 90
},
"roles" : [ "Member", "Admin" ],
"name" : "John Doe",
"admin" : true,
"email" : "john.doe#example.com"
}
I wanted to both read the values of the keys and add them to an Array List.
try {
// create object mapper instance
ObjectMapper mapper = new ObjectMapper();
// convert JSON file to map
Map<?, ?> map = mapper.readValue(Paths.get("user.json").toFile(), Map.class);
ArrayList<String> data = new ArrayList<String>();
// print map entries
for (Map.Entry<?, ?> entry : map.entrySet()) {
System.out.println((entry.getClass()) + " " + entry.getValue());
data.add((String)entry.getValue()); // trying to add entry values to arraylist
}
} catch (Exception ex) {
ex.printStackTrace();
}
I'm able to print out the data type of the value along with the value itself. All the values are part of class java.util.LinkedHashMap$Entry. I'm not able to cast the values to a String to add them to an ArrayList. How should I go about doing this? Thanks
From the jackson-databind documentation you can convert your json to a Map<String, Object> map with the following line (you have boolean, list, number and string values in your json) :
Map<String, Object> map = mapper.readValue(json, Map.class);
// it prints {meta={stock=AWS, date modified=90}, roles=[Member, Admin], name=John Doe, admin=true, email=john.doe#example.com}
System.out.println(map);
If you want to save your map values string representation into an ArrayList data you can iterate over them with a loop :
List<String> data = new ArrayList<>();
for (Object value : map.values()) {
data.add(value.toString());
}
//it will print [{stock=AWS, date modified=90}, [Member, Admin], John Doe, true, john.doe#example.com]
System.out.println(data);
Your data type of entries will be like:
meta: Map<String:Object>
roles: List<String>
admin: Boolean
So you will get an exception when casting to string for each entry value.
You should handle different data type and convert it according to your request:
Object value = entry.getValue();
I highly recommend you write more few functions to check and convert map/list/primitive variables to expected data (String):
boolean isList(Object obj);
boolean isMap(Object obj);
...
public List<String> convertMap(Map<String,Object> map);
...
Following is the structure of my MongoDB document userActivity.
{
"_id" : ObjectId("5e49569f93e956eeb28eb8a6"),
"userId" : "123",
"likes" : {
"videos" : [
{
"_id" : "abc",
"title" : "This video is part of test setup",
}
]
}
}
I am using Spring Data MongoOperations to manipulate MongoDB collections. And below is the code to remove a video from videos array in likes sub-document. I have tried to first filter the document as per the user's userId. And then apply filter to update function as per videoId.
public UpdateResult removeVideoLike(String videoId, String userId) {
Query queryUser = Query.query( Criteria.where("userId").is(userId) );
Query queryVideo = Query.query( Criteria.where("id").is(videoId) );
Update update = new Update().pull("likes.videos", queryVideo );
return mongoOperations.updateFirst( queryUser , update, UserActivity.class );
}
This runs without errors but the entry is not removed. The UpdateResult has following values
matchedCount = 1
modifiedCount = 0
upsertedId = null
I am confused if it is able to match the entry in the array, why it is not removing it? What I am missing?
I am a little confused as to how to add an element to an array in an exisiting mongodb document, or why my results are not showing properly and how I expect.
There is only one document in the collection and will only ever be one. The mongo document looks like when I do a db.collection-name.find.pretty() command in a mongo session on the command line:
{
"_id" : ObjectID("1234567890"),
"details" : {
...
},
"calculations" : [
{
"count" : 1,
"total" : 10,
"mean" : 2.5
},
{
"count" : 2,
"total" : 20,
"mean" : 6.4
}
]
}
I want to add another object to the calculations list.
The Java code I am running is based upon THIS example:
// Get the database and collection
MongoDatabase database = mongo.getDatabase(dataBaseName);
MongoCollection<Document> collection = database.getCollection(collectionName);
Document document = collection.find().first(); // will only ever be one document
// The object comes in as a Map
Map<String, Object> incomingMap = new HashMap<>();
incomingMap.put("count", 3);
incomingMap.put("total", 4);
incomingMap.put("mean", 7.9);
// convert to a Document
Document newDocument = new Document();
incomingMap.forEach((k, v) -> {
newDocument.append(k, v);
});
// append this to the collection - this is where I am confused
// for this example just hardcoding the _id value for simplicity
collection.updateOne(new Document("_id", "1234567890"), Updates.push("calculations", newDocument));
However when I do a System.out.println(collection.find().first()) in the code after this or db.collection-name.find.pretty() in a mongo session the new document has not been added. There are no errors thrown and completes fine.
What I am wondering is
Is the line collection.updateOne(new Document("_id", "1234567890"), Updates.push("calculations", newDocument)); correct?
Has it been added but not been saved - if so how do I save?
Can I do this at a document level, for example document.update(new Documen(Updates.push("calculations", newDocument)); or similar?
I have also tried collection.findAndUpdateOne(new Document("_id", "1234567890"), Updates.push("calculations", newDocument)); with the same result
Is how I am getting/hardcoding the document ID incorrect?
You have filter condition issue (your _id is ObjectId type)
new Document("_id", ObjectId("1234567890"))`
Always make sure your documents updated correctly. Look code fagment:
UpdateResult result = collection.updateOne(filter, update);
log.info("Update with date Status : " + result.wasAcknowledged());
log.info("Nº of Record Modified : " + result.getModifiedCount());
https://api.mongodb.com/java/3.1/com/mongodb/client/result/UpdateResult.html
I am using Java driver for mongo-db and trying to add multiple query criteria using BasicDBObjectBuilder. I have a text field where an XML is stored as String so we are using regex to form the query.
Below is my query and the output I am getting:
regexQuery.put("REQUEST_XML",BasicDBObjectBuilder
.start("$regex", ".*Main>[\r\n]<.?.?.?.?action>"+MainValue+".*")
.add("$regex", ".*Details>[\r\n]<.?.?.?.?action>" + DetailValue+ ".*").get());
regexQuery.put("NAME", "Video");
What I am getting as query is :
{ "REQUEST_XML" : { "$regex" : ".*Details>[\r\n]<.?.?.?.?action>Change.*"} , "NAME" : "Video"}
The first part with .start("$regex", ".Main>[\r\n]<.?.?.?.?action>"+MainValue+".") is not getting added to query.
Can you please let me know what is the issue ?
You are overwriting the key value pair. "$regex", ".*Details>[\r\n]<.?.?.?.?action>" + DetailValue+ ".*" overwrites "$regex", ".*Main>[\r\n]<.?.?.?.?action>"+MainValue+".*".
Use $or to pass both regex expression.
Something like
BasicDBObject regexQuery = new BasicDBObject();
regexQuery.put("$or", Arrays.asList(new BasicDBObject("REQUEST_XML", new BasicDBObject("$regex", ".*Main>[\r\n]<.?.?.?.?action>"+".*")),
new BasicDBObject("REQUEST_XML", new BasicDBObject("$regex", ".*Details>[\r\n]<.?.?.?.?action>"+".*"))));
regexQuery.put("NAME", "Video");
This should output query like
{ "$or" : [{ "REQUEST_XML" : { "$regex" : ".*Main>[\r\n]<.?.?.?.?action>.*" } }, { "REQUEST_XML" : { "$regex" : ".*Details>[\r\n]<.?.?.?.?action>.*" } }], "NAME" : "Video" }
Using 3.x driver
import static com.mongodb.client.model.Filters.or;
import static com.mongodb.client.model.Filters.regex;
Bson regexQuery = or(regex("REQUEST_XML", ".*Main>[\r\n]<.?.?.?.?action>"+".*"), regex("$regex", ".*Details>[\r\n]<.?.?.?.?action>"+".*"));
I have a mongo collection with objects like these:
[
{
"_id" : "a2d",
"entityType" : "Location",
"type" : "STRING",
},
{
"_id" : "a1_order",
"entityType" : "Order",
"type" : "STRING",
}
]
Trying to append the _entityType to all document's id where it is not present at the end id the id (the first object in the above case).
Using mongo with Spring, but I'm already stuck with the first step, to get all the objects with no entityType in id.
Thinking about something like this, with regex, but I'm not sure how should it look like:
Query query = new Query();
query.addCriteria( Criteria.where( "id" ).regex( "here i need the entity type of the current document" ) );
You can build your regex by '^' ('starts with' Regex).
So you need a function who point in all documents and check this filter
List<Document> result = new ArrayList<Document>();
StringBuilder idPrefix = new StringBuilder();
idPrefix.append("^");
idPrefix.append(idCode);
idPrefix.append("_");
List<Bson> filters = new ArrayList<Bson>();
filters.add(Filters.regex("_id", keyPrefix.toString()));
for (Document d : yourCollections.find(Filters.and(filters)))
list.add(d);
You actually want a "reverse regex" here, as you need to use the data in the document in order to match on another field.
Presently you can really only do this with MongoDB using $where, which evaluates JavaScript on the server. So for spring mongo, you need the BasicQuery instead, so we can construct from BasicDBObject and Code primatives:
BasicDBObject basicDBObject = new BasicDBObject("$where",
new Code("!RegExp('_' + this.entityType + '$','i').test(this.id)"));
BasicQuery query = new BasicQuery(basicDBObject);
That will test the "id" field in the document to see if it matches the value from entityType at the "end of the string" and without considering "case". The ! is a Not condition, so the "reverse" of the logic is applied to "not match" where the field actually did end that way.