Develop Reference

Exclusive Start Key causing issue with local secondary index in DynamoDbMapper

I have a table in dynamo db.
Below is the key for this table.
partitionKey - campaignId
sortKey - email
I have created a local secondary index for this table.Below is the key for local secondary index
partitionKey - campaignId
sortKey - subStatus
As per the dynamo db you should include primary keys of the table and the index (as key), with last evaluated values (as attribute value) when you setting ExclusiveStartKey.
Below is the code snippet of code.
public QueryResultPage<Prospect> getPaginatedProspectsByCampaignIdAndSubStatus(String campaignId, ProspectSubStatus subStatus, Integer limit, Map<String, AttributeValue> lastEvaluatedKey, String email) {
Prospect prospect = new Prospect();
prospect.setCampaignId(campaignId);
Map<String, AttributeValue> exclusiveStartKey = new HashMap<String, AttributeValue>();
if(!lastEvaluatedKey.isEmpty()) {
exclusiveStartKey.put("campaingId", lastEvaluatedKey.get("campaingId"));
exclusiveStartKey.put("subStatus", lastEvaluatedKey.get("subStatus"));
exclusiveStartKey.put("email", new AttributeValue().withS(email));
}
Condition rangeKeyCondition = new Condition()
.withComparisonOperator(ComparisonOperator.EQ)
.withAttributeValueList(new AttributeValue().withS(subStatus.toString()));
DynamoDBQueryExpression queryExpression = null;
if(exclusiveStartKey.isEmpty()) {
queryExpression = new DynamoDBQueryExpression<Prospect>()
.withHashKeyValues(prospect)
.withIndexName("subStatus-index")
.withRangeKeyCondition("subStatus", rangeKeyCondition)
.withConsistentRead(false).withLimit(limit);
} else {
queryExpression = new DynamoDBQueryExpression<Prospect>()
.withHashKeyValues(prospect)
.withIndexName("subStatus-index")
.withExclusiveStartKey(exclusiveStartKey)
.withRangeKeyCondition("subStatus", rangeKeyCondition)
.withConsistentRead(false).withLimit(limit);
}
QueryResultPage<Prospect> prospects = mapper.queryPage(Prospect.class, queryExpression);
return prospects;
}
But when run it, I am getting below error. Can someone please help me out to understand what I am doing wrong and how can I achieve the pagination in it.

Arithmetic operation in condition expression DynamoDb Using the AWS SDK for Java

I have a dynamodb Table and I want to do a conditional update operation on that using aws sdk java library. I have the table with hashkey named as "Id" and sortkey named as "Sk" . I have 5 other fields in the table and i have to update 2 fields based on the condition on the remaining fields. The condition is "total_record_count = record_passed + records_failed" , where total_record_count, record_passed and records_failed are the fields in the table.
Below is the code
DynamoDB dynamoDB = new DynamoDB(amazonDynamoDB);
Table table = dynamoDB.getTable("ciw_ocrResponse_upload");
Map<String, String> expressionAttributeNames = new HashMap<String, String>();
expressionAttributeNames.put("#updatedAt", "updatedAt");
expressionAttributeNames.put("#statusOfTracker","statusOfTracker");
Timestamp timestamp = new Timestamp(System.currentTimeMillis());
Map<String, Object> eav = new HashMap<>();
eav.put(":val1", value);
eav.put(":val2", timestamp.getTime());
UpdateItemSpec updateItemSpec = new UpdateItemSpec().withPrimaryKey("Id", requestId, "Sk", sortKey)
.withUpdateExpression("set #updatedAt = :val2, #statusOfTracker = :val1")
.withValueMap(eav)
.withNameMap(expressionAttributeNames)
.withConditionExpression(" total_record_count = record_passed + records_failed")
.withReturnValues(ReturnValue.UPDATED_NEW);
table.updateItem(updateItemSpec);
I am getting the error as Invalid ConditionExpression: Syntax error; token: "+", near: "record_passed + records_failed"

According to the docs this arithmetic operation is not supported.
Workaround is to create a total_record_count field which you need to calculate and add on every new item created. Also to add it to all existing items.

Fetch String of map in dynamodb

I have dynamodb table structure as follows:
{
"id": "1",
"skills": {
"skill1": "html",
"skill2": "css"
}
}
I have task to filter by skills value, In order to complete my task wrote java logic as follows:
AmazonDynamoDB client = dynamoDBService.getClient();
DynamoDB dynamoDB = new DynamoDB(client);
Table table = dynamoDB.getTable("dummy");
Map<String, String> attributeNames = new HashMap<String, String >();
attributeNames.put("#columnValue", "skills.skill1");
Map<String, AttributeValue> attributeValues = new HashMap<String, AttributeValue>();
attributeValues.put(":val1", new AttributeValue().withS("html"));
ScanSpec scanSpec = new ScanSpec().withProjectionExpression("skills.skill1")
.withFilterExpression("#columnValue = :val1 ").withNameMap(new NameMap().with("#columnValue", "skills.skill1"))
.withValueMap(new ValueMap().withString(":val1", "html"));
ItemCollection<ScanOutcome> items = table.scan(scanSpec);
Iterator<Item> iter = items.iterator();
while (iter.hasNext()) {
Item item = iter.next();
System.out.println("--------"+item.toString());
}
The mentioned code does not help me out. Any solution ?

You can use a ProjectionExpression to retrieve only specific attributes or elements, rather than an entire item. A ProjectionExpression can specify top-level or nested attributes, using document paths.
for example from AWS:
GetItemSpec spec = new GetItemSpec()
.withPrimaryKey("Id", 206)
.withProjectionExpression("Id, Title, RelatedItems[0], Reviews.FiveStar")
.withConsistentRead(true);
Item item = table.getItem(spec);
System.out.println(item.toJSONPretty());

Simple solution to this problem is:
First fetch all the records from the table.
Then iterate over the list of that object.
Extract the skills from each object.
Wrote your logic to do filtering.
Repeat the loop till the last record.

I found solution,scanSpec should be as follows:
ScanSpec scanSpec = new ScanSpec()
.withFilterExpression("#category.#uid = :categoryuid").withNameMap(new NameMap().with("#category","skills").with("#uid",queryString))
.withValueMap(new ValueMap().withString(":categoryuid", queryString));

DynamoDB - Query instead of Scan - how to achive with random generated key

Setup
I have the following table in java and DynamoDB (DynamoDB Mapper) annotations:
#DynamoDBTable(tableName="myentity")
public class MyEntity {
#DynamoDBHashKey
#DynamoDBAutoGeneratedKey
private String id;
#DynamoDBIndexHashKey()
//#DynamoDBIndexRangeKey
private String userId;
private Date eventDate;
The id is randomly generated when saved and the same userId can occur for multiple entities.
The tables where defined via Web GUI like this:
Primary partition key: id (String)
Primary sort key: userId (String)
Issue
I would like to have all the entities for one userId with a query instead of a scan.
Query - does not work like this:
public List<MyEntity> findByUserIdQuery(String userId) {
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBQueryExpression<MyEntity> queryExpression = new DynamoDBQueryExpression<MyEntity>()
.withKeyConditionExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.query(MyEntity.class, queryExpression);
of course I get:
com.amazonaws.AmazonServiceException: Query condition missed key schema element: id (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: 123) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1305)
because I don't know the generated id when I want to read data.
Workaround
Instead I use:
public List<MyEntity> findByOwner(String userId){
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBScanExpression scanExpression = new DynamoDBScanExpression()
.withFilterExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.scan(MyEntity.class, scanExpression);
Possible solutions
The issue is that I never know the randomly generated key for the elements I have. When I want to read I only know the userId and I want to have all entities for them.
I watched the examples here:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBMapper.QueryScanExample.html
...but they know always the key for what they are searching. They are using a combinded key like this:
String partitionKey = forumName + "#" + threadSubject;*
eav.put(":val1", new AttributeValue().withS(partitionKey))
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
DynamoDBQueryExpression<Reply> queryExpression = new DynamoDBQueryExpression<Reply>()
.withKeyConditionExpression("Id = :val1 and ReplyDateTime > :val2").withExpressionAttributeValues(eav);
Question
Is there something like a wildcard for this combinded partition key that I could use? (Something like: "* # myGreatUser")
Other suggestions for db setup and other annotations as key element?
Maybe an Index where I could use query instead of scan?
Maybe somebody has a working example for this kind of use case?

You just need to switch your partition key and sort key around.
#DynamoDBTable(tableName="myentity")
public class MyEntity {
#DynamoDBRangeKey
#DynamoDBAutoGeneratedKey
private String id;
#DynamoDBHashKey
private String userId;
private Date eventDate;
And then query using
public List<MyEntity> findByUserIdQuery(String userId) {
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBQueryExpression<MyEntity> queryExpression = new DynamoDBQueryExpression<MyEntity>()
.withKeyConditionExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.query(MyEntity.class, queryExpression);
By the way, you mentioned that you thought your sort key was userId, but in the example supplied you have a primary hash key of id and a global secondary index hash key of userid. There is no sort key.

How to update a Map or a List on AWS DynamoDB document API?

The new AWS DynamoDB document API allows 2 new data types that correspond directly to the underlying JSON representation: Map (aka JSON object) and List (aka JSON array).
However, I can't find a way to update attributes of these data types without completely overwriting them. In contrast, a Number attribute can be updated by ADDing another number, so in Java you can do something like:
new AttributeUpdate("Some numeric attribute").addNumeric(17);
Similarly you can addElements to an attribute of a Set data type. (In the old API you would use AttributeAction.ADD for both purposes.)
But for a Map or a List, it seems you must update the previous value locally, then PUT it instead of that value, for example in Java:
List<String> list = item.getList("Some list attribute");
list.add("new element");
new AttributeUpdate("Some list attribute").put(list);
This is much less readable, and under some circumstances much less efficient.
So my questions are:
Is there a way to update an attribute of a Map or a List data type without overwriting the previous value? For example, to add an element to a List, or to put an element in a Map?
How would you implement it using the Java API?
Do you know of plans to support this in the future?

Please take a look at UpdateExpression in the UpdateItem API
For example given an item with a list:
{
"hashkey": {"S" : "my_key"},
"my_list" : {"L":
[{"N":"3"},{"N":"7"} ]
}
You can update the list with code like the following:
UpdateItemRequest request = new UpdateItemRequest();
request.setTableName("myTableName");
request.setKey(Collections.singletonMap("hashkey",
new AttributeValue().withS("my_key")));
request.setUpdateExpression("list_append(:prepend_value, my_list)");
request.setExpressionAttributeValues(
Collections.singletonMap(":prepend_value",
new AttributeValue().withN("1"))
);
dynamodb.updateItem(request);`
You can also append to the list by reversing the order of the arguments in the list_append expression.
An expression like: SET user.address.zipcode = :zip would address a JSON map element combined with expression attribute values {":zip" : {"N":"12345"}}

Base on DynamoDB examples, this also work (scala)
val updateItemSpec:UpdateItemSpec = new UpdateItemSpec()
.withPrimaryKey("hashkey", my_key)
.withUpdateExpression("set my_list = list_append(:prepend_value, my_list)")
.withValueMap(new ValueMap()
.withList(":prepend_value", "1"))
.withReturnValues(ReturnValue.UPDATED_NEW)
println("Updating the item...")
val outcome: UpdateItemOutcome = table.updateItem(updateItemSpec)
println("UpdateItem succeeded:\n" + outcome.getItem.toJSONPretty)

A generic function to add or update a key/value pairs. attribute updateColumn should be of type map.
Update tableName attribute name should be passed as attributeName under key:value pairs where primaryKey = primaryKeyValue
public boolean insertKeyValue(String tableName, String primaryKey, String
primaryKeyValue, String attributeName, String newKey, String newValue) {
//Configuration to connect to DynamoDB
Table table = dynamoDB.getTable(tableName);
boolean insertAppendStatus = false;
try {
//Updates when map is already exist in the table
UpdateItemSpec updateItemSpec = new UpdateItemSpec()
.withPrimaryKey(primaryKey, primaryKeyValue)
.withReturnValues(ReturnValue.ALL_NEW)
.withUpdateExpression("set #columnName." + newKey + " = :columnValue")
.withNameMap(new NameMap().with("#columnName", attributeName))
.withValueMap(new ValueMap().with(":columnValue", newValue))
.withConditionExpression("attribute_exists("+ attributeName +")");
table.updateItem(updateItemSpec);
insertAppendStatus = true;
//Add map column when it's not exist in the table
} catch (ConditionalCheckFailedException e) {
HashMap<String, String> map = new HashMap<>();
map.put(newKey, newValue);
UpdateItemSpec updateItemSpec = new UpdateItemSpec()
.withPrimaryKey(primaryKey,primaryKeyValue)
.withReturnValues(ReturnValue.ALL_NEW)
.withUpdateExpression("set #columnName = :m")
.withNameMap(new NameMap().with("#columnName", attributeName))
.withValueMap(new ValueMap().withMap(":m", map));
table.updateItem(updateItemSpec);
insertAppendStatus = true;
} catch(Exception e) {
e.printStackTrace();
}
return insertAppendStatus;
}