I have a table in dynamo db.
Below is the key for this table.
partitionKey - campaignId
sortKey - email
I have created a local secondary index for this table.Below is the key for local secondary index
partitionKey - campaignId
sortKey - subStatus
As per the dynamo db you should include primary keys of the table and the index (as key), with last evaluated values (as attribute value) when you setting ExclusiveStartKey.
Below is the code snippet of code.
public QueryResultPage<Prospect> getPaginatedProspectsByCampaignIdAndSubStatus(String campaignId, ProspectSubStatus subStatus, Integer limit, Map<String, AttributeValue> lastEvaluatedKey, String email) {
Prospect prospect = new Prospect();
prospect.setCampaignId(campaignId);
Map<String, AttributeValue> exclusiveStartKey = new HashMap<String, AttributeValue>();
if(!lastEvaluatedKey.isEmpty()) {
exclusiveStartKey.put("campaingId", lastEvaluatedKey.get("campaingId"));
exclusiveStartKey.put("subStatus", lastEvaluatedKey.get("subStatus"));
exclusiveStartKey.put("email", new AttributeValue().withS(email));
}
Condition rangeKeyCondition = new Condition()
.withComparisonOperator(ComparisonOperator.EQ)
.withAttributeValueList(new AttributeValue().withS(subStatus.toString()));
DynamoDBQueryExpression queryExpression = null;
if(exclusiveStartKey.isEmpty()) {
queryExpression = new DynamoDBQueryExpression<Prospect>()
.withHashKeyValues(prospect)
.withIndexName("subStatus-index")
.withRangeKeyCondition("subStatus", rangeKeyCondition)
.withConsistentRead(false).withLimit(limit);
} else {
queryExpression = new DynamoDBQueryExpression<Prospect>()
.withHashKeyValues(prospect)
.withIndexName("subStatus-index")
.withExclusiveStartKey(exclusiveStartKey)
.withRangeKeyCondition("subStatus", rangeKeyCondition)
.withConsistentRead(false).withLimit(limit);
}
QueryResultPage<Prospect> prospects = mapper.queryPage(Prospect.class, queryExpression);
return prospects;
}
But when run it, I am getting below error. Can someone please help me out to understand what I am doing wrong and how can I achieve the pagination in it.
Related
Creation of table:
final ArrayList<KeySchemaElement> indexKeySchema = new ArrayList<KeySchemaElement>();
indexKeySchema.add(new KeySchemaElement().withAttributeName("expiresOn").withKeyType(KeyType.HASH));
indexKeySchema.add(new KeySchemaElement().withAttributeName("createdAt").withKeyType(KeyType.RANGE));
GlobalSecondaryIndex index = new GlobalSecondaryIndex()
.withIndexName("expiresOnIndex")
.withProvisionedThroughput(new ProvisionedThroughput()
.withWriteCapacityUnits((long) 10)
.withReadCapacityUnits((long) 1))
.withProjection(new Projection().withProjectionType(ProjectionType.ALL));
index.setKeySchema(indexKeySchema);
dynamoDbclient.createTable(new CreateTableRequest()
.withProvisionedThroughput(new ProvisionedThroughput(1000L, 1000L))
.withTableName(tableName)
.withAttributeDefinitions(Arrays.asList(new AttributeDefinition(hashColumn, ScalarAttributeType.S),
new AttributeDefinition("expiresOn", ScalarAttributeType.S),
new AttributeDefinition("createdAt", ScalarAttributeType.S)))
.withKeySchema(Arrays.asList(new KeySchemaElement(hashColumn, KeyType.HASH)))
.withGlobalSecondaryIndexes(index));
And this is the query:
final Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(startTime.toString()));
eav.put(":val2", new AttributeValue().withS(endTime.toString()));
final DynamoDBQueryExpression<MyEntity> queryExpression =
new DynamoDBQueryExpression<MyEntity>()
.withKeyConditionExpression("expiresOn BETWEEN :val1 AND :val2")
.withExpressionAttributeValues(eav)
.withConsistentRead(false)
.withIndexName("expiresOnIndex");
return dynamoDBMapper.query(MyEntity.class, queryExpression);
Always getting Query key condition not supported. I'm not sure what's wrong here. I read couple of stackoverflow answers which suggested to use GSI, but doesn't work for me.
When you Query a DDB table or index,
The BETWEEN key condition can only be used on the Sort (aka Range) Key, the hash key must be an always be an EQUAL condition...
In a DynamoDB table where CustomerStatus is a column in which item may contain 'Active','Inactive' or 'Deleted' as CustomerStatus. I want to fetch all customers whose status is 'Active' as well 'Inactive'.
Here is the code sample I am using, but I am not sure how to get it done.
private Object data(MuleEventContext eventContext) {
List<Object> finalJson = new ArrayList<Object>();
String tableName = "Customers";
NameMap nameMap = new NameMap();
nameMap.put("#v_status", "CustomerStatus");
ValueMap valueMap = new ValueMap();
valueMap.put(":v_statusval", "Deleted");
BasicAWSCredentials cre = new BasicAWSCredentials(accesKey,secretKey);
AmazonDynamoDB dynamoDB1 = AmazonDynamoDBClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(cre)).withRegion(Regions.EU_WEST_1).build();
DynamoDB dynamoDB = new DynamoDB(dynamoDB1);
Table table = dynamoDB.getTable(tableName);
QuerySpec querySpec = new QuerySpec().withKeyConditionExpression("#v_status != :v_statusval")
.withNameMap(nameMap)
.withValueMap(valueMap);
ItemCollection<QueryOutcome> items = null;
Iterator<Item> iterator = null;
Item item = null;
try {
items = table.query(querySpec);
iterator = items.iterator();
while (iterator.hasNext()) {
item = iterator.next();
finalJson.add(item.asMap());
}
} catch (Exception e) {
logger.info(e.getMessage());
}
return finalJson;
}
You can't filter on a HASH key.
I would need to know which attribute is the HASH key and which is the RANGE key.
If the CustomerStatus is not the HASH key, the best way is to create a secondary index for this attribute and do 2 different queries:
One for the Active;
Another one for the Inactive.
This is the most efficient way to do this kind of query.
Another way is doing a scan and filtering the attribute, using the IN operand, as stated in Syntax for Condition Expressions. But this will read all data from your table, can be slow, and consume a lot capacity.
I have a dynamodb Table and I want to do a conditional update operation on that using aws sdk java library. I have the table with hashkey named as "Id" and sortkey named as "Sk" . I have 5 other fields in the table and i have to update 2 fields based on the condition on the remaining fields. The condition is "total_record_count = record_passed + records_failed" , where total_record_count, record_passed and records_failed are the fields in the table.
Below is the code
DynamoDB dynamoDB = new DynamoDB(amazonDynamoDB);
Table table = dynamoDB.getTable("ciw_ocrResponse_upload");
Map<String, String> expressionAttributeNames = new HashMap<String, String>();
expressionAttributeNames.put("#updatedAt", "updatedAt");
expressionAttributeNames.put("#statusOfTracker","statusOfTracker");
Timestamp timestamp = new Timestamp(System.currentTimeMillis());
Map<String, Object> eav = new HashMap<>();
eav.put(":val1", value);
eav.put(":val2", timestamp.getTime());
UpdateItemSpec updateItemSpec = new UpdateItemSpec().withPrimaryKey("Id", requestId, "Sk", sortKey)
.withUpdateExpression("set #updatedAt = :val2, #statusOfTracker = :val1")
.withValueMap(eav)
.withNameMap(expressionAttributeNames)
.withConditionExpression(" total_record_count = record_passed + records_failed")
.withReturnValues(ReturnValue.UPDATED_NEW);
table.updateItem(updateItemSpec);
I am getting the error as Invalid ConditionExpression: Syntax error; token: "+", near: "record_passed + records_failed"
According to the docs this arithmetic operation is not supported.
Workaround is to create a total_record_count field which you need to calculate and add on every new item created. Also to add it to all existing items.
Setup
I have the following table in java and DynamoDB (DynamoDB Mapper) annotations:
#DynamoDBTable(tableName="myentity")
public class MyEntity {
#DynamoDBHashKey
#DynamoDBAutoGeneratedKey
private String id;
#DynamoDBIndexHashKey()
//#DynamoDBIndexRangeKey
private String userId;
private Date eventDate;
The id is randomly generated when saved and the same userId can occur for multiple entities.
The tables where defined via Web GUI like this:
Primary partition key: id (String)
Primary sort key: userId (String)
Issue
I would like to have all the entities for one userId with a query instead of a scan.
Query - does not work like this:
public List<MyEntity> findByUserIdQuery(String userId) {
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBQueryExpression<MyEntity> queryExpression = new DynamoDBQueryExpression<MyEntity>()
.withKeyConditionExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.query(MyEntity.class, queryExpression);
of course I get:
com.amazonaws.AmazonServiceException: Query condition missed key schema element: id (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: 123) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1305)
because I don't know the generated id when I want to read data.
Workaround
Instead I use:
public List<MyEntity> findByOwner(String userId){
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBScanExpression scanExpression = new DynamoDBScanExpression()
.withFilterExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.scan(MyEntity.class, scanExpression);
Possible solutions
The issue is that I never know the randomly generated key for the elements I have. When I want to read I only know the userId and I want to have all entities for them.
I watched the examples here:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBMapper.QueryScanExample.html
...but they know always the key for what they are searching. They are using a combinded key like this:
String partitionKey = forumName + "#" + threadSubject;*
eav.put(":val1", new AttributeValue().withS(partitionKey))
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
DynamoDBQueryExpression<Reply> queryExpression = new DynamoDBQueryExpression<Reply>()
.withKeyConditionExpression("Id = :val1 and ReplyDateTime > :val2").withExpressionAttributeValues(eav);
Question
Is there something like a wildcard for this combinded partition key that I could use? (Something like: "* # myGreatUser")
Other suggestions for db setup and other annotations as key element?
Maybe an Index where I could use query instead of scan?
Maybe somebody has a working example for this kind of use case?
You just need to switch your partition key and sort key around.
#DynamoDBTable(tableName="myentity")
public class MyEntity {
#DynamoDBRangeKey
#DynamoDBAutoGeneratedKey
private String id;
#DynamoDBHashKey
private String userId;
private Date eventDate;
And then query using
public List<MyEntity> findByUserIdQuery(String userId) {
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBQueryExpression<MyEntity> queryExpression = new DynamoDBQueryExpression<MyEntity>()
.withKeyConditionExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.query(MyEntity.class, queryExpression);
By the way, you mentioned that you thought your sort key was userId, but in the example supplied you have a primary hash key of id and a global secondary index hash key of userid. There is no sort key.
I have a DynamoDB attribute whose value is a map from a Number to a String. I am trying to put in a new key-value pair. From what I've read, this seems to be possible, but I don't know how.
I assume the solution is similar to the one in the link below:
How to update a Map or a List on AWS DynamoDB document API?
But I do not believe the example is on putting in a new item to a map. Could someone show me how to put in an item to a map?
Thanks.
EDIT:
I do not want to get the item, locally make the changes, and put it back. I am working with multiple clients who might interact concurrently (and I assume the update by dynamo ensures there will be no race conditions).
With the following arguments to UpdateItem, you can condition adding a map entry at #number when map.#number does not exist in the map already:
UpdateExpression = "SET map.#number = :string"
ExpressionAttributeNames = { "#number" : "1" }
ExpressionAttributeValues = { ":string" : "the string to store in the map at key value 1" }
ConditionExpression = "attribute_not_exists(map.#number)"
Visit http://www.tryyourskill.com/aws/insert-or-append-key-values-map-in-dynamodb-using-java may help you with the code snippet to add or append values in map column
public boolean insertKeyValue(String tableName, String primaryKey, String
primaryKeyValue, String updateColumn, String newKey, String newValue) {
//Configuration to connect to DynamoDB
Table table = dynamoDB.getTable(tableName);
boolean insertAppendStatus = false;
try {
//Updates when map is already exist in the table
UpdateItemSpec updateItemSpec = new UpdateItemSpec()
.withPrimaryKey(primaryKey, primaryKeyValue)
.withReturnValues(ReturnValue.ALL_NEW)
.withUpdateExpression("set #columnName." + newKey + " = :columnValue")
.withNameMap(new NameMap().with("#columnName", updateColumn))
.withValueMap(new ValueMap().with(":columnValue", newValue))
.withConditionExpression("attribute_exists("+ updateColumn +")");
table.updateItem(updateItemSpec);
insertAppendStatus = true;
//Add map column when it's not exist in the table
} catch (ConditionalCheckFailedException e) {
HashMap<String, String> map = new HashMap<>();
map.put(newKey, newValue);
UpdateItemSpec updateItemSpec = new UpdateItemSpec()
.withPrimaryKey(primaryKey,primaryKeyValue)
.withReturnValues(ReturnValue.ALL_NEW)
.withUpdateExpression("set #columnName = :m")
.withNameMap(new NameMap().with("#columnName", updateColumn))
.withValueMap(new ValueMap().withMap(":m", map));
table.updateItem(updateItemSpec);
insertAppendStatus = true;
} catch(Exception e) {
e.printStackTrace();
}
return insertAppendStatus;
}