I am currently trying to load info from a dynamo DB table using a LSI and dynamoDB mapper class. Assume i have the following code
Conisder this class
class Employee {
#DynamoDBHashKey
public String getID() {
return ID;
}
#DynamoDBRangeKey
public String getFirstName() {
return firstName;
}
#DynamoDBIndexRangeKey(localSecondaryIndexName = "my-lsi")
public String getLastName() {
return lastName;
}
public String getMyotherfield() {
return myotherfield;
}
}
Consider this piece of code to retrieve info using LSI and dynamoDB mapper
Employee obj = new Employee();
obj.setID("221");
obj.setLastName("SOMEONE");
DynamoDBQueryExpression<Employee> queryExpression = new DynamoDBQueryExpression<>();
queryExpression.setHashKeyValues(obj);
queryExpression.setIndexName("my-lsi");
queryExpression.setProjectionExpression("myotherfield");
PaginatedQueryList<Employee> paginatedQueryList = mapper.query(Employee.class, queryExpression);
Iterator<Employee> iterator = paginatedQueryList.iterator();
while (iterator.hasNext()) {
Employee pp = iterator.next();
// some code
}
[EDIT]
Is this the right way to query LSI with mapper?
If not what is a better way to fetch data from the table using LSI and dynamoDBMapper? (in terms of performance)
i also found this https://java.awsblog.com/post/Tx3GYZEVGO924K4/The-DynamoDBMapper-Local-Secondary-Indexes-and-You but its a 2013 link. I am not able to find latest documentation on querying with LSI and mapper.
Note that in the code you have given, you haven't set the range key at all. Use queryExpression.setRangeKeyConditions() to do so.
Here is a code sample:
Map<String, Condition> rangeKeyConditions = new HashMap<>();
rangeKeyConditions.put("lastName", new Condition()
.withComparisonOperator(ComparisonOperator.EQ)
.withAttributeValueList(new AttributeValue().withS("SOMEONE")));
queryExpression.setRangeKeyConditions(rangeKeyConditions);`
[Update]:
It is not sufficient (nor necessary) to set the range key value ("SOMEONE") in the object (obj), since that object is used only for setting the hashKey. Have a look at the examples here
Related
I have next table:
#DynamoDBTable(tableName = "Table")
#Getter
#Setter
public class Table {
#DynamoDBHashKey
private String id;
#DynamoDBAttribute
private List<String> list;
}
I need to add values to list from Java. I tried next approach:
UpdateItemRequest request = new UpdateItemRequest()
.withTableName("Table")
.withKey(Collections.singletonMap("id", new AttributeValue().withS("123")))
.withUpdateExpression("SET #ri = list_append(:vals, #ri)")
.withExpressionAttributeNames(Collections.singletonMap("#ri","list"))
.withExpressionAttributeValues(Collections.singletonMap(":vals",
new AttributeValue().withSS("1","2","3")));
dynamoDB.updateItem(request);
But it fails without providing exact reason:
Unable to unmarshall exception response with the unmarshallers provided
I got two questions:
What is wrong with my request ?
Why I get unmarshall exception and how to fix it to get real reason ?
P.S. Other update requests work fine e.g. I can update id with same approach without problem.
Updated:
I've found a way to update values from Java
UpdateItemRequest request = new UpdateItemRequest()
.withTableName("Table")
.withKey(Collections.singletonMap("id", new AttributeValue().withS("123")))
.withUpdateExpression("SET #ri = list_append(:vals, #ri)")
.withExpressionAttributeNames(Collections.singletonMap("#ri","list"))
.withExpressionAttributeValues(Collections.singletonMap(":vals",
new AttributeValue().withL(new AttributeValue().withS("1"),AttributeValue().withS("2"),AttributeValue().withS("3"))));
dynamoDB.updateItem(request);
But it only allows to add one by one values in list, but I need to add several.
private void updateList(String id, String field, List<AttributeValue> values) {
UpdateItemRequest request = new UpdateItemRequest()
.withTableName("Table")
.withKey(Collections.singletonMap("id", new AttributeValue().withS(id)))
.withUpdateExpression("SET #field = list_append(:values, #field)")
.withExpressionAttributeNames(Collections.singletonMap("#field", field))
.withExpressionAttributeValues(Collections.singletonMap(":values",
new AttributeValue().withL(values)));
dynamoDB.updateItem(request);
}
And use next method to convert to list of AttributeValue
private List<AttributeValue> buildAttributeValuesList(List<String> values) {
return values
.stream()
.map(it -> new AttributeValue().withS(it))
.collect(Collectors.toList());
}
I'm using HIbernate search(5.8.2). The search works smoothly on every field except for the primary key. It returns an empty list when I pass anything to it. I followed the hibernate documentation, used #documentId annotation for primary key. What am I missing?
Here's my model:
#SuppressWarnings("serial")
#Entity
#Indexed
#Table(name = "MAIN",schema="maindb")
public class MAIN implements Serializable {
#Id
#DocumentId
private String poNo; // my primary key which has values like "PO123"
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
private String postatus;
My search function:
public List<?> search(String poNumber, String status) {
QueryBuilder qb =
fullTextSession.getSearchFactory().buildQueryBuilder().forEntity(MAIN.class).get();
BooleanQuery.Builder finalLuceneQuery = new BooleanQuery.Builder();
org.hibernate.query.Query hibQuery =
fullTextSession.createFullTextQuery(finalLuceneQuery.build(),MAIN.class);
org.apache.lucene.search.Query querypono1 =
qb2.simpleQueryString().onField("poNo").matching(poNumber)
.createQuery();
org.apache.lucene.search.Query queryStatus =
qb.simpleQueryString().onField("po01_status")
.matching("postatus").createQuery();
finalLuceneQuery.add(querypono1, BooleanClause.Occur.MUST);
finalLuceneQuery.add(queryStatus , BooleanClause.Occur.MUST);
hibQuery.setFirstResult(0);
List<?> resultArchive = new ArrayList<String>();
try {
result = hibQuery.getResultList();
} catch (Exception e) {
e.printStackTrace();
// log.log(ERROR, "ERROR FETCHING RESULT LIST FROM DATABASE");
}
return result;
}
The issue is that "PO123" is transformed to "po123" by the simple query string parser. I wonder why, I have to check that, it's probably a bug, or it's at least an unexpected behavior.
That being said, you shouldn't use the simpleQuery() entry point for an exact matching.
Replace:
org.apache.lucene.search.Query querypono1 = qb2.simpleQueryString().onField("poNo").matching(poNumber).createQuery();
By:
org.apache.lucene.search.Query querypono1 = qb2.keyword().onField("poNo").matching(poNumber).createQuery();
(keyword() instead of simpleQueryString())
I will follow up on this issue though as it's not the behavior I would have expected. Thanks for raising it.
-> JIRA issue: https://hibernate.atlassian.net/browse/HSEARCH-3039 , will be included in the upcoming 5.10.0.Final.
Setup
I have the following table in java and DynamoDB (DynamoDB Mapper) annotations:
#DynamoDBTable(tableName="myentity")
public class MyEntity {
#DynamoDBHashKey
#DynamoDBAutoGeneratedKey
private String id;
#DynamoDBIndexHashKey()
//#DynamoDBIndexRangeKey
private String userId;
private Date eventDate;
The id is randomly generated when saved and the same userId can occur for multiple entities.
The tables where defined via Web GUI like this:
Primary partition key: id (String)
Primary sort key: userId (String)
Issue
I would like to have all the entities for one userId with a query instead of a scan.
Query - does not work like this:
public List<MyEntity> findByUserIdQuery(String userId) {
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBQueryExpression<MyEntity> queryExpression = new DynamoDBQueryExpression<MyEntity>()
.withKeyConditionExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.query(MyEntity.class, queryExpression);
of course I get:
com.amazonaws.AmazonServiceException: Query condition missed key schema element: id (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: 123) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1305)
because I don't know the generated id when I want to read data.
Workaround
Instead I use:
public List<MyEntity> findByOwner(String userId){
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBScanExpression scanExpression = new DynamoDBScanExpression()
.withFilterExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.scan(MyEntity.class, scanExpression);
Possible solutions
The issue is that I never know the randomly generated key for the elements I have. When I want to read I only know the userId and I want to have all entities for them.
I watched the examples here:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBMapper.QueryScanExample.html
...but they know always the key for what they are searching. They are using a combinded key like this:
String partitionKey = forumName + "#" + threadSubject;*
eav.put(":val1", new AttributeValue().withS(partitionKey))
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
DynamoDBQueryExpression<Reply> queryExpression = new DynamoDBQueryExpression<Reply>()
.withKeyConditionExpression("Id = :val1 and ReplyDateTime > :val2").withExpressionAttributeValues(eav);
Question
Is there something like a wildcard for this combinded partition key that I could use? (Something like: "* # myGreatUser")
Other suggestions for db setup and other annotations as key element?
Maybe an Index where I could use query instead of scan?
Maybe somebody has a working example for this kind of use case?
You just need to switch your partition key and sort key around.
#DynamoDBTable(tableName="myentity")
public class MyEntity {
#DynamoDBRangeKey
#DynamoDBAutoGeneratedKey
private String id;
#DynamoDBHashKey
private String userId;
private Date eventDate;
And then query using
public List<MyEntity> findByUserIdQuery(String userId) {
Map<String, AttributeValue> eav = new HashMap<String, AttributeValue>();
eav.put(":val1", new AttributeValue().withS(userId));
DynamoDBQueryExpression<MyEntity> queryExpression = new DynamoDBQueryExpression<MyEntity>()
.withKeyConditionExpression("userId = :val1").withExpressionAttributeValues(eav);
List<MyEntity> list = mapper.query(MyEntity.class, queryExpression);
By the way, you mentioned that you thought your sort key was userId, but in the example supplied you have a primary hash key of id and a global secondary index hash key of userid. There is no sort key.
The ask: a function to retrieve a single Entity from the Google App Engine Datastore based on a property that is not its Key or otherwise return null if no such object is found.
Here is the function I have currently:
public Entity find(DatastoreService datastore, String kind, String property, String value) {
Filter propertyFilter =
new FilterPredicate(property, FilterOperator.EQUAL, value);
Query q = new Query(kind).setFilter(propertyFilter);
List<Entity> results =
datastore.prepare(q).asList(FetchOptions.Builder.withDefaults());
if (results.isEmpty()) {
return null;
}
return results.get(0);
}
Is there a one-shot API I could use instead or are there any optimization suggestions?
You could use PreparedQuery#asSingleEntity():
public Entity find(DatastoreService datastore,
String kind, String property, String value)
throws TooManyResultsException {
Filter propertyFilter =
new FilterPredicate(property, FilterOperator.EQUAL, value);
Query q = new Query(kind).setFilter(propertyFilter);
return datastore.prepare(q).asSingleEntity();
}
The only real difference between this and your code is that the underlying query sets a LIMIT of 2.
I am using Morphia, the Pojo mapper for MongoDB, and I find difficult a task that in my view should be very simple: getting an object by id. I am able to find all the objects in a collection but I cannot figure out the simple task of querying using an id I got from the list. I am actually talking about the ObjectId. If I try to render it in JSON I see
This question seems incomplete.
It also seems like the answer to you question is on the Morphia QuickStart page. Seems to be as simple as follows.
Datastore ds = morphia.createDatastore("testDB");
String hotelId = ...; // the ID of the hotel we want to load
// and then map it to our Hotel object
Hotel hotel = ds.get(Hotel.class, hotelId);
So you'll definitely need more details.
Datastore ds = morphia.createDatastore("testDB");
String hotelId = "516d41150364a6a6697136c0"; // the ID of the hotel we want to load
ObjectId objectId = new ObjectId(hotelId);
// and then map it to our Hotel object
Hotel hotel = ds.get(Hotel.class, objectId);
If you're finding by id and the id is provided by the user (means that it could be whatever type of data), you shouldn't use the solutions given above.
As explained in the documentation, an ObjectId consists of 12 bytes, so if you pass something else to new ObjectId(myValue), your code will throw an IllegalArgumentException.
Here is how I implemented the method to find by id :
public Model findById(String id) throws NotFoundException {
if (!ObjectId.isValid(id)) {
throw new NotFoundException();
}
ObjectId oid = new ObjectId(id);
Model m = datastore().find(Model.class).field("_id").equal(oid).get();
if (m == null) {
throw new NotFoundException();
}
return m;
}
The previous explanation used the deprecated methods: datastore().find(Model.class).field("_id").equal(oid).get(); My variant of it on Morphia 2.2.10:
public Optional<Hotel> findById(final String id) {
if (!ObjectId.isValid(id)) {
throw new SomeException();
}
final Query<Hotel> query = datastore.find(Hotel.class)
.filter(eq("_id", new ObjectId(id)));
return Optional.ofNullable(query.first());
}