I am using DynamoDBMapper for a class, let's say "User" (username being the primary key) which has a field on it which says "Status". It is a Hash+Range key table, and everytime a user's status changes (changes are extremely infrequent), we add a new entry to the table alongwith the timestamp (which is the range key). To fetch the current status, this is what I am doing:
DynamoDBQueryExpression expr =
new DynamoDBQueryExpression(new AttributeValue().withS(userName))
.withScanIndexForward(false).withLimit(1);
PaginatedQueryList<User> result =
this.getMapper().query(User.class, expr);
if(result == null || result.size() == 0) {
return null;
}
for(final User user : result) {
System.out.println(user.getStatus());
}
This for some reason, is printing all the statuses a user has had till now. I have set scanIndexForward to false so that it is in descending order and I put limit of 1. I am expecting this to return the latest single entry in the table for that username.
However, when I even look into the wire logs of the same, I see a huge amount of entries being returned, much more than 1. For now, I am using:
final String currentStatus = result.get(0).getStatus();
What I am trying to understand here is, what is whole point of the withLimit clause in this case, or am I doing something wrong?
In March 2013 on the AWS forums a user complained about the same problem.
A representative from Amazon sent him to use the queryPage function.
It seems as if the limit is not preserved for elements but rather a limit on chunk of elements retrieved in a single API call, and the queryPage might help.
You could also look into the pagination loading strategy configuration
Also, you can always open a Github issue for the team.
Related
Here I have a dilemma.
Let's imagine that we have a sql table like this
enter image description here
It could be a problem when two or more users overwrite data in the table.
How should I check if the place hasn't been taken before update data?
I have two options
in SQL query:
UPDATE ticket SET user_user_id = ? WHERE place = ? AND user_user_id is NULL
or in Service layer:
try {
Ticket ticket = ticketDAO.read(place)
if (ticket.getUser() == null) {
ticket.setUser(user)
ticketDAO.update(ticket)
}else {
throw new DAOException("Place has been already tooken")
}
What way is safer and commonly used in practice?
Please, share your advice.
Possible approach here is to go ahead with SQL query. After query execution check number of rows modified in ticketDAO.update method. If 0 rows modified then throw exception DAOException.
I'm writing an android app that supports exporting the app database to various formats. I don't want to run out of memory, but I want to page the results easily without receiving updates when it changes. So I put it in a service, and came up with the following method of paging.
I use a limit clause in my query to limit the number of results returned and I'm sorting on the primary key. So it should be fast. I use a set of nested for loops to execute the series of queries until no results are returned, and walk through the given results, so that's linear. It's in a service, so it doesn't matter that I'm using immediate result things here.
I feel like I might be doing something bad here. Am I?
// page through all results
for (List<CountedEventType> typeEvents = dao.getEventTypesPaged2(0);
typeEvents.size() > 0;
typeEvents = dao.getEventTypesPaged2(typeEvents.get(typeEvents.size() - 1).uid)
) {
for (CountedEventType type : typeEvents) {
// Do something for every result.
}
}
Here's my dao method.
#Dao
interface ExportDao {
#Query("SELECT * FROM CountedEventType WHERE uid > :lastUid ORDER BY uid ASC LIMIT 4")
List<CountedEventType> getEventTypesPaged2(int lastUid);
}
I have the following data structure in Firebase Firestore to represent a many to many relationship between clients and users:
Clients
clientId1 {
users (object): {
userId1: true
userId2: true
}
}
clientId2 {
users (object): {
userId1: true
}
}
I query it on Android using the following query:
db.collection("clients").whereEqualTo("users."+uid, true);
For userId2, the query should only return clientId1.
If I set the rule to (allow read: if true;) when I execute the query above I get the correct clients returned.
I would also like to set up a database rule to prevent userId2 from seeing clientId2.
I tried this rule but I get no results returned:
match /clients/{clientId} {
//Allow read if the user exists in the user collection for this client
allow read: if users[request.auth.uid] == true;
}
I also tried:
match /clients/{clientId} {
//Allow read if the user exists in the user collection for this client
allow read: if resource.data.users[request.auth.uid] == true;
}
But neither of the above rules returns any clients.
How do I write the rule?
I am going to answer my own question as I was just doing something silly.
My data structure is fine and the correct syntax for my rule is this one:
match /clients/{clientId} {
//Allow read if the user exists in the user collection for this client
allow read: if resource.data.users[request.auth.uid] == true;
}
Given this:
Cloud Firestore evaluates a query against its potential result set
instead of the actual field values for all of your documents. If a
query could potentially return documents that the client does not have
permission to read, the entire request fails.
This Android query does correctly implement the right filter for the rule:
db.collection("clients").whereEqualTo("users."+uid, true);
I am yet to implement my adapter properly. I wanted to see if I could get the correct data structure / rules / query working first. I was calling it from another listener that was listening on the entire client collection (which fails the rule) and therefore this query was not being called. Earlier when I set the rule to (allow read: if true;) the initial listener was executing my query and returning the correct results. This lead me to believe my rule was incorrect, when it wasn't.
As per the official documentation regarding Firestore Security Rules:
When writing queries to retrieve documents, keep in mind that security rules are not filters—queries are all or nothing. To save you time and resources, Cloud Firestore evaluates a query against its potential result set instead of the actual field values for all of your documents. If a query could potentially return documents that the client does not have permission to read, the entire request fails.
So you cannot filter the documents that exist in your database using security rules.
I am using spring, hibernate and postgreSQL.
Let's say I have a table looking like this:
CREATE TABLE test
(
id integer NOT NULL
name character(10)
CONSTRAINT test_unique UNIQUE (id)
)
So always when I am inserting record the attribute id should be unique
I would like to know what is better way to insert new record (in my spring java app):
1) Check if record with given id exists and if it doesn't insert record, something like this:
if(testDao.find(id) == null) {
Test test = new Test(Integer id, String name);
testeDao.create(test);
}
2) Call straight create method and wait if it will throw DataAccessException...
Test test = new Test(Integer id, String name);
try{
testeDao.create(test);
}
catch(DataAccessException e){
System.out.println("Error inserting record");
}
I consider the 1st way appropriate but it means more processing for DB. What is your opinion?
Thank you in advance for any advice.
Option (2) is subject to a race condition, where a concurrent session could create the record between checking for it and inserting it. This window is longer than you might expect because the record might be already inserted by another transaction, but not yet committed.
Option (1) is better, but will result in a lot of noise in the PostgreSQL error logs.
The best way is to use PostgreSQL 9.5's INSERT ... ON CONFLICT ... support to do a reliable, race-condition-free insert-if-not-exists operation.
On older versions you can use a loop in plpgsql.
Both those options require use of native queries, of course.
Depends on the source of your ID. If you generate it yourself you can assert uniqueness and rely on catching an exception, e.g. http://docs.oracle.com/javase/1.5.0/docs/api/java/util/UUID.html
Another way would be to let Postgres generate the ID using the SERIAL data type
http://www.postgresql.org/docs/8.1/interactive/datatype.html#DATATYPE-SERIAL
If you have to take over from an untrusted source, do the prior check.
I have a JUnit test, which includes use of Mybatis. At the start of the test I'm retrieving a count of records in the table.
At the end of the test I expect an additional record to be present in the table, and have an assert to verify this condition. However I find that the second query returns exactly the same number of records as the first one did.
I know a new record has definitely been inserted in the table.
I thought this may be related to caching, so I tried flushing all caches associated with the session. I also tried using setCacheEnabled(false), but still the same result.
Here's my code fragment -
#Test
public void config_0_9() {
session.getConfiguration().setCacheEnabled(false);
cfgMapper = session.getMapper(CfgMapper.class);
int oldRecords = cfgMapper.countByExample(null);
messageReprocessor.processSuspendedMessages();
session.commit();
int newRecords = cfgMapper.countByExample(null);
assertTrue(newRecords == oldRecords + 1);
}