I'm using Google App Engine and Objectify 3.1 and slowly learning about denormalization and designing entities based on their usage, but I'm still struggling with some aspects.
I'm currently building a system in which a User entity can participate in a Game entity and need to be able to find all games a user participates in. The way I see it, there's two basic solutions:
Solution 1)
Store a list of user keys (participantKeys) on the game and find the games a member participates in like this:
List<Key<User>> userList = new ArrayList<Key<User>>();
userList.add(new Key<User>(User.class, myUserId));
Collection<Game> games = ofy().query(Game.class).filter("participantKeys in" , userList).list();
Solution 2)
In addition to storing a participant list on the game entity, also store a list of games the user has participated in on the user entity and find the games like this:
User myUser = userDao.getUser(myUserId);
Collection<Game> games = user.getParticipatedGameKeys();
Solution 1 would become pretty slow once there's a lot of games in the system.
Solution 2 would make finding the games faster, but I'll need to constantly keep the list updated as users join and leave games.
The list of games would also become large once a user has been using the system for a long time. I only want to return all games the user is currently participating in, so that would require traversing the list and excluding "historical" games.
Am I missing a more elegant solution? Neither of the above seem very attractive.
Any and all help is greatly appreciated!
Edit:
I decided to try something like Mikl suggested after thinking about alternatives for a long time.. so it's good to hear a solution that's very pretty much exactly like it :-)
I have created a GameParticipation entity which contains a link to the game, a link to the user and all other information that I need to be able to get.
Every time a game is joined, I update the GameParticipation entity to reflect the current state of the game. When a game is left, I delete the entity. Also, when a game is changed, I update all related GameParticipation entities.
I've done a little performance testing and it seems to work reasonably well!
you can also have an another entity I don't know how you can call it (UserGame?) , but in this entity you will store the game key, the user key and also some information you want to access to, for instance the user name, the game name and so on. Then when the user enter a game you will create this entity.
With this entity you can easily retrieve the games a member participates in and also all the users that partipate to a game.
The inconvenient with this method is that, if a user property or a game property change you need to update also the information you stored in the USerGame entity like the userName.
I don't know if it's a good solution but it should work.
Related
I'm writing in Java.
Trying to write my first user time-management mobile app.
I create a user class and a great function which allows the user to create a group of users in which he is admin, from which he has capabilities to track the employees—-
Does it require separate arrays?
Or is there something about databases that I don't yet understand?
Where does all the info from a single user being stored?
Sorry in advance for the noob question.
There are many solutions possible, you can have a separate table called admin where he has the entries of all of his associated employees (user) and yes programmatically you need to access them via an array. More information like uml diagram will make it more clear.
I'm maintaining a system (in Java, with Tomcat, Spring MVC, and Hibernate) where I have to set access rules for user groups. These rules are saved in a database (PostgreSQL) as records / rows. The logic is very simple. Each user of a company's team belongs (is connected) to a group, and each group has a set of rules.
I have to allow administrators to configure (through a web application) rules for groups, so that each rule has a logic and this is recognized and reproduced on the server side.
I need to define rules with parameters, such as:
Authentications only weekends.
Authentications only on weekdays.
Authentications only at a certain time (from time X to time Y).
X authentications per day.
Account expiration from date X
And so on...
My intention is that the company team can organize itself dynamically, just setting up any rules they want at any time, without the need for maintenance every time their policies change.
I've been searching on google and found nothing about it. I know I can do this in Java code, I would have to tie Java code with values of rules names present in database, something that could change in the future (or between companies), and this does not seem right to me. I'm not sure if this is correct, or preferable (maintainable). I appreciate any suggestions, ideas, or corrections (for real).
Note: Team/Groups names may change, but its rules should remain the same (if desired).
EDIT
The database is already modeled and ready. Groups and rules represent values from two different tables, with no logic at all. Querying these values works trivial. However, as I'm maintaining a web application, I'm in charge of creating a code or procedure that applies logic to the choice of rule values.
I was very clear in my question, but I will add more things:
Imagine that my clients (companies) want a website (a web application) that can manage their employees. Every company has teams of employees (groups), each with its function. Otherwise, some employees are sometimes hired as temporary employees.
My duty is to restrict access to the accounts of users who are part of company teams. This will allow business leaders to restrict things according to their policies.
For any company, the process works something like this:
The person in charge defines groups (with names and descriptions).
The same person defines restrictions rules for each group.
User accounts are created and linked to groups with rules.
The accounts are given (assigned) to each person part of the company
team, each according to their function.
Why should this be done?
Management
Control
Security
Speaking more technically now, I do not know where or how I should implement this properly. I know of a way to accomplish this, which is in programming code (Java, in my case), but again, I do not know if this is appropriate.
I also know that it is possible to define users and groups on the database side. But creating and deleting such definitions for each time an employee is hired or his or her length of service is expired can not become practical. My intention is to avoid to the maximum that companies have to spend more money on maintenance (Although sometimes this is obviously impossible).
My question based on a real case can be answered indicating to me an ideal way / approach for this type of scenario, either the solution being something that should be implemented in the database, or something done in the application layer, or both, or something else (I do not have experience to solve this kind of situation properly, so I'm here).
For practical purposes, I have decided to describe what technologies I am using in this system. If you want more information, I'll be happy to show you here.
Also, as this is a question that covers a larger context, not specifically databases, and also not specifically web applications, I have decided to put it here (instead of other StackExchange communities).
Thank you.
Google datastore started off looking so good and has become so frustrating, but maybe it's just that I'm used to relational databases. I'm pretty new to datastore and nosql in general and have done a ton of research but can't seem to find a solution to this problem.
Assume I have a User class that looks like this
class User{
#Id
Long id;
String firstName, lastName;
List<Key<User>> friends;
}
I have another class that will model Events that users have done like so
class Event{
Key<User> user;
Date eventTime;
List<Key<User>> receivers;
}
and now what I'm trying to do is query for events that my friends have done.
In a usual relational way I would say :
select * from Event where user in (select friends from User where id = ?)
Taking that as a starting point I tried doing
// Key<User> userKey = ...
User user = ofy.load.type(User.class).key(userKey).first.now;
List<Key<User>> friends = user.getFriends();
ofy.load.type(Event.class).filter("user in", friends).order("-eventTime")list();
But I heard about this 30 sub-query limit making this unsustainable since I assume eventually someone will have more than 30 friends, not to mention using an 'in' clause will guarantee that you cannot get a cursor to continue loading events. I've done so much research and tried so many options but have yet to find a good way to approach this problem except to say "why Google, why."
Things I've considered :
add an extra field in event that is a copy of the users friendlist and use a single equals on MVP to find events (extremely wasteful since there may be many many events.
split event query up into batches of 30 friends at a time and somehow determine a way to ensure continued retrieval from a synthetic cursor based on time, and merge them (problem is waay too many edge cases and makes reading events very difficult.)
I would really appreciate any input you could offer since I am 100% out of ideas
TL;DR ~ GAE has limit on how many items an in-clause can handle and fml.
You come from a relational database background, so the concept of denormalization is probably a bit painful - I know it was for me.
Right now you have a single table that contains all events from all users. This approach works well in relational databases but is a nightmare in the datastore for the reasons you named.
So to solve this concrete problem you could restructure your data as follows:
All users have two timelines. One for their own posts and one from friends' posts. (There could be a third timeline for public stuff.)
When a new event is published, it is written to the timeline of the user who created it, and to all the timelines of the receiving users. (You may want to add references of the third-party timelines in the user's timeline, so you know what to delete when the user decides to delete an event)
Now every user has access to complete timelines, his/her own and the timeline that was created by third-party events. Those timelines are easy to query and you will not require sub-selects at all.
There are downsides to this approach:
Writing cost is higher. You have to write way more timelines than you had to until now. You will probably have to put this in a task queue to have enough time to write to all those timelines.
You're using a lot more storage, BUT storage is really cheap, I'm guessing the storage will be cheaper than running expensive queries in the long run.
What you get in return though is lightning fast responses with simple queries through this denormalization. All that remains is to merge the responses from the different timelines in the UI (you can do it on the server side, but i would do it in the UI)
My goal with this project is to create a quiz app that has the ability to quiz from different banks of data. I would like the user to have the option to select which subjects they wish to be tested on, and then once selected the different banks can be combined and randomized for testing purposes. This is my first time utilizing databases and I was wondering if I could get a little input on the most economic way for this to happen. Are multiple tables my best option? Later down the road I hope to implement the ability to keep records of how well was done on each subject. If someone could point me in the right direction or give me suggestions for this I would be very appreciative.
I literally just finished creating my quiz app and I created one table to manage the questions and another to manage the scores. As long as you use a category column, you can set the cursor to read through questions with the categories you've selected. The cursor can return how many questions are in the categories selected, and you can also quite easily shuffle the numbers to get a random assortment of questions. If you've got any specific questions, just ask
(As for the how well people have done, I suggest putting that in another different database. It's easier as that database is probably never going to be upgraded, you don't have to modify it once it is created).
I am developing a facebook type application for my institute.
and I am stuck at the friends module. i.e. How to know if the particular users are one's friends.
I googled a lot but didn't get any satisfactory answers.
What I got is : there will be many friends of a person and implementing users and their friends in seperate table will only increase redundancy and large DB size.
I thought of using a graph with vertices as users and edges as connection .
But how to implement something like that in db.
Or How Facebook handles such huge amount of relationships?
Personally, I would have a dedicated table for it:
You could have a table with just two columns: userID and friendID
Since the relationships between users in the db will be many-to-many, normalizing it requires a link table which breaks it into many-to-one-to-many
http://dev.mysql.com/tech-resources/articles/intro-to-normalization.html#03
This kind of problems are usually solved by using a different type of database. For a social network, a graph database should make sense, as nodes and relationships are first class citizens in it. There's a social network example for the Neo4j graph database, the full source code of the example is included in the standard dowload package. I've also written a blog post on this theme, with another example as starting point.