I am currently writing a program for which I need to select an entity from the database which has multiple one to many relationships. In this program the main object a Route is made of many Route Points has many Comments and many Reports. This means if i want to select a Route from the database along with its Route Points, Comments and Reports as a result of the join I am going to get multiple rows back with duplicate information in.
As far as I am aware this will mean I will have to loop on each row and keep a set of if I have already "seen" this RoutePoint, Report or Comment. As a Route can have a large number of Route Points I imagine this would be quite slow as well as been quite complicated and messy. I have included below what this would look like.
HashSet<String> commentIds = new HashSet<String>();
HashSet<String> routePointIds = new HashSet<String>();
HashSet<String> reportIds = new HashSet<String>();
Route route = null;
while (resultSet.next()) {
String commentId = resultSet.getString("commentId");
String routePointId = resultSet.getString("routePointId");
String reportId = resultSet.getString("reportId");
if(route == null){
// create route from result set
}
if(!commentIds.contains(commentId)){
// create comment object from result set
// add to route object
commentIds.add(commentId);
}
if(!routePointIds.contains(routePointId)){
// create route point object from result set
// add to route object
routePointIds.add(routePointId);
}
if(!reportIds.contains(reportId)){
// create comment object from result set
// add to route
reportIds.add(reportId);
}
}
My question is, is there a simpler way of processing a query with multiple one to many joins in JDBC?
Another way I have thought of approaching this is to do multiple selects. One for each table rather than joining. I am not sure if this is a suitable way to do this as it would mean querying the database multiple times rather than just the once.
Thanks for your replies.
Query for distinct rows for each independent result set. Since you are only querying for one route's data, I'm assuming you are not storing this in a cache or memory, and will need to perform this action any time you need the route data. It will more efficient to use an optimized query, and avoid java logic for each row returned to weed out duplicate data.
Related
I have been given a task to return the data for a list of items whose ids I have. The table has an id defined as:
...
"KeySchema" [
0: {
"AttributeName":"id"
"KeyType":"HASH"
"TableStatus":"ACTIVE"
I have a list of say 100 of these ids and want to query to return the details in a similar way to how IN works in SQL.
I have tried many approaches but cant see a way I can make a single query to the DynamoDB instance where it will return all documents for the Ids in a supplied list.
I hope to use DynamoDBMapper.
I thought I'd hit the jackpot when I found withHashKeyValues on DynamoDBQueryExpression.
e.g.
DynamoDBQueryExpression<MyObject> ddqe;
...
for (String idStr : idList) {
MyObject mo= new MyObject();
mo.setId(idStr);
ddqe.withHashKeyValues(mo);
}
but looking at the code although the method is plural there is a note that says:
Note 1: Currently the DynamoDBMapper supports only one value per hash key.
I have also tried a Condition (amongst many other things)
Condition condition = new Condition();
condition.withAttributeValueList(filters);
condition.withComparisonOperator(ComparisonOperator.EQ);
Where filters is an ArrayList of the ids.
Is there a way to do this in DynamoDb or have I got to query the DB for every known id individually, e.g. issue 100 query's to the data store?
Queries start at a particular Partition key, so you would have to look up one key at a time with the Query API. Instead, you could use the BatchGetItem API and request 25 items a time.
I am trying to figure out an elegant way to query all of the entities in an AppEngine datastore that have a certain property. Since entities that lack a property aren't included in an index, basically what I want to do is to retrieve the index for a given property. I'm sure it's possible to do something like:
Filter bigger = new FilterPredicate(PROPERTY,
FilterOperator.GREATER_THAN_OR_EQUAL,
0);
Filter smaller = new FilterPredicate(PROPERTY,
FilterOperator.LESS_THAN_OR_EQUAL,
0);
Filter present = CompositeFilterOperator.or(bigger, smaller);
Query q = new Query(KIND).setFilter(present);
but it doesn't look like a very elegant (or efficient) solution. Does anyone have a better idea?
If you don't need entities with null values, you can use this:
Filter filter = new FilterPredicate(PROPERTY, FilterOperator.NOT_EQUAL, null);
It may look simpler, but NOT_EQUAL filter actually results in two separate queries - just as your solution, unless App Engine is smart about null values - that I don't know.
The property I am interested in is actually a list which may have lots of values, so I decided to leave it unindexed, and add a separate boolean property HAS_LIST. That allows me to do a single query for results at the cost of a slightly bigger entity. Still not a very elegant solution, but perhaps a little more efficient.
i am going to type my code here and then I will explain my problem below.
for (int i = 0; i < sales.totalSales(); i++) {
EntidadGeo gec = sales.getSale(i).getCustomer();
EntidadGeo get = sales.getSale(i).getStore();
int[] c = geo.georeferenciar(sales.getSale(i).getCustomer().getCP(), ventas.getVenta(i).getCCustomer().getCalle(), ventas.getVenta(i).getCCustomer().getProvincia());
gec.setX(c[0]);
gec.setY(c[1]);
int[] c2 = geo.georeferenciar(ventas.getSale(i).getStore().getCP(), ventas.getVenta(i).getStore().getCalle(), ventas.getSale(i).getStore().getProvincia());
get.setX(c2[0]);
get.setY(c2[1]);
mapaventas.representar(gec, get);
}
I have that for loop, what i want to do in my project is to print in a map. The point is what I need to draw in the map are customers and stores and one store can sell to many customers at the same time. In my project I am using MVC pattern, this part belongs to Controller part, and in the model part i draw the map.
It works now but the problem is that my project draw one customer and one store instead of 4 customers per 1 store.
Thanks
Your problem is here:
mapaventas.representar(gec, get);
So it looks like you have a Map<Vendor, Client> which will only associate only one client per vendor. I have to guess at this because we have no knowledge what the method above does. If I am correct a better solution perhaps is to use a Map<Vendor, ArrayList<Client>>. so that a Vendor can be associated with multiple clients. Then you would do something like
ArrayList<Client> getList = mapaventas.get(gec);
// if the above is null, create the arraylist first and put it
// and the gec into the map.
getList.add(get);
Note that my variable names and types will not be the same as yours, but hopefully you will understand the concept I'm trying to get across. If not, please ask.
It sounds like your database has a one-to-many relation between Store and Customer. A corresponding object model might be List<Map<Store, List<Customer>>>. Because a Customer may trade at more than one Store, you want "to check there if there is an IdStore already drawn, and then I don't want to draw it."
One approach would be to iterate through the List and add entries to a Set<Location>. Because implementations of Set reject duplicate elements, only one copy would be present, and no explicit check would be required. As a concrete example using JMapViewer, you would add a MapMarker to the mapViewer for each Location in the Set, as shown here.
I have Employee table and an Entity class for it,
My task is such that i need the data of employee table within result set(of type scrollable) two times,
in such case what would be better of the following for using the data second time ->
1: create instance of entity class and store it in List while iterating through result set for first time.
OR
2: after first iteration call the first() method of result set to go back to first row and use data for second time.
Which option will consume less time and resources.
If You have better suggestions, please provide.
Thanks.
Unless this is about very large resultsets, you're probably much better off consuming the whole JDBC ResultSet into memory and operating on a java.util.List rather than on a java.sql.ResultSet for these reasons:
The List is more user-friendly for developers
The database resource can be released immediately (which is less error-prone)
You will probably not run out of memory in Java on 1000 rows.
You can make many many mistakes when operating on a scrollable ResultSet, as various JDBC drivers implement this functionality just subtly differently
You can use tools for consuming JDBC result sets. For instance Apache DbUtils:
QueryRunner run = new QueryRunner(dataSource);
ResultSetHandler<List<Person>> h
= new BeanListHandler<Person>(Person.class);
List<Person> persons = run.query("SELECT * FROM Person", h);
jOOQ (3.0 syntax):
List<Person> list =
DSL.using(connection, sqldialect)
.fetch("SELECT * FROM Person");
.into(Person.class);
Spring JdbcTemplate:
JdbcTemplate template = // ...
List result = template.query("SELECT * FROM Person", rowMapper);
Cache the data you retrieve from database. It's always better than polling it, even if driver provides caching on its own level. You can always withdraw it if it's not needed anymore.
Maybe using a Map of employees by their primary key would help?
If You'd describe why You think You need to iterate the list more than once, than we'd see if there's a better algorithm there to get rid of that second interation in the first place.
I want to compare two lists in java and build two lists oneFound and another one NotFound.
Here's my code so far, can you please advice for the best results and efficient way to do it.
So bascially i am getting a list of emails to be added or update. Now what i am try to achieve is I want to check those emails against the ones in the DB. if they exist in DB then simply update them otherwise add them.
For that I get the list of emails from the DB and then try to compare that list with the list of emails which are to be added or updated.
Then I build 2 lists one Found and another one not found. The foundones are updated and not found ones are which will be inserted into DB.
The following code builds not found with the lists which are already in db resulting in the DB insertion (causing duplicate records).
So if I correctly build found and notfound then the update and insertion will be working correctly.
public updateData (List<String> emailToAddList, List<String> emailToDeleteList)
{
List<String> emailsFromDB = Service.getEmailsFromDB();
List<String> emailToUpdateFound = new ArrayList<String>( );
List<String> emailToUpdateNotFound = new ArrayList<String>();
/**
** compare emailToAddList with emailsFromDB, if found populate
** emailToUpdateFound for data update, otherwise populate emailToUpdateNotFound for data insetion
**/
Collections.sort(emailListToAdd);
Collections.sort(emailListfromDB);
if(emailListToAdd.size() > emailListfromDB.size()
{
for(String addStr: emailListToAdd)
{
if(emailListfromDB.contains(addStr))
{
emailToUpdateFound.add(addStr);
}
else
{
emailToUpdateNotFound.add(addStr);
}
}
}
else
{
for(String str: emailListfromDB)
{
if(emailListToAdd.contains(str))
{
emailToUpdateFound.add(str);
}
else
{
emailToUpdateNotFound.add(str);
}
}
}
}
Thanks
I think you will like the org.apache.commons.collections.CollectionUtils.subtract(a,b) method. This addresses your requirement in a simple way.
Here's my sample code:
public static void main(String[] args) {
List<String> input = new ArrayList<String>();
input.add("a");
input.add("b");
input.add("c");
input.add("d");
List<String> existing = new ArrayList<String>();
existing .add("d");
existing .add("b");
existing .add("z");
Collection<String> newStuff = CollectionUtils.subtract(input, existing);
Collection<String> updateStuff = CollectionUtils.subtract(input,newStuff);
System.out.println(newStuff);
System.out.println(updateStuff);
}
Judicious use of removeAll() and retainAll() is way simpler than what you're trying.
Collection<String> c1;
Collection<String> c2;
c1.removeAll(c2); // set of elements in c1 not in c2
c1.retainAll(c2); // set of elements in c1 also in c2
Treat each list as a queue. Load the "head" of each queue in a variable. Loop, examining the two head values. If left > right, left is unique (assuming you started with the lowest), so put it in the list of unique left entries, then reload left with a new value from the left list. Likewise if right > left only the other way around. If left == right then you have a match -- put them in the list of matches, then refresh both variables.
Keep this up until one list is empty.
How about adding a unique key to the email column, and then just insert anything. If the email already exist then the DB will ignore it. Then there is no need to compare 2 lists.
To approach the problem in another way...
As more an more emails are put into the database, it may become unwieldy to get the entire list from the database and then run the comparison.
If you don't need to know what emails were actually added, you can use an INSERT IGNORE or INSERT ON DUPLICATE KEY UPDATE (or the equivalent depending on your database) to add the entire list to the database.
If you do need to know which emails you're adding, you can do a SELECT WHERE email IN (list of new emails) to retrieve the ones that already exist, and then subtract that list from the ones your have to get the ones that don't exist. Extending it even further, you can create a temp table, put all the emails in it, and then remove the ones that do exist via a similar query.
The above two solutions assume that the database list grows to a large number, whereas the new emails list remains at a smaller size.