I want to extract data by joining tables from two different postgres hosted on different servers using java.
ResultSet resA = statement_A.executeQuery("select issue_id from Server_A.table_name");
ResultSet resB = statement_B.executeQuery("select issue_id from Server_B.table_name");
How can I get join query executed to get result set in this case ? Any pointers would be highly appreciated..
You can't do it in any automatic/magical way. What you can do is define a class that will have the union of properties of the two tables like:
public class JoinedResult{
private int id;
private int name;
// all other common properties to both
...
// properties exclusive to first table
...
// properteis exclusive to second table
...
}
and construct a list of these object that will contain the joined result of both tables.
To make the actual construction you have a few options:
The first one and the easiest one (but not efficient) is to iterate both results with nested loops, and once the ids (or whatever key is used) match you should construct a JoinedResult.
The second one is a bit more complex but also more efficient:
Iterate first result set and construct a map that will map the id to the object.
Iterate second result set and construct a map that will map the id to the object.
Run a loop over the keys of one of the maps you constructed and use that key to access the matching values in both maps, finally construct the joined object.
Related
I have a Java Object, Record . It represents a single record as a result of SQL execution. Can CQEngine index collection of Record ?
My class is of the form
public class Record {
private List<String> columnNames;
private List<Object> values;
... Other getters
}
I have looked through some examples, but I have no luck there.
I want to index only specific column(s) with its name and corresponding value. Can this be achived using cqengine or is there any other alternatives to achieve the same.
Thanks.
That seems to be a strange way to model data, but you can use CQEngine with that model if you wish.
(First off, CQEngine will have no use for your column names so you can remove that field.)
To do this, you will need to define a CQEngine virtual attribute for each of the indexes in your list of values.
Each attribute will need to be declared with the data type which will be stored in that column/index, and will need to be able to cast the object at that index in your list of values, to the appropriate data type (String, Double, Integer etc.).
So let's say your Record has a column called 'price', which is of type Double, and is stored at index 5 in the list of values. You could define an attribute which reads it as follows:
public static final Attribute<Record, Double> PRICE =
attribute("PRICE", record -> ((Double) record.values.get(5));
If this sounds complicated, it's because that way of modelling data makes things a bit complicated :) It's usually easier to work with a data model which leverages the Java type system (which your model does not). As such, you will need to keep track of the data types etc. of each field programmatically yourself.
CQEngine itself will work fine with that model though, because at the end of the day CQEngine attributes don't need to read fields, the attributes are just functions which are programmed to fetch values.
There's a bunch of stuff not covered above. For example can your values be null? (if so, you should use the nullable variety of attributes as discussed in the CQEngine docs. Or, might each of your Record objects have different sets of columns? (if so, you can create attributes on-the-fly when you encounter a new column, but you should probably cache the attributes you have created somewhere).
Hope that helps,
Niall (CQEngine author)
I have the following problem:
There is a Set<C> s of objects of class C. C is defined as follows:
class C {
A a;
B b;
...
}
Given A e, B f, ..., I want to find from s all objects o such that o.a = e, o.b = f, ....
Simplest solution: stream over s, filter, collect, return. But that takes a long time.
Half-assed solution: create a Map<A, Set<C>> indexA, which splits the set by a's value. Stream over indexA.get(e), filter for the other conditions, collect, return.
More-assed solution: create index maps for all fields, select for all criteria from the maps, stream over the shortest list, filter for other criteria, collect, return.
You see where this is going: we're accidentally building a database. The thing is that I don't want to serialize my objects. Sure I could grab H2 or HSQLDB and stick my objects in there, but I don't want to persist them. I basically just want indices on my regular old on-the-heap Java objects.
Surely there must be something out there that I can reuse.
Eventually, I found a couple of projects which tackle this problem including CQEngine, which seems like the most complete and mature library for this purpose.
HSQLDB provides the option of storing Java objects directly in an in-memory database without serializing them.
The property sql.live_object=true is used as a property on the connection URL to a mem: database, for example jdbc:hsqldb:mem:test;sql.live_object=true. A table is created with a column of type OTHER to store the object. Extra columns in this table duplicate any fields in the object that need indexing.
For example:
CREATE TABLE OBJECTLIST (ID INTEGER IDENTITY, OBJ OTHER, TS_FIELD TIMESTAMP, INT_FIELD INTEGER)
CREATE INDEX IDX1 ON OBJECTLIST(TS_FIELD)
CREATE INDEX IDX2 ON OBJECTLIST(INT_FIELD)
The object is stored in the OBJ column, and the timestamp and integer values for the fields that are indexed are stored the the extra columns. SQL queries such as SELECT * FROM OBJECTLIST WHERE INT_FILED = 1234 return the rows containing the relevant objects.
http://hsqldb.org/doc/2.0/guide/dbproperties-chapt.html#dpc_sql_conformance
Since data stored in Mysql always has a column name, I thought it will be great if there is a way to retrieve the records in a key-value map.
The Hibernate mechanism for join queries can't help too much, as we could only retrieve the Object[] result and deserialize the result ourself, or we could use this:
This is most useful when used together with select new map:
select new map( max(bodyWeight) as max, min(bodyWeight) as min, count(*) as n )
from Cat cat
The query above gives a way of retrieving a map, but we have to define the key of the map(with as), I think it's just a case-by-case solution, it is not very elegant to hard-code the field name.
Isn't it better and generic that we find a way to automatically define the key of the retrieved map with just the column name? So can we send a map with all the fields and let the application choose which field to use?
Even if there was such a feature how could Hibernate decide which is the Map key and which columns from the Map value.
The most flexible solution is to define a DTO which will be used to hold the selected projection:
select new my.package.DTO(field1, field2, .., fieldn) from ...
And you simply pass the result to a custom result transformer, which contains the exact result set key mapping logic.
My retrived data from SQL looks like this
ID Name
1 abc
2 xyz
3 def
Which Java collection variable shall I use to store and retrieve them based on postion alter.
I tried with list...But is was only including ID values,not the Name column.
Any help would be highly Appriciated
You might want to use Map collection which is useful to store key value pairs where Id is your key and value is your name.
Map<Long,String> map = new HashMap<Long,String>();
Like Suresh ATTA said Map is the best solution for this scenario. But in case you are going to get more columns from the DB, then it is better if you write a class around it.
An object of that class will represent a row, and the list of object will represent the result set you queried for.
For your current scenario it looks like:
class Name {
String ID;
String Name;
}
List<Name> allNames = // query from DB and resultset mapped to classes properly.
allNames.get(0) will give you the first row, and allNames.get(allNames.size()-1) will give you the last row.
I have two relations A and B, both with all integer attributes (A {a1,a2,a3,...} B{b1,b2,b3,..}. How would I hash-join these two in java? The user will pick the two joining attributes. Do I make two hashtables and then proceed to join them?
Well, what form do your relations have? Are they in a relational database? If so, just use an SQL JOIN - the DBMS will probably do a hash join, but you don't have to care about that.
If they're not in a relational database, why not?
If some weird constraint prevents you form using the best tool for the job then yes, doing a hash join is as simple as putting each tuple into a hashtable keyed on the join attribute and then iterating over the entries of one and looking up matches in the other. If all your data fits into main memory, that is.
Here his a way to do an hash-join in Java
The best is to hash one table A with an hashmap.
HashMap<Sting, Object[]> hash = new HashMap<String, Object[]>();
for (Object[] a : as) {
hash.put(a.a1, a);
}
Then loop in B using the hash and regroup the matched.
ArrayList joined = new ArrayList();
for(Objec[] b : bs){
A a = hash.get(b.b1);
joined.add(new Object[]{a, b});
}
This will work only if each element of A table has an unique a1.