Java remove duplicate values from result set - java

I have requirement to remove the duplicate values from result set based on some unique identifier.
I need to remove the duplicates from the result set.
while(resultSet.next())
{
int seqNo = resultSet.getInt("SEQUENCE_NO");
String tableName = resultSet.getString("TABLE_NAME");
String columnName = resultSet.getString("COLUMN_NAME");
String filter = resultSet.getString("FILTER");
}
from the above iteration, i m getting 2 rows from result set. There is same seq no,same table name, different columnname, same filter.
1 PRODUCTFEES CHARGETYPE PRODUCTID
1 PRODUCTFEES PRODUCTCODE PRODUCTID
My requirement is to remove the duplicate table name, duplicate seq no, duplicate filter.
I want to get output something below,
1 PRODUCTFEES CHARGETYPE PRODUCTCODE PRODUCTID

By the example you provide, it seems like you want to output all distinct values for each column indidivually (there are 4 columns in the table, but you output 5 values).
Being the question tagged java, an approach you could take would be using an implementation of Set for each of the columns, so that duplicates won't get through. Then output all the elements of each Set.
LinkedHashSet[] sets = new LinkedHashSet[]{
new LinkedHashSet(),
new LinkedHashSet(),
new LinkedHashSet(),
new LinkedHashSet() };
while(resultSet.next()) {
sets[0].add(resultSet.getInt("SEQUENCE_NO"));
sets[1].add(resultSet.getString("TABLE_NAME")););
sets[2].add(resultSet.getString("COLUMN_NAME"));
sets[3].add(resultSet.getString("FILTER"));
}
StringBuilder buf = new StringBuilder();
for (LinkedHashSet set : sets) {
// append to buf all elements of each set
}
But it might be simpler to address this from the very same SQL query and just make SELECT DISTINCT columnX for each of the columns and output the result without further manipulation. Or use an aggregation function that will concatenate all distinct values. The implementation will be highly dependent on the DBMS you're using (GROUP_CONCAT for MySQL, LISTAGG for Oracle, ...). This would be a similar question for Oracle: How to use Oracle's LISTAGG function with a unique filter?

Based on the different outputs I'd say, that you not just need to remove duplicates, but also reorder the data from the duplicates.
In that case you need to fill a new data-array (or similar structure) in the while(resultSet.next()), and after that loop over the newly arranged data-object and output accordingly.
In Meta-Lang this would be as follows:
while resultset.next()
if newdata-array has unique key
add column-name to found entry in newdata-array
else
create new entry in newdata-array with column-name
while newdata-array.next()
output seq, table-name
while entry.column-names.next()
output column-name
output product-id

Related

How to get data from raw query in spring boot?

I have a query:
Query q = em.createNativeQuery("select DISTINCT id, rating, random() as ordering FROM table\n" +
" WHERE id not in (1,2) ORDER BY ordering LIMIT 10");
List data = q.getResultList();
Every element of this list is array like object:
I want to retrieve that "8" and "16" and compose a comma separated string (to use it in my query in "not in" section in future):
for (Object x : data) {
System.out.println(Arrays.asList(x).get(0));
}
But it produces strings:
[Ljava.lang.Object;#ee93cd3
[Ljava.lang.Object;#62f3c3e1
I don't know how to get that IDs ("8" and "16")
1.I think this is what you are looking for...
Convert JPA query.getResultList() to MY Objects.
or
List<Object[]> rows = q.getResultList(); for (Object[] row : rows) { System.out.println(row[0]); }
in this line
List<Object[]> data = q.getResultList();
data is list of Object of form
[ [1,233, 0.000333], [1,233, 0.000333] ]
for (Object[] x : data) {
// x is [1,233, 0.000333]
System.out.println(x[0]);
}
If I understood it correctly, you are looking for comma separated string of ID's.
If so, then follow these steps might help you to solve the issue.
Create a constructor in table which has only one parameter ID. (If you want you can add more parameters as well but make sure the value which you want it must be in constructor as well as in query.)
Write sql query and execute it.
It returns result and gather it in List which contains the object of the table.
Get the string
dataList.stream().map(obj -> obj.getId()).collect(Collectors.joining(", "))
This will give you the comma separated string.

How to compare list of records against database in Java?

How to compare list of records against database? I have more than 1000 records in list and need to validate against database. How to validate each record from list to database? Select all the data from database and stored in list, then have to compare the values? Please advise...
The below code lists values to validate against database.
private void validatepart(HttpServletRequest req, Vector<String> errors) {
Parts Bean = (Parts)req.getAttribute("partslist");
Vector<PartInfo> List = Bean.getPartList();
int sz = partList.size();
for (int i = 0; i < sz; i++) {
PartInfo part = (PartInfo)partList.elementAt(i);
System.out.println(part.getNumber());
System.out.println(part.getName());
}
}
This depends on what you mean by compare. If it's just one field then executing a query such as select * from parts_table where part_number = ?. It's not that much of a stretch to add more fields to that query. If nothing is returned you know it doesn't exist.
If you need to compare and know exactly which values are different then you can try something like this
List<String> compareObjects(PartInfo filePart, PartInfo dbPart) {
List<String> different = new LinkedList<String>();
if (!filePart.getNumber().equals(dbPart.getNumber())) {
different.add("number");
}
//repeat for all your fields
return different;
}
If your list of objects that you need to validate against the database includes a primary key, then you could just build a list of those primary key values and run a query like:
SELECT <PRIMARY KEY FIELD> FROM <TABLE> WHERE <PRIMARY_KEY_FIELD> IN <LIST OF PRIMARY KEYS> SORT BY <PRIMARY KEY FIELD> ASC;
Once you get that list back, you can compare the results. My instinct would be to put your data (and the query results too) into a Set object and then call removesAll() to get the items not in the database (reverse this for items in the database but not in your set):
yourDataSet.removeAll(queryResults);
This assumes that you have an equals() method implemented in your PartInfo object. You can see the Java API documentation for more details.

ORMLite createOrUpdate() records while preserving specific column?

I'm using ORMLite to manage database tables which contain lists of lookup values for a data collection application. These lookup values are periodically updated from a remote server. However, I'd like to be able to preserve the data in a specific column while creating or updating the records, since I would like to store usage counts (specific to the device) associated with each lookup value. Here's how I'm updating the records:
//build list of new records
final List<BaseLookup> rows = new ArrayList<BaseLookup>();
for (int i = 0; i < jsonRows.length(); i++) {
JSONObject jsonRow = jsonRows.getJSONObject(i);
//parse jsonRow into a new BaseLookup object and add to rows
...
}
//add the new records
dao.callBatchTasks(new Callable<Void>() {
public Void call() throws Exception {
for (BaseLookup row : rows) {
//this is where I'd like to preserve the existing
//value (if any) of the "usageCount" column
Dao.CreateOrUpdateStatus result = dao.createOrUpdate(row);
}
return null;
}
});
I've considered attempting to fetch and merge each record individually within the loop, but this seems like it would perform poorly (some tables are a few thousand records). Is there a simpler or more integrated way to accomplish this?
I'd like to be able to preserve the data in a specific column while creating or updating the records, since I would like to store usage counts (specific to the device) associated with each lookup value
If you have to update certain columns from the JSON data but you want to set the usageCount to usageCount + 1 then you have a couple of options.
You could build an update statement using the dao.updateBuilder(); method and the UpdateBuilder class and then update the columns to their new values and usageCount to usageCount + 1 where the id matches. You should watch the return value to make sure a row was updated. If none were then you create the object.
However, it would be easier to just:
get the BaseLookup from the database
if null, call dao.create() to persist a new entry
otherwise update columns and increment the usageCount
and save it back with a dao.update(...)

Simple Java Insert using JDBC

My Database Table Schema is something like
DocumentID: Name1: Name2: Name3: Name4: Location1: Location2: Location3....:Organization1:..
Now I have 3 Hashset's available having the above values (i.e one for name, one for location and one for organization)
In each single iteration of loop these hashset are being populated with above values.
At the end of each iteration the data from these hashset's is removed and new one's are created.
Now my problem is at each iteration I have to populate the sql table row (just 1 row each iteration) from these hashset values.
What I am not able to understand is if I have hard coded strings than simply I can use something like:
String sql = "INSERT INTO Table " +
"VALUES ('100', 'Zara', 'Akli', '18','100', 'Zara', 'Ali', '18')";
However I need to iterate through each hashset and insert (something like above) the data of all 3 hashset's in a single row. I am confused of how to write such statement. Remember my table is initially completely empty so I cant't use the where clause (something like "insert into.....where documentID="23423")
Assuming you have created these Sets these way:
long DocumentId
names {"A", "B", "C"}
location {"1", "2", "3"}
and so on...
I think the easiest is to build dinamically the SQL to execute:
{
...
StringBuilder sb = new StringBuilder("insert into mytable (");
List<Object> params = new ArrayList<Object>();
addColumnAndValue(sb, "DocumentID", docIdYouHaveSomewhere, params);
int i = 0;
for (String name: names)
addColumnAndValue(sb, ",name" + i, name, params);
i = 0;
for (String location: locations)
addColumnAndValue(sb, ",location" + i, location, params);
// now close the thing
sb.append(") values (?");
for (i = 1; i<params.size(); i++)
sb.append(",?");
sb.append("=");
// and now sb.toString() will contain valid SQL and the param values for a PreparedStatement will be in params array-list:
PreparedStatement ps = conn.prepareStatement(sb.toString());
for (i=0; i<params.size(); i++)
ps.setObject(i+1, params.get(i));
ps.executeUpdate(); // voi là!
...
}
private void add addColumnAndValue(StringBuilder sb, String columnName, Object value, List<Object> params) {
sb.append(columnName);
params.add(value);
}
i guess you need to first do some work on your 3 "HashSet"s.
Since you said the data in 3 Sets will finally go to single row in database. so I suggest that convert your 3 hashset into a Map, or at least a List, with same order as the fields in your insert statement. so that later you could set those values by name or index as parameters to your PS.
and I have never seen an insert statement like "Insert into table values (....) where id=123" are u sure it will work?

Any better approach rather than result.getString() while iterating a result set in java

I have 6 columns in a table. I have a select query which selects some records from the table. While iterating over the result set, im using the following logic to extract the values in the columns:
Statement select = conn.createStatement();
ResultSet result = select.executeQuery
("SELECT * FROM D724933.ECOCHECKS WHERE ECO = '"+localeco+"' AND CHK_TOOL = '"+checknames[i]+"'");
while(result.next()) { // process results one row at a time
String eco = result.getString(1);
mapp2.put("ECO", eco);
String chktool = result.getString(2);
mapp2.put("CHECK_TOOL", chktool);
String lastchktime = result.getString(3);
mapp2.put("LAST_CHECK_TIME", lastchktime);
String status = result.getString(4);
mapp2.put("STATUS", status);
String statcmts = result.getString(5);
mapp2.put("STATUS_COMMENTS", statcmts);
String details = result.getString(6);
mapp2.put("DETAILS_FILE", details);
}
I have 2 questions here:
1. Is there any better approach rather than using result.getString()???
2. Lets say, another column gets added to the table at a later point. Is there any way my code handles this new addition without making change to the code at that point of time
You can use ResultSetMetaData to determine the number and names of the columns in your ResultSet and deal with it this way. Note however that changing the number of columns in the database - affecting your code - and having the code still work may not always be a good idea.
Additionally, note that you're overwriting the values in your map on each iteration of the loop. You probably want to add those maps to some sort of List?
Finally, you need to make sure that your getString methods will not return null anywhere, otherwise putting it into a map will throw an exception.
Statement select = conn.createStatement();
ResultSet result = select.executeQuery("SELECT * FROM D724933.ECOCHECKS WHERE ECO = '"+localeco+"' AND CHK_TOOL = '"+checknames[i]+"'");
ResultSetMetaData rsmd = result.getMetaData();
int numberOfColumns = rsmd.getColumnCount();
List data = new ArrayList<Map>();
Map mapp2;
while(result.next()) { // process results one row at a time
mapp2 = new HashMap<String, String>();
for(int i=1; i<=numberOfColumns; i++) {
mapp2.put(rsmd.getColumnName(i), rs.getString(i));
}
data.add(mapp2);
}
Each of the get family of methods on ResultSet has an overloaded variant that takes a column name as argument. You can use this instead to reduce reliance on ordering of columns.
ResultSet results = ...;
results.getString(1);
You could do this:
results.getString("name");
But the preferred way of handling this sort of problem is to impose an ordering of your own on the result set, by explicitly selecting the columns you want in the initial query.
If your table adds a new column, then obviously you have to change your code, because in your code you use hardcoded value, I mean getString(1).
Instead use ResultSetMetaData's getColumnCount and do some other logic to get that many column values dynamically.
Another thing for your first question, ResultSet contains getXXX() methods with two types of parameters, String column name and int column index. You used the index instead of column name which will perform little faster.
It is bad practice to use SELECT *, instead you should select only the columns you are interested in. The reason is exactly what you mentioned: What happens if your DB changes. you don't want to go trhough the whole code and find and edit all SELECT * statements.
You don't need to put the result into your own map because you can already do:
result.getString("DETAILS_FILE");
But there are already other answers explaining that.
It would be further helpful to use a constant instead of the string "DETAILS_FILE". You can use the constant in the SELECT and in the result.getString(). In case your DB changes you only need to introduce a new constant or change an existing one.

Categories

Resources