I have a list which is a java object like below.
public class TaxIdentifier {
public String id;
public String gender;
public String childId;
public String grade,
public String isProcessed;
////...///
getters and setters
///....///
}
Records in DB looks like below,
id gender childId grader isProcessed
11 M 111 3 Y
12 M 121 4 Y
11 M 131 2 Y
13 M 141 5 Y
14 M 151 1 Y
15 M 161 6 Y
List<TaxIdentifier> taxIdentifierList = new ArrayList<TaxIdentifier>();
for (TaxIdentifier taxIdentifier : taxIdentifierList) {
}
while I process for loop and get the id = 11, i have to check if there are other records with id = 11 and process them together and do a DB operation and then take the next record say in this case 12 and see if there are other records with id = 12 and so on.
One option is i get the id and query the DB to return all id = 11 and so on.
But this is too much back and forth with the DB.
What is the best way to do the same in java? Please advice.
If you anyway need to process all the records in the corresponding database table - you should retrieve all of them in 1 database roundtrip.
After that, you can collect all your TaxIdentifier records in dictionary data structure and process in whatever way you want.
The brief example may look like this:
Map<String, List<TaxIdentifier>> result = repositoty.findAll().stream().collect(Collectors.groupingBy(TaxIdentifier::getId));
Here all the TaxIdentifier records are grouped by TaxIdentifier's id (all the records with id equals "11") can be retrieved and processed this way:
List<TaxIdentifier> taxIdentifiersWithId11 = result.get("11");
I would leverage the power of your database. When you query, you should order your query by id in ascending order. Not sure which database you are using but I would do:
SELECT * FROM MY_DATABASE WHERE IS_PROCESSED = 'N' ORDER BY ID ASC;
That takes the load of sorting it off of your application and onto your database. Then your query returns unprocessed records with the lowest id's on top. Then just sequentially work through them in order.
Related
How can I update with one query?
I want to do something like this:
update customer
set balance = (400,150) where customer_id IN ('2','3');
customer 2 will get a new balance of 400 and customer 3 will get 150.
I want 1 query because I'm using spring-boot, JPA
#Modifying
#Query("update customer set balance = (400,150) where customer_id IN ('2','3');")
Can I do here 2 queries? for each customer?
what is recommended? what is acceptable?
thanks.
You can do by this way -
Update customer
SET balance = (case when customer_id = '2' then '400'
when customer_id = '3' then '150'
end)
WHERE
customer_id IN ('2','3');
The CASE statement may be what you are looking for.
UPDATE customer
SET balance = (case
when customer_id = 1 then 150
when customer_id = 2 then 300
end)
WHERE ID in (1,2);
If your customer_id is of type string, add quotes to the customer_id numbers.
My example is just a modified version of:
Example Code:
UPDATE students
SET JavaScore = (case
when ID = 1 then 75
when ID = 2 then 80
when ID = 3 then 86
when ID = 4 then 55
end),
PythonScore = (case
when ID = 1 then 70
when ID = 2 then 85
when ID = 3 then 94
when ID = 4 then 75
end)
WHERE ID in (1,2,3,4);
From this website:
DelftStack
Hibernate can do this for you, no need to write your own query.
The steps.
Set hibernate.jdbc.batch_size to some reasonable size.
Enable insert/update query ordering
Enable statement rewrites for MySQL (set rewriteBatchedStatements to true)
In your application.properties add the following
spring.jpa.properties.hibernate.jdbc.batch_size=50
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
# spring.jpa.properties.hibernate.jdbc.batch_versioned_data=true # Needed when using versioned data
spring.datasource.hikari.dataSourcePoperties.rewriteBatchedStatements=true
See also this and this for a bit more background.
Now in your code you can just update and save your customers and the database will receive only 1 query.
if you want to use the spring-data way, you have to use complex SQL/JPQL as less as possible.
#Entity
class CustomerEntity {
}
#Modifying
#Query("update CustomerEntity customer set customer.balance = :balance where customer.id = :customerId")
int updateCustomerBalance(#Param("customerId") String customerId, #Param("balance") String balance);
customerRepository.updateCustomerBalance("2", "400");
customerRepository.updateCustomerBalance("3", "150");
Common transaction
if you want to update happens in one transaction
#Transactional
void doUpdate() {
customerRepository.updateCustomerBalance("2", "400");
customerRepository.updateCustomerBalance("3", "150");
}
Keep in mind that you have to call service.doUpdate() from outside. if you call the method from another service method, transaction will not be created.
Check that update has happened
int count = customerRepository.updateCustomerBalance("2", "400");
if (count == 0) {
log.error("Customer not updated customerId=2 customerBalance=400");
}
This is my dataset,
Name Group Age
A [123] 10
B. [123,456] 20
C. [456,789] 30
D. [900] 40
E. [800,900] 50
F. [1000] 60
Now I want to merge Group such that, the result looks like
Name Group Age
A,B,C [123,456,789] 10,20,30
D,E [900,800] 40,50
F. [1000] 60
I tried arrays contains but that is not giving me what I want. I tried self join too. Anyone can help with a java solution.
Edit:
I found ReduceFunction which can do something similar.
dataset.reduce(new ReduceFunction<Grouped>(){
private static final long serialVersionUID = 8289076985320745158L;
#Override
public Grouped call(final Grouped v1, final Grouped v2) {
if (!Collections.disjoint(v1.getGroup(), (v2.getGroup())))
{
v1.getAge().addAll(v2.getAge());
v1.getGroup().addAll(v2.getGroup());
v1.getName().addAll(v2.getName());
}
}
}
But how to do this for all rows???
This is able to give me first 2 rows reduced to :
Name Group Age
A,B [123,456] 10,20
I want to implement java application that can connect to any sql server and load any table from it. For each table I want to create histogram based on some arbitrary columns.
For example if I have this table
name profit
------------
name1 12
name2 14
name3 18
name4 13
I can create histogram with bin size 4 based on min and max value of profit column and count number of records for each bin.
result is:
profit count
---------------
12-16 3
16-20 1
My solution for this problem is retrieving all the data based on required columns and after that construct the bins and group by the records using java stream Collectors.groupingBy.
I'm not sure if my solution is optimized and for this I want some help to find the better algorithm specially when I have big number of records.(for example use some benefits of sql server or other frameworks that can be used.)
Can I use better algorithm for this issue?
edit 1:
assume my sql result is in List data
private String mySimpleHash(Object[] row, int index) {
StringBuilder hash = new StringBuilder();
for (int i = 0; i < row.length; i++)
if (i != index)
hash.append(row[i]).append(":");
return hash.toString();
}
//index is index of column for histogram
List<Object[]> histogramData = new ArrayList<>();
final Map<String, List<Object[]>> map = data.stream().collect(
Collectors.groupingBy(row -> mySimpleHash(Arrays.copyOfRange(row, index))));
for (final Map.Entry<String, List<Object[]>> entry : map.entrySet()) {
Object[] newRow = newData.get(rowNum);
double result = entry.getValue().stream()
.mapToDouble(row ->
Double.valueOf(row[index].toString())).count();
newRow[index] = result;
histogramData.add(newRow);
}
As you have considered, performing the aggregation after getting all the data out of SQL server is going to be very expensive if the number of rows in your tables increase. You can simply do the aggregation within SQL. Depending on how you are expressing your histogram bins, this is either trivial or requires some work. In your case, the requirement that the lowest bin start at min value requires a little bit of setup as opposed to binning starting from 0. See sample below. The inner query is mapping values to a bin number, the outer query is aggregating and computing the bin boundaries.
CREATE TABLE Test (
Name varchar(max) NOT NULL,
Profit int NOT NULL
)
INSERT Test(Name, Profit)
VALUES
('name1', 12),
('name2', 14),
('name3', 18),
('name4', 13)
DECLARE #minValue int = (SELECT MIN(Profit) FROM Test)
DECLARE #binSize int = 4
SELECT
(#minValue + #binSize * Bin) AS BinLow,
(#minValue + #binSize * Bin) + #binSize - 1 AS BinHigh,
COUNT(*) AS Count
FROM (
SELECT
((Profit - #minValue) / #binSize) AS Bin
FROM
Test
) AS t
GROUP BY Bin
| BinLow | BinHigh | Count |
|--------|---------|-------|
| 12 | 15 | 3 |
| 16 | 19 | 1 |
http://sqlfiddle.com/#!18/d093c/9
I am using Java, MySQL and iReport-3.7.6. I need to create a report with two tables. One contains Machine Production details other contains employees working in a shift. It should print date and shift wise.
I can merge employees against machines. Because there are one or more employees works in two or three same machines.
I need to get a report like the one below.
Date:04-03-2015 Shift - I
Sno Supervisor Machines Employees
-----------------------------------------------------------
1 Arun 1,2,4 Siva,Raj,Ram,James
2 Kumar 3,5 Balu,Mano,Stephan
Sno Machines WorkMins Production_kg
--------------------------------------------
1 1 480 800
2 2 300 500
3 3 480 1200
4 4 480 900
5 5 480 1000
and then only Date:04-03-2015 Shift - II, and so on.
If I use sub report concept, for example, I am giving for one day report means, it prints like employee table for all shift and then production table for shift on that day. But I need to print as above.
Can someone give me a solution to overcome this problem?
I think you can use a sub report, I know that you need a HashMap to build a sub report, then I propose the following:
1.- Create three classes:
public class RowEmployee {
private int sno;
private String supervisor;
private int [] machines;
private String [] employees;
// getters and setters
}
public class RowMachinesDetails {
private int sno;
private int machine;
private int workMins;
private int productinKg;
// getters and setters
}
public class Shift {
private Date dateShift;
private List<TableEmployee> listTableEmployee;
private List<TableMachinesDetails> listTableMachinesDetails;
// getters and setters
}
The RowEmployee class is for the first table, the RowMachinesDetails is for the second table and the Shift class is for each shift of your report. As you can see, the Shift class has a list of RowEmployee and a list of RowMachinesDetails, because these lists correspond to each table, also it has a date which corresponds to date of the shift.
2.- Fill your lists with data of employee and data of production
List<TableEmployee> listTableEmployee = new ArrayList<TableEmployee>();
List<TableMachinesDetails> listTableMachinesDetails = new ArrayList<TableMachinesDetails>();
//Create instances of TableEmployee and TableMachinesDetails, and fill your lists
listTableEmployee.add(TableEmployee);
listTableMachinesDetails(TableMachinesDetails);
3.- Create instances of Shift and fill your HashMap with these instances, put the number of shift as key in the HashMap.
//Create instances of Shift
Shift shift = new Shift();
shift.setDateShitf (dateShift);
shift.setListTableEmployee(listTableEmployee);
shift.setListTableMachinesDetails(listTableMachinesDetails);
//Fill the HashMap
hashMapShift.add("I", shift);
4.- Finally, create your datasource as HashMap in iReport and use hashMapShift to fill your datasource.
NOTE: Maybe the type of variables aren't appropiate, the most important is the concept of the solution.
I hope this helps you.
Good Luck.
I have a table where user_id and parent_user_id is stored. For example:
user_id parent_user_id calls designation
---------------------------------------------------
1 0 10 Tech Support
2 1 5 Sr. Tech Support
3 2 11 Tech Support
4 2 12 Tech Support
5 4 10 Tech Support
Scenario is, if a user who has 2 children with 10 calls each, he will get an designation change like Sr. Tech Support. And If he has 10 such callers, it will be Manager.
To do this so far what I have done(java),
#Override
public boolean updateDesignation(int userId, int depth) {
// check whether maximum depth is reached
if (depth == 0)
return false;
depth--;
int userIds = getIds(userId);//Will get parent_id
String LOCAL_SQL = SQLconstants.getSQL("get-total-calls.sql");
if(userIds>0) {
int calls = jdbcTemplate.queryForObject(LOCAL_SQL, Integer.class, userIds);
// I get 4's calls with which I need to see if I have 2 users with 10 calls each!
updateDesignation(userIds, depth);
}
//updateRanks(userId, depth);
return true;
}
If I pass 5 as user_id, and 4 as depth. It will go till user_id and update values. And how it works is 5->4, 4->2, 2->1. But what I need to achieve is 5->4, and check 4's child's calls. same like 3, 2, 1. How can I do this? Please help me.
if(userIds>=0) { // process 0 too, as it is a parent
/* execute this sql
SELECT COUNT(*) FROM tablename WHERE parent_user_id=userIds AND calls>=10;
then check if the returned value is >= 2 or other chekings...
and update designations..
*/
updateDesignation(userIds, depth);
}
In this way u dont need to get calls of each parent. So this line is not needed anymore:
int calls = jdbcTemplate.queryForObject(LOCAL_SQL, Integer.class, userIds);