Develop Reference

Java/Mysql: Get all resulted lines from a stored procedure and not only the last

I have a stored procedure in mysql that returns more than one lines.
My java code to execute it is:
preparedStmt = conn.prepareCall(queryString);
preparedStmt.setString(1, String.valueOf(patient_id));
//System.out.print("select patient data java file 1 ");
boolean results = preparedStmt.execute();
int rowsAffected = 0;
// Protects against lack of SET NOCOUNT in stored procedure
while (results || rowsAffected != -1) {
if (results) {
rs = preparedStmt.getResultSet();
break;
} else {
rowsAffected = preparedStmt.getUpdateCount();
}
results = preparedStmt.getMoreResults();
}
int i = 0;
obj = new JSONObject();
while (rs.next()) {
JSONArray alist = new JSONArray();
alist.put(rs.getString("patient_id"));
alist.put(rs.getString("allergy"));
alist.put(rs.getString("allergy_description"));
alist.put(rs.getString("allergy_onset_date"));
alist.put(rs.getString("agent_description"));
alist.put(rs.getString("agent"));
alist.put(rs.getString("severity"));
obj.put("ps_allergies", alist);
i++;
}
conn.close();
At the end, ps_allergies json object contains only the last line of the query. This is the print output:
["1","week",null,"2017-07-07","vacation home","test2","mobile contact"]
I want ps_allergies to contain something similar to
[["1","hydrogen peroxide","Nuts","2017-07-04","Nursing profressionals","43","Paramedical practinioners"],["1","week",null,"2017-07-07","vacation home","test2","mobile contact"]...]
Do you know how to fix this?

Not exactly knowing what library you use, but it might have something to do with this line:
obj.put("ps_allergies", alist);
A put method in general associates the specified value with the specified key in a map. Since you are constantly overwriting you key 'ps_allergies' in the loop it will only retain the last value.
You might want to associate a list/array to ps_allergies and you then add every alist object in this list/array.

I found the solution. Instead of put I'm using append method.
obj.append("ps_allergies", alist);
The resulted output now is:
[["1","hydrogen peroxide","Nuts","2017-07-04","Nursing professionals","43","Paramedical practitioners"],["1","chlorhexidine","test123","2017-07-15","mobile contact","test232","pager"],["1","Resistance to unspecified antibiotic","Feb3","2017-03-02","mobile contact","test232","pager"],["1","week",null,"2017-07-07","vacation home","test2","mobile contact"]]

Spark - Java UDF returning multiple columns

I'm using sparkSql 1.6.2 (Java API) and I have to process the following DataFrame that has a list of value in 2 columns:
ID AttributeName AttributeValue
0 [an1,an2,an3] [av1,av2,av3]
1 [bn1,bn2] [bv1,bv2]
The desired table is:
ID AttributeName AttributeValue
0 an1 av1
0 an2 av2
0 an3 av3
1 bn1 bv1
1 bn2 bv2
I think I have to use a combination of the explode function and a custom UDF function.
I found the following resources:
Explode (transpose?) multiple columns in Spark SQL table
How do I call a UDF on a Spark DataFrame using JAVA?
and I can successfully run an example that read the two columns and return the concatenation of the first two strings in a column
UDF2 combineUDF = new UDF2<Seq<String>, Seq<String>, String>() {
public String call(final Seq<String> col1, final Seq<String> col2) throws Exception {
return col1.apply(0) + col2.apply(0);
}
};
context.udf().register("combineUDF", combineUDF, DataTypes.StringType);
the problem is to write the signature of a UDF returning two columns (in Java).
As far as I understand I must define a new StructType as the one shown below and set that as return type, but so far I didn't manage to have the final code working
StructType retSchema = new StructType(new StructField[]{
new StructField("#AttName", DataTypes.StringType, true, Metadata.empty()),
new StructField("#AttValue", DataTypes.StringType, true, Metadata.empty()),
}
);
context.udf().register("combineUDF", combineUDF, retSchema);
Any help will be really appreciated.
UPDATE: I'm trying to implement first the zip(AttributeName,AttributeValue) so then I will need just to apply the standard explode function in sparkSql:
ID AttName_AttValue
0 [[an1,av1],[an1,av2],[an3,av3]]
1 [[bn1,bv1],[bn2,bv2]]
I built the following UDF:
UDF2 combineColumns = new UDF2<Seq<String>, Seq<String>, List<List<String>>>() {
public List<List<String>> call(final Seq<String> col1, final Seq<String> col2) throws Exception {
List<List<String>> zipped = new LinkedList<>();
for (int i = 0, listSize = col1.size(); i < listSize; i++) {
List<String> subRow = Arrays.asList(col1.apply(i), col2.apply(i));
zipped.add(subRow);
}
return zipped;
}
};
But when I run the code
myDF.select(callUDF("combineColumns", col("AttributeName"), col("AttributeValue"))).show(10);
I got the following error message:
scala.MatchError: [[an1,av1],[an1,av2],[an3,av3]] (of class java.util.LinkedList)
and it looks like the combining has been performed correctly but then the return type is not the expected one in Scala.
Any Help?

Finally I managed to get the result I was looking for but probably not in the most efficient way.
Basically the are 2 step:
Zip of the two list
Explode of the list in rows
For the first step I defined the following UDF Function
UDF2 concatItems = new UDF2<Seq<String>, Seq<String>, Seq<String>>() {
public Seq<String> call(final Seq<String> col1, final Seq<String> col2) throws Exception {
ArrayList zipped = new ArrayList();
for (int i = 0, listSize = col1.size(); i < listSize; i++) {
String subRow = col1.apply(i) + ";" + col2.apply(i);
zipped.add(subRow);
}
return scala.collection.JavaConversions.asScalaBuffer(zipped);
}
};
Missing the function registration to SparkSession:
sparkSession.udf().register("concatItems",concatItems,DataTypes.StringType);
and then I called it with the following code:
DataFrame df2 = df.select(col("ID"), callUDF("concatItems", col("AttributeName"), col("AttributeValue")).alias("AttName_AttValue"));
At this stage the df2 looks like that:
ID AttName_AttValue
0 [[an1,av1],[an1,av2],[an3,av3]]
1 [[bn1,bv1],[bn2,bv2]]
Then I called the following lambda function for exploding the list into rows:
DataFrame df3 = df2.select(col("ID"),explode(col("AttName_AttValue")).alias("AttName_AttValue_row"));
At this stage the df3 looks like that:
ID AttName_AttValue
0 [an1,av1]
0 [an1,av2]
0 [an3,av3]
1 [bn1,bv1]
1 [bn2,bv2]
Finally to split the attribute name and value into two different columns, I converted the DataFrame into a JavaRDD in order to use the map function:
JavaRDD df3RDD = df3.toJavaRDD().map(
(Function<Row, Row>) myRow -> {
String[] info = String.valueOf(myRow.get(1)).split(",");
return RowFactory.create(myRow.get(0), info[0], info[1]);
}).cache();
If anybody has a better solution feel free to comment.
I hope it helps.

JDBC getString(i) significant slower in server

I have an Oracle 12c database query, which pulls a table of 13 columns and more than 114470 rows in a daily basis.
I was not concerned with this issue until I moved the same code from my DEV server to my PROD server.
On my DEV environment the query takes 3 min:26 sec to complete its execution.
However on PROD the exact same code takes 15 min:34 sec for finishing.
These times were retrieved adding logs on the following code execution:
private List<Map<String, String>> getFieldInformation(ResultSet sqlResult) throws SQLException {
//Map between each column name and the designated data as String
List<Map<String, String>> rows = new ArrayList<Map<String,String>>();
// Count number of returned records
ResultSetMetaData rsmd = sqlResult.getMetaData();
int numberOfColumns = rsmd.getColumnCount();
boolean continueLoop = sqlResult.next();
// If there are no results we return an empty list not null
if(!continueLoop) {
return rows;
}
while (continueLoop) {
Map<String, String> columns = new LinkedHashMap<String, String>();
// Reset Variables for data
String columnLabel = null;
String dataInformation = null;
// Append to the map column name and related data
for(int i = 1; i <= numberOfColumns; i++) {
columnLabel = rsmd.getColumnLabel(i);
dataInformation = sqlResult.getString(i);
if(columnLabel!=null && columnLabel.length()>0 && (dataInformation==null || dataInformation.length() <= 0 )) {
dataInformation = "";
}
columns.put(columnLabel, dataInformation);
}
rows.add(columns);
continueLoop = sqlResult.next();
}
return rows;
}
I understand that "getString" is not the best way for retrieving non TEXT data, but due to the nature of the project I not always know the data type.
Furthermore, I checked in PROD under task manager, that "Memory (Private Working Set)" is being reserved very slowly.
So I would appreciate if you could help in the following questions:
Why there is a discrepancy in the execution timings for both environments? Can you please highlight some ways for checking this issue?
Is there a way were I can see my result set required memory and reserve the same upfront? Will this have some improvements in Performance?
How can I improve the performance for getString(i) method?
Thank you in advance for your assistance.
Best regards,
Ziza

How to get all data from row wise from a resultset in jdbc

I have a table which is showing one row at a time, i.e in the first iteration it will give one row information and 2nd iteration and so on. Now I want to get all the row data in result set. How I can do that??
This is the structure of my table:
name s e p f
Allan 2 3 8 9
I am doing:
rsServeResource6 = st.executeQuery(sqlForIndividualMileStone);
while(rsServeResource6.next()){
if(rsServeResource6.getString(2)!=null){
engageActual = Integer.parseInt(rsServeResource6.getString(2));
System.out.println("Results :"+engageActual);
}else if(rsServeResource6.getString(3)!=null){
qualificationActual = Integer.parseInt(rsServeResource6.getString(3));
System.out.println("Results :"+qualificationActual);
}else if(rsServeResource6.getString(4)!=null){
isSubmissionActual = Integer.parseInt(rsServeResource6.getString(4));
System.out.println("Results :"+isSubmissionActual);
}else if(rsServeResource6.getString(5)!=null){
presentActual = Integer.parseInt(rsServeResource6.getString(5));
System.out.println("Results :"+presentActual);
}else if(rsServeResource6.getString(6)!=null){
interviewActual = Integer.parseInt(rsServeResource6.getString(6));
System.out.println("Results :"+interviewActual);
}
}
like that.
How can I achieve that ??

Use if instead of using else if when fetching results

How to sort data in a table data structure in Java?

I need to sort data based on the third column of the table data structure. I tried based on the answers for the following question. But my sorting does not work. Please help me in this.
Here goes my code.
Object[] data = new Object[y];
rst.beforeFirst();
while (rst.next()) {
int p_id = Integer.parseInt(rst.getString(1));
String sw2 = "select sum(quantity) from tbl_order_detail where product_id=" + p_id;
rst1 = stmt1.executeQuery(sw2);
rst1.next();
String sw3 = "select max(order_date) from tbl_order where tbl_order.`Order_ID` in (select tbl_order_detail.`Order_ID` from tbl_order_detail where product_id=" + p_id + ")";
rst2 = stmt2.executeQuery(sw3);
rst2.next();
data[i] = new Object[]{new String(rst.getString(2)), new String(rst.getString(3)), new Integer(rst1.getString(1)), new String(rst2.getString(1))};
i++;
}
ColumnComparator cc = new ColumnComparator(2);
Arrays.sort(data, cc);
if (i == 0) {
table.addCell("");
table.addCell("");
table.addCell("");
table.addCell("");
} else {
for (int j = 0; j < y; j++) {
Object[] theRow = (Object[]) data[j];
table.addCell((String) theRow[0]);
table.addCell((String) theRow[1]);
table.addCell((String) theRow[2]);
table.addCell((String) theRow[3]);
}
Sample Expected Output:
Product_code Product_name Quantity Order_date
FK Cake 3000 2010-12-09
CK Jelly 100 2010-09-23
F juice 30 2010-12-09
but what I get is:
Product_code Product_name Quantity Order_date
CK Jelly 100 2010-09-23
F juice 30 2010-12-09
FK Cake 3000 2010-12-09

You have far too much going on here. You're mingling database access and UI all into a single method. I'd separate those concerns.
I'd also recommend having the database do the sorting. Add an ORDER BY to the SELECT and let the database do the work.
I'd map the data from the SELECT into an object that had a Comparator for sorting. Load the ResultSet into a List of that object; you can have all your wishes that way.

Is the problem the data or the comparator? In the other posting you where shown how to create a simple test program using hard coded data. The code you posted here doesn't help us because we don't have access to your database and we don't know if you are accessing the data correctly.
The output looks like it is sorted in ascending order by "String" value. So it does indeed look like the data is wrong. I don't know what the problem is since it looks like you are adding an Integer value to the array.
You want the output in descending order by amount, so you need to set a Comparator property to do this.
Anyway to make sure the problem wasn't with my Comparator I created a simple test:
import java.util.*;
public class SortSIJ
{
public static void main(String args[])
{
Object[] data = new Object[3];
data[0] = new Object[] {"CK", "Jelly", new Integer(100), "2010-09-23"};
data[1] = new Object[] {"FK", "Cake", new Integer(3000), "2010-12-09"};
data[2] = new Object[] {"F", "juice", new Integer(30), "2010-12-09"};
ColumnComparator cc = new ColumnComparator(2);
cc.setAscending( false );
Arrays.sort(data, cc);
for (Object row: data)
{
Object[] theRow = (Object[])row;
System.out.println( Arrays.asList(theRow) );
}
}
}
The output looks fine to me. All I can suggest is that you modify the ColumnComparator to add the following line of code to verify the Object type that is being sorted.
System.out.println(o1.getClass());
When I do that I get the following output:
class java.lang.Integer
class java.lang.Integer
[FK, Cake, 3000, 2010-12-09]
[CK, Jelly, 100, 2010-09-23]
[F, juice, 30, 2010-12-09]