Develop Reference

How to find many item in one parameter Spring data jpa

I want to do a data search on spring data jpa, with only one parameter but can search several items. I have list data, like this.
{
"id": 105,
"roomId": 43,
"floor": "1",
"roomNumber": "001",
"description": "Normal",
"status": "Vacant Clean"
},
{
"id": 11,
"bedTypeName": "King size"
},
{
"id": 39,
"categoryName": "President Suite"
}
I want to search by roomNumber,bedTypeName, and categoryName. But, in one parameter. For example i have #Param("roomNumber") for search 3 item. Example "roomNumber":"001"then show filter by rooomNumber. Example "roomNumber":"king size" then show filter by bedType. Example "roomNumber":"president suite" then show filter by categoryName. In one parameter can search many item.
Here is my repository
#Query("select a,b.bedType,b.roomCategory from RoomDetail a left outer join RoomMaster b on a.roomId = b.id where lower(a.roomNumber) interests like %:roomNumber% OR interests LIKE %:bedTypeName% OR interests LIKE %:categoryName%")
Page<RoomDetail> findByRoomNumberBedTypeRoomCategor(
#Param("roomNumber") String roomNumber, Pageable paging);

Have you tried using custom query methods via JpaRepository interface. With custom query methods you can generate your repos like :
public interface RoomDetailRepository extends PagingAndSortingRepository<RoomDetail, Integer> {
Page<RoomDetail> findByRoomNumberOrBedTypeOrRoomCategoryContains(String search, Pageable pageable);}
You can include sorting keywords in your method names as well.

You can use Specification instead of writing a painful query and vague JpaRepository method.
JpaSpecificationExecutor
At first extend JpaSpecificationExecutor in your repository interface,
interface RoomRepository extends JpaRepository<RoomDetail, Long>, JpaSpecificationExecutor<RoomDetail> {
// your other queries
}
Specification
You need to create Specification to lookup the value on needed columns,
public static Specification<RoomDetail> search(String value) {
String search = "%" + value + "%";
return (root, query, builder) -> builder.or(
builder.like(root.get("roomNumber"), search),
builder.like(root.get("bedTypeName"), search),
builder.like(root.get("categoryName"), search)
);
}
Search :)
Now you can use it where you want to search from.
Page<RoomDetail> filteredRooms = roomDetailRepository.findAll(search(value), paging);
FYI
As it looks at every value on roomNumber, bedTypeName, categoryName columns, there might be some cases on which multiple rooms will be filtered if the same value exists in different rooms columns unless you have unique values.
| id | roomid | floor | roomNumber | bedTypeName | categoryName | description | status |
|-----|--------|-------|------------|-------------|-----------------|------------------|--------------|
| 105 | 43 | 1 | 001 | Single | Junior Suite | Normal Size | Vacant Clean |
| 106 | 16 | 1 | 002 | Double | Business Suite | Couple Suitable | Booked |
| 121 | 23 | 2 | 020 | King Size | President Suite | Very comfortable | Vacant Clean |
In the above case if you give Size as a filter value you will get two rooms because id with 105 has Normal Size value on categoryName column and id with 121 has King Size on your bedTypeName column.
Moreover: to match the exact value, you may need to update search specification using builder.equals instead of builder.like but make sure that % should not be appended before and after the value.
Explore
Spring official documentation
spring-data-criteria-queries
spring-data-jpa-specification

The answer like this.. i have found the answer myself. And it works :)
#Query("select a,b.bedType,b.roomCategory from RoomDetail a left outer join RoomMaster b on a.roomId = b.id where lower(a.roomNumber) like %:filter% "
+ "or lower(b.bedType.bedTypeName) like %:filter% "
+ "or lower(b.roomCategory.categoryName) like %:filter%")
Page<RoomDetail> findByBedTypeOrRoomCategoryOrRoomNumber(#Param("filter") String Filter, Pageable paging);

retrieve histogram from mssql table using java

I want to implement java application that can connect to any sql server and load any table from it. For each table I want to create histogram based on some arbitrary columns.
For example if I have this table
name profit
------------
name1 12
name2 14
name3 18
name4 13
I can create histogram with bin size 4 based on min and max value of profit column and count number of records for each bin.
result is:
profit count
---------------
12-16 3
16-20 1
My solution for this problem is retrieving all the data based on required columns and after that construct the bins and group by the records using java stream Collectors.groupingBy.
I'm not sure if my solution is optimized and for this I want some help to find the better algorithm specially when I have big number of records.(for example use some benefits of sql server or other frameworks that can be used.)
Can I use better algorithm for this issue?
edit 1:
assume my sql result is in List data
private String mySimpleHash(Object[] row, int index) {
StringBuilder hash = new StringBuilder();
for (int i = 0; i < row.length; i++)
if (i != index)
hash.append(row[i]).append(":");
return hash.toString();
}
//index is index of column for histogram
List<Object[]> histogramData = new ArrayList<>();
final Map<String, List<Object[]>> map = data.stream().collect(
Collectors.groupingBy(row -> mySimpleHash(Arrays.copyOfRange(row, index))));
for (final Map.Entry<String, List<Object[]>> entry : map.entrySet()) {
Object[] newRow = newData.get(rowNum);
double result = entry.getValue().stream()
.mapToDouble(row ->
Double.valueOf(row[index].toString())).count();
newRow[index] = result;
histogramData.add(newRow);
}

As you have considered, performing the aggregation after getting all the data out of SQL server is going to be very expensive if the number of rows in your tables increase. You can simply do the aggregation within SQL. Depending on how you are expressing your histogram bins, this is either trivial or requires some work. In your case, the requirement that the lowest bin start at min value requires a little bit of setup as opposed to binning starting from 0. See sample below. The inner query is mapping values to a bin number, the outer query is aggregating and computing the bin boundaries.
CREATE TABLE Test (
Name varchar(max) NOT NULL,
Profit int NOT NULL
)
INSERT Test(Name, Profit)
VALUES
('name1', 12),
('name2', 14),
('name3', 18),
('name4', 13)
DECLARE #minValue int = (SELECT MIN(Profit) FROM Test)
DECLARE #binSize int = 4
SELECT
(#minValue + #binSize * Bin) AS BinLow,
(#minValue + #binSize * Bin) + #binSize - 1 AS BinHigh,
COUNT(*) AS Count
FROM (
SELECT
((Profit - #minValue) / #binSize) AS Bin
FROM
Test
) AS t
GROUP BY Bin
| BinLow | BinHigh | Count |
|--------|---------|-------|
| 12 | 15 | 3 |
| 16 | 19 | 1 |
http://sqlfiddle.com/#!18/d093c/9

Java compare int arrays, filter and insert or update to DB

I have written an app which helps organize home bills. The problem is that in one home can live more than one person, and one person can have more than one home (e.g. me - in both cases :) ). So I've decided to give the user a possibility to bind a contractor (payment receiver) to multiple users and multiple homes.
In my data base there are concatenation tables between accounts and contractors and between homes and contractors. Great, isn't it?
Now, the point is that I'm getting a list of related users (or houses) as sql array, and I finally keep it as Integer[] array. I've made some dummy database entries, so I can test the functionality and it works fine.
But... I have completely no idea how should I properly store changed values in database. The structure of my tables are:
Users
id | username | ....
1 | user1 | ...
2 | user2 | ...
Contractors
id | name | ...
1 | contractor1 | ...
users_contractors
user_id | contractor_id | is_deleted
1 | 1 | false
1 | 2 | false
etc .....
So what I have is: an array of users related to contractor and the second array of users related to contrator (the modified one). Now I need to store the values in DB. When user + contractor does not exists - i need to insert that relation. If it already exists in database, but does not exist in my array (e.g. the connection was deleted) - i need to update the relation table and marked as deleted=true.
I've found some solutions on how to compare two arrays, but they all assume that the arrays are the same length, and they compare values with the same index only.
So what I need is to compare not arrays as we speak, but the array values (if one array contains values from another array, or the opposite). Can this be achieved without forloop-in-forloop ?
Thank you in advance.
Tom

Is there any reason why you are using arrays instead of Lists/Collections? These can help you search for items and make it easier to compare two of them.
I don't have an IDE at hand now, so here is some pseudocode:
// Create a list with all the values (maybe use a hashset to prevent duplicates)
List<int> all = new List();
all.addAll(A);
all.addAll(B);
//for each loop
for (int i : all) {
boolean inA = A.contains(i);
boolean inB = B.contains(i);
if (inA && inB) {
// You can figure out the rest of these statements I think
}
}

Thanks to #DrIvol - I've managed to solve the issue using the code:
List<Integer> allUsers = new ArrayList<Integer>();
allUsers.addAll(bean.getUserId());
allUsers.addAll(bean.getNewUserId());
for(Integer i : allUsers) {
Boolean oldValue = bean.getUserId().contains(i);
Boolean newValue = bean.getNewUserId().contains(i);
if(oldValue && newValue) {
System.out.println(i + " value in both lists");
// Nothing to do
} else if (oldValue && !newValue) {
System.out.println(i + " value removed");
// Set value as deleted
} else if(!oldValue && newValue) {
System.out.println(i + " value added");
// Insert new value to concat table
}
}
It has one problem: If the value was on the first list, and it still is in the second list (no modification) - it's checked twice. But, since I don't need to do anything with this value - it's acceptable for now. Someday, when I'll finish beta version - I'll be doing some optimisations, so I'll make some deduplicator for the list :)
Thank you very much!
Tom

Create dataframe from rdd objectfile

What is the method to create ddf from an RDD which is saved as objectfile. I want to load the RDD but I don't have a java object, only a structtype I want to use as schema for ddf.
I tried retrieving as Row
val myrdd = sc.objectFile[org.apache.spark.sql.Row]("/home/bipin/"+name)
But I get
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to
org.apache.spark.sql.Row
Is there a way to do this.
Edit
From what I understand, I have to read rdd as array of objects and convert it to row. If anyone can give a method for this, it would be acceptable.

If you have an Array of Object you only have to use the Row apply method for an array of Any. In code will be something like this:
val myrdd = sc.objectFile[Array[Object]]("/home/bipin/"+name).map(x => Row(x))
EDIT
you are rigth #user568109 this will create a Dataframe with only one field that will be an Array to parse the whole array you have to do this:
val myrdd = sc.objectFile[Array[Object]]("/home/bipin/"+name).map(x => Row.fromSeq(x.toSeq))
As #user568109 said there are other ways to do this:
val myrdd = sc.objectFile[Array[Object]]("/home/bipin/"+name).map(x => Row(x:_*))
No matters which one you will because both are wrappers for the same code:
/**
* This method can be used to construct a [[Row]] with the given values.
*/
def apply(values: Any*): Row = new GenericRow(values.toArray)
/**
* This method can be used to construct a [[Row]] from a [[Seq]] of values.
*/
def fromSeq(values: Seq[Any]): Row = new GenericRow(values.toArray)

Let me add some explaination,
suppose you have a mysql table grocery with 3 columns (item,category,price) and its contents as below
+------------+---------+----------+-------+
| grocery_id | item | category | price |
+------------+---------+----------+-------+
| 1 | tomato | veg | 2.40 |
| 2 | raddish | veg | 4.30 |
| 3 | banana | fruit | 1.20 |
| 4 | carrot | veg | 2.50 |
| 5 | apple | fruit | 8.10 |
+------------+---------+----------+-------+
5 rows in set (0.00 sec)
Now, within spark you want to read it, your code will be something like below
val groceryRDD = new JdbcRDD(sc, ()=> DriverManager.getConnection(url,uname,passwd), "select item,price from grocery limit ?,?",1,10,2,r => r.getString("item")+"|"+r.getString("price"))
Note :
In the above statement i converted the ResultSet into String r => r.getString("item")+"|"+r.getString("price")
So my JdbcRDD will be as
groceryRDD: org.apache.spark.rdd.JdbcRDD[String] = JdbcRDD[29] at JdbcRDD at <console>:21
now you save it.
groceryRDD.saveAsObjectFile("/user/cloudera/jdbcobject")
Answer to your question
while reading the object file you need to write as below,
val newJdbObjectFile = sc.objectFile[String]("/user/cloudera/jdbcobject")
In a blind manner ,just substitute the type Parameter of RDD you are saving.
In my case, groceryRDD has a type parameter as String, hence i have used the same
UPDATE:
In your case, as mentioned by jlopezmat, you need to use Array[Object]
Here each row of RDD will be Object, but since you have converted that using ObjectArray each row with its contents will be again saved as Array,
i.e, In my case , if save above RDD as below,
val groceryRDD = new JdbcRDD(sc, ()=> DriverManager.getConnection(url,uname,passwd), "select item,price from grocery limit ?,?",1,10,2,r => JdbcRDD.resultSetToObjectArray(r))
when i read the same using and collect data
val newJdbcObjectArrayRDD = sc.objectFile[Array[Object]]("...")
val result = newJdbObjectArrayRDD.collect
result will be of type Array[Array[Object]]
result: Array[Array[Object]] = Array(Array(raddish, 4.3), Array(banana, 1.2), Array(carrot, 2.5), Array(apple, 8.1))
you can parse the above based on your column definitions.
Please let me know if it answered you question

Changing Mocked ResultSet value each time a SQL statement is run

I'm using Mockrunner to create a mock result set for a select statement. I have a loop that executes the select statement (which returns a single value). I want to have the result set return a different value each time, but I have been unable to find anything about how to specify the result set return value based on the times the statement has been called. Here's a pseudocode snippet of the code:
In the test Code:
String selectSQL = "someselectStmt";
StatementResultSetHandler stmtHandler = conn.GetStatementResultSetHandler();
MockResultSet result = stmtHandler.createResultSet();
result.addRow(new Integer[]{new Integer(1)});
stmtHandler.prepareResultSet(selectSQL, result);
In the Actual Target Class:
Integer[] Results = getResults(selectSQL);
while(Results.length != 0){
//do some stuff that change what gets returned in the select stmt
Results = getResults(selectSQL)
}
So essentially I'd like to return something like 1 on the first time through, 2 on the 2nd and nothing on the 3rd. I haven't found anything so far that I'd be able to leverage that could achieve this. The mocked select statement will always return whatever the last result set was to be associated with it (for instance if I created two MockResultSets and associated both with the same select stmt). Is this idea possible?

Looping Control Flow Working Within Java and SQL
If you're coding this one in Java, a way to make your code execution calls return different, sequenced results can be accomplished throughh a looping control flow statement such as a do-while-loop. This Wikipedia reference has a good discussion using the contrast of the do-while-loop between implementations in Java and also in different programming lanugages.
Some Additional Influences through Observation:
A clue from your work with the Mockrunner tool:
The mocked select statement will always return whatever the last result set was to be associated with it (for instance if I created two MockResultSets and associated both with the same select stmt)
This is the case because the SELECT statement must actually change as well or else repeating the query will also repeat the result output. A clue is that your SQL exists as a literal string value throughout the execution of the code. Strings can be altered through code and simple string manipulations.
String selectSQL = "someselectStmt";
StatementResultSetHandler stmtHandler = conn.GetStatementResultSetHandler();
MockResultSet result = stmtHandler.createResultSet();
result.addRow(new Integer[]{new Integer(1)});
stmtHandler.prepareResultSet(selectSQL, result);
in addition to the selectSQL variable, also add a line for a numeric variable to keep track of how many times the SQL statement is executed:
Int queryLoopCount = 0;
In the following target class:
Integer[] Results = getResults(selectSQL);
while(Results.length != 0){
//do some stuff that change what gets returned in the select stmt
Results = getResults(selectSQL)
}
Try rewriting this WHILE loop control following this example. In your pseudocode, you will keep pulling the same data from the call to getResults(selectSQL); because the query remains the same through every pass made through the code.
Setting up the Test Schema and Example SQL Statement
Here is a little workup using a single MySQL table that contains "testdata" output to be fed into some result set. The ID column could be a way of uniquely identifying each different record or "test case"
SQL Fiddle
MySQL 5.5.32 Schema Setup:
CREATE TABLE testCaseData
(
id int primary key,
testdata_col1 int,
testdata_col2 varchar(20),
details varchar(30)
);
INSERT INTO testCaseData
(id, testdata_col1, testdata_col2, details)
VALUES
(1, 2021, 'alkaline gab', 'First Test'),
(2, 322, 'rebuked girdle', '2nd Test'),
(3, 123, 'municipal shunning', '3rd Test'),
(4, 4040, 'regal limerick', 'Skip Test'),
(5, 5550, 'admonished hundredth', '5th Test'),
(6, 98, 'docile pushover', '6th Test'),
(7, 21, 'mousiest festivity', 'Last Test');
commit;
Query 1 A Look at All the Test Data:
SELECT id, testdata_col1, testdata_col2, details
FROM testCaseData
Results:
| ID | TESTDATA_COL1 | TESTDATA_COL2 | DETAILS |
|----|---------------|----------------------|------------|
| 1 | 2021 | alkaline gab | First Test |
| 2 | 322 | rebuked girdle | 2nd Test |
| 3 | 123 | municipal shunning | 3rd Test |
| 4 | 4040 | regal limerick | Skip Test |
| 5 | 5550 | admonished hundredth | 5th Test |
| 6 | 98 | docile pushover | 6th Test |
| 7 | 21 | mousiest festivity | Last Test |
Query 2 Querying Only the First Record in the Table:
SELECT id, testdata_col1, testdata_col2, details
FROM testCaseData
WHERE id = 1
Results:
| ID | TESTDATA_COL1 | TESTDATA_COL2 | DETAILS |
|----|---------------|---------------|------------|
| 1 | 2021 | alkaline gab | First Test |
Query 3 Querying a Specific Test Record Within the Table:
SELECT id, testdata_col1, testdata_col2, details
FROM testCaseData
WHERE id = 2
Results:
| ID | TESTDATA_COL1 | TESTDATA_COL2 | DETAILS |
|----|---------------|----------------|----------|
| 2 | 322 | rebuked girdle | 2nd Test |
Query 4 Returning and Limiting the Output Set Size:
SELECT id, testdata_col1, testdata_col2, details
FROM testCaseData
WHERE id < 5
Results:
| ID | TESTDATA_COL1 | TESTDATA_COL2 | DETAILS |
|----|---------------|--------------------|------------|
| 1 | 2021 | alkaline gab | First Test |
| 2 | 322 | rebuked girdle | 2nd Test |
| 3 | 123 | municipal shunning | 3rd Test |
| 4 | 4040 | regal limerick | Skip Test |
Writing a Parameterized SQL Statement
I do not know if this difference in syntax yields the exact same results as your pseudocode, but I am recommending it from references of code structures that I know already work.
set condition value before loop
do{
// do some work
// update condition value
}while(condition);
The WHILE condition is instead at the end of the statement and should be based on a change to a value made within the looping block. We will now introduce the second variable, an int which tracks the number of times that the loop is iterated over:
String selectSQL = "someselectStmt";
String[] Results; = getResults(selectSQL);
// set condition value before loop
queryLoopCount = 0
do{
// do some work
Results = getResults(selectSQL);
// update condition value
queryLoopCount = queryLoopcount + 1;
}while(queryLoopCount < 6);
Where selectSQL comes from:
SELECT id, testdata_col1, testdata_col2, details
FROM testCaseData
WHERE id = 2;
And adapts with a built in parameter to:
selectSQL = 'SELECT id, testdata_col1, testdata_col2, details
FROM testCaseData
WHERE id = ' + queryLoopCount;
Mixing the string and integer values may not be a problem as in this reference on concatenated(+) values suggests: Anything concatenated to a string is converted to string (eg, "weight = " + kilograms).
Ideas for Specialized Case Requirements
You could introduce your own numbering sequence to get the records of each case to cycle through the reference table. There are a lot of possibilities by introducing an ORDER BY statement and altering the key ORDER BY value.
The "Skip" case. Within the Do-While loop, add a IF-THEN statement to conditionally skip a specific record.
set condition value before loop
do{
if ( queryLoopCount <> 4 ) {
// do some work}
// update condition value
queryLoopCount = queryLoopCount + 1;
}while(condition);
Using an if-then loop, this code sample will process all test records but will skip over the record of ID = 4 and continue through until the while loop condition is met.