Java - MariaDB Perfoming slow with Hibernate criteria - java

Environment:
mariadb-java-client-2.7.0
DB : MariaDB 10.5.7
ojdbc8 - Oracle 11.2.0.3.0 JDBC 4.0
DB : Oracle Database 11g
Hibernate 4.3.8
Code :
Session session = sessionFactory.openSession();
Criteria fetchCriteria = session.createCriteria("Student");
Disjunction disjunction = Restrictions.disjunction();
for (int i = 1; i <= 10000; i++) {
Conjunction conjunction = Restrictions.conjunction();
conjunction.add(Restrictions.eq("RollNumber", i+""));
disjunction.add(conjunction);
}
fetchCriteria.add(disjunction);
long start1 = System.currentTimeMillis();
List resultList = fetchCriteria.setFirstResult(0).setResultTransformer(Criteria.ALIAS_TO_ENTITY_MAP).list();
long end1 = System.currentTimeMillis();
System.out.println("Time took :"+(end1-start1) +"ms");
Issue
If i run above code with Hibernate 4.3.8 + Oracle 8 it taking less than 5000 milliseconds.
If i run above code with Hibernate 4.3.8 +mariadb-java-client-2.7.0 it taking more than 40,000 milliseconds.
Extra Configuration :
I have set hibernate.jdbc.fetch_size to 100 in hibernate.cfg.xml
along with jdbc URL ,username and password.
Findings:
The query generated in both cases are same and if i execute those
query with SQL Client it takes 10-11 seconds for ORACLE and 41-42 seconds for MariaDB.
The query which is generated by both database if i invoke using JDBC
program (both for ORACLE and MariaDB) it is taking approx 600 milliseconds
Note: Both tables (Oracle and MariaDB) have 15,000 records.
Can anyone help me why MariaDB is taking time?
or some extra settings are required to improve the MariaDB performance.
I have tried defaultFetchSize which is mentioned in https://mariadb.com/kb/en/about-mariadb-connector-j/ but no luck.
SQL Query Generated by the databases:
select this_.rollNo as RollNo1_0_0_, this_.VersionID as Version2_0_0_,
this_.Name as Name3_0_0_, this_.dept as dept4_0_0_,
this_.favSubj as favSubj5_0_0_,
this_.ID as ID33_0_0_
from Student this_
where ((this_.ID='1')
or (this_.ID='2')
or (this_.ID='3')
or ....
or (this_.ID='10000')
MariaDB DDL
CREATE TABLE `student` (
`RollNo` bigint(20) NOT NULL ,
`VersionID` bigint(20) NOT NULL,
`Name` varchar(100) COLLATE ucs2_bin DEFAULT NULL,
`dept` varchar(100) COLLATE ucs2_bin DEFAULT NULL,
`favSubj` varchar(100) COLLATE ucs2_bin DEFAULT NULL,
`ID` varchar(100) COLLATE ucs2_bin DEFAULT NULL,
PRIMARY KEY (`RollNo`),
UNIQUE KEY `UK_student` (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=20258138 DEFAULT CHARSET=ucs2 COLLATE=ucs2_bin
Oracle DDL
CREATE TABLE student (
RollNo NUMBER(19,0),
VersionID NUMBER(19,0) NOT NULL ENABLE,
Name VARCHAR2(100),
dept VARCHAR2(100),
favSubj VARCHAR2(100),
ID VARCHAR2(100),
PRIMARY KEY ("RollNo"),
CONSTRAINT "UK_student" UNIQUE ("ID")
)
MariaDB explain select query output
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
this_
range
UK_Student
UK_Student
203
NULL
10000
Using index condition

An OR with 10K items takes a long time to parse. Faster would be an IN:
where this_.ID IN ('1', '2', ..., '10000')
However, even that is likely to take a long time to run.
In the case of MariaDB, I think the Optimizer will say
Oh, that's too many items for me to look up each one, so
I will, instead, simply scan the table, checking each row for an ID in that list (using some kind of efficient lookup in the 10K-long list).
However, if there are 20M rows in the table, that will take a long time.
Can you provide the query plan (EXPLAIN) so we can confirm what I am hypothecating?
This seems logical and faster, but will not work correctly:
where this_.ID BETWEEN '1' AND '10000'
because it is a VARCHAR!!
Performance -- Make id an INT, not a VARCHAR!

Related

JDBC. Replace in all rows value of the column

Java 11. PostgreSQL.
Having following table in db:
TABLE public.account (
id bigserial NOT NULL,
account_id varchar(100) NOT NULL,
display_name varchar(100) NOT NULL,
is_deleted bool NULL DEFAULT false,
);
There are about 1000 rows in this table. In the code I have a static method, which return random string - Helper.getRandomName()
How, using JDBC, in this table (public.account) for all rows replace "display_name" value with value of Helper.getRandomName()?
This is a SQL question. You need to run an update query:
UPDATE public.account set display_name = ?
And provide the new name as the parameter. The absence of a WHERE clause means that all rows will be affected.
If you want to do this for each row individually, then it's harder. You'll want to do a select statement to find all the IDs, and then you can prepare a batch of updates using JDBC, adding a where clause for each ID.
JDBC is just a thin Java wrapper around plain SQL execution.

Mysql Update query optimize for single table

I am using MySql database for my Project,
I have one table having 6 Millions records, Out of which I am updating around 5 Millions records, my table structure and query are as below.
Table
CREATE TABLE `temp` (
`ref_id_1` int(11) NOT NULL,
`ref_id_2` varchar(32) NOT NULL,
`dna_id` int(11) DEFAULT NULL,
`product_id` varchar(16) DEFAULT NULL,
PRIMARY KEY (`ref_id_1`,`ref_id_2`),
KEY `product_id` (`product_id`),
KEY `dna_id` (`dna_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Update Query
UPDATE temp SET dna_id = 8 WHERE product_id ='Dr_23' AND ref_id_1 = 4;
There are 5 Million records for Product Id Dr_23
Above query taking around 2 minutes to execute. I have 32 GB RAM and SSD Harddisk.
Does anyone know how to optimize this query

SQL syntax error initializing DB before unit Tests

I have created a .sql for preparing my DB for testing.
here a part of the .sql which produces the error:
DROP TABLE IF EXISTS announcement;
CREATE TABLE announcement (
id int(11) NOT NULL AUTO_INCREMENT,
date datetime DEFAULT NULL,
title varchar(255) DEFAULT NULL,
content mediumtext,
PRIMARY KEY (id)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;
LOCK TABLES announcement WRITE;
INSERT INTO announcement VALUES (1,'2015-07-29 10:59:16','Test Anno ','some text');
UNLOCK TABLES;
when executed in Mysql Workbench this scipt works fine.
however when executed via hibernate:
String sqlScript = readFile("dump.sql", Charset.forName("UTF8"));
//System.err.println(sqlScript);
Query q = em.createNativeQuery("BEGIN " + sqlScript + "END;");
q.executeUpdate();
I get:
2015-08-06 16:15:55 ERROR SqlExceptionHelper:146 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'DROP TABLE IF EXISTS announcement;
CREATE TABLE announcement (
id int(11) NO' at line 1
I am using:
Properties jpaProperties = new Properties();
jpaProperties.put("hibernate.dialect","org.hibernate.dialect.MySQL5Dialect");
jpaProperties.put("hibernate.enable_lazy_load_no_trans", true);
Can someone help me out here?
thanks.
As stated by mustaccio BEGIN and END statements are not allowed outside of stored procedueres. See: Hibernate multiple native SQL statements

Java DB How to Insert Values for Foreign Keys into Table Column

I am using java DB database and NetBeans 8.0 for a desktop application
I am also using a PreparedStatement to query the database.
below is the code for creating the tables.
CREATE TABLE ALUMNUS (
ALUMNUA_ID INT NOT NULL PRIMARY KEY
GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1),
FIRST_NAME VARCHAR (45),
LAST_NAME VARCHAR (45),
OTHER_NAME VARCHAR (100)
);
CREATE TABLE DUES (
ID INT NOT NULL PRIMARY KEY
GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1),
PAYMENT_YEAR DATE,
AMOUNT DOUBLE,
ALUMNUS_ID INT
);
--FOREIGN KEY
ALTER TABLE APP.DUES
ADD FOREIGN KEY (ALUMNUS_ID) REFERENCES APP.ALUMNUS(ID);
Now I want to insert, delete and update the foreign key values in APP.DUES table. what is the best option; trigger , stored procedure or the preparedstatement?
An example will be good.
If you want to primarily insert into the DUES table, you would use a sub select in SQL. I havent tested it with Java DB, but it basically looks like:
INSERT INTO DUES(PAYMENT_YEAR, AMOUNT,ALUMNUS_ID)
VALUES(2014, 100.0,
(SELECT ALUMNUA_ID from ALUMNUS where ...));
You need to catch the "not found" error case and prepend a INSERT (and need to catch the duplicate case for that as well).
See also: Insert Data Into Tables Linked by Foreign Key

Assign ID number automatically

strSQL = "INSERT INTO emp(NO, EMP_NAME, EMP_TEL)VALUES(088000, 'JIMMY', *****)";
stmt.executeUpdate(strSQL);
I have this statement to insert a new employee into the database.
What if I want the employee NO to be automatically generated by adding 1 to the previous employee NO? How can this be done in JSP?
While not JSP, a possible solution would be to create an auto generated incrementing column (known as an identity column) in the database. Importantly, this avoids the race condition that exists with a solution that retrieves the current maximum and increments it.
MySQL example:
create table emp (
emp_id integer not null auto_increment,
...
);
Apache Derby example:
create table emp (
emp_id integer not null generated always as identity,
...
);
MS SQL Server 2008 R2 example:
create table emp (
emp_id integer not null identity,
...
);
The INSERT statements do not include the emp_id column. See Statement.getGeneratedKeys() for obtaining generated id if required.
Depending of your DB... I give you a mysql example.
create table emp{
NO int unsigned auto_increment,
EMP_NAME varchar(30) not null,
...
}
insert into emp(EMP_NAME,...) values ("Jimmy", ...);
Now you can ask mysql the last inserted id with
LAST_INSERT_ID()
Yes of course, you can do this by setting "employee no" to be unique and A_I (auto_increament) in this column properties
Check database Schema where you are creating table emp with ID int NOT NULL AUTO_INCREMENT
Then update the schema strSQL = "INSERT INTO emp(EMP_NAME, EMP_TEL) VALUES('ABC_NAME', '321321')";
Though it is possible BUT we should not do any logical operation into JSP. Forward all input in Servlet and do there.
There are several way to do.
Some of databases like Oracle has features like sequence, which allows you to increment numbers sequently and operates as atomic.
Set the column (possibly primary key) to auto increment ( database option ), and do not specify that "NO" in your query. That way, the NO column you didn't add will be added by database automatically.
You can get max values from database table and add 1 for new NO, or you can save those latest value even in file, memcached, whatever you want. The problem of this #3 is, if you don't make program to be atomic between GET LATEST VALUE, ADD 1, CALL DATABASE INSERT QUERY, multiple query can have same NO to use. It's OK, however, if NO is primary key since only very first update/insert query will executed and others query will be failed due to primary key unique violation... but problematic in some cases.
You can use the AUTOINCREMENT option on the field NO on the database, or execute a query like SELECT MAX(NO) FROM emp
and get the max value
I think this will be going to solve your doubt in database and use this following query as:
CREATE TABLE:
CREATE TABLE `test` (
`id` INT(5) UNSIGNED NOT NULL AUTO_INCREMENT,
`emp_name` VARCHAR(50) NOT NULL,
`emp_tel` INT(5) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=INNODB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8
INSERT TABLE METHOD:1
INSERT INTO test
VALUES (0,jmail,1234567)OR(?,?,?);
INSERT TABLE METHOD:2
INSERT INTO test (id,emp_name,emp_tel)
VALUES (0,jmail,1234567);
If you had any doubt give me comment.
And if your using the sqlyog to use the shortcut.
if your wants this method like following as:
PreparedStatement ps = con.prepareStatement("INSERT INTO test(id,emp_name,emp_tel)
VALUES (0,jmail,1234567)");
ps.executeUpdate();
PreparedStatement ps = con.prepareStatement("INSERT INTO test(id,emp_name,emp_tel)
VALUES (?,?,?)");
ps.setString(1, id );
ps.setString(2, name);
ps.setString(3, tel);
ps.executeUpdate();

Categories

Resources