I have a mysql table like:
create table table (
id bigint auto_increment
s_id varchar(30),
col1 varchar(50),
col2 varchar(50),
col3 varchar(30),
col4 varchar(30),
col5 int,
primary key(id),
index idx_search(col1, col2, col3)
)
The table have more than 90 million rows of data. In the previous query, I get top 100 s_id. Now, I have a query to get the info of the s_id:
select s_id, col1, col2, col3, col4, max(col5) from table where col1 = 'XXX' and col2 = 'XXX' and s_id in (?, ?, ?, ...) group by s_id
This query speed is unstable, range 100ms to 4000ms.
When the col1='full' and col2='full', it has more than 10 million rows of data in MySQL.
I also tried to add index like idx_sid(s_id, col1, col2), it's helpless.
How can I speed up this query? Or any other solution?
The sql indexes work when their order is from left to right. Because you have created your index to be (s_id,col1,col2), so your WHERE clause should contain these columns in the defined order itself. Using below query definitely speeds up your response time.
select s_id, col1, col2, col3, col4 from table where s_id in (?, ?, ?, ...) and col1 = 'XXX' and col2 = 'XXX'
Related
I have list of data and a table like below.
Data : {ABC111,ABC112,ABC113,111111,111112,111113}
Table:
Column 1 Column 2
ABC111 ABC115
ABC115 111333
111111 ABC112
111111 111112
ABC123 111113
111113 ABC113
My result should be like below
Result : {ABC115,111333,111113,ABC113,ABC123,ABC112,111112}
Explanation:
Data - ABC111 is associated to ABC115 which is also associated to 111333. Hence the result is ABC115,111333.
Similarly, ABC113 is associated to 111113 which is also associated to ABC123. Result = 111113,ABC123.
Is it possible to implement the above using a query. Looking for Prepared statement to which I will pass the Data that I mentioned above.
Try this:-
/*CREATING RECORDS FOR THE MAIN TABLE*/
CREATE TABLE TABLES(Column1 text, Column2 text);
INSERT INTO TABLES VALUES('ABC111','ABC115');
INSERT INTO TABLES VALUES('ABC115','111333');
INSERT INTO TABLES VALUES('111111','ABC112');
INSERT INTO TABLES VALUES('111111','111112');
INSERT INTO TABLES VALUES('ABC123','111113');
INSERT INTO TABLES VALUES('111113','ABC113');
COMMIT;
/*CREATE LIST TO BE SUPPLIED*/
CREATE TABLE LIST(Column1 text);
INSERT INTO LIST VALUES('ABC111');
INSERT INTO LIST VALUES('ABC112');
INSERT INTO LIST VALUES('ABC113');
INSERT INTO LIST VALUES('111111');
INSERT INTO LIST VALUES('111112');
INSERT INTO LIST VALUES('111113');
COMMIT;
/*CODE TO GET THE RESULT*/
SELECT CONCAT('{',GROUP_CONCAT(RESULT),'}') AS RESULT FROM
(
SELECT DISTINCT COLUMN1 AS RESULT FROM
(
Select a.column2 as column1,b.column2
from
(
Select * from
tables
where column1 in (SELECT COLUMN1 FROM LIST)
) a
inner join
tables b
on a.column2=b.column1
UNION ALL
Select b.column2 as column1,b.column1 as Column2
from
(
Select * from
tables a
where column2 in (SELECT COLUMN1 FROM LIST)
) a
inner join
tables b
on a.column1=b.column2
) a
UNION ALL
SELECT DISTINCT COLUMN2 AS RESULT FROM
(
Select a.column2 as column1,b.column2
from
(
Select * from
tables
where column1 in (SELECT COLUMN1 FROM LIST)
) a
inner join
tables b
on a.column2=b.column1
UNION ALL
Select b.column2 as column1,b.column1 as Column2
from
(
Select * from
tables a
where column2 in (SELECT COLUMN1 FROM LIST)
) a
inner join
tables b
on a.column1=b.column2
) a
UNION ALL
SELECT DISTINCT COLUMN2 AS RESULT FROM
(
Select distinct a.Column1, a.column2
from
(
Select * from
tables a
where column2 in (SELECT COLUMN1 FROM LIST)
) a
inner join
(
Select * from
tables a
where column2 in (SELECT COLUMN1 FROM LIST)
) b
on a.column1=b.column1
where a.column2 not in
(
Select column1 from
(
Select b.column2 as column1,b.column1 as Column2
from
(
Select * from
tables a
where column2 in (SELECT COLUMN1 FROM LIST)
) a
inner join
tables b
on a.column1=b.column2
) a
)
) a
) a;
Hope this helps:-)
I'm trying to get the automatically incremented column after I input a new row. I have the following columns in my table:
+----+---------+---------+-------+------------+--------------+------------+---------------+-----------+
| ID | Minkita | Maxkita | Meser | MetodasNum | MainQuestion | MetodaData | RequiresZiood | Nispachim |
+----+---------+---------+-------+------------+--------------+------------+-----
Where ID is an auto-incremented, not null mediumint.
My JSP code for this:
PreparedStatement s = connection.prepareStatement(
"INSERT INTO Threads"
+"(`MinKita`, `MaxKita`, `Meser`, `MetodasNum`, `MainQuestion`, `MetodaData`, `RequiresZiood`, `Nispachim`)"
+" VALUES(?, ?, ?, ?, ?, ? ,? ,?)"
);
//Blah blah, setting strings and values for the prepared statement... Then:
s.executeUpdate();
Now, I understand that executeUpdate returns the row number of my new row, but what I want to do is to get the ID from my row number.
The thing is, the IDs aren't consistent and they may be in an order like so: 1, 2, 5, 7, 13... as rows may be deleted.
So, how do I get the data in the ID column from the row number?
Thanks in advance,
~ NonameSL
You can get the generated ID when doing an INSERT with the following:
First create a PreparedStatement which does return generated values of auto-increment columns:
PreparedStatement s = connection.prepareStatement(
"INSERT INTO <your sql>",
Statement.RETURN_GENERATED_KEYS);
After you have execute the statement you can retrieve the ID as follows:
s.executeUpdate();
ResultSet result = stmt.getGeneratedKeys();
if (result.next())
id = result.getObject(1);
If you have multiple AUTO_INCREMENT columns in the table you can also specify the columns which should be returned:
PreparedStatement s = connection.prepareStatement(
"INSERT INTO <your sql>",
new String[] { "ID" }); // the ID column only
"INSERT into FOLDET (FOLDER_ID, FIELD_NAME, OP_VALUE, "
+ "FIELD_VALUE, FOLDER_FIELD_TYPE, DISPLAY_FLAG ) values \n"
+ "( ?, ?, ?, ?, ?, ?) \n" ;
we use the above query to insert values into the table FOLDET
for column FIELD_VALUE the datatype is varchar(32000) , if the string length is greater than 32000 , we want to store the string in new column of CLOB type eg:- say if the new column name is FLD_VAL_EXT of CLOB type added to the table FOLDET
i want a query to add the string to the new column if the value is greater than "32K"
The Oracle JDBC tutorial covers this well: Blob Insert Example
Basically setup the Clob by writing to its character stream before insert.
I have a table like this:
create table images (
image_id serial primary key,
user_id int references users(user_id),
date_created timestamp with time zone
);
I then have a tag table for tags that images can have:
create table images_tags (
images_tag_id serial primary key,
image_id int references images(image_id),
tag_id int references tags(tag_id)
);
To get the results I want, I run a query like this:
select image_id,user_id,tag_id from images left join images_tags using(image_id)
where (?=-1 or user_id=?)
and (?=-1 or tag_id in (?, ?, ?, ?)) --have up to 4 tag_ids to search for
order by date_created desc limit 100;
The problem is, I want to limit based on the number of unique image_ids because my output will look like this:
{"images":[
{"image_id":1, "tag_ids":[1, 2, 3]},
....
]}
Notice how I group the tag_ids into an array for output, even though the SQL returns a row for each tag_id and image_id combo.
So, when I say limit 100, I want it to apply to 100 unique image_ids.
Maybe you should put one image on each row? If that works, you can do:
select image_id, user_id, string_agg(cast(tag_id as varchar(2000)), ',') as tags
from images left join
images_tags
using (image_id)
where (?=-1 or user_id=?) and
(?=-1 or tag_id in (?, ?, ?, ?)) --have up to 4 tag_ids to search for
group by image_id, user_id
order by date_created desc
limit 100;
If that doesn't work, then use a CTE:
with cte as (
select image_id, user_id, tag_id,
dense_rank() over (order by date_created desc) as seqnum
from images left join
images_tags
using (image_id)
where (?=-1 or user_id=?) and
(?=-1 or tag_id in (?, ?, ?, ?)) --have up to 4 tag_ids to search for
)
select *
from cte
where seqnum <= 100
order by seqnum;
Select 100 qualifying images first, and then join images_tags.
Use an EXISTS semi-join to satisfy the condition on images_tags and take care to get the parentheses right.
SELECT i.*, t.tag_id
FROM (
SELECT i.image_id, i.user_id
FROM images i
WHERE (? = -1 OR i.user_id = ?)
AND (? = -1 OR EXISTS (
SELECT 1
FROM images_tags t
WHERE t.image_id = i.image_id
AND t.tag_id IN (?, ?, ?, ?)
))
ORDER BY i.date_created DESC
LIMIT 100
) i
LEFT JOIN images_tags t
ON t.image_id = i.image_id
AND (? = -1 OR t.tag_id in (?, ?, ?, ?)) -- repeat condition
This should be faster than a solution with window functions and CTEs.
Test performance with EXPLAIN ANLAYZE. As always run a couple of times to warm up cache.
Can JOOQ do 'insert into select' syntax for specified columns? I run several different tries..
The Tables:
table1 (Expected insert)
id column1 column2 column3 ... timestamp
1 John Leo null 2012/1/28 23:32:23 (Expected insert)
table2
id col1 col2 col3
101 John xxx xxx (from table2.col1)
102 xxx xxx xxx
table3
id col1 col2 col3
101 xxx Leo xxx (from table3.col2)
102 xxx xxx xxx
INSERT INTO table1 ( column1, column2 )
SELECT table2.col1, table3.col2
FROM table2 join table3 t3 on table2.id = table3.id
where table2.id = 101;
JOOQ code:
create.insertInto(table1, column1, column2 )
.values( create.select( table2.col1, table3.col2 )
.from(table2)
.join(table3)
.on( table12.id.equal(table3.id) )
.where( table2.id.equal(101) ))
.execute(); //.getSQL();
JOOQ show error message:
The number of values must match the number of fields
Anyone know what problem I make and how I can fix my JOOQ code.
thanks, Pay.
Reference:
Example: INSERT SELECT syntax support
You're using the INSERT .. VALUES syntax, instead of the INSERT .. SELECT syntax. Your subquery provides values column1 but you don't provide any value for column2. What you want to do is described further down in the manual at "Example: INSERT SELECT syntax support". In your case, this would read (jOOQ 2.x syntax, no longer available in jOOQ 3.x):
create.insertInto(table1,
create.select( table2.col1, table3.col2 )
.from(table2)
.join(table3)
.on( table12.id.equal(table3.id) )
.where( table2.id.equal(101) ))
.execute(); //.getSQL();
Or with a custom projection (jOOQ 3.x syntax):
create.insertInto(table1, column1, column2)
.select(create.select(...))
.execute();