I have a table like this:
create table images (
image_id serial primary key,
user_id int references users(user_id),
date_created timestamp with time zone
);
I then have a tag table for tags that images can have:
create table images_tags (
images_tag_id serial primary key,
image_id int references images(image_id),
tag_id int references tags(tag_id)
);
To get the results I want, I run a query like this:
select image_id,user_id,tag_id from images left join images_tags using(image_id)
where (?=-1 or user_id=?)
and (?=-1 or tag_id in (?, ?, ?, ?)) --have up to 4 tag_ids to search for
order by date_created desc limit 100;
The problem is, I want to limit based on the number of unique image_ids because my output will look like this:
{"images":[
{"image_id":1, "tag_ids":[1, 2, 3]},
....
]}
Notice how I group the tag_ids into an array for output, even though the SQL returns a row for each tag_id and image_id combo.
So, when I say limit 100, I want it to apply to 100 unique image_ids.
Maybe you should put one image on each row? If that works, you can do:
select image_id, user_id, string_agg(cast(tag_id as varchar(2000)), ',') as tags
from images left join
images_tags
using (image_id)
where (?=-1 or user_id=?) and
(?=-1 or tag_id in (?, ?, ?, ?)) --have up to 4 tag_ids to search for
group by image_id, user_id
order by date_created desc
limit 100;
If that doesn't work, then use a CTE:
with cte as (
select image_id, user_id, tag_id,
dense_rank() over (order by date_created desc) as seqnum
from images left join
images_tags
using (image_id)
where (?=-1 or user_id=?) and
(?=-1 or tag_id in (?, ?, ?, ?)) --have up to 4 tag_ids to search for
)
select *
from cte
where seqnum <= 100
order by seqnum;
Select 100 qualifying images first, and then join images_tags.
Use an EXISTS semi-join to satisfy the condition on images_tags and take care to get the parentheses right.
SELECT i.*, t.tag_id
FROM (
SELECT i.image_id, i.user_id
FROM images i
WHERE (? = -1 OR i.user_id = ?)
AND (? = -1 OR EXISTS (
SELECT 1
FROM images_tags t
WHERE t.image_id = i.image_id
AND t.tag_id IN (?, ?, ?, ?)
))
ORDER BY i.date_created DESC
LIMIT 100
) i
LEFT JOIN images_tags t
ON t.image_id = i.image_id
AND (? = -1 OR t.tag_id in (?, ?, ?, ?)) -- repeat condition
This should be faster than a solution with window functions and CTEs.
Test performance with EXPLAIN ANLAYZE. As always run a couple of times to warm up cache.
Related
I have a mysql table like:
create table table (
id bigint auto_increment
s_id varchar(30),
col1 varchar(50),
col2 varchar(50),
col3 varchar(30),
col4 varchar(30),
col5 int,
primary key(id),
index idx_search(col1, col2, col3)
)
The table have more than 90 million rows of data. In the previous query, I get top 100 s_id. Now, I have a query to get the info of the s_id:
select s_id, col1, col2, col3, col4, max(col5) from table where col1 = 'XXX' and col2 = 'XXX' and s_id in (?, ?, ?, ...) group by s_id
This query speed is unstable, range 100ms to 4000ms.
When the col1='full' and col2='full', it has more than 10 million rows of data in MySQL.
I also tried to add index like idx_sid(s_id, col1, col2), it's helpless.
How can I speed up this query? Or any other solution?
The sql indexes work when their order is from left to right. Because you have created your index to be (s_id,col1,col2), so your WHERE clause should contain these columns in the defined order itself. Using below query definitely speeds up your response time.
select s_id, col1, col2, col3, col4 from table where s_id in (?, ?, ?, ...) and col1 = 'XXX' and col2 = 'XXX'
I have a Car table with columns carID, brand, color, price. I'm copying carID and color to another table Detail which have columns carID, finish, color (with carID being a foreign key from Car table and the finish column is not NULL).
I have an SQL Prepared Statement:
public void insertToDetail(int carID, String finish){
String sql = "INSERT INTO detail (carID, finish, color) SELECT ?, ?, color FROM car WHERE carID = ?;";
PreparedStatement psmt = connect.prepareStatement(sql);
psmt.setString(1, carID);
psmt.setInt(2, finish);
psmt.setInt(3, carID);
psmt.executeUpdate();
psmt.close();
}
How may I check if an exact same carID value is not already inserted into my Detail table?
I tried something like:
"INSERT INTO detail (carID, finish, color)
SELECT ?, ?, color FROM car
WHERE NOT EXISTS {
SELECT carID
FROM detail
WHERE carID = ?
} ;";
And it gave me the error:
MySQLSyntaxError You have an error in your SQL syntax; check the
manual that corresponds to your MySQL server version for the right
syntax to use near '{ SELECT carID FROM detail WHERE carID = 123 }' at
line 1
The query you are trying to write is:
INSERT INTO detail (carID, finish, color)
SELECT ?, ?, color
FROM car
WHERE NOT EXISTS (SELECT carID
FROM detail
WHERE carID = ?
) ;
However, NOT EXISTS is the wrong approach. Let the database do the checking. So create a unique index:
CREATE UNIQUE INDEX unq_detail_carid on detail(carID);
This will generate duplicate car ids in the table. An insert will return an error if a duplicate is generated.
To prevent the error, use ON DUPLICATE KEY UPDATE:
INSERT INTO detail (carID, finish, color)
SELECT ?, ?, color
FROM car
ON DUPLICATE KEY UPDATE carID = VALUES(carID);
Your sql have syntax error,need to change {} to () and remove ; at the end
so change
"INSERT INTO detail (carID, finish, color) SELECT ?, ?, color FROM car WHERE NOT EXISTS {
SELECT carID FROM detail WHERE carID = ?;";
to
"INSERT INTO detail (carID, finish, color) SELECT ?, ?, color FROM car WHERE NOT EXISTS (
SELECT carID, finish, color FROM detail WHERE carID = ?)";
I want to query my database (H2) using the following code:
PreparedStatement x = connection.prepareStatement("INSERT INTO tree(ancestor, desc) (select ancestor,? from tree where desc = ? union all select ?,? )");
x.setInt(1,99219);
x.setInt(2,4);
x.setInt(3,99219);
x.setInt(4,99219);
but it doesn't work and I get the following error:
org.h2.jdbc.JdbcSQLException: Unknown data type: "?, ?"; SQL statement:insert into tree(ancestor, desc) (select ancestor,? from tree where desc = ? union all select ?,? ) [50004-192]
Executing the query in the console (without JDBC) works fine. What is wrong with this code?
"INSERT INTO tree(ancestor, desc) (select ancestor,? from tree where desc = ? union all select ?,? )"
This SQL statement is incomplete. it is mising a FROM part after this:
union all select ?,?
The column returned by UNION select have to match by type the columns in the first select. You can do an explicit cast.
Here is my code to check to see if a record already exists in the system before entering ther new record to the sql database.
String sql = "INSERT INTO Stock (name, cost_price, selling_price, numberinstock, supplier) VALUES (?, ?, ?, ?,?) "
+ "Select name"
+ " from Stock"
+ "Where not exists (select * from Stock"
+ "where name = "+NameTextField+")";
I am using Java, my sql and a derby database.
What I am trying to do is when a new item is entered into the system, the sql statement will check to see if that items is already in the system.
What is wrong with this sql statement
You cannot do what you want using insert . . . values. So use insert . . . select instead. The code should look like this:
INSERT INTO Stock(name, cost_price, selling_price, numberinstock, supplier)
Select ?, ?, ?, ?, ?
from sysibm.sysdummy1
Where not exists (select * from Stock where name = "+NameTextField+");
However, you should pass the second reference to name as a parameter, just like all the others:
INSERT INTO Stock(name, cost_price, selling_price, numberinstock, supplier)
Select ?, ?, ?, ?, ?
from sysibm.sysdummy1
Where not exists (select 1 from Stock where name = ?);
try using trigger
Create Trigger Modified_Order_Trigger
On Orders
After Update --as per youe need "insert, delete, update"
AS
Insert Into tablename (tables fileds)
SELECT tablefileds
FROM INSERTED
"INSERT into FOLDET (FOLDER_ID, FIELD_NAME, OP_VALUE, "
+ "FIELD_VALUE, FOLDER_FIELD_TYPE, DISPLAY_FLAG ) values \n"
+ "( ?, ?, ?, ?, ?, ?) \n" ;
we use the above query to insert values into the table FOLDET
for column FIELD_VALUE the datatype is varchar(32000) , if the string length is greater than 32000 , we want to store the string in new column of CLOB type eg:- say if the new column name is FLD_VAL_EXT of CLOB type added to the table FOLDET
i want a query to add the string to the new column if the value is greater than "32K"
The Oracle JDBC tutorial covers this well: Blob Insert Example
Basically setup the Clob by writing to its character stream before insert.