I have a large data set(250,000 rows) that I need to upload to a SQL table. I am currently doing this from a Java application I created, and as expected, it is extremely slow. I have to insert each row from a text file into a table, with an auto_increment column that stores an id for that new row. Now I want to use that same id to insert a modified version of the string I entered to the first table, into a second table, where one of the columns holds that original incremented id, so that I can tie the two rows together. Obviously I can query the string I just inserted, and pull the id, but that is an extra query per row I'm trying to avoid. Is there a way to pull the incremented id value from the sql table to my java class at the time the insert statement is executed?
Thanks,
I guess you are using JDBC so use getGeneratedKeys() of java.sql.PreparedStatement. The returned Resulset contains the auto-generated key.
Related
I'm fairly new to developing web apps in Java. I just connected the database, and as seen in the pictures, my ID_patient is auto_increment, but in Netbeans it looks generated.
INSERT INTO sys.patient values('5','elif','nil','er','elif#hotmail.com','11111111111','1234a','istanbul')
The new record inserted wants this value, while i want it to take
INSERT INTO sys.patient values('elif','nil','er','elif#hotmail.com','11111111111','1234a','istanbul')
and auto-increment and give the id as 1,2,3,4...etc.
how can I fix this?
thank you
in netbeans
in mysql
If you are expecting to create an insert query where you don't want to provide an id and the database should generate it automatically based on table defination then you need to create insert query where you have to mention columns names.
Example
INSERT INTO <TABLENAME>(COLUMN1, COLUMN2, COLUMN3, COLUMN4)
VALUES
(VALUE1,VALUE2, VALUE 3,VALUE 4);
So in ur case it should be
INSERT INTO PATIENT(FirstName,MiddleName,LastName,E_mail)
values
('myname','mymiddlename','mylastname','myemailid');
Sequence of column names and their values are very important. They should match exactly. If you don't provide a value for one of column and if its is auto increment, then DB will add an value to it else it will add null value.
I want to fetch a particular value from database in java. I used the following command in prepared statement:
Select Pname from table where pid=458;
The table contains around 50,000 rows and taking more time to fetch, please help me to get the data faster.
i used index and then i bind the variable also but it reduce the execution time only few seconds, i need more efficient. Is there any way to retrieve data faster???
Index your database table for pid, it will make the search faster.
Indexes are used to quickly locate data without having to search every row in a database table every time a database table is accessed. Indexes can be created using one or more columns of a database table, providing the basis for both rapid random lookups and efficient access of ordered records.
SQL Server
CREATE TABLE MyCustomers (CustID int, CompanyName nvarchar(50));
CREATE UNIQUE INDEX idxCustId ON MyCustomers (CustId);
References
https://msdn.microsoft.com/en-us/library/ms188783.aspx
https://technet.microsoft.com/en-us/library/ms345331(v=sql.110).aspx
Create index on field pid in your table.
Use bind variables in queries.
Use prepared statement instead of statement in Java, that will use bind variables.
pstatement = conn.prepareStatement("Select Pname from table where pid = ?");
This ensures that the SQl is pre compiled and hence runs faster.
However, you are likely to gain more performance improvement by index than bind variables .
So I have a MySQL database schema where there is a USERS table which contains the ID as a primary key for that table, I also have a USER_PASSWORDS table which references the USERS table where the USER_ID will act as a foreign key in this table.
The issue that I am facing is that I am writing an application where the user will be able to sign up and specify a username and password. But I would like to insert the user into the database with one query.
I was thinking I had to insert the username first into the USERS table and see what ID has been given to that username and then insert the hash of the password that the user has entered into the USER_PASSWORDS table and specifying the ID that was queried.
I dont like this approach because it means that I have to:
INSERT into the database
QUERY the database
INSERT into the database again
Is there a better way of doing this?
Thanks
You can't insert into two tables with one insert statement, and you would have to query the users table anyway to get the ID value to insert as a foreign key for the user_passwords table.
Really the only way to do what you want is the solution you've already identified:
Insert into the Users table
Query to get the ID of the User you just inserted
Insert into the USER_PASSWORDS table with the ID you obtained for the User.
You could wrap all this up in a stored procedure that takes user data and password as parameters, which would be the "better" way of doing it.
As you didnt really tell for what system / programming language you need this and you did not provide any code example either, I can only give you some theory what you could do:
Its impossible to insert data into two different MySQL tables with one queries but you can reduce your script atleast by the SELECT query:
1.) There is a function in most mysql apis (Depending on what programming language and MySQL Library you are using) that says "getLastInsertId()", "lastInsertId()" or similar.
This will return the ID that was inserted by the auto-increment of the table after the insert is completed.
Just check the docs of your MySQL-api it will have such a function.
2.)
The second possibility is using a UUID - a very large (commonly 128-bit) long string which is generated totally random. There are more so many possible combinations it will happen more probably that you win in the lottery 10 times in a row then you generate two times the same UUID that is already in your table.
So you just generate the UUID and insert it as a key in both tables and you are done.
Just use google to find out-of-the-box libraries to generate UUID's you dont need to build the alogrithm on your own.
An UUID could looks like this:
4a34fe87-f577-4ea9-9557-1bc8f779a68c
One solution: since the hash is unique you can use the hash as a primary key in the USERS table. Then you know what the primary key (id) is at the time of the insert and can reuse when INSERTING in the USER_PASSWORDS table.
That way you can avoid the id query at least.
I am using Excel sheet for reading the values and updating in DB. I have two questions:
How to avoid duplicates in DB table when same value is added in Excel Sheet?
If a new value/s is updated in Excel sheet, I will run the java source console once again and execute the query in the DB to see the results. But I don't want that... Instead of that, if any values is modified/updated in Excel sheet, it should automatically reflect in DB Table.
Is there any ways to do that?
1) to avoid duplicates in DB table, just make the column unique. Non-unique updates/inserts will just fail.
create table mytable (
id int primary key,
name varchar(255) unique not null
);
2) if you want it to reflect directly in the DB, I suggest you just link MS Access directly to the DB table. It looks very much like Excel and is probably what you want.
You can also try a free Access like OpenOffice.org Base.
In my project are two matrices using Microsoft Access database, sourced from two different tables (for example TABLE_A and TABLE_B). I need to retrieve a row and a column from that database.
Selecting COL1 from one table means selecting a corresponding row from the other.
I would like to retrieve COL1 and the corresponding row at the same time.
How is this possible using SQL and a JDBC call?
Thank you!
In order to work with two separate tables of the same database you will need one Connection and a Statement and a ResultSet for each database table.
Since you do not clarify how you have stored your matrix data in the database i can not give you more details here. But, whatever you do, in order to access the contents of the first matrix from the database table you will have to replicate for the second. Just change the table name in your SQL select.
However it will not happen "at the same time", one query will follow the other.