i need to insert csv into mysql database in proper column.
let say csv has header and then data
A B C
and Mysql has table with column C A B
i need to know best way to insert csv data to mysql table
Use tools like OpenCsv.Jar
Example are given here - http://opencsv.sourceforge.net/
It handles large volume of data.
I believe you can use the following syntax for mysql:
"INSERT INTO users (username, password, email, firstName, lastName, createDate) VALUES ('test', 'test', 'test', 'test', 'test', 'test')"
So you can build up your query, using the header and the column it falls into, like so (pseudocode):
"Insert into table (header1, header2, header3) values (column1, column2, column3)"
Regardless of what order the data is in the table, that will insert data into the correct column.
To load CSV data into MySQL database in the correct order, regardless of column order in the file, do this:
LOAD DATA INFILE path/to/datafile.csv INTO TABLE tablename (colA, colB, colC)
Ordinarily, the column list (colA, colB, colC) is optional. However, when (as in this question) the order of columns is different between the CSV file and the DB table, then arrange the column names in the order you want them inserted into the table, like so:
LOAD DATA INFILE path/to/datafile.csv INTO TABLE tablename (colC, colB, colA)
To see other options for loading CSV data into mysql, check out http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Related
I have an SQLite database. I am trying to insert values (users_id, lessoninfo_id) in table bookmarks, only if both do not exist before in a row.
INSERT INTO bookmarks(users_id,lessoninfo_id)
VALUES(
(SELECT _id FROM Users WHERE User='"+$('#user_lesson').html()+"'),
(SELECT _id FROM lessoninfo
WHERE Lesson="+lesson_no+" AND cast(starttime AS int)="+Math.floor(result_set.rows.item(markerCount-1).starttime)+")
WHERE NOT EXISTS (
SELECT users_id,lessoninfo_id from bookmarks
WHERE users_id=(SELECT _id FROM Users
WHERE User='"+$('#user_lesson').html()+"') AND lessoninfo_id=(
SELECT _id FROM lessoninfo
WHERE Lesson="+lesson_no+")))
This gives an error saying:
db error near where syntax.
If you never want to have duplicates, you should declare this as a table constraint:
CREATE TABLE bookmarks(
users_id INTEGER,
lessoninfo_id INTEGER,
UNIQUE(users_id, lessoninfo_id)
);
(A primary key over both columns would have the same effect.)
It is then possible to tell the database that you want to silently ignore records that would violate such a constraint:
INSERT OR IGNORE INTO bookmarks(users_id, lessoninfo_id) VALUES(123, 456)
If you have a table called memos that has two columns id and text you should be able to do like this:
INSERT INTO memos(id,text)
SELECT 5, 'text to insert'
WHERE NOT EXISTS(SELECT 1 FROM memos WHERE id = 5 AND text = 'text to insert');
If a record already contains a row where text is equal to 'text to insert' and id is equal to 5, then the insert operation will be ignored.
I don't know if this will work for your particular query, but perhaps it give you a hint on how to proceed.
I would advice that you instead design your table so that no duplicates are allowed as explained in #CLs answer below.
For a unique column, use this:
INSERT OR REPLACE INTO tableName (...) values(...);
For more information, see: sqlite.org/lang_insert
insert into bookmarks (users_id, lessoninfo_id)
select 1, 167
EXCEPT
select user_id, lessoninfo_id
from bookmarks
where user_id=1
and lessoninfo_id=167;
This is the fastest way.
For some other SQL engines, you can use a Dummy table containing 1 record.
e.g:
select 1, 167 from ONE_RECORD_DUMMY_TABLE
I am trying to insert values by extracting from another table but I got exceptionsqlsyntax error while trying this way
int row=st.executeUpdate("insert into bug_history (bug_h_id,type,summary,desc,ans)
select bug_id,type,summary,desc,solution
from bug_details
where bug_id="+bug_id);
Change your query as follow. Use Backtick symbol
INSERT INTO bug_history (`bug_h_id`, `type`, `summary`, `desc`, `ans`)
SELECT `bug_id`, `type`, `summary`, `desc`, `solution`
FROM bug_details
WHERE bug_id = bugID
I have a java Map (Map) and a JDBC connection to hive server.
The schema of the table at the server contains a column of type Map.
Is it possible to insert the java Map to the hive table column with similar datatype using JDBC?
I tried:
"create table test(key string, value Map<String, String>)"
"insert into table test values ('keywer', map('subkey', 'subvalue')) from dummy limit 1;"
ref: Hive inserting values to an array complex type column
but the insert failed with:
"Error: Error while compiling statement: FAILED: ParseException line 1:69 missing EOF at 'from' near ')' (state=42000,code=40000)"
[EDIT]
hive version is : 0.14.0
Thanks
The manual clearly says you cannot insert into a Map data type using SQL:
"Hive does not support literals for complex types (array, map, struct, union), so it is not possible to use them in INSERT INTO...VALUES clauses. This means that the user cannot insert data into a complex datatype column using the INSERT INTO...VALUES clause.”
I think the correct DDL and query would be:
CREATE TABLE test(key STRING, value MAP<STRING, STRING>);
INSERT INTO TABLE test VALUES('keywer', map('subkey', 'subvalue')) from dummy limit 1;
A working method to put a complex type from jdbc client is:
insert into table test select "key",map("key1","value1","key2","value2") from dummy limit 1;
where dummy is another table which has at least one row.
It is possible in MS SQL Server to store the results of query into a table, and most importantly, have the query create the table:
SELECT an_existing_column
INTO table_xyz
FROM an_existing_table
This is also possible in MySQL using:
CREATE TABLE table_xyz
SELECT an_existing_column
FROM an_existing_table
I have searched the Apache Derby Reference Guide and cannot see a method for achieving similar behaviour.
Does anyone know if this possible in Apache Derby?
Store the results of a query into a table:
INSERT INTO table_xyz (an_existing_column) SELECT an_existing_column FROM an_existing_table;
Create a table from another table:
All Columns:
CREATE TABLE table_xyz AS SELECT * FROM an_existing_table WITH NO DATA;
Specific Column:
CREATE TABLE table_xyz AS SELECT an_existing_column FROM an_existing_table WITH NO DATA;
It does not work in JAVA DB, the correct way to do it is:
For all columns:
Step 1: Create a new table with a different name. for example, my_new_table:
CREATE TABLE my_new_table AS SELECT * FROM original_table WITH NO DATA;
This statement creates a new table from original table in the same format and no data copied. It is required to specify WITH NO DATA for it creates a new table with the same columns.
Step 2: Copy data from orig_table to my_new_table using INSERT INTO.
INSERT INTO my_new_table SELECT * FROM orig_table.
Then you will have all the data copied.
I have created a connection with Mysql and java program via jdbc. Now I want to populate the tables in the mysql database. How do I parse the data into the tables from the java code?
I have two input data files.The format of file is like:
"AAH196","17:13:00","02:49:00",287,166.03,"Austin","TX","Virginia Beach","VA"
"AAH3727","21:38:00","03:04:00",273,176.44,"Los Angeles","CA","Colorado Springs","CO"
You can use the LOAD DATA INFILE SQL command.
An easy solution: read a single line an use the content as a part of an SQL INSERT command:
List<String> lines = getAllLinesFromFile(file);
for (String line: lines) {
String query = "INSERT INTO \"TABLE\" (COL1, ..., COL9) VALUES("+line+");";
stmt.executeUpdate(query);
}
Replace TABLE with your actual tablename and COL1, ..., COL9 with an enumeration of your column names. There may be solutions with a better database performance (like using prepared statements) but the algorithm is easy and sufficient to get some data into the database.