This question already has answers here:
How can I list the tables in a SQLite database file that was opened with ATTACH?
(17 answers)
Closed 7 years ago.
I'm new to SQLite. I'm using it in Eclipse(Java)just in case this is relevant.
Now my problem is that i have a *.db file and know nothing about its content. I would like to know which way i can get some information about the tables inside. Otherwise it seems to be imposible to read in the database correctly by a SELECT Query. So basically my problem is just this part
stmt = c.createStatement();
ResultSet rs = stmt.executeQuery( "SELECT * FROM ???????;" );
while ( rs.next() ) {
int id = rs.getInt("id");
..
1. Understand the Schema of your Database
Open the terminal in the location of your .db file.
Enter the following command to start the SQLite Console.
sqlite3 NameOfDatabase.db
1.1 All tables
Then give the following command to the console:
.schema
This will give you all the information you need about all of your tables, including the data type of the fields. In other words, the above command will give you, your database schema.
An output example of the above command is the following:
CREATE TABLE log (ID INTEGER PRIMARY KEY AUTOINCREMENT, userID INTEGER, cardID INTEGER, eventID INTEGER, nameOnTicket TEXT, pricePaid REAL);
CREATE TABLE card (cardID INTEGER PRIMARY KEY AUTOINCREMENT, cardNum TEXT, securityCode TEXT, expiryMonth INTEGER, expiryYear INTEGER, addressID INTEGER, userID INTEGER);
It actually returns the command to re-create the tables, so that is also handy if you would like to output the queries to re-create your tables or to create a documentation for your database/application, but also to understand the structure and the table of your database.
1.2 Specific table
Additionally, you can see the schema of a specific table, using the following command:
.schema TableName
Which will return back the schema of the TableName table.
2. Integrate SQLite with Eclipse
Another option is to integrate your SQLite Database with Eclipse bellow you can find the steps to do that. The steps bellow have been copied here from the official Eclipse wiki, that you can find here.
1) Download the SQLite drivers from here. The actual zip file with the
driver is at 3. Expand the zip somewhere locally and note the
location.
2) Put the sqlite_jni.dll from the zip into your JRE's bin directory.
The driver requires this file to be in the java library path.
3) In Eclipse with DTP 1.0 installed (preferably the final build or a
nightly build dated 110806 or later), go to the Preferences
(Window->Preferences) and select the Connectivity->Driver Definitions
page.
4) Select the "Generic JDBC" category in the Available Driver
Definitions tree and click "Add...".
5) Select "Generic JDBC Driver->Generic JDBC Driver" in the Available
Driver Templates tree. Give the new generic JDBC driver a name like
"javasqlite JDBC driver". Click OK.
6) Click "Add Jar/Zip" and select the sqlite.jar from the driver zip
you expanded in step 1. Click Open.
7) In the Properties table, select the Driver Class property and click
the "..." button. If the jar is accessible, you will see a dialog
appear with at lease one class in the list. Select
"SQLite.JDBCDriver". Click OK.
8) Also in the Properties table, select the Driver URL property and
type the following: jdbc:sqlite:/DRIVE:/dirA/dirB/dbfile
9) Click OK on the Edit Driver Definition dialog. You should see your
new driver appear in the driver list on the Driver Definitions
preference page.
10) Click OK to close the Preferences dialog.
11) If the Data Source Explorer is not open, open the
Connectivity->Data Source Explorer view from the Window->Show View
menu or open the Database Development perspective from the
Window->Open Perspective.
12) In the Data Source Explorer, right-click on the Databases category
and select New...
13) In the New Connection Profile wizard's Wizard Selection Page,
choose the SQL Model-JDBC Connection entry in the list and click Next.
14) Give your new profile a name like "SQLiteTestDB". Click Next.
15) In the "Select a driver from the drop-down" combo box, select your
new SQLite driver definition. Modify the file path in the sample URL
to match the path to your local SQLite database.
16) Click "Test Connection" to verify you can connect to your
database.
17) Click Finish to create the profile.
18) In the Data Source Explorer, right-click on the new profile and
select Connect. You should see content appear in the tree beneath the
profile. Browse through your database to view available tables and
their columns.
I have an unnormalized events-diary CSV from a client that I'm trying to load into a MySQL table so that I can refactor into a sane format. I created a table called 'CSVImport' that has one field for every column of the CSV file. The CSV contains 99 columns , so this was a hard enough task in itself:
CREATE TABLE 'CSVImport' (id INT);
ALTER TABLE CSVImport ADD COLUMN Title VARCHAR(256);
ALTER TABLE CSVImport ADD COLUMN Company VARCHAR(256);
ALTER TABLE CSVImport ADD COLUMN NumTickets VARCHAR(256);
...
ALTER TABLE CSVImport Date49 ADD COLUMN Date49 VARCHAR(256);
ALTER TABLE CSVImport Date50 ADD COLUMN Date50 VARCHAR(256);
No constraints are on the table, and all the fields hold VARCHAR(256) values, except the columns which contain counts (represented by INT), yes/no (represented by BIT), prices (represented by DECIMAL), and text blurbs (represented by TEXT).
I tried to load data into the file:
LOAD DATA INFILE '/home/paul/clientdata.csv' INTO TABLE CSVImport;
Query OK, 2023 rows affected, 65535 warnings (0.08 sec)
Records: 2023 Deleted: 0 Skipped: 0 Warnings: 198256
SELECT * FROM CSVImport;
| NULL | NULL | NULL | NULL | NULL |
...
The whole table is filled with NULL.
I think the problem is that the text blurbs contain more than one line, and MySQL is parsing the file as if each new line would correspond to one databazse row. I can load the file into OpenOffice without a problem.
The clientdata.csv file contains 2593 lines, and 570 records. The first line contains column names. I think it is comma delimited, and text is apparently delimited with doublequote.
UPDATE:
When in doubt, read the manual: http://dev.mysql.com/doc/refman/5.0/en/load-data.html
I added some information to the LOAD DATA statement that OpenOffice was smart enough to infer, and now it loads the correct number of records:
LOAD DATA INFILE "/home/paul/clientdata.csv"
INTO TABLE CSVImport
COLUMNS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES;
But still there are lots of completely NULL records, and none of the data that got loaded seems to be in the right place.
Use mysqlimport to load a table into the database:
mysqlimport --ignore-lines=1 \
--fields-terminated-by=, \
--local -u root \
-p Database \
TableName.csv
I found it at http://chriseiffel.com/everything-linux/how-to-import-a-large-csv-file-to-mysql/
To make the delimiter a tab, use --fields-terminated-by='\t'
The core of your problem seems to be matching the columns in the CSV file to those in the table.
Many graphical mySQL clients have very nice import dialogs for this kind of thing.
My favourite for the job is Windows based HeidiSQL. It gives you a graphical interface to build the LOAD DATA command; you can re-use it programmatically later.
Screenshot: "Import textfile" dialog
To open the Import textfile" dialog, go to Tools > Import CSV file:
Simplest way which I have imported 200+ rows is below command in phpmyadmin sql window
I have a simple table of country with two columns
CountryId,CountryName
here is .csv data
here is command:
LOAD DATA INFILE 'c:/country.csv'
INTO TABLE country
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
Keep one thing in mind, never appear , in second column, otherwise your import will stop
I Used this method to import more than 100K records (~5MB) in 0.046sec
Here's how you do it:
LOAD DATA LOCAL INFILE
'c:/temp/some-file.csv'
INTO TABLE your_awesome_table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
(field_1,field_2 , field_3);
It is very important to include the last line , if you have more than one field i.e normally it skips the last field (MySQL 5.6.17)
LINES TERMINATED BY '\n'
(field_1,field_2 , field_3);
Then, assuming you have the first row as the title for your fields, you might want to include this line also
IGNORE 1 ROWS
This is what it looks like if your file has a header row.
LOAD DATA LOCAL INFILE
'c:/temp/some-file.csv'
INTO TABLE your_awesome_table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(field_1,field_2 , field_3);
phpMyAdmin can handle CSV import. Here are the steps:
Prepare the CSV file to have the fields in the same order as the MySQL table fields.
Remove the header row from the CSV (if any), so that only the data is in the file.
Go to the phpMyAdmin interface.
Select the table in the left menu.
Click the import button at the top.
Browse to the CSV file.
Select the option "CSV using LOAD DATA".
Enter "," in the "fields terminated by".
Enter the column names in the same order as they are in the database table.
Click the go button and you are done.
This is a note that I prepared for my future use, and sharing here if someone else can benefit.
If you are using MySQL Workbench (currently 6.3 version) you can do this by:
Right click on "Tables";
Chose Table Data Import Wizard;
Chose your csv file and follow the instructions (JSON also could be used);
The good thing is that you can create a new table based on the csv file you want to import or load data to an existing table
You can fix this by listing the columns in you LOAD DATA statement. From the manual:
LOAD DATA INFILE 'persondata.txt' INTO TABLE persondata (col1,col2,...);
...so in your case you need to list the 99 columns in the order in which they appear in the csv file.
Try this, it worked for me
LOAD DATA LOCAL INFILE 'filename.csv' INTO TABLE table_name FIELDS TERMINATED BY ',' ENCLOSED BY '"' IGNORE 1 ROWS;
IGNORE 1 ROWS here ignores the first row which contains the fieldnames. Note that for the filename you must type the absolute path of the file.
I see something strange. You are using for ESCAPING the same character you use for ENCLOSING. So the engine does not know what to do when it founds a '"' and I think that is why nothing seems to be in the right place.
I think that if you remove the line of ESCAPING, should run great. Like:
LOAD DATA INFILE "/home/paul/clientdata.csv"
INTO TABLE CSVImport
COLUMNS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES;
Unless you analyze (manually, visually, ... ) your CSV and find which character uses for escape. Sometimes is '\'. But if you do not have it, do not use it.
The mysql command line is prone to too many problems on import. Here is how you do it:
use excel to edit the header names to have no spaces
save as .csv
use free Navicat Lite Sql Browser to import and auto create a new table (give it a name)
open the new table insert a primary auto number column for ID
change the type of the columns as desired.
done!
Yet another solution is to use csvsql tool from amazing csvkit suite.
Usage example:
csvsql --db mysql://$user:$password#localhost/$database --insert --tables $tablename $file
This tool can automatically infer the data types (default behavior), create table and insert the data into the created table. --overwrite option can be used to drop table if it already exists. --insert option — to populate the table from the file.
To install the suite
pip install csvkit
Prerequisites: python-dev, libmysqlclient-dev, MySQL-python
apt-get install python-dev libmysqlclient-dev
pip install MySQL-python
In case if you using Intellij
https://www.jetbrains.com/datagrip/features/importexport.html
I use mysql workbench to do the same job.
create new schema
open newly created schema
right click on "Tables" and select "Table Data Import Wizard"
give the csv file path and table name and finally configure your column type because the wizard set default column type based on their values.
Note: take a look at mysql workbench's log file for any errors by using "tail -f [mysqlworkbenchpath]/log/wb*.log"
How to import csv files to sql tables
Example file: Overseas_trade_index data CSV File
Steps:
Need to create table for overseas_trade_index.
Need to create columns related to csv file.
SQL Query:
( id int not null primary key auto_increment,
series_reference varchar (60),
period varchar (60),
data_value decimal(60,0),
status varchar (60),
units varchar (60),
magnitude int(60),
subject text(60),
group text(60),
series_title_1 varchar (60),
series_title_2 varchar (60),
series_title_3 varchar (60),
series_title_4 varchar (60),
series_title_5 varchar (60),
);
Need to connect mysql database in terminal.
=>show databases;
=>use database;
=>show tables;
Please enter this command to import the csv data to mysql tables.
load data infile '/home/desktop/Documents/overseas.csv' into table trade_index fields terminated by ',' lines terminated by '\n' (series_reference,period,data_value,status,units,magnitude,subject,series_title1,series_title_2,series_title_3,series_title_4,series_title_5);
Find this overseas trade index data on sqldatabase:
select * from trade_index;
If you are using a windows machine with Excel spreadsheet loaded, the new mySql plugin to Excel is phenomenal. The folks at Oracle really did a nice job on that software. You can make the database connection directly from Excel. That plugin will analyse your data, and set up the tables for you in a format consistent with the data. I had some monster big csv files of data to convert. This tool was a big time saver.
http://dev.mysql.com/downloads/windows/excel/
You can make updates from within Excel that will populate to the database online. This worked exceedingly well with mySql files created on ultra inexpensive GoDaddy shared hosting. (Note when you create the table at GoDaddy, you have to select some off-standard settings to enable off site access of the database...)
With this plugin you have pure interactivity between your XL spreadsheet and online mySql data storage.
I know that my answer is late, but I'd like to mention a few other ways to do it.
The easiest one is using command line. The steps will be the following:
Accessing the MySQL CLI by entering the below command:
mysql -u my_user_name -p
Creating a table in the database
use new_schema;
CREATE TABLE employee_details (
id INTEGER,
employee_name VARCHAR(100),
employee_age INTEGER,
PRIMARY KEY (id)
);
Importing the CSV file into a table. We can either mention the file path or store the file in the default directory of the MySQL server.
LOAD DATA INFILE 'Path to the exported csv file'
INTO TABLE employee_details
FIELDS TERMINATED BY ','
IGNORE 1 ROWS;
It's the only one of many solutions, I found it in this tutorial
If loading CSV files into MySQL database is your daily task, then it'll be better to automate this process. In this case you can use some 3rd-party tools that allows you to load data in schedule.
PHP Query for import csv file to mysql database
$query = <<<EOF
LOAD DATA LOCAL INFILE '$file'
INTO TABLE users
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(name,mobile,email)
EOF;
if (!$result = mysqli_query($this->db, $query))
{
exit(mysqli_error($this->db));
}
**Sample CSV file data **
name,mobile,email
Christopher Gritton,570-686-3439,ChristopherKGritton#inbound.plus
Brandon Wilson,541-309-5149,BrandonMWilson#inbound.plus
Craig White,516-795-8065,CraigJWhite#inbound.plus
David Whitney,713-214-3966,DavidCWhitney#inbound.plus
Here is sample excel file screen shot:
Save as and choose .csv.
And you will have as shown below .csv data screen shot if you open using notepad++ or any other notepad.
Make sure you remove header and have column alignment in .csv as in mysql Table.
Replace folder_name by your folder name
LOAD DATA LOCAL INFILE
'D:/folder_name/myfilename.csv'
INTO TABLE mail
FIELDS TERMINATED BY ','
(fname,lname ,email, phone);
If big data, you can take coffee and have it load!.
Thats all you need.
Change servername,username, password,dbname,path of your file, tablename and the field which is in your database you want to insert
<?php
$servername = "localhost";
$username = "root";
$password = "";
$dbname = "bd_dashboard";
//For create connection
$conn = new mysqli($servername, $username, $password, $dbname);
$query = "LOAD DATA LOCAL INFILE
'C:/Users/lenovo/Desktop/my_data.csv'
INTO TABLE test_tab
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(name,mob)";
if (!$result = mysqli_query($conn, $query)){
echo '<script>alert("Oops... Some Error occured.");</script>';
exit();
//exit(mysqli_error());
}else{
echo '<script>alert("Data Inserted Successfully.");</script>'
}
?>
I did it in simple way using phpmyadmin. I followed the steps by #Farhan but all data were eltered in single column.
How I did:
Created a CSV file and deleted the header row with column names. Kept only data.
I created a table with column names matching the csv columns.
Remember to assign appropriate types to each column.
I just selected the import and went to import tab.
In browse I selected the CSV file and kept all options as it is.
To my surprise all the data got imported successfully in their appropriate columns.
When executing MySQL Query to import CSV I was getting error
'Error Code: 1290. The MySQL server is running with the --secure-file-priv option so it cannot execute this statement'
So I moved file to secure file location
LOAD DATA INFILE 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/Orders.csv'
INTO TABLE orderdetails.orders
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
Where location of file is 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/Orders.csv' this is because, I moved my CSV file to 'secure_file_priv' location otherwise I was getting above error
You can get your secure_file_priv using query SHOW VARIABLES LIKE "secure_file_priv";
Source: Import CSV file to MySQL (Query or using Workbench)
What SQL can be used to list the tables, and the rows within those tables in an SQLite database file - once I have attached it with the ATTACH command on the SQLite 3 command line tool?
There are a few steps to see the tables in an SQLite database:
List the tables in your database:
.tables
List how the table looks:
.schema tablename
Print the entire table:
SELECT * FROM tablename;
List all of the available SQLite prompt commands:
.help
The .tables, and .schema "helper" functions don't look into ATTACHed databases: they just query the SQLITE_MASTER table for the "main" database. Consequently, if you used
ATTACH some_file.db AS my_db;
then you need to do
SELECT name FROM my_db.sqlite_master WHERE type='table';
Note that temporary tables don't show up with .tables either: you have to list sqlite_temp_master for that:
SELECT name FROM sqlite_temp_master WHERE type='table';
It appears you need to go through the sqlite_master table, like this:
SELECT * FROM dbname.sqlite_master WHERE type='table';
And then manually go through each table with a SELECT or similar to look at the rows.
The .DUMP and .SCHEMA commands doesn't appear to see the database at all.
To show all tables, use
SELECT name FROM sqlite_master WHERE type = "table"
To show all rows, I guess you can iterate through all tables and just do a SELECT * on each one. But maybe a DUMP is what you're after?
Use .help to check for available commands.
.table
This command would show all tables under your current database.
There is a command available for this on the SQLite command line:
.tables ?PATTERN? List names of tables matching a LIKE pattern
Which converts to the following SQL:
SELECT name FROM sqlite_master
WHERE type IN ('table','view') AND name NOT LIKE 'sqlite_%'
UNION ALL
SELECT name FROM sqlite_temp_master
WHERE type IN ('table','view')
ORDER BY 1
To list the tables you can also do:
SELECT name FROM sqlite_master
WHERE type='table';
I use this query to get it:
SELECT name FROM sqlite_master WHERE type='table'
And to use in iOS:
NSString *aStrQuery=[NSString stringWithFormat:#"SELECT name FROM sqlite_master WHERE type='table'"];
Try PRAGMA table_info(table-name);
http://www.sqlite.org/pragma.html#schema
According to the documentation, the equivalent of MySQL's SHOW TABLES; is:
The ".tables" command is similar to setting list mode then executing
the following query:
SELECT name FROM sqlite_master
WHERE type IN ('table','view') AND name NOT LIKE 'sqlite_%'
UNION ALL
SELECT name FROM sqlite_temp_master
WHERE type IN ('table','view')
ORDER BY 1;
However, if you are checking if a single table exists (or to get its details), see LuizGeron's answer.
As of the latest versions of SQLite 3 you can issue:
.fullschema
to see all of your create statements.
The easiest way to do this is to open the database directly and use the .dump command, rather than attaching it after invoking the SQLite 3 shell tool.
So... (assume your OS command line prompt is $) instead of $sqlite3:
sqlite3> ATTACH database.sqlite as "attached"
From your OS command line, open the database directly:
$sqlite3 database.sqlite
sqlite3> .dump
Via a union all, combine all tables into one list.
select name
from sqlite_master
where type='table'
union all
select name
from sqlite_temp_master
where type='table'
Use:
import sqlite3
TABLE_LIST_QUERY = "SELECT * FROM sqlite_master where type='table'"
Use .da to see all databases - one is called 'main'.
Tables of this database can be seen by:
SELECT distinct tbl_name from sqlite_master order by 1;
The attached databases need prefixes you chose with AS in the statement ATTACH, e.g., aa (, bb, cc...) so:
SELECT distinct tbl_name from **aa.sqlite_master** order by 1;
Note that here you get the views as well. To exclude these add:
where type = 'table'
before ' order'
Since nobody has mentioned about the official reference of SQLite, I think it may be useful to refer to it under this heading:
https://www.sqlite.org/cli.html
You can manipulate your database using the commands described in this link. Besides, if you are using Windows OS and do not know where the command shell is, that is in the SQLite's site:
https://www.sqlite.org/download.html
After downloading it, click sqlite3.exe file to initialize the SQLite command shell. When it is initialized, by default this SQLite session is using an in-memory database, not a file on disk, and so all changes will be lost when the session exits. To use a persistent disk file as the database, enter the ".open ex1.db" command immediately after the terminal window starts up.
The example above causes the database file named "ex1.db" to be opened and used, and created if it does not previously exist. You might want to use a full pathname to ensure that the file is in the directory that you think it is in. Use forward-slashes as the directory separator character. In other words use "c:/work/ex1.db", not "c:\work\ex1.db".
To see all tables in the database you have previously chosen, type the command .tables as it is said in the above link.
If you work in Windows, I think it might be useful to move this sqlite.exe file to same folder with the other Python files. In this way, the Python file writes to and the SQLite shell reads from .db files are in the same path.
The ".schema" commando will list available tables and their rows, by showing you the statement used to create said tables:
sqlite> create table_a (id int, a int, b int);
sqlite> .schema table_a
CREATE TABLE table_a (id int, a int, b int);