I need to insert about 700 records(name,id) into sqlite permanently ,because app will get user's name from the database.
I think ,reading text file is a solution but not know this is the best.
Can you show me other options to insert about 700 records into database?
thanks
The best practice to add multiple inserts into database shown in this video tutorial, you can watch it from 10.15
[Android Sqlite3 video tutoridal][inserting multiple values into database using fast way]
https://www.youtube.com/watch?v=dBnOn17pI7c&list=PLGLfVvz_LVvQUjiCc8lUT9aO0GsWA4uNe&index=14
U have Sqlite browser in order to view sqlite database.Insert data using the browser and u can permanently use that database.
Or try adding data to database using webservices.
It really depends on what you want to do and why you want to do it. That being said, text files can work. I had a similar case where I stored a few thousand items into an SQLite database. I used a text file and a CSVReader to parse the text file.
InputStream is = new ByteArrayInputStream(theContent.getBytes());
BufferedReader br = new BufferedReader(new InputStreamReader(is));
CSVReader<String[]> csvReader = new CSVReaderBuilder<String[]>(br).strategy(new CSVStrategy('\t', '\b', '#', true, true)).entryParser(new EntryParser()).build();
while ((nextLine = csvReader.readNext()) != null) {
// Do Parsing work and Store to SQLite Database
}
If you know the data won't change and want the fastest solution, then a text file is sufficient. If the data will change frequently, then you're probably going to want to access a web service to update your data. The speed of this method will be affected by the internet speed of the user.
Related
I'm trying to find the most efficient way to parse a large file, and save the results in a database.
The file in question is a 12_500_000 line text file, with logs from a server.
The log looks like this
[notice] 2021-03-10T16:19:26.102551Z couchdb#127.0.0.1 <0.8999.68> 351ac014dd 87.92.211.148:5984 125.129.113.37 user1 GET /userdb 200 ok 8
I'm parsing the database name (userdb) and the verb (GET).
while ((line = reader.readLine()) != null) {
String[] data = line.split("\\s+");
service.save(new Request(data[8], data[9].substring(1)));
}
The parsing time(under 1ms) is insignificant compared to the time it takes to save it to the database (78.6ms).
I wanted to do this async, but from what I understand, you can't save records to a database asynchronously.
Any idea which way to go, to it faster?
I am developing an app which after installation has to insert into database a big massive of data (about 5 Mb). But before to start inserting data, first app has to get these data somewhere. For that I created a class where I store all data into arrays. Below I give an example:
public class Data {
public static String[] table_1 = {"text1","text2","text3","text4"..."text500"};
public static String[] table_2 = {"text1","text2","text3","text4"..."text356"};
public static String[] table_3 = {"text1","text2","text3","text4"..."text485"};
..........................
public static String[] table_35 = {"text1","text2","text3","text4"..."text267"};
}
Than in onCreate(), app loops through all these arrays and inserts data into database. But here I encounter with problem called: error: code too large. I know that cause of problem is Data class because code is limited to 64K bytes. I thought to separate data between two classes, but I don't think its a good idea.
So, my question is how correctly to store data for database. What can you advise in this situation ?
you can load that content from a xml file, and once loaded do your insert statements.
You can use SQLite database .
https://developer.android.com/training/basics/data-storage/databases.html
two ways:
load that content from the internet and store it in your local SQLite database.
Create a File e.g. XML or Txt File with JSON string. Place it inside of your APK and load that file in your application and persist your text. But maybe it is not required to persist that text in your database because you can use your xml file as static data storage too.
I have tried various different ways which i read from other questions but none of them seem working for me ...
I am using NetBeans 7.4 as front-end and Sqlite as Backend , iReport 5.5
I am making college ic-ards in which i want photo to be fetched from database ,In Database images are stored as blob , i am storing them as bytes..
My Requirement is getting student images from Database for ICard So i can print them..
Below is code i am using to insert image into Database
File image= new File(logPath2);
FileInputStream fis=new FileInputStream(image);
ByteArrayOutputStream bos=new ByteArrayOutputStream();
byte buf[]=new byte[1024];
for(int readNum;(readNum=fis.read(buf))!=-1; ) {
bos.write(buf,0,readNum);
}
person_image=bos.toByteArray();
I have already tried various image expressions and class expressions, but none seem working ..
Please note i am talking about sqlite, i know ways to do it by mysql but sqlite, its not rendering. But i don't wanna shift from sqlite to mysql.
Error i get is
Error filling print... Unable to get value for field 'Photo' of class 'java.io.InputStream' net.sf.jasperreports.engine.JRException: Unable to get value for field 'Photo' of class 'java.io.InputStream' at net.sf.jasperreports.engine.JRResultSetDataSource.getFieldValue(JRResultSetDataSource.java:319) at net.sf.jasperreports.engine.fill.JRFillDataset.setOldValues(JRFillDataset.java:1356) at net.sf.jasperreports.engine.fill.JRFillDataset.next(JRFillDataset.java:1257) at net.sf.jasperreports.engine.fill.JRFillDataset.next(JRFillDataset.java:1233) –
I have an sqback file respresenting an sqlite db file. I want to extract the data from this sqback file, ie, table names and contents, and convert it into csv file format. I want to do this using Java.
** The sqback file will have already been uploaded from android device to pc prior to processing. So I need a solution that is appropriate for taking place server side.
Does anyone have any leads on how to perform such a task?
If using Android you can take advantage of the built SQLiteDatabase and SQLiteOpenHelper. You'll find all the info you need here.
After parsing everything you can export to CSV the way you want by using File.
EDIT: So basically what you need to do is to parse the bytes by reading them and that way have access to the content. In some cases you don't even need to convert them to a String, since could be that you only need the value of the byte. (ie.: Offset: 44 Length:4 Description:The schema format number. Supported schema formats are 1, 2, 3, and 4.).
You can always check if your values are correct with any HEX editor, even opening the sqlite file with a text editor of any kind would help.
Let's start from scratch. First, reading the file. Two approaches
a. Read the whole file and parse it after
b. Read and parse the whole file in blocks (recommended, specially for bigger files)
Both approaches would share most of the following code:
File file = new File("YOUR FILE ROUTE");
int len = 1024 * 512; //512KB
try {
fis = new FileInputStream(file);
} catch (FileNotFoundException e1) {
fis = null;
e1.printStackTrace();
}
byte[] b = new byte[len];
int bread;
try {
while((bread = fis.read(b, 0, len))!=-1)
{
if(parseBlock(b, bread)==1)
return;
}
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
The difference would be between getting partial blocks and parsing them on the fly (which I guess works for you) and getting the whole thing would be to just put:
fis.read(b, 0, fis.available);
Instead of the while loop.
Ultimately your approach is right, and that's the way to get bytes into a String. (new String(b)). Moreover the first characters are likely to represent weird symbols, if you have a look to the file format of SQL, these are reserved bytes for storing some metadata.
Open the sqlite file with any text editor and check that what you see there matches with what comes out of your code.
This website indicates which extra libraries to make use of, as well as provides examples of how to interact with the sqlite files (http://www.xerial.org/trac/Xerial/wiki/SQLiteJDBC#Usage)
important things to note:
1) make sure to include load the sqlite-JDBC driver using the current class loader. This is done with the line
2) the sqlite file IS a db, even if its not sitting on a server somewhere. So you still must create a connection to the file to interact with it. And you must open the connection and close the connection as well.
Connection connection = null;
connection = DriverManager.getConnection("jdbc:sqlite:" + path); // path is a String to the sqlite file (sqlite or sqback)
connection.close(); // after you are done with the file
3) Information can be extracted by using sql code to query the file. This returns a processable object of type ResultSet that holds your data pertaining to the query
Statement statement = connection.createStatement();
statement.setQueryTimeout(30);
ResultSet rs = statement.executeQuery("SELECT * FROM " + tblName);
4) from the ResultsSet you can grab data using the get commands with either the column index or the column header key
rs.getString("qo")
Hope that helps anyone having the same issue as I was having
I have written a application that parses the html code of some web pages. My problem is with inserting that data into my mysq database. So for example i want to insert ľščťžýáíé and when i look into the table i get ?š??žýáíé.
I guess the problem could be that the html pages i'm downloading are encoded in cp1250. but the database is utf8.
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream(),"cp1250"));
and this is how i download the data.
Do you have some ideas how to fix this problem? Because i allready ran out.
Edit: oh and when i write the data out to the console (with System.out, i know i shouldn't use it... :) ) then every character is showing up correctly.
issue a set names CP1251; just after your connect to mysql and before any inserts
So i found out what works.
As i'm connecting to via JDBC to MySQL i used the following connection string
conString = "jdbc:mysql://"+host+"/"+database+"?useUnicode=true&characterEncoding=utf8";
And this did the trick. I had to force JDBC to use utf8 for the connection using ?useUnicode=true&characterEncoding=utf8