Develop Reference

Cannot parse file using JDBC

Im trying to parse a pipe delimited file and insert fields into a table. when i start the application nothing happens in my DB. My DB has 4 columns (account_name, command_name, and system_name, CreateDt). The file i am parsing has the date in the first row then extra data. The rows following i only need the first 3 fields in each the rest is extra data. the last row is the row count. i skipped the inserting date because for now but want to get back to it after at least able to insert the first 3 fields. I have little experience with parsing a file and storing data in a DB and have looked through jdbc examples to get to this point but im struggling and am sure there is a better way.
File Example
20200310|extra|extra|extra||
Mn1223|01192|windows|extra|extra|extra||
Sd1223|02390|linux|extra|extra|extra||
2
table format
account_name command_name system_name createDt
Mn1223 01192 windows 20200310
Sd1223 02390 linux 20200310
Code to parse and insert into DB
public List insertZygateData (List<ZygateEntity> parseData) throws Exception {
String filePath = "C:\\DEV\\Test_file.xlsx";
List<String> lines = Files.readAllLines(Paths.get(filePath));
// remove date and amount
lines.remove(0);
lines.remove(lines.size() - 1);
for (ZygateEntity zygateInfo : parseData){
new MapSqlParameterSource("account_name", zygateInfo.getAccountName())
.addValue("command_name", zygateInfo.getCommandName())
.addValue("system_name", zygateInfo.getSystemName())
.getValues();
}
return lines.stream()
.map(s -> s.split("[|]")).map(val -> new ZygateEntity(val[0],val[1],val[2])).collect(Collectors.toList());
}
public boolean cleantheTable() throws SQLException {
String sql = "INSERT INTO Landing.midrange_xygate_load (account_name,command_name,system_name)"+
"VALUES (:account_name,:command_name,:system_name)";
boolean truncated = false;
Statement stmt = null;
try {
String sqlTruncate = "truncate table Landing.midrange_xygate_load";
jdbcTemplate.execute(sqlTruncate);
truncated = true;
} catch (Exception e) {
e.printStackTrace();
truncated = false;
return truncated;
} finally {
if (stmt != null) {
jdbcTemplate.execute(sql);
stmt.close();
}
}
log.info("Clean the table return value :" + truncated);
return truncated;
}
}
Entity/Model
public ZygateEntity(String accountName, String commandName, String systemName){
this.accountName=accountName;
this.commandName=commandName;
this.systemName=systemName;
}
//getters and setters
#Override
public String toString() {
return "ZygateEntity [accountName=" + accountName + ", commandName=" + commandName + ", systemName=" + systemName + ", createDt=" + createDt +"]";
}
}

Taking a look at what you've provided, it seems you have a jumbled collection of bits of code, and while most of it is there, it's not all there and not quite all in the right order.
To get some kind of clarity, try to break down what it is you're doing into separate steps, and have a method that focuses on each step. In particular, you write
Im trying to parse a pipe delimited file and insert fields into a table
This naturally breaks down into two parts:
parsing the pipe-delimited file, and
inserting fields into a table.
For the first part, you seem to have most of the parts already in your insertZygateData method. In particular, this line reads all the lines of a file into a list:
List<String> lines = Files.readAllLines(Paths.get(filePath));
These lines then remove the first and last lines from the list of lines read:
// remove date and amount
lines.remove(0);
lines.remove(lines.size() - 1);
You then have some code that looks a bit out of place: this seems to be something to do with inserting into the database, but we haven't created our list of ZygateEntity objects as we haven't yet finished reading the file. Let's put this for loop to one side for the moment.
Finally, we take the list of lines we read, split them using pipes, create ZygateEntity objects from the parts and create a List of these objects, which we then return.
return lines.stream()
.map(s -> s.split("[|]")).map(val -> new ZygateEntity(val[0],val[1],val[2])).collect(Collectors.toList());
Putting this lot together, we have a useful method that parses the file, completing the first part of the task:
private List<ZygateEntity> parseZygateData() throws IOException {
String filePath = "C:\\DEV\\Test_file.xlsx";
List<String> lines = Files.readAllLines(Paths.get(filePath));
// remove date and amount
lines.remove(0);
lines.remove(lines.size() - 1);
return lines.stream()
.map(s -> s.split("[|]")).map(val -> new ZygateEntity(val[0],val[1],val[2])).collect(Collectors.toList());
}
(Of course, we could add a parameter for the file path to read, but in the interest of getting something working, it's OK to stick with the current hard-coded file path.)
So, we've got our list of ZygateEntity objects. How do we write a method to insert them into the database?
We can find a couple of the ingredients we need in your code sample. First, we need the SQL statement to insert the data. This is in your cleanThetable method:
String sql = "INSERT INTO Landing.midrange_xygate_load (account_name,command_name,system_name)"+
"VALUES (:account_name,:command_name,:system_name)";
We then have this loop:
for (ZygateEntity zygateInfo : parseData){
new MapSqlParameterSource("account_name", zygateInfo.getAccountName())
.addValue("command_name", zygateInfo.getCommandName())
.addValue("system_name", zygateInfo.getSystemName())
.getValues();
}
This loop creates a MapSqlParameterSource out of each ZygateEntity object, and then converts it to a Map<String, Object> by calling the getValues() method. But then it does nothing with this value. Effectively you're creating these objects and getting rid of them again without doing anything with them. This isn't ideal.
A MapSqlParameterSource is used with a Spring NamedParameterJdbcTemplate. Your code mentions a jdbcTemplate, which appears to be a field within the class that parses data and inserts into the database, but you don't show the full code of this class. I'm going to have to assume it's a NamedParameterJdbcTemplate rather than a 'plain' JdbcTemplate.
A NamedParameterJdbcTemplate contains a method update that takes a SQL string and a SqlParameterSource. We have a SQL string, and we're creating MapSqlParameterSource objects, so we can use these to carry out the insert. There's not a lot of point in creating one of these MapSqlParameterSource objects only to convert it to a map, so let's remove the call to getValues().
So, we now have a method to insert the data into the database:
public void insertZygateData(List<ZygateEntity> parseData) {
String sql = "INSERT INTO Landing.midrange_xygate_load (account_name,command_name,system_name)"+
"VALUES (:account_name,:command_name,:system_name)";
for (ZygateEntity zygateInfo : parseData){
SqlParameterSource source = new MapSqlParameterSource("account_name", zygateInfo.getAccountName())
.addValue("command_name", zygateInfo.getCommandName())
.addValue("system_name", zygateInfo.getSystemName());
jdbcTemplate.update(sql, source);
}
}
Finally, let's take a look at your cleanThetable method. As with the others, let's keep it focused on one task: it looks like at the moment you're trying to delete the data out of the table and then insert it in the same method, but let's have it just focus on deleting the data as we've now got a method to insert the data.
We can't immediately get rid of the String sql = ... line, because the finally block in your code uses it. If stmt is not null, then you attempt to run the INSERT statement and then close stmt.
However, stmt is never assigned any value other than null, so it remains null. stmt != null is therefore always false, so the INSERT statement never runs. Your finally block never does anything, so you would be best off removing it altogether. With your finally block gone, you can also get rid of your local variable stmt and the sql string, leaving us with a method whose focus is to truncate the table:
public boolean cleantheTable() throws SQLException {
boolean truncated = false;
try {
String sqlTruncate = "truncate table Landing.midrange_xygate_load";
jdbcTemplate.execute(sqlTruncate);
truncated = true;
} catch (Exception e) {
e.printStackTrace();
truncated = false;
return truncated;
}
log.info("Clean the table return value :" + truncated);
return truncated;
}
I'll leave it up to you to write the code that calls these methods. I wrote some code for this purpose, and it ran successfully and inserted into a database.
So, in summary, no data was being written to your database because you were never making a call to the database to insert any. In your insertZygateData method you were creating the parameter-source objects but not doing anything useful with them, and in your cleanThetable method, it looked like you were trying to insert data, but your line jdbcTemplate.execute(sql) that attempted to do this never ran. Even if stmt wasn't null, this line wouldn't work as you didn't pass the parameter values in anywhere: you would get an exception from the database as it would be expecting values for the parameters but you never gave it any.
Hopefully my explanation gives you a way of getting your code working and helps you understand why it wasn't.

checking the database value using sugarRecord

i am having a problem in checking the value passed by the user every time,is already there in the sugarORM database. I am using this line of code to retrieve the value.but this line of doesn't get execute
private void postDataToSQLite() {
User.find(User.class, "email = ? and password = ?", textInputEditTextEmail.getText().toString().trim(),textInputEditTextPassword.getText().toString().trim());

Inserting data with UCanAccess from big text files is very slow

I'm trying to read text files .txt with more than 10.000 lines per file, splitting them and inserting the data in Access database using Java and UCanAccess. The problem is that it becomes slower and slower every time (as the database gets bigger).
Now after reading 7 text files and inserting them into database, it would take the project more than 20 minutes to read another file.
I tried to do just the reading and it works fine, so the problem is the actual inserting into database.
N.B: This is my first time using UCanAccess with Java because I found that the JDBC-ODBC Bridge is no longer available. Any suggestions for an alternative solution would also be appreciated.

If your current task is simply to import a large amount of data from text files straight into the database, and it does not require any sophisticated SQL manipulations, then you might consider using the Jackcess API directly. For example, to import a CSV file you could do something like this:
String csvFileSpec = "C:/Users/Gord/Desktop/BookData.csv";
String dbFileSpec = "C:/Users/Public/JackcessTest.accdb";
String tableName = "Book";
try (Database db = new DatabaseBuilder()
.setFile(new File(dbFileSpec))
.setAutoSync(false)
.open()) {
new ImportUtil.Builder(db, tableName)
.setDelimiter(",")
.setUseExistingTable(true)
.setHeader(false)
.importFile(new File(csvFileSpec));
// this is a try-with-resources block,
// so db.close() happens automatically
}
Or, if you need to manually parse each line of input, insert a row, and retrieve the AutoNumber value for the new row, then the code would be more like this:
String dbFileSpec = "C:/Users/Public/JackcessTest.accdb";
String tableName = "Book";
try (Database db = new DatabaseBuilder()
.setFile(new File(dbFileSpec))
.setAutoSync(false)
.open()) {
// sample data (e.g., from parsing of an input line)
String title = "So, Anyway";
String author = "Cleese, John";
Table tbl = db.getTable(tableName);
Object[] rowData = tbl.addRow(Column.AUTO_NUMBER, title, author);
int newId = (int)rowData[0]; // retrieve generated AutoNumber
System.out.printf("row inserted with ID = %d%n", newId);
// this is a try-with-resources block,
// so db.close() happens automatically
}
To update an existing row based on its primary key, the code would be
Table tbl = db.getTable(tableName);
Row row = CursorBuilder.findRowByPrimaryKey(tbl, 3); // i.e., ID = 3
if (row != null) {
// Note: column names are case-sensitive
row.put("Title", "The New Title For This Book");
tbl.updateRow(row);
}
Note that for maximum speed I used .setAutoSync(false) when opening the Database, but bear in mind that disabling AutoSync does increase the chance of leaving the Access database file in a damaged (and possibly unusable) state if the application terminates abnormally while performing the updates.

Also, if you need to use slq/ucanaccess, you have to call setAutocommit(false) on the connection at the begin, and do a commit each 200/300 record. The performances will improve drammatically (about 99%).

BaseAdapter with sections, listview - adding cursor list data

I'm new here and would be great if someone could help me with this small crisis I have been having. I have been following Jeff Sharkeys Separate List Adapter tutorial which can be found here and I got it all working appropriately to how he explains it.
My problem is I have a database, pre made one which has a table that I am trying to put the extracted data into a list view using Jeffs adapter. I need this data to be put in different sections according to the Category ID column.
The table I am currently extracting data from is the Food table, which has 6 columns, categoriID, menuID, Item, Description, price and a PK _id
My database works properly as in other activities I use a SimpleCursorAdapter to bind data to a list view. (from other tables)
I use the following method in the Database Helper class to retrieve the data I need
public List<FoodModel> getData(String catid, String menuid) {
List<FoodModel> FoodListModel = new ArrayList<FoodModel>();
Cursor cursor = myDataBase.query(FOOD_TABLE, new String [] {FOODITEM_COLUMN, FOODITEMDESCRIPTION_COLUMN,FOODPRICE_COLUMN}, "catid = ? AND menuid = ?",
new String[] { catid, menuid},null, null, null, null);
if (cursor.moveToFirst()){
do{
FoodModel FoodModel = new FoodModel();
FoodModel.setItem(cursor.getString(0));
FoodModel.setdescrription(cursor.getString(1));
FoodModel.setprice(Double.parseDouble(cursor.getString(2)));
FoodListModel.add(FoodModel);} while (cursor.moveToNext());
}
return FoodListModel;
}
the Food Model class is the standard get set class to store the cursors data.
This public method works accordingly as in my main activity (jeffs ListSample.java) I output to the log cat the required information using
Log.d("Reading: ", "Testing Cursor");
List<FoodModel> Data1 = dba.getData("1", "1");
for (FoodModel fd : Data1){
String log = "Item: "+fd.getitem()+" ,Description: " + fd.getdescription() + " ,Price: " + fd.getprice();
Log.d("Name: ", log);
This outputs a list of all the data I need to put into 1 section of the SeperateListAdapter but to the log cat
The 3 things i am trying to output to the list view are the Item, description and price.
My problem is How do I add this data to the list view under the correct section?
as oposed to manually inserting it as Jeff shows
List<Map<String,?>> security = new LinkedList<Map<String,?>>();
security.add(createItem("Remember passwords", "Save usernames and passwords for Web sites"));
security.add(createItem("Clear passwords", "Save usernames and passwords for Web sites"));
security.add(createItem("Show security warnings", "Show warning if there is a problem with a site's security"));
I didn't want to start my first experience of using this site by pasting all my classes is as this is my first time using this website, my English is not the best and it took me a while to write this I hope it is acceptable as a question, I would be really great full if someone could kindly help me or point me in the right direction.
EDIT: Will gladly post the rest of my code just didnt whant to bombard everyone with loads of information

Error on Updating Table in Sqlite

When trying to update a record for one of my records I am using this code
private void UpdateCattleRecord(UpdateCattleRecord updateRecord){
mDB.beginTransaction();
String where = "_ID=";
String[] RecordToUpdate = {Cattle._ID};
Toast.makeText(this,"Updating Animal "+ RecordToUpdate, Toast.LENGTH_LONG).show();
try {
ContentValues CattleFieldsToUpdate = new ContentValues();
CattleFieldsToUpdate.put(Cattle.CATTLE_ANIMALID,updateRecord.getCattleName());
CattleFieldsToUpdate.put(Cattle.CATTLE_TYPE, updateRecord.getCattleType());
CattleFieldsToUpdate.put(Cattle.CATTLE_LOCATION, updateRecord.getCattleLocation());
CattleFieldsToUpdate.put(Cattle.CATTLE_DOB, updateRecord.getCattleDob());
CattleFieldsToUpdate.put(Cattle.CATTLE_DAM, updateRecord.getCattleDam());
CattleFieldsToUpdate.put(Cattle.CATTLE_SEX, updateRecord.getCattleSex());
mDB.update(Cattle.CATTLE_TABLE_NAME,CattleFieldsToUpdate, where, RecordToUpdate);
mDB.setTransactionSuccessful();
} finally {
mDB.endTransaction();
}
}
My log shows
Tag Database sqlite returned: error code =1, msg = near "=": syntax error
After researching this, I think I have everything in the right place but obviously I don't,
when I look at the next error in the log it's of course in 'red' and it shows me all the correct data,
03-27 15:15:29.291: E/Database(12011): Error updating date_of_birth=March 27, 2012 animaltype=Calf sex=F location=Eastern dam=601 animal_id=601A using UPDATE cattle SET date_of_birth=?, animaltype=?, sex=?, location=?, dam=?, animal_id=? WHERE _ID=
I've obviously got a problem with the value for _ID but can't seem to locate it. Can someone please point out where my Syntax error is?
Update
The problem occurred because I was failing to pass the actual value of the record (_ID) that I wanted to update. Once I passed that as a parameter to my updaterecords function the update went as scheduled.
Thanks for the input, it helped me narrow down what I was doing wrong.

Check your database creation, your probably have a column named _id(although you refer to it by _ID, its name is _id) and not _ID:
String where = "_id= ?"; // ? represent the value from the selection arguments String array
or better:
String where = Cattle._ID + "= ?";
Edit:
In your where selection argument you put:
String[] RecordToUpdate = {Cattle._ID};
you probably want to put in there some id you get from somewhere(of the record you want to update, a long number), right now you're doing:
WHERE _ID = _ID (or _id)
and this will fail.

try:
mDB.update(Cattle.CATTLE_TABLE_NAME,CattleFieldsToUpdate, "_ID="+Cattle._ID, null);
try:
mDB.update(Cattle.CATTLE_TABLE_NAME,CattleFieldsToUpdate, "_ID="+updateRecord.getId(), null);