Develop Reference

Cannot parse file using JDBC

Im trying to parse a pipe delimited file and insert fields into a table. when i start the application nothing happens in my DB. My DB has 4 columns (account_name, command_name, and system_name, CreateDt). The file i am parsing has the date in the first row then extra data. The rows following i only need the first 3 fields in each the rest is extra data. the last row is the row count. i skipped the inserting date because for now but want to get back to it after at least able to insert the first 3 fields. I have little experience with parsing a file and storing data in a DB and have looked through jdbc examples to get to this point but im struggling and am sure there is a better way.
File Example
20200310|extra|extra|extra||
Mn1223|01192|windows|extra|extra|extra||
Sd1223|02390|linux|extra|extra|extra||
2
table format
account_name command_name system_name createDt
Mn1223 01192 windows 20200310
Sd1223 02390 linux 20200310
Code to parse and insert into DB
public List insertZygateData (List<ZygateEntity> parseData) throws Exception {
String filePath = "C:\\DEV\\Test_file.xlsx";
List<String> lines = Files.readAllLines(Paths.get(filePath));
// remove date and amount
lines.remove(0);
lines.remove(lines.size() - 1);
for (ZygateEntity zygateInfo : parseData){
new MapSqlParameterSource("account_name", zygateInfo.getAccountName())
.addValue("command_name", zygateInfo.getCommandName())
.addValue("system_name", zygateInfo.getSystemName())
.getValues();
}
return lines.stream()
.map(s -> s.split("[|]")).map(val -> new ZygateEntity(val[0],val[1],val[2])).collect(Collectors.toList());
}
public boolean cleantheTable() throws SQLException {
String sql = "INSERT INTO Landing.midrange_xygate_load (account_name,command_name,system_name)"+
"VALUES (:account_name,:command_name,:system_name)";
boolean truncated = false;
Statement stmt = null;
try {
String sqlTruncate = "truncate table Landing.midrange_xygate_load";
jdbcTemplate.execute(sqlTruncate);
truncated = true;
} catch (Exception e) {
e.printStackTrace();
truncated = false;
return truncated;
} finally {
if (stmt != null) {
jdbcTemplate.execute(sql);
stmt.close();
}
}
log.info("Clean the table return value :" + truncated);
return truncated;
}
}
Entity/Model
public ZygateEntity(String accountName, String commandName, String systemName){
this.accountName=accountName;
this.commandName=commandName;
this.systemName=systemName;
}
//getters and setters
#Override
public String toString() {
return "ZygateEntity [accountName=" + accountName + ", commandName=" + commandName + ", systemName=" + systemName + ", createDt=" + createDt +"]";
}
}

Taking a look at what you've provided, it seems you have a jumbled collection of bits of code, and while most of it is there, it's not all there and not quite all in the right order.
To get some kind of clarity, try to break down what it is you're doing into separate steps, and have a method that focuses on each step. In particular, you write
Im trying to parse a pipe delimited file and insert fields into a table
This naturally breaks down into two parts:
parsing the pipe-delimited file, and
inserting fields into a table.
For the first part, you seem to have most of the parts already in your insertZygateData method. In particular, this line reads all the lines of a file into a list:
List<String> lines = Files.readAllLines(Paths.get(filePath));
These lines then remove the first and last lines from the list of lines read:
// remove date and amount
lines.remove(0);
lines.remove(lines.size() - 1);
You then have some code that looks a bit out of place: this seems to be something to do with inserting into the database, but we haven't created our list of ZygateEntity objects as we haven't yet finished reading the file. Let's put this for loop to one side for the moment.
Finally, we take the list of lines we read, split them using pipes, create ZygateEntity objects from the parts and create a List of these objects, which we then return.
return lines.stream()
.map(s -> s.split("[|]")).map(val -> new ZygateEntity(val[0],val[1],val[2])).collect(Collectors.toList());
Putting this lot together, we have a useful method that parses the file, completing the first part of the task:
private List<ZygateEntity> parseZygateData() throws IOException {
String filePath = "C:\\DEV\\Test_file.xlsx";
List<String> lines = Files.readAllLines(Paths.get(filePath));
// remove date and amount
lines.remove(0);
lines.remove(lines.size() - 1);
return lines.stream()
.map(s -> s.split("[|]")).map(val -> new ZygateEntity(val[0],val[1],val[2])).collect(Collectors.toList());
}
(Of course, we could add a parameter for the file path to read, but in the interest of getting something working, it's OK to stick with the current hard-coded file path.)
So, we've got our list of ZygateEntity objects. How do we write a method to insert them into the database?
We can find a couple of the ingredients we need in your code sample. First, we need the SQL statement to insert the data. This is in your cleanThetable method:
String sql = "INSERT INTO Landing.midrange_xygate_load (account_name,command_name,system_name)"+
"VALUES (:account_name,:command_name,:system_name)";
We then have this loop:
for (ZygateEntity zygateInfo : parseData){
new MapSqlParameterSource("account_name", zygateInfo.getAccountName())
.addValue("command_name", zygateInfo.getCommandName())
.addValue("system_name", zygateInfo.getSystemName())
.getValues();
}
This loop creates a MapSqlParameterSource out of each ZygateEntity object, and then converts it to a Map<String, Object> by calling the getValues() method. But then it does nothing with this value. Effectively you're creating these objects and getting rid of them again without doing anything with them. This isn't ideal.
A MapSqlParameterSource is used with a Spring NamedParameterJdbcTemplate. Your code mentions a jdbcTemplate, which appears to be a field within the class that parses data and inserts into the database, but you don't show the full code of this class. I'm going to have to assume it's a NamedParameterJdbcTemplate rather than a 'plain' JdbcTemplate.
A NamedParameterJdbcTemplate contains a method update that takes a SQL string and a SqlParameterSource. We have a SQL string, and we're creating MapSqlParameterSource objects, so we can use these to carry out the insert. There's not a lot of point in creating one of these MapSqlParameterSource objects only to convert it to a map, so let's remove the call to getValues().
So, we now have a method to insert the data into the database:
public void insertZygateData(List<ZygateEntity> parseData) {
String sql = "INSERT INTO Landing.midrange_xygate_load (account_name,command_name,system_name)"+
"VALUES (:account_name,:command_name,:system_name)";
for (ZygateEntity zygateInfo : parseData){
SqlParameterSource source = new MapSqlParameterSource("account_name", zygateInfo.getAccountName())
.addValue("command_name", zygateInfo.getCommandName())
.addValue("system_name", zygateInfo.getSystemName());
jdbcTemplate.update(sql, source);
}
}
Finally, let's take a look at your cleanThetable method. As with the others, let's keep it focused on one task: it looks like at the moment you're trying to delete the data out of the table and then insert it in the same method, but let's have it just focus on deleting the data as we've now got a method to insert the data.
We can't immediately get rid of the String sql = ... line, because the finally block in your code uses it. If stmt is not null, then you attempt to run the INSERT statement and then close stmt.
However, stmt is never assigned any value other than null, so it remains null. stmt != null is therefore always false, so the INSERT statement never runs. Your finally block never does anything, so you would be best off removing it altogether. With your finally block gone, you can also get rid of your local variable stmt and the sql string, leaving us with a method whose focus is to truncate the table:
public boolean cleantheTable() throws SQLException {
boolean truncated = false;
try {
String sqlTruncate = "truncate table Landing.midrange_xygate_load";
jdbcTemplate.execute(sqlTruncate);
truncated = true;
} catch (Exception e) {
e.printStackTrace();
truncated = false;
return truncated;
}
log.info("Clean the table return value :" + truncated);
return truncated;
}
I'll leave it up to you to write the code that calls these methods. I wrote some code for this purpose, and it ran successfully and inserted into a database.
So, in summary, no data was being written to your database because you were never making a call to the database to insert any. In your insertZygateData method you were creating the parameter-source objects but not doing anything useful with them, and in your cleanThetable method, it looked like you were trying to insert data, but your line jdbcTemplate.execute(sql) that attempted to do this never ran. Even if stmt wasn't null, this line wouldn't work as you didn't pass the parameter values in anywhere: you would get an exception from the database as it would be expecting values for the parameters but you never gave it any.
Hopefully my explanation gives you a way of getting your code working and helps you understand why it wasn't.

How to configure the #CosmosDBtrigger using java?

I'm setting up #CosmosDBTrigger, need help with the below code and also what needs to be in the name field?
I'm using below Tech stack,
JDK 1.8.0-211
apache maven 3.5.3
AzureCLI 2.0.71
.net core 2.2.401
Java:
public class Function {
#FunctionName("CosmosTrigger")
public void mebershipProfileTrigger(
#CosmosDBTrigger(name = "?", databaseName =
"*database_name*", collectionName = "*collection_name*",
leaseCollectionName = "leases",
createLeaseCollectionIfNotExists = true,
connectionStringSetting = "DBConnection") String[] items,
final ExecutionContext context) {
context.getLogger().info("item(s) changed");
}
}
What do we need to provide in the name field?
local.settings.json
{
"IsEncrypted": false,
"Values": {
"DBConnection": "AccountEndpoint=*Account_Endpoint*"
}
}
Expected: function starts
Result:
"Microsoft.Azure.WebJobs.Host: Error indexing method 'Functions.Cosmostrigger'. Microsoft.Azure.WebJobs.Extensions.CosmosDB: Cannot create Collection Information for collection_name in database database_name with lease leases in database database_name : Unexpected character encountered while parsing value: <. Path '', line 0, position 0. Newtonsoft.Json: Unexpected character encountered while parsing value: <. Path '', line 0, position 0."

Follow this:- https://github.com/microsoft/inventory-hub-java-on-azure/blob/master/function-apps/Notify-Inventory/src/main/java/org/inventory/hub/NotifyInventoryUpdate.java
#CosmosDBTrigger(name = "document", databaseName = "db1",collectionName = "col1", connectionStringSetting = "dbstr",leaseCollectionName = "lease1", createLeaseCollectionIfNotExists = true) String document,
Now when you publish put the value for dbstr as your connection string in Application Settings of Azure portal, after setting the properties just restart

See the official samples here: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-cosmosdb-v2#trigger---java-example
name is just some identifier for your Function. The error you are getting is because you are telling the Trigger that the Collection you want to monitor your changes in is called "collection_name" and it's inside a database called "database_name".
Please use the real correct values for them, they should be pointing to an existing Collection, and your connection string DBConnection needs to be in the correct format of: AccountEndpoint=https://<your-account-name>.documents.azure.com:443/;AccountKey=<your-account-key>;(you can get it from the Azure Portal).

SQLite database saves String as "0"

I've just learned how to use SQLite database in Android. And i have one error which i cant figure out.
When i try to save string in my sqlite database, it saves it always as "0", no matter which string i use as an input. This is my code example.
ContentValues values = new ContentValues();
Log.i("CrimeLab", "put : "+crime.getId().toString());
values.put(CrimeDbSchema.CrimeTable.Cols.UUID, "31b0a98a-4089-46de-8325-4ec673bbd713"); // crime.getId().toString()
Log.i("CrimeLab", "take: "+values.getAsString(CrimeDbSchema.CrimeTable.Cols.UUID));
values.put(CrimeDbSchema.CrimeTable.Cols.TITLE, crime.getTitle());
values.put(CrimeDbSchema.CrimeTable.Cols.DATE, crime.getDate().getTime());
values.put(CrimeDbSchema.CrimeTable.Cols.UUID, crime.isSolved() ? 1 : 0);
I have class Crime which have it's unique UUID, title, date and if crime is solved. Everything is saved perfectly but only UUID.toString() is saved in database as "0". No matter which string i try to put it will be saved as "0" in my database, here's the picture.
Here is how it is displayed in the database, everything is good except this string. I have one book, from where i follow my lectures, and it is the same code from the book and it doesn't work. This is how i get my value back from the database.
public class CrimeCursorWrapper extends CursorWrapper
{
public CrimeCursorWrapper(Cursor cursor)
{
super(cursor);
}
public Crime getCrime(){
String uuidString = getString(getColumnIndex(CrimeDbSchema.CrimeTable.Cols.UUID));
String title = getString(getColumnIndex(CrimeDbSchema.CrimeTable.Cols.TITLE));
long date = getLong(getColumnIndex(CrimeDbSchema.CrimeTable.Cols.DATE));
int isSolved = getInt(getColumnIndex(CrimeDbSchema.CrimeTable.Cols.SOLVED));
// And so on.

Everything is saved perfectly but only UUID.toString() is saved in database as "0".
That is because you have not solved any crimes, and you probably have a typo in your code.
In your code snippet, you are setting CrimeDbSchema.CrimeTable.Cols.UUID twice. Once it is the UUID. Once it is 0 or 1 depending on isSolved().
I would assume that the second occurrence needs a different column name.

How to store Single user login data for Android?

Referring to the question and answer on Best way to store a single user in an Android app?
How does shared preferences actually work?
What I'd like to do is:
First time user opens app adds a login id and password
Next time user opens app uses the previous id/password and data to login. (I don't want automatic login because data in my app will be sensitive and thus even a friend taking the mobile shouldn't be able to see it.)
Ability for the user to change this id/password
Is this possible through Shared Preferences? Or do I need to use SQLlite?
I am completely new to Android so I'd really appreciate it if you attach a working code and explanation.

You can do this with shared preferences, as long as you are comfortable storing reasonably confidential data there. You will need some shared codes between storing and retrieving:
final static String pfName = "com.super.stuff.preffile.name";
final static String pfCodeForID = "com.super.stuff.pf.id";
final static String pfCodeForPassword = "com.super.stuff.pf.passwd";
final static String pfNoStringPresent = "NO-STRING-PRESENT-HERE";
final static pfCodes = MODE_PRIVATE; // See http://developer.android.com/reference/android/content/Context.html#getSharedPreferences(java.lang.String, int)
To store the information:
String ID = //whatever;
String password = //whatever;
SharedPreferences settings = context.getSharedPreferences(pfName, pfCodes);
SharedPreferences.Editor editor = settings.edit();
editor.putString(pfCodeForID, ID);
editor.putString(pfCodeForPassword, password);
editor.commit();
To retrieve the information:
SharedPreferences settings = context.getSharedPreferences(pfName, pfCodes);
String ID = editor.getString(pfCodeForID, pfNoStringPresent);
String password = editor.getString(pfCodeForPassword, pfNoStringPresent);
if (ID.contentEquals(pfNoStringPresent) && password.contentEquals(pfNoStringPresent)) {
// Handle the case of nothing stored, ie get ID and password
}
Obviously this fails if both the username and the password are the same as pfNoStringPresent!
If you are concerned about storing sensitive data in this way, then you will need to store it either in a database, or encrypt it in some way. You will need to decide how critical it is for the information to be protected when it is being stored on a device belonging to the person who is giving you the ID information, how important getting this information from the phone would be to a thief, etc etc.

Use Sqlite, its fairly simple. Follow this:
public SQLiteDatabase sampleDB;
sampleDB = this.openOrCreateDatabase(TABLE_NAME, MODE_PRIVATE, null);
sampleDB.execSQL("CREATE TABLE IF NOT EXISTS " +
TABLE_NAME+ "(" + COLUMN_ID
+ " integer primary key autoincrement, " + COLUMN1
+ " text not null,"+ COLUMN2
+ " text not null);");
Here i have three fields where column1 and column2 are strings having values "username" and "password". After this creation you can execute query as what ever you need.

Integrate with AccountManager then use setUserData for it...the best way i think. :)

Error on Updating Table in Sqlite

When trying to update a record for one of my records I am using this code
private void UpdateCattleRecord(UpdateCattleRecord updateRecord){
mDB.beginTransaction();
String where = "_ID=";
String[] RecordToUpdate = {Cattle._ID};
Toast.makeText(this,"Updating Animal "+ RecordToUpdate, Toast.LENGTH_LONG).show();
try {
ContentValues CattleFieldsToUpdate = new ContentValues();
CattleFieldsToUpdate.put(Cattle.CATTLE_ANIMALID,updateRecord.getCattleName());
CattleFieldsToUpdate.put(Cattle.CATTLE_TYPE, updateRecord.getCattleType());
CattleFieldsToUpdate.put(Cattle.CATTLE_LOCATION, updateRecord.getCattleLocation());
CattleFieldsToUpdate.put(Cattle.CATTLE_DOB, updateRecord.getCattleDob());
CattleFieldsToUpdate.put(Cattle.CATTLE_DAM, updateRecord.getCattleDam());
CattleFieldsToUpdate.put(Cattle.CATTLE_SEX, updateRecord.getCattleSex());
mDB.update(Cattle.CATTLE_TABLE_NAME,CattleFieldsToUpdate, where, RecordToUpdate);
mDB.setTransactionSuccessful();
} finally {
mDB.endTransaction();
}
}
My log shows
Tag Database sqlite returned: error code =1, msg = near "=": syntax error
After researching this, I think I have everything in the right place but obviously I don't,
when I look at the next error in the log it's of course in 'red' and it shows me all the correct data,
03-27 15:15:29.291: E/Database(12011): Error updating date_of_birth=March 27, 2012 animaltype=Calf sex=F location=Eastern dam=601 animal_id=601A using UPDATE cattle SET date_of_birth=?, animaltype=?, sex=?, location=?, dam=?, animal_id=? WHERE _ID=
I've obviously got a problem with the value for _ID but can't seem to locate it. Can someone please point out where my Syntax error is?
Update
The problem occurred because I was failing to pass the actual value of the record (_ID) that I wanted to update. Once I passed that as a parameter to my updaterecords function the update went as scheduled.
Thanks for the input, it helped me narrow down what I was doing wrong.

Check your database creation, your probably have a column named _id(although you refer to it by _ID, its name is _id) and not _ID:
String where = "_id= ?"; // ? represent the value from the selection arguments String array
or better:
String where = Cattle._ID + "= ?";
Edit:
In your where selection argument you put:
String[] RecordToUpdate = {Cattle._ID};
you probably want to put in there some id you get from somewhere(of the record you want to update, a long number), right now you're doing:
WHERE _ID = _ID (or _id)
and this will fail.

try:
mDB.update(Cattle.CATTLE_TABLE_NAME,CattleFieldsToUpdate, "_ID="+Cattle._ID, null);
try:
mDB.update(Cattle.CATTLE_TABLE_NAME,CattleFieldsToUpdate, "_ID="+updateRecord.getId(), null);