I have a working sqlite db which I have place in my /src folder.
I then went onto the Codename One website and followed their doc example
Database db = null;
Cursor cur = null;
try {
db = Display.getInstance().openOrCreate("MyDb.db");
if(query.getText().startsWith("select")) {
cur = db.executeQuery(query.getText());
int columns = cur.getColumnCount();
frmMain.removeAll();
if(columns > 0) {
boolean next = cur.next();
if(next) {
ArrayList<String[]> data = new ArrayList<>();
String[] columnNames = new String[columns];
for(int iter = 0 ; iter < columns ; iter++) {
columnNames[iter] = cur.getColumnName(iter);
}
while(next) {
Row currentRow = cur.getRow();
String[] currentRowArray = new String[columns];
for(int iter = 0 ; iter < columns ; iter++) {
currentRowArray[iter] = currentRow.getString(iter);
}
data.add(currentRowArray);
next = cur.next();
}
Object[][] arr = new Object[data.size()][];
data.toArray(arr);
frmMain.add(BorderLayout.CENTER, new Table(new DefaultTableModel(columnNames, arr)));
} else {
frmMain.add(BorderLayout.CENTER, "Query returned no results");
}
} else {
frmMain.add(BorderLayout.CENTER, "Query returned no results");
}
} else {
db.execute(query.getText());
frmMain.add(BorderLayout.CENTER, "Query completed successfully");
}
frmMain.revalidate();
} catch(IOException err) {
frmMain.removeAll();
frmMain.add(BorderLayout.CENTER, "Error: " + err);
frmMain.revalidate();
} finally {
Util.cleanup(db);
Util.cleanup(cur);
}
However when I run the example and try and execute a simple select query I get this error ...
java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: MyTable)
So I have added the DB
I have used the 'openOrCreate' statement
Have I missed a step?
Thanks
Are you shure that you current working directory at execution is ./src ?
Try
db = Display.getInstance().openOrCreate("./src/MyDb.db");
or open with absolute filename:
db = Display.getInstance().openOrCreate("/path/to/src/MyDb.db");
You can try this cn1lib https://github.com/shannah/cn1-data-access-lib
I have used it and it works a charm, except doesn't work for 2 tables in the same query and can't perform delete operations.
Cheers
Thanks for all the input guys.
Unfortunately none of the advice worked for me.
However I did solve it in the end.
It turns out that there is a folder in my home directory called '.cn1/database'. Once I placed the DB into this folder it worked.
Two things:
1] If the db does not exist then it will create it and place it into this directory
2] The db does not show up anywhere in Netbeans (well not that I could see anyway)
Thanks again
From the developer guide:
Some SQLite apps ship with a "ready made" database. We allow you to
replace the DB file by using the code:
String path = Display.getInstance().getDatabasePath(“databaseName”);
You can then use the FileSystemStorage class to write the content of
your DB file into the path. Notice that it must be a valid SQLite
file!
Important: getDatabasePath() is not supported in the Javascript port. It
will always return null.
This is very useful for applications that need to synchronize with a
central server or applications that ship with a large database as part
of their core product.
You are relying on paths that make sense in the simulator, in the device you need to copy a resource into location. Check out the SQL demo where this is implemented: https://www.codenameone.com/blog/sql-demo-revisited.html
Use following code to copy database to Codenameone storage after put the database file in src folder, the database will copy to directory ".cn1/database" after running
String DB_NAME = "DBNAME.db";
Database temp_db = Database.openOrCreate(DB_NAME); //To create an empty file before copy, otherwise Android will throw file not found exception
temp_db.close();
String p = Database.getDatabasePath(DB_NAME);
OutputStream o = FileSystemStorage.getInstance().openOutputStream(p);
InputStream i = Display.getInstance().getResourceAsStream(getClass(), "/" + DB_NAME);
Util.copy(i, o);
This doesn't seem like a complete answer.
I'm going through the demo, and something seems to be missing:
Shouldn't a build task copy this db resource to the correct target folder? Otherwise how can a deployment ever work? (if the db doesnt get packaged up then how will it get deployed?) without it the app cant run
Related
What is the difference between those two queries:
SELECT my_fun(col_name) FROM my_table;
and
CREATE TABLE new_table AS SELECT my_fun(col_name) FROM my_table;
Where my_fun is a java UDF.
I'm asking, because when I create new table (second query) I receive a java error.
Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
...
Caused by: org.apache.hadoop.hive.ql.exec.UDFArgumentException: Unable to instantiate UDF implementation class com.company_name.examples.ExampleUDF: java.lang.NullPointerException
I found that the source of error is line in my java file:
encoded = Files.readAllBytes(Paths.get(configPath));
But the question is why it works when table is not created and fails if table is created?
The problem might be with the way you read the file. Try to pass the file path as the second argument in the UDF, then read as follows
private BufferedReader getReaderFor(String filePath) throws HiveException {
try {
Path fullFilePath = FileSystems.getDefault().getPath(filePath);
Path fileName = fullFilePath.getFileName();
if (Files.exists(fileName)) {
return Files.newBufferedReader(fileName, Charset.defaultCharset());
}
else
if (Files.exists(fullFilePath)) {
return Files.newBufferedReader(fullFilePath, Charset.defaultCharset());
}
else {
throw new HiveException("Could not find \"" + fileName + "\" or \"" + fullFilePath + "\" in inersect_file() UDF.");
}
}
catch(IOException exception) {
throw new HiveException(exception);
}
}
private void loadFromFile(String filePath) throws HiveException {
set = new HashSet<String>();
try (BufferedReader reader = getReaderFor(filePath)) {
String line;
while((line = reader.readLine()) != null) {
set.add(line);
}
} catch (IOException e) {
throw new HiveException(e);
}
}
The full code for different generic UDF that utilizes file reader can be found here
I think there are several points unclear, so this answer is based on assumptions.
First of all, it is important to understand that hive currently optimize several simple queries and depending on the size of your data, the query that is working for you SELECT my_fun(col_name) FROM my_table; is most likely running locally from the client where you are executing the job, that is why you UDF can access your config file locally available, this "execution mode" is because the size of your data. CTAS trigger a job independent on the input data, this job runs distributed in the cluster where each worker fail accessing your config file.
It looks like you are trying to read your configuration file from the local file system, not from the HDSFS Files.readAllBytes(Paths.get(configPath)), this means that your configuration has to either be replicated in all the worker nodes or be added previously to the distributed cache (you can use add file from this, doc here. You can find another questions here about accessing files from the distributed cache from UDFs.
One additional problem is that you are passing the location of your config file through an environment variable which is not propagated to worker nodes as part of your hive job. You should pass this configuration as a hive config, there is an answer for accessing Hive Config from UDF here assuming that you are extending GenericUDF.
For bilingual support in an application I am working on, we are using Spring messaging which uses two files, ApplicationResources.properties and ApplicationResources_fr.properties. This works well.
Now I am trying to expand on this by making it a little more dynamic. The application will read key value pairs from the database and insert them, which gives me the following error:
java.io.FileNotFoundException: \ApplicationResources.properties (Access is denied)
I am able to check on the key value pairs so I know the path I am using is correct. I have also checked the files in Eclipse properties by right clicking, and by visiting the actual file on my system, and they are not read-only. I do not believe they are encrypted because I am able to open and view with notepad++.
Here is my testing code which shows I can view them
Properties test_prop = null;
InputStream is = null;
try {
test_prop = new Properties();
is = this.getClass().getResourceAsStream(en_path);
test_prop.load(is);
Set<Object> keys = test_prop.keySet();
boolean key_found = false;
for(Object k:keys) {
String key = (String)k;
if(key.equals("f12345"))
{
key_found=true;
break;
}
}
System.out.println("Language Properties Test in DAO:" + (key_found? "Key Found" : "Key not found"));
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (NullPointerException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
Here is where I try to write to the file, and get the error:
ResultSet rs = null;
try (
Connection connection = jdbcTemplate.getDataSource().getConnection();
CallableStatement callableStatement = connection.prepareCall(test_prod_cur);
)
{
callableStatement.registerOutParameter(1, OracleTypes.CURSOR);
callableStatement.executeUpdate();
rs = (ResultSet) callableStatement.getObject(1);
while (rs.next())
{
String thead = rs.getString(1);
//System.out.println(thead + " " + rs.getString(2) + " " + rs.getString(3));
en_prop.setProperty(keyheader+thead, rs.getString(2));
fr_prop.setProperty(keyheader+thead, rs.getString(3));
}
}
catch (SQLException e)
{
System.out.println("SQLException - bilingual values - CLUDAOImpl");
System.out.println(e.getMessage());
}
//add to properties files
//*
try (OutputStream en_os = new FileOutputStream(en_path);)
{
en_prop.store(en_os, null);
} catch (IOException e) {
e.printStackTrace();
}
try(OutputStream fr_os = new FileOutputStream(en_path);)
{
fr_prop.store(fr_os, null);
} catch (IOException e) {
e.printStackTrace();
}
So the database query is successful, that was tested with the commented out system.out.println. It is the following lines that end up throwing the error:
en_prop.store(en_os, null);
fr_prop.store(fr_os, null);
Update: I did a search on the java.util.Properties which lead me to the javadocs on it and wow does that simplify many things. I can now grab a property value or check if the key exists in 6 lines of code (not counting try catch).
Properties prop = null;
InputStream is = null;
this.prop = new Properties();
is = this.getClass().getResourceAsStream(path);
prop.load(is);
this.prop.getProperty("key name"); //returns value of key, or null
this.prop.containsKey("key name"); //returns true if key exists
Update2: There is an issue using java.util.Properties and that is you lose all formatting of the original file, so white-space, comments, and ordering are all lost. In another answer someone suggested using Apache's Commons Configuration API. I plan on trying it out.
So I ended up creating a class to handle interactions with the ApplicationResources(_fr).properties files instead of doing it in the DAO. This was because I plan on using it in more places. I also started using methods from the java.util.Properties Javadocs which proved very helpful and simplified many areas.
Below is my new file write/properties store code.
try (
OutputStream en_os = new FileOutputStream(getClass().getResource(en_path).getFile(),false);
OutputStream fr_os = new FileOutputStream(getClass().getResource(fr_path).getFile(), false);
)
{
en_prop.store(en_os, null);
fr_prop.store(fr_os, null);
} catch (IOException e) {
e.printStackTrace();
}
Lets compare the new and original OutputStreams:
OutputStream en_os = new FileOutputStream(getClass().getResource(en_path).getFile(),false); //new
OutputStream en_os = new FileOutputStream(en_path); //original, Access is Denied
This answer is incomplete for the following reasons.
I am unable to explain why the original method failed and resulted in a "Access is denied error".
More concerning reason to me, this doesnt actually alter the file I am expecting or wanting. I expected to alter the file that appears in my project navigator, but when viewed changes are not observed. If I use an absolute path (C:\...) and overwrite the file then I can alter it as expected, but this path would have to be changed as servers are changed and its bad programming and dangerous. This working method is altering some kind of temp or running file (as confirmed via the path as the file that shows the new values is in the tmp0 folder). After some testing, this temporary file is overwritten on startup only when the original file has been changed, otherwise the new values persist across application starting.
I am also unsure as to the scope of this file. I am unable to tell if all users interacting with the website would cause changes to the same file. If all users are interacting with the file, then potential leakage across sessions could occur. It is also possible that each session has isolated values and could lead to missing information. I suspect that all users are interacting with the same resource but have not performed the testing required to be absolutely positive about this. UPDATE: I have confirmed that all users interact with the same temporary file.
I want to fetch files from SFTP which are created after a given timestamp(time of last pull) in java. I am using j2ssh as of now. Please let me know if some other API supports such a feature.
Jsch supports the ls command which will bring you back all the attributes of the remote file. You can write a little code to eliminate the files you want to retrieve from there.
Java Doc: http://epaul.github.io/jsch-documentation/javadoc/
This example compares the remote file timestamps to find the oldest file, it wouldn't be much of a stretch to modify it to compare your last run date against the remote file date, then do the download as part of the loop.
Code from Finding file size and last modified of SFTP file using Java
try {
list = Main.chanSftp.ls("*.xml");
if (list.isEmpty()) {
fileFound = false;
}
else {
lsEntry = (ChannelSftp.LsEntry) list.firstElement();
oldestFile = lsEntry.getFilename();
attrs = lsEntry.getAttrs();
currentOldestTime = attrs.getMTime();
for (Object sftpFile : list) {
lsEntry = (ChannelSftp.LsEntry) sftpFile;
nextName = lsEntry.getFilename();
attrs = lsEntry.getAttrs();
nextTime = attrs.getMTime();
if (nextTime < currentOldestTime) {
oldestFile = nextName;
currentOldestTime = nextTime;
}
}
I want to create something like "PDF Viewer app". Application will search for all *.pdf files in location chosen by user. User can choose this folder by this function:
Intent intent = new Intent(Intent.ACTION_OPEN_DOCUMENT_TREE);
startActivityForResult(intent, REQUEST_CODE);
Then I get DocumentFile (folder):
#Override
public void onActivityResult(int requestCode, int resultCode, Intent data) {
if (resultCode == getActivity().RESULT_OK && requestCode == REQUEST_CODE) {
Uri uriTree = data.getData();
DocumentFile documentFile = DocumentFile.fromTreeUri(getActivity(), uriTree);
//rest of code here
}
}
Why I chose this method of selecting folder? Because I want to make possible to choose Secondary Storage (you know, in Android >= 5.0, you can't access Secondary Storage with Java.io.file).
Ok, so I get folder with all *.pdf as DocumentFile. Then I call:
for(DocumentFile file: documentFile.listFiles()){
String fileNameToDisplay = file.getName();
}
And this is VERY SLOW. It takes almost 30 seconds when there are ~600 files in chosen folder. To prove it, I chose directory from External Storage (not secondary storage), and then I tried two solutions: DocumentFile and File.
File version looks like it:
File f = new File(Environment.getExternalStorageDirectory()+"/pdffiles");
for(File file: f.listFiles()){
String fileNameToDisplay = file.getName();
}
}
Second version works about 500x faster. There is almost no time in displaying all files on List View.
Why is DocumentFile so slow?
If you read the source code of TreeDocumentFile, you will find that each call to listFiles() and getName() invokes ContentResolver#query() under the hood. Like CommonsWare said, this would perform hundreds of queries, which is very inefficient.
Here is the source code of listFiles():
#Override
public DocumentFile[] listFiles() {
final ContentResolver resolver = mContext.getContentResolver();
final Uri childrenUri = DocumentsContract.buildChildDocumentsUriUsingTree(mUri,
DocumentsContract.getDocumentId(mUri));
final ArrayList<Uri> results = new ArrayList<>();
Cursor c = null;
try {
c = resolver.query(childrenUri, new String[] {
DocumentsContract.Document.COLUMN_DOCUMENT_ID }, null, null, null);
while (c.moveToNext()) {
final String documentId = c.getString(0);
final Uri documentUri = DocumentsContract.buildDocumentUriUsingTree(mUri,
documentId);
results.add(documentUri);
}
} catch (Exception e) {
Log.w(TAG, "Failed query: " + e);
} finally {
closeQuietly(c);
}
final Uri[] result = results.toArray(new Uri[results.size()]);
final DocumentFile[] resultFiles = new DocumentFile[result.length];
for (int i = 0; i < result.length; i++) {
resultFiles[i] = new TreeDocumentFile(this, mContext, result[i]);
}
return resultFiles;
}
In this function call, listFiles() made a query that only selects the document ID column. However, in your case you also want the file name for each file. Therefore, you can add the column COLUMN_DISPLAY_NAME to the query. This would retrieve the filename and document ID (which later you will convert it into Uri) in a single query and is much more efficient. There are also many other columns available such as file type, file size, and last modified time, which you may want to retrieve them as well.
c = resolver.query(mUri, new String[] {
DocumentsContract.Document.COLUMN_DOCUMENT_ID,
DocumentsContract.Document.COLUMN_DISPLAY_NAME
}, null, null, null);
Within the while loop, retrieve the filename by
final String filename = c.getString(1);
The above modified code is able to instantly retrieve the Uri and filename of a directory with 1000+ files.
In summary, my recommendation is to avoid using DocumentFile if you are working with more than just a few files. Instead use ContentResolver#query() to retrieve the Uri and other information by selecting multiple columns in the query. For file operations, use the static methods in the DocumentsContract class by passing the appropriate Uri's.
By the way, it seems that the sortOrder parameter of ContentResolver#query() gets completely ignored in the above code snippet when tested on Android 11 and Android 9. I would manually sort the results instead of relying on the query order.
Why is DocumentFile so slow?
For ~600 files you are performing ~600 requests of a ContentProvider to get the display name, which means ~600 IPC transactions.
Instead, use MediaStore to query for all indexed media with the application/pdf MIME type.
In case someone comes up here still looking for a solution,
I built a wrapper over this with some pretty good performance.
You can check the wrapper & performance info. here:
https://github.com/ItzNotABug/DocumentFileCompat
To use DocumentsContract to obtain children documents, see https://developer.android.com/reference/android/provider/DocumentsContract.html#buildChildDocumentsUriUsingTree(android.net.Uri, java.lang.String).
The Uri returned from ACTION_OPEN_DOCUMENT_TREE is a tree document URI. Use the above method to build the Uri to query all children documents.
The root document ID can be obtained using https://developer.android.com/reference/android/provider/DocumentsContract.html#getTreeDocumentId(android.net.Uri) with the Uri returned from ACTION_OPEN_TREE_DOCUMENT.
To obtain a speed little less than the one from File use DocumentsContract instead of DocumentFile to list the content of trees obtained with Intent.ACTION_OPEN_DOCUMENT_TREE.
I am in the middle of creating an app that allows users to apply for job positions and upload their CVs. I`m currently stuck on trying to make a search box for the admin to be able to search for Keywords. The app will than look through all the CVs and if it finds such keywords it will show up a list of Cvs that contain the keyword. I am fairly new to Gui design and app creation so not sure how to go about doing it. I wish to have it done via java and am using the Eclipse Window builder to help me design it. Any help will be greatly appreciated, hints, advice anything. Thank You.
Well, this not right design approach as real time search of words in all files of given folder will be slow and not sustainable in long run. Ideally you should have indexed all CV's for keywords. The search should run on index and then get the associated CV for that index ( think of indexes similar to tags). There are many options for indexing - simples DB indexing or using Apache Lucene or follow these steps to create a index using Maps and refer this index for search.
Create a map Map<String, List<File>> for keeping the association of
keywords to files
iterate through all files, and for each word in
each file, add that file to the list corresponding to that word in
your index map
here is the java code which will work for you but I would still suggest to change your design approach and use indexes.
File dir = new File("Folder for CV's");
if(dir.exists())
{
Pattern p = Pattern.compile("Java");
ArrayList<String> list = new ArrayList<String>(); // list of CV's
for(File f : dir.listFiles())
{
if(!f.isFile()) continue;
try
{
FileInputStream fis = new FileInputStream(f);
byte[] data = new byte[fis.available()];
fis.read(data);
String text = new String(data);
Matcher m = p.matcher(text);
if(m.find())
{
list.add(f.getName()); // add file to found-keyword list.
}
fis.close();
}
catch(Exception e)
{
System.out.print("\n\t Error processing file : "+f.getName());
}
}
System.out.print("\n\t List : "+list); // list of files containing keyword.
} // IF directory exists then only process.
else
{
System.out.print("\n Directory doesn't exist.");
}
Here you get the files list to show now for "Java". As I said use indexes :)
Thanks for taking your time to look into my problem.
I have actually come up with a solution of my own. It is probably very amateur like but it works for me.
JButton btnSearch = new JButton("Search");
btnSearch.addActionListener(new ActionListener()
{
public void actionPerformed(ActionEvent arg0)
{
list.clear();
String s = SearchBox.getText();
int i = 0,present = 0;
int id;
try
{
Class.forName(driver).newInstance();
Connection conn = DriverManager.getConnection(url+dbName,userName,password);
Statement st = conn.createStatement();
ResultSet res = st.executeQuery("SELECT * FROM javaapp.test");
while(res.next())
{
i = 0;
present = 0;
while(i < 9)
{
String out = res.getString(search[i]);
if(out.toLowerCase().contains(s.toLowerCase()))
{
present = 1;
break;
}
i++;
}
if(tglbtnNormalshortlist.isSelected())
{
if(present == 1 && res.getInt("Shortlist") == 1)
{
id = res.getInt("Candidate");
String print = res.getString("Name");
list.addElement(print+" "+id);
}
}
else
{
if(present == 1 && res.getInt("Shortlist") == 0)
{
id = res.getInt("Candidate");
String print = res.getString("Name");
list.addElement(print+" "+id);
}
}
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
});