Develop Reference

Vaadin EasyUpload add-on sometimes could not open or find file specified

We are currently using the EasyUpload add-on, and we have specified the criteria for this component:
a) only CSV files are allowed, with a cap size of 1MB per file.
b) only one file can be submitted at a time.
We just did an uploading test on small-sized CSV files that are below 100Kb. Usually, the upload process completes successfully. Occasionally, the error of "Could not open file, The system cannot find the file specified" is displayed although the file is already inside the temp folder, and we found that this happens either when:
a) If the same file is uploaded again after making a small change and within a few seconds after the file has been uploaded successfully.
b) If there are two tabs of the web app, logged under different users were uploading their respective csv files and they also do the same thing of changing values in the csv before uploading them again.
We tried forcing the file upload through (as another testing method) and noticed after a while that the files sometimes get stuck in the queue although we have imposed a one file at a submission time rule. It was displayed in a message "There are too many files over the count limit". We also considered of putting a sleep / wait command of 3-5 seconds after the file submission.
MultiFileUpload multiFileUpload = new MultiFileUpload() {
#Override
protected void handleFile(File tmpFile, String fileName, String mimeType, long length) {
String[] header = {"EOD_NUM","OUTLET_NAME","POSM_NAME","EOD_DATE","TOTAL_SALES","GROSS_SALES",
"TRAN_COUNT","VOID_COUNT","SERVICE_CHARGE","DISCOUNT_AMT","VAT_TAX_AMT","SVC_TAX_AMT","ROUNDING_ADJ"};
uploadLogger.debug("File: " + tmpFile.getAbsolutePath());
uploadLogger.debug("FileName: " + fileName);
uploadLogger.debug("MimeType: " + mimeType);
uploadLogger.debug("File Length: " + length);
DateTimeFormatter dtf = DateTimeFormatter.ofPattern("ddMMyyyyHHmmss");
LocalDateTime now = LocalDateTime.now();
File f2 = null;
f2 = new File(busId+"_"+dtf.format(now)+".csv");
tmpFile.renameTo(f2);
try {
///var/lib/tomcat8/ in linux
///D:\\home\\site\\wwwroot\\ in Windows
uploadLogger.debug("f2 absolutepath: " + f2.getAbsolutePath());
uploadLogger.debug("f2 canonical path: " + f2.getCanonicalPath());
CloudBlockBlob blob = container.getBlockBlobReference(f2.getName());
if(f2.length() > 0){
blob.uploadFromFile(f2.getAbsolutePath());
Notification.show("File upload completed.",Notification.Type.TRAY_NOTIFICATION);
}
CSVReader reader = new CSVReader(new FileReader(f2.getAbsolutePath()), ',' , '"' , 0);
//read header name
//String[] myheader = reader.readNext();
//NOTE :: Store all row and column from csv info List of String Array
myEntries = reader.readAll();
if (myEntries != null && !myEntries.isEmpty()) {
boolean success = uploadDAO.insertUploaderEntry(myEntries,busId, userId,"");
uploadLogger.debug("SUCCESSS??? " + success);
if(success){
Notification successNotify = new Notification("Record has been created successfully.","Upload Successful!");
successNotify.setDelayMsec(3000);
successNotify.setStyleName(ValoTheme.NOTIFICATION_SUCCESS);
successNotify.setPosition(Position.MIDDLE_CENTER);
successNotify.show(Page.getCurrent());
}else {
Notification.show("Error in submitting uploaded record.","Upload failed!"
, Notification.Type.ERROR_MESSAGE).setDelayMsec(3000);
}
Thread.sleep(3000); //added to see if the delay solves the problem or not.
}
} catch (URISyntaxException | StorageException | IOException ex) {
new Notification("Could not open file",ex.getMessage(),Notification.Type.ERROR_MESSAGE).show(Page.getCurrent());
uploadLogger.debug(ex);
} catch (InterruptedException ix) {
uploadLogger.debug("Interrupted Exception found: " + ix.getMessage());
}
}
#Override
protected boolean supportsFileDrops() {
return false;
}
};
multiFileUpload.setMaxFileCount(1);
multiFileUpload.setUploadButtonCaption("Upload CSV file here");
multiFileUpload.setMaxFileSize(fileSizeLimit); // 2MB
multiFileUpload.setAcceptFilter(".csv");
We are unsure whether if this problem is a known limitation of the component or not.
Some of the questions we have discovered along the way are:
a) Is there a better way or to control on the file uploading and to avoid the open file / file not found error?
b) Are the values in the setAcceptedFilter method the mime/type values or something else. We noticed for images, it's "images/*" but for csv, we had to put in as ".csv"

Answering to your second question. The acceptFilter is directly passed to upload inputs "accept" attribute, so both .csv and text/csv should do fine. See https://www.w3schools.com/tags/att_input_accept.asp for more instructions.

java open file xlsx

how do I open the excel file located inside my project?
I would like to press a jbutton and open file1

What would you like to do with it? If you just want to handle the data of the Excel-file, I would export my Excel-file to a csv-file (in Excel 2016: File > Export > Change File Type > CSV (Comma delimited)).
The delimiter used for separating data depends on your system settings (mine is set on semicolon to eliminate annoying situations with commas in the cells).
The advantage of a CSV-file is that you can handle the file as any other text file.
Inside the actionPerformed-method of your JButton, you can open the file using:
try (Scanner sc = new Scanner(new File("my_csv_file.csv"))) {
// do anything you want with the file using the scanner object
// for example, print all data to the screen:
// make sure you import java.util.Scanner, java.io.File and java.io.FileNotFoundException
// and catch a FileNotFoundException or throw it to be handled anywhere else
while (sc.hasNextLine()) {
String data[] = sc.nextLine().split(";");
for (String s : data) {
System.out.print(s + "\t");
}
System.out.println();
}
}

How to load multiple csv into multiple tables using sql loader

I use SQL loader to load multiple CSV files into multiple tables.
Example: I have the a.csv, b.csv, c.csv files loaded in the Employee table, and the d.csv, e.csv, and f.csv files load into the Student table.
I have written a Java script to implement SQL loader
#RestController
public class FileUploadController {
// Using ServletContext
#Autowired
ServletContext context;
String folderUpload = context.getRealPath("/WEB-INF/uploaded");
Process p = Runtime.getRuntime().exec(new String[] { "cmd", "/C", "all.bat" }, null,
new File(folderUpload));
String sqlldrCmd = "Sqlldr baotrung/baotrung1192 control = " + folderUpload + "/full.ctl"
+ "log=d:/bt.log skip=1";
System.out.println(sqlldrCmd.replace("\\", "/"));
System.out.println("SQLLDR Started ....... ");
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(sqlldrCmd.replace("\\", "/"));
System.out.println(proc.waitFor());
System.out.println("SQLLDR Ended ........ ");
I know in SQL loadder there is an option to load:
load data
INFILE 'loader2.csv'
INTO TABLE articles_formatted
APPEND
FIELDS TERMINATED BY ','
(article_id SEQUENCE (MAX, 1),
author CHAR (30)
format,
pub_date SYSDATE,
title,
ext_fname FILLER CHAR (80)
text LOBFILE (ext_fname) TERMINATED BY EOF)
However, if I use the load data command
INFILE only loads a file into a table. If I want to continue loading I have to continue writing that command. This is impossible to do because my system has too many CSV files. I have an idea of ​​joining the CSV file into a single CSV file but my CSV files are for multiple tables rather than one table so the work This connection is not implemented. I have two questions:
How to determine which CSV file is loaded into the table when the
number of CSV is so large. I made the CSV name match the table name
but it did not fit my system.
How to load multiple CSV into multiple tables. Is there a way to
insert all the files in my directory into the LOAD DATA INFILE
statement and load it one by one?
I have written a Java to do this problem but it can not be read sequentially
package com.baotrung;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public class ListFile {
public static void main(String[] args) {
File dir = new File("E:\\xls");
File[] files = dir.listFiles((d, name) -> name.endsWith(".csv"));
for (File file : files) {
file.getName();
}
Path content = Paths.get("E://xls//final.ctl");
Stream<String> stream;
try {
stream = Files.lines(content);
String line1 = Files.readAllLines(Paths.get("E://xls//final.ctl")).get(1);
System.out.println(line1);
List<String> replaced = stream.map(line -> line.replace(line1, files[0].getName().toString()))
.collect(Collectors.toList());
Files.write(content, replaced);
stream.close();
System.out.println("Find and Replace done!!!");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The code executes the list of CSV files in the directory, then opens the final.ctl file and executes the contents of the CSV file. But I can not duplicate it to make the week of the files in the folder.
How can I read the files at [0] files in turn.
Please help.

If you merge all your CSV files, I'd suggest you to set - in its first column (or an identifier) a value that distinguishes records. For example:
first;1;little;20;111 --> should go to table called "first"
first;2;foot;30;111
second;100;donald;50;43 --> should go to table called "second"
Then you'd utilize control file's WHEN clause (search for "Loading records based on condition", here) and specify which identifier's records go to which table. The following example is adjusted from Case Study 5: Loading data into multiple tables:
LOAD DATA
INFILE ...
INTO TABLE first
WHEN identifier= 'first'
(empno POSITION(1:4) INTEGER EXTERNAL,
ename POSITION(6:15) CHAR,
deptno POSITION(17:18) CHAR,
mgr POSITION(20:23) INTEGER EXTERNAL)
INTO TABLE second
WHEN identifier= 'second'
(empno POSITION(1:4) INTEGER EXTERNAL,
projno POSITION(25:27) INTEGER EXTERNAL)
INTO TABLE third
WHEN identifier= 'third'
(empno POSITION(1:4) INTEGER EXTERNAL,
projno POSITION(29:31 INTEGER EXTERNAL)
I guess that it should work just fine. Moreover, now you know the keywords to search for (WHEN, MULTIPLE TABLES) so that you could Google yourself for some more info. Good luck!

SQL do not accept AbsolutePath Java

I get a text file via:
JFileChooser dialog = new JFileChooser();
FileNameExtensionFilter filter = new FileNameExtensionFilter("Текстовый файл", "txt");
dialog.removeChoosableFileFilter(dialog.getFileFilter());
dialog.addChoosableFileFilter(filter);
dialog.setFileSelectionMode(JFileChooser.FILES_ONLY);
dialog.setDialogTitle("Выберите текстовый файл");
dialog.setDialogType(JFileChooser.OPEN_DIALOG);
dialog.setMultiSelectionEnabled(false);
int ret = dialog.showDialog(null, "Открыть");
if (ret == JFileChooser.APPROVE_OPTION) {
File file = dialog.getSelectedFile();
pach = file.getAbsolutePath();
} else return;
System.out.println(pach);
Last command shows:
D:\data\streets.txt
Now I make a request:
try {
querySQL = "LOAD DATA LOCAL INFILE '" + pach + "' INTO TABLE " + DB_NAME + "." + TABLE_NAME + ";";
stSQL.execute(querySQL);
} catch (SQLException e) {
ErrorMsg(e, querySQL);
isError = true;
break;
}
And I do my ErrorMsg issues:
Unable to process the query:
LOAD DATA LOCAL INFILE 'D:\data\streets.txt' INTO TABLE base2.streets;
java.sql.SQLException: Unable to open file 'D:datastreets.txt'for 'LOAD DATA LOCAL INFILE' command.Due to underlying IOException:
Where it removes the skew, and why all this is happening? In fact, if such a request is inserted into Workbench, the query is executed without error. Please tell me the solution of this problem, it is very necessary. Thank you in advance.
P.S. The text is translated into English by Google Translate

You may want to escape the backslashes contained in your path. For you database a backslash ( \ ) can change the value in a way that the string cannot be saved in the database. This can be a problem with other special characters like ' also. Concatenating a string containing ' to you query, adds an additional ' which will break the query.
You can use StringEscapeUtils for doing this. https://commons.apache.org/proper/commons-lang/javadocs/api-2.6/org/apache/commons/lang/StringEscapeUtils.html

Store and retrieve word documents with MySQL

I need to store and retrieve MS Word documents into MySQL 5.1 with Servlets. I've the code to upload a file, but I don't know can I feed into the table. I've used BLOB for the field I've to insert .doc files.
Here's my code snippet to upload files:
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.setContentType("text/html;charset=UTF-8");
PrintWriter out = response.getWriter();
try {
// get access to file that is uploaded from client
Part p1 = request.getPart("file");
String type=p1.getContentType();
String name=p1.getName();
long size = p1.getSize();
InputStream is = p1.getInputStream();
//FileInputStream fis = is.
// read filename which is sent as a part
Part p2 = request.getPart("name");
Scanner s = new Scanner(p2.getInputStream());
String filename = s.nextLine(); // read filename from stream
// get filename to use on the server
String outputfile = this.getServletContext().getRealPath(filename); // get path on the server
FileOutputStream os = new FileOutputStream (outputfile);
// write bytes taken from uploaded file to target file
int ch = is.read();
while (ch != -1) {
os.write(ch);
ch = is.read();
}
os.close();
out.println("<h3>File : '" + name + "' Type : '" + type + "' "
+ "of Size : " + ((double) size/1024) + "KB uploaded successfully!</h3>");
}
catch(Exception ex) {
out.println("Exception -->" + ex.getMessage());
}
finally {
out.close();
}
}
Here, I've used Servlets 3.0 feature for uploading a file...
My table schema :
resources
- UserID [varchar(15)]
- Document [mediumblob]
Could anyone help me how can I store the document into the table and though BLOB is a type representing binary data, how can I retrieve as a Word Document (*.doc)?

I agree with Archimedix... Instead of putting them into MySQL as BLOB, you can store the file on the disk and store its path in MYSQL as TEXT field. This way your retrieval time will be low. If you are space conscious then you can zip the doc and save it on the disk and on request uncompress and send it.
UPDATE
From your code it appears that you already have the handle of the file and you are able to save it on the server.
Now to save space you can zip it using default java zip utility.
You might face a problem when two people upload two different files with the same name. To avoid scenarios like this you can either rename your archived document with an uuid (use java 6 uuid class) or you can generate SHA1 for that file and use that for name.
Now you can use the absolute path of the archived (and renamed file) for storing in the MySQL.
Instead of table schema
resources
UserID [varchar(15)]
Document [mediumblob]
You can use this
resources
UserID [varchar(15)]
Document [varchar(512)]
So for a query like this:
Select Document from table Documents WHERE UserID = 'abcd';
you will now get an absolute path for the zipped file. Uncompress this file and send it.

A partial answer on storing the Word documents in files:
You don't need any additional column to save the file name as the document's record ID can serve as the file name.
When saving a new document, do in a database transaction so that you can undo the process when something goes wrong.
In pseudo code, this would look like this:
begin transaction;
try {
save new record for document;
save Word document in predefined directory, using record's ID as the filename;
} catch (Exception e) {
rollback transaction;
throw e; // Rethrow exception
}
commit transaction;
The code above assumes that an exception is thrown when an error occurs.