I am trying to use asynchronousFileChannel to write the date into a text file. I made 3 jar file of the program with the AsynchronousFileChannel and compiled all 3 jars simultaneously through command prompt to read 3 different text files and output to one common temporary file
I have 2000 records in my test files(3) to be read,but the output in the common temporary file is missing some of the records,the output should have 6000 records but it shows only 5366 or 5666 or sometimes less than that.
I am not able to figure out why some data is lost as it is the functionality of a asynchronousFileChannel.
Here is the code for the java program using asynchronousfilechannel.
class Writer(){
public void writeOut(ReadableData fileData)
throws InterruptedException {
Path file = null;
AsynchronousFileChannel asynchFileChannel = null;
String filePath = tempFileName;
try {
file = Paths.get(filePath);
asynchFileChannel = AsynchronousFileChannel.open(file,
StandardOpenOption.WRITE, StandardOpenOption.CREATE);
CompletionHandler<Integer, Object> handler = new CompletionHandler<Integer, Object>() {
#Override
public void completed(Integer result, Object attachment) {
if (result == Integer.MAX_VALUE) {
log.debug("Attachment: " + attachment + " " + result
+ " bytes written");
log.debug("CompletionHandler Thread ID: "
+ Thread.currentThread().getId());
}
result++;
}
#Override
public void failed(Throwable e, Object attachment) {
try {
throw e;
} catch (Throwable e1) {
e1.printStackTrace();
}
log.debug("File Write Failed Exception:");
e.printStackTrace();
}
};
String printData = fileData.getId() + "|"
+ fileData.getName() + "|" + fileData.getEmpId()
+ "|" + fileData.getServieId() + "|" + "\n";
asynchFileChannel.write(ByteBuffer.wrap(printData.getBytes()),
asynchFileChannel.size(), "file write", handler);
log.debug(printData);
}
}
catch (IOException e) {
e.printStackTrace();
log.error(e.getMessage());
} finally {
}
}
}
}
and this is my class to read data from 3 files:
public class FileReader1 {
static Logger log = Logger.getLogger(FileHandlerNorthBoundMain.class
.getName());
Writer wrO=new Writer();
public static void main(String[] args) throws IOException,
IllegalFileFormatException, InterruptedException {
String filePath = "C:\\Users\\Public\\testdata1.csv"; //"C:\\Users\\Public\\testdata2.csv"; "C:\\Users\\Public\\testdata3.csv";
File file = new File(filePath);
log.info("Fetching data.... from: " + filePath);
ArrayList<ReadableData> list = new ArrayList<ReadableData>();
FileInputStream fs = null;
BufferedReader reader = null;
String Name;
int Id, EmpId, ServiceId;
ReadableData readableData = null;
int count = 0;
fs = new FileInputStream(file);
reader = new BufferedReader(new InputStreamReader(fs));
String line = reader.readLine();
while (line != null) {
StringTokenizer st = new StringTokenizer(line, "\\|");
while (st.hasMoreTokens()) {
try {
Id = Integer.parseInt(st.nextToken());
Name = st.nextToken();
EmpId = Integer.parseInt(st.nextToken());
ServiceId = Integer.parseInt(st.nextToken());
readableData = new ReadableData(Id,
, Name, EmpId,ServiceId);
wrO.writeOut(readableData);
list.add(count, readableData);
count = count++;
} catch (Exception ex) {
log.error("Illegal File Format");
throw new IllegalFileFormatException("Illegal File Format");
}
}
line = reader.readLine();
}
reader.close();
}
Modify your Writer class with the following code part with asynchronousFileChannel lock()
byte[] test = printData.getBytes();
Future<FileLock> featureLock = asynchFileChannel.lock();
log.info("Waiting for the file to be locked ...");
FileLock lock = featureLock.get();
if (lock.isValid()) {
log.debug(printData);
Future<Integer> featureWrite = asynchFileChannel.write(
ByteBuffer.wrap(test), asynchFileChannel.size());
log.info("Waiting for the bytes to be written ...");
int written = featureWrite.get();
log.info("I’ve written " + written + " bytes into "
+ file.getFileName() + " locked file!");
lock.release();
}
This might be because asynchronousFileChannel is thread safe but Bytebuffer is not,care should be taken to ensure that the buffer is not accessed until after the operation has completed.
check the documentation http://openjdk.java.net/projects/nio/javadoc/java/nio/channels/AsynchronousFileChannel.html
Related
Faced such a problem. I wrote data to a file and on the first attempt everything is written well, but subsequent ones output Null. With what it can be connected ?
public static String ExportTofile (boolean archive) throws Exception {
if (!Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)) {
throw new Exception("Нет доступа к SDCard");}
File sdFile;
File sdPath = Environment.getExternalStorageDirectory();
sdPath = new File(sdPath.getAbsolutePath() + "/" + GlobalVars.DirectoryPath);
if (archive) {
sdFile = new File(sdPath +".txt");
} else
sdFile = new File(GlobalVars.AppPathData(), GlobalVars.DirectoryFileName);
Cursor cDeliveryTable;
try {
BufferedWriter br = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(sdFile),
"windows-1251"));
Log.d(GlobalVars.LOG_TAG, "ExpotToFile: " + sdFile.getAbsolutePath());
cDeliveryTable = GlobalVars.db.rawQuery("select * from CustDeliveryTable order by _id", null);
cDeliveryTable.moveToFirst();
while (!cDeliveryTable.isAfterLast()) {
CustDeliveryTable st = new CustDeliveryTable(cDeliveryTable.getLong(cDeliveryTable.getColumnIndex("_id")));
br.write(st.CargoStr() + "\r\n");
cDeliveryTable.moveToNext();}
br.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
throw e;
} catch (IOException e) {
e.printStackTrace();
throw e;}
return sdFile.getAbsolutePath();}
public String CargoStr() {
String sLine;
sLine = this.Name
+ ";"
+ this.Address
+ ";"
+ this.Number
+ ";";
return sLine.substring(0, sLine.length() - 1);}
First write to file
Subsequent entries
`public abstract class TV implements Logger {
protected int currentChannel;
protected int currentVolume;
protected String model;
public TV(String model)
{
this.model = model;
currentChannel = 2;
currentVolume = 10;
}
public void incChannel()
{
getcurrentChannel();
currentChannel = currentChannel + 1;
writeToLogFile("Increasing channel to " + currentChannel);
System.out.println("Increasing channel to " + currentChannel);
}
public void decChannel()
{
getcurrentChannel();
currentChannel = currentChannel - 1;
writeToLogFile("Decreasing channel to " + currentChannel);
System.out.println("Decreasing channel to " + currentChannel);
}
public void incVolume()
{
getcurrentVolume();
currentVolume = currentVolume + 1;
writeToLogFile("Increasing volume to " + currentVolume);
System.out.println("Increasing volume to " + currentVolume);
}
public void decVolume()
{
getcurrentVolume();
currentVolume = currentVolume - 1;
writeToLogFile("Decreasing volume to " + currentVolume);
System.out.println("Decreasing volume to " + currentVolume);
}
public void changeChannel(int currentChannel)
{
this.currentChannel = currentChannel;
writeToLogFile("Changing channel to " + currentChannel);
System.out.println("Changing channel to " + currentChannel);
}
public void writeToLogFile(String message)
{
model = getModel();
try {
FileReader fr = new FileReader(new File("./model.txt"));
BufferedReader br = new BufferedReader(fr);
FileWriter fw = new FileWriter(new File("./model.txt"), true);
BufferedWriter bw = new BufferedWriter(fw);
PrintWriter pw = new PrintWriter(bw);
String line = br.readLine();
while (line != null) {
Scanner scanLine = new Scanner(line);
String mess = scanLine.next();
pw.println(message);
pw.println(mess);
line = br.readLine();
}
br.close();
pw.flush();
pw.close();
}
catch (FileNotFoundException e)
{
System.out.println("File not found.");
}
catch (IOException e)
{
System.out.println("An IO error occurred.");
}
}`
When I call run my program my writeToLogFile method is not working correctly. When its called it is not making the text file as needed. I need the writeToLogFile method to create the file and then append it when its called in other methods.
Also, I have two models of TVs in this program. When I am using one of the TVs (Sony) I want it to write to its own log file named Sony.txt. I then have another TV (LG) that needs to have its own log file as well. Do I need to write two separate if statements to figure out the make of the tv then point it at its own log file?
The issue was with using the reader and writer on the same file at the same time. Once I got rid of the code where you open the file the method worked fine. I edited the code to reflect the proper changes.
I would guess there is a problem using a reader and a writer on the same file at the same time. I would recommend that you eliminate the reader and just open the file for append. Alternatively, you could open a different file for the writer, then once you're done writing to it, delete the original and rename your new file. – bcr666 6 mins ago
I'm uploading bulk files (say 50000) to Box through Box Java API's. To do that we call the uploadFile method 50000 times. The method also creates and close the file input stream.
Is there a way to keep the stream open until the bulk load is done? Even if I close the stream in finally block, it will close it every time I call the method.
private static String uploadFile(String pathFileName, BoxAPIConnection api, BoxFolder folder) {
boolean fileExists = false;
String fileId = null;
FileInputStream stream = null;
log.debug("Invoked uploadFileAsBoxAppUser-uploadFile :" + pathFileName + ":" + api + ":" + folder);
try {
String fileName = pathFileName.substring(pathFileName.lastIndexOf("\\") + 1, pathFileName.length());
log.debug("fileName :" + fileName);
for (BoxItem.Info itemInfo : folder) {
if (itemInfo instanceof BoxFile.Info) {
BoxFile.Info fileInfo = (BoxFile.Info) itemInfo;
if (fileName.equals(fileInfo.getName())) {
fileExists = true;
fileId = fileInfo.getID();
log.debug("fileExists in Destination box Folder fileID " + fileId);
}
}
}
if (!fileExists) {
log.debug("uploading new file: " + fileName);
stream = new FileInputStream(pathFileName);
BoxFile.Info boxInfo = folder.uploadFile(stream, pathFileName);
fileId = boxInfo.getID();
} else {
log.debug("uploading new version of file: " + fileName);
BoxFile file = new BoxFile(api, fileId);
stream = new FileInputStream(pathFileName);
file.uploadVersion(stream);
}
} catch (IOException e) {
log.debug("Exception in uploadFileAsBoxAppUser :" + e);
}
finally {
if (stream != null)
try {
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return fileId;
}
I'm retrieving files list from FTPClient using ftpClient.listFiles(); and it returns an array with correct file names and appropriate quantity.
I'm trying to read content from files, but only ftpClient.retrieveFileStream(fileName) is available for that purpose. And it somehow breaks after reading the first file and returns null.
Is there a way to convert FTPFile directly into String?
ftp.connect();
ftp.changeWorkingDirectoryToFrom();
List<String> idsList = new ArrayList<String>();
List<String> names = ftp.listFileNames();
for (String fileName : names) {
String content = fromFTPFileToString(fileName);
Matcher matcher = FILES_PATTERN.matcher(content);
String id = extractId(content);
if (matcher.find()) {
boolean duplicate = idsList.contains(id);
LOG.info("MATCHED: " + fileName);
if (!duplicate) {
ftp.moveFileFromTo(fileName);
idsList.add(id);
} else {
LOG.info("DUPLICATE: " + fileName);
duplicated++;
ftp.deleteFileOnFromFtp(fileName);
}
}
processed++;
}
ftp.disconnect();
private String fromFTPFileToString(String fileName) {
String content = "";
try {
InputStream is = ftp.readContentFromFTPFile(fileName);
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, ENCODING);
content = writer.toString();
IOUtils.closeQuietly(is);
IOUtils.closeQuietly(writer);
} catch (Exception ex) {
LOG.error(ex);
}
return content;
}
void deleteFileOnFromFtp(String fileName) {
changeWorkingDirectory(properties.getProperty(PropertiesType.FOLDER_FROM.toString()));
deleteFile(fileName);
}
InputStream readContentFromFTPFile(String fileName) {
changeWorkingDirectory(properties.getProperty(PropertiesType.FOLDER_FROM.toString()));
InputStream inputStream = null;
try {
inputStream = ftpClient.retrieveFileStream(fileName);
} catch (IOException ex) {
LOG.error("Unable to extract content from file:" + O_Q + fileName + C_Q);
}
return inputStream;
}
void moveFileFromTo(String fileName) {
String from = FORWARD_SLASH + properties.getProperty(PropertiesType.FOLDER_FROM.toString()) + FORWARD_SLASH + fileName;
String to = FORWARD_SLASH + properties.getProperty(PropertiesType.FOLDER_TO.toString()) + FORWARD_SLASH + fileName;
try {
ftpClient.rename(from, to);
} catch (IOException ex) {
LOG.error("Unable to move file file from:" + O_Q + from + C_Q + " to: " + O_Q + to + C_Q);
throw new RuntimeException(ex.getCause());
}
}
3 Hints
1) You can use retrieveFile
And use a ByteArrayOutputStream as the 2nd parameter.
To get a String from it simply "new String(baos.toByteArray(),[The Charset you used]);"
2) Check for ftpClient.completePendingCommand();
If some commands are still pending
3) I had once a similar issue setting ftp.enterLocalPassiveMode() helped
i have an application that want to write a file using fileoutputstream
here's the code, method patch
public static Response patch() {
try {
System.out.println("PATCH");
System.out.println(request.contentType);
String file = params.get("filename");
System.out.println("patch file: " + file);
Map<String, Header> MapOffset = request.headers;
for (Entry<String, Header> entry : MapOffset.entrySet()) {
System.out.println("Header['" + entry.getKey() + "]: "
+ entry.getValue().value());
}
Header offsetParam = MapOffset.get("offset");
Long offset = 0L;
if (offsetParam != null) {
offset = Long.parseLong(offsetParam.value());
}
InputStream input = request.body;
File f = new File(UPLOAD_DIR + System.getProperty("file.separator")
+ file);
System.out.println("address: " + f.getAbsolutePath());
System.out.println("offset: " + offset);
System.out.println("length: " + f.length());
fileBasicUpload(f, offset, input);
Response respon = new Response();
respon.status = OK;
return respon;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
and this is where i write a file
private static void fileBasicUpload(File f, Long offset, InputStream input)
throws IOException {
FileOutputStream output = null;
try {
int c = -1;
byte[] b = new byte[1024];
try {
output = new FileOutputStream(f, true);
while ((c = input.read(b)) != -1) {
output.write(b, 0, c);
}
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
} finally {
output.close();
}
}
but when my application called, then stream closed error is show up at while ((c = input.read(b)) != -1) that line.
i don't know how that error is called. sorry for my poor english and thanks
i found the answer. in my application i found like this
public static Response upload(File file){
System.out.println("Appliaction.upload");
response = ResumableUpload.post();
return response;
// render(response);
}
the parameter file, it must be delete, then it work!