I'm using an IntentService to append single NMEA strings to a log file stored in the external public storage on an Android device, but I'm getting inconsistent behavior.
First off the log file does not appear when the device is connected for USB debugging. I read that many Android devices cannot write to external storage when connected via USB, but even when it is run disconnected from USB, it may take several times of turning it on and off and rerunning the applications before the log file appears in the file system. If I do not clear the log, I still have to restart the phone before it will begin appending to the file again.
How can I get the file to appear consistently everytime?
import android.app.IntentService;
import android.content.Intent;
public class LogFileService extends IntentService {
DebugLog debug = new DebugLog("LogFileService");
public static final String GPS_STR = "GPS_STR";
public static final String FILE_PATH = "FILE_PATH";
public static final String FILE_NAME = "gps_data.txt";
public LogFileService() {
super("LogFileService");
}
#Override
protected void onHandleIntent(Intent intent) {
debug.log("Handling intent");
String data = intent.getStringExtra(GPS_STR);
debug.log("Writing " + data);
GpsLogFile logFile = new GpsLogFile(FILE_NAME);
logFile.open();
logFile.write(data);
logFile.close();
}
#Override
public void onCreate() {
super.onCreate();
debug.log("Created");
}
}
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintWriter;
import android.content.Context;
import android.os.Environment;
public class GpsLogFile {
DebugLog debug = new DebugLog("LogFile");
private File filePath = Environment.getExternalStoragePublicDirectory(
Environment.DIRECTORY_DOWNLOADS
);
private File logFile = null;
private String fileName = null;
private PrintWriter fileOut = null;
public GpsLogFile(String name) {
fileName = name;
}
public void open() {
try {
filePath.mkdirs();
if (!isExternalMediaAvailable()) {
throw new IOException("External media not available.");
}
logFile = new File(filePath, fileName);
//logFile.mkdirs();
if (!logFile.exists()) {
logFile.createNewFile();
}
fileOut = new PrintWriter(
new FileOutputStream(logFile.getAbsolutePath(), true)
);
} catch (IOException e) {
debug.log("Unable to open file.");
debug.log(e.getMessage());
}
}
public void write(String data) {
debug.log("Writing to " + logFile.getAbsolutePath() + " " + data);
try {
fileOut.write(data);
//fileOut.newLine();
checkPrintWriterError(fileOut);
fileOut.flush();
} catch (IOException e) {
debug.log("Unable to write");
debug.log(e.getMessage());
}
}
public void close() {
if (null != fileOut) {
try {
fileOut.close();
checkPrintWriterError(fileOut);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
private void checkPrintWriterError(PrintWriter writer) throws IOException {
if (true == writer.checkError()) {
throw new IOException("Print writer error.");
}
}
}
Related
I am trying to make a base file plugin which other threads will inherit. But I am stuck at a point where the file exists and can be read from a normal thread but when I try to read that file from an abstract Base file, it says File not found. Here's my base class :-
package com.evol.fp;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
public abstract class BaseFilePlugin extends Thread implements BaseFileReader{
String filename = "";
File file = null;
boolean fileStarted = false;
boolean fileEnded = false;
public BaseFilePlugin() {
file = new File(filename);
}
public void readFile() {
BufferedReader br = null;
System.out.println("Base call: " + filename);
try {
System.out.println("inbside ");
br = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
if(br.readLine().trim().isEmpty()) {
endFile();
return;
} else {
startFile(filename);
String record;
while((record = br.readLine().trim()) != null) {
parseRecord(record);
}
endFile();
}
} catch(Exception ioe) {
ioe.printStackTrace();
} finally {
try {
br.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public abstract void parseRecord(String record);
public void startFile(String filename) {
this.fileStarted = true;
this.fileEnded = false;
}
public void endFile() {
file.delete();
this.fileEnded = true;
this.fileStarted = false;
}
public void run() {
while(true) {
System.out.println("Inside run, fileName: " + filename);
System.out.println("Filestarted: " + fileStarted + ", file exists: " + file.exists());
if(!fileStarted) {
readFile();
}
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
/**
* #return the filename
*/
public String getFilename() {
return filename;
}
/**
* #param filename the filename to set
*/
public void setFilename(String filename) {
this.filename = filename;
}
}
I am aware of multithreading but never implemented with base class to parse records from a file, if someone tells me what's the problem that will be great. I know that the file exists for sure. Here's my child class: -
package com.evol.fp;
public class FileReaderThread extends BaseFilePlugin {
public FileReaderThread() {
super.setFilename("E:\\soham\\soham.txt");
}
#Override
public void parseRecord(String record) {
System.out.println(record);
}
}
But its not calling the child's parseRecord method, using a simple main method:-
package com.evol.fp;
public class StartProcess {
public static void main(String[] args) {
FileReaderThread thd = new FileReaderThread();
thd.start();
}
}
I think it's because the parent constructor (BaseFilePlugin.class) is called first before you set your filename in super.setFile("E:\\soham\\soham.txt");
If you can remove the parent constructor instead and replace your setFileName into setFile where file is iniatilize .e.g
// public BaseFilePlugin() {
// file = new File(filename);
// }
....
....
/**
* #return the file
*/
public String getFile() {
return file
}
/**
* #param file the file to set
*/
public void setFile(String file) {
file = new File(file);
}
and in your subclass
public FileReaderThread() {
super.setFile("E:\\soham\\soham.txt");
}
BaseFilePlugin's constructor creates its file with an empty string since initially String filename = "";.
The client calls setFilename(...) which updates filename. However, file is still the same instance when the object was first created (which is using an empty string as the file name).
I would suggest to pass the file name as part of the constructor so file is properly initialized:
public BaseFilePlugin(String filename) {
this.filename = filename;
file = new File(filename);
}
Optionally, if it makes sense that an instance can read only 1 file, then make those class attributes final, and remove the setFilename() method.
I'm working on a project that copies files with four threads.
I create List and store in there name of files to copy.
I want to use this 4 thread to work together, but I don't really get it how make it happend.
public class CopyingFiles implements Runnable
{
static File source = new File("C:\\test\\1\\");
static File dest = new File("C:\\test\\2\\");
#Override
public void run()
{
try
{
CopyingFromList(source, dest);
}
catch (IOException e)
{
e.printStackTrace();
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
public static void CopyFile(File sourceFile, File destination) throws IOException
{
InputStream inputStream = null;
OutputStream outputStream = null;
try {
inputStream = new FileInputStream(sourceFile);
outputStream = new FileOutputStream(destination);
byte[] buffer = new byte[1024];
int length;
while ((length = inputStream.read(buffer)) > 0)
{
outputStream.write(buffer, 0 ,length);
}
} finally {
if(inputStream != null)
{
inputStream.close();
}
if(inputStream != null)
{
outputStream.close();
}
}
}
public static void CopyingFromList(File source, File dest) throws IOException, InterruptedException
{
List<String> fileList = FilesList.CreateFilesList(source);
for(String file : fileList)
{
System.out.println(Thread.currentThread().getName() + " > " + FilesList.DestinationOfFile(source) + file + " > " + FilesList.DestinationOfFile(dest) + file );
CopyFile(new File(FilesList.DestinationOfFile(source) + file), new File(FilesList.DestinationOfFile(dest) + file));
}
}
}
AND SECOND CLASS
public class FilesList
{
static File source = new File("C:\\test\\1\\");
static File source1 = new File("C:\\test\\3\\");
static File dest = new File("C:\\test\\2\\");
static File dest1 = new File("C:\\test\\4\\");
public static List<String> CreateFilesList(File source) throws InterruptedException, IOException
{
List<String> fileList = new ArrayList<>(Arrays.asList(source.list()));
return fileList;
}
public static String DestinationOfFile(File source)
{
return new String(source + "\\");
}
public static void PrintWholeList(File source) throws IOException, InterruptedException
{
List<String> fileList = CreateFilesList(source);
for(String file : fileList)
{
System.out.println(DestinationOfFile(source) + file);
}
}
public static void main(String []args) throws IOException, InterruptedException
{ /*
//CopyingFiles.CopyFile(new File(source+"\\file1.txt"), new File(dest+"\\file1.txt"));
//CopyingFiles.CopyingFromList(source,dest);
CopyingFiles t1 = new CopyingFiles();
CopyingFiles t2 = new CopyingFiles();
CopyingFiles t3 = new CopyingFiles();
CopyingFiles t4 = new CopyingFiles();
t1.start();
t2.start();
t3.start();
t4.start();
*/
ExecutorService executorService = Executors.newFixedThreadPool(5);
System.out.println(Thread.activeCount());
executorService.submit(() -> {
try
{
CopyingFiles.CopyingFromList(source,dest);
}
catch (IOException e)
{
e.printStackTrace();
}
catch (InterruptedException e)
{
e.printStackTrace();
}
finally
{
executorService.shutdown();
}
});
}
}
Could anyone help me,or show some other way to solve my problem.
I don't actually get what the problem is. This one works for me (and it is almost your program, just cleaned up a bit):
package multicp;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import static java.nio.file.StandardCopyOption.COPY_ATTRIBUTES;
/**
* Copy files using threads.
*/
public class Multicp {
public static void main(String[] args) {
// List of source/dest pairs ("tasks")
List<CopierCallable<Void>> opsList = new ArrayList<>();
opsList.add(new CopierCallable<>(Paths.get("f1src.dat"), Paths.get("f1dest.dat")));
opsList.add(new CopierCallable<>(Paths.get("f2src.dat"), Paths.get("f2dest.dat")));
opsList.add(new CopierCallable<>(Paths.get("f3src.dat"), Paths.get("f3dest.dat")));
opsList.add(new CopierCallable<>(Paths.get("f4src.dat"), Paths.get("f4dest.dat")));
ExecutorService execSvc = Executors.newFixedThreadPool(2); // 4 in your case. 2 is just for testing
try {
execSvc.invokeAll(opsList);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
execSvc.shutdown();
}
}
}
/**
* Performs actual copying from one source to one destination.
*/
class CopierCallable<Void> implements Callable<Void> {
private Path pathFrom;
private Path pathTo;
public CopierCallable(Path pathFrom, Path pathTo) {
this.pathFrom = pathFrom;
this.pathTo = pathTo;
}
#Override
public Void call() {
try {
// REPLACE_EXISTING is destructive, uncomment at your own risk
Files.copy(pathFrom, pathTo, COPY_ATTRIBUTES /*, REPLACE_EXISTING*/);
System.out.println(pathFrom + " copied");
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
}
I can see files being copied simultaneously in groups (by 2 for testing; replace with 4, but it would make initialization code larger for no profit).
I am working on a Java application in which I am trying to create a Multipart file out of downloaded InputStream. Unfortunately, it is not working and the Multipart file is empty. I checked the size of savedFile on disk before copying it to Multipart, and it has correct size, attributes, content.
What am I doing wrong in the conversion, there is no stacktrace, as I am catching it.
Code :
// InputStream contains file data.
byte[] bytes = IOUtils.toByteArray(inputStream);
File file = new File(msg + "temp");
if (file.exists() && file.isDirectory()) {
OutputStream outputStream = new FileOutputStream(new File(msg + "temp" + "/" +
groupAttachments.getFileName()));
outputStream.write(bytes);
outputStream.close();
}
java.io.File savedFile = new java.io.File(msg + "temp" + "/" +
groupAttachments.getFileName());
DiskFileItem fileItem = new DiskFileItem("file", "text/plain", false,
savedFile.getName(), (int) savedFile.length(), savedFile.getParentFile());
fileItem.getOutputStream();
MultipartFile multipartFile = new CommonsMultipartFile(fileItem);
System.out.println("Saved file size is "+savedFile.length());
if (multipartFile.isEmpty()) {
System.out.println("Dropbox uploaded multipart file is empty");
} else {
System.out.println("Multipart file is not empty.");
}
this.dropboxTask.insertFile(multipartFile, "",
savedPersonalNoteObject.getNoteid(), (long) 0, true);
Path path = Paths.get(msg + "temp" + "/" + groupAttachments.getFileName());
Console output :
Multipart file is not empty
Bytes are not null
File path is /My Group
Input stream is not null
Saved file size is 4765
Dropbox uploaded multipart file is empty
Multipart file is empty
Bytes are not null
What am I doing wrong in the conversion? Any help would be nice. Thanks a lot.
The DiskFileItem uses a DeferredFileOutputStream which uses an in-memory byte-array that is only filled when bytes are actually transferred.
Since files are used directly and no bytes are actually copied,
the byte-array is never filled. See for yourself in the source code:
Source code CommonsMultipartFile
Source code DiskFileItem
Source code DeferredFileOutputStream
So, instead of just calling fileItem.getOutputStream();, transfer the bytes to fill the in-memory byte-array:
try (OutputStream out = fileItem.getOutputStream();
InputStream in = Files.newInputStream(file.toPath())) {
IOUtils.copy(in, dfos);
}
and then the tranferTo call will work.
This appears to be a bit cumbersome for just moving a file: CommonsMultipartFile only calls fileItem.write((File)dest) in the transferTo method.
Below are two test cases, one using the DiskFileItem and one using the LocalFileItem. The code for LocalFileItem is shown further below.
I used dependencies org.springframework:spring-web:4.2.2.RELEASE, commons-fileupload:commons-fileupload:1.3.1 and junit:junit:4.12
Test class CommonMp:
import static org.junit.Assert.*;
import java.io.*;
import java.nio.charset.*;
import java.nio.file.*;
import org.apache.commons.fileupload.disk.DiskFileItem;
import org.apache.commons.io.IOUtils;
import org.junit.Test;
import org.springframework.web.multipart.commons.CommonsMultipartFile;
public class CommonMp {
private final Charset CS = StandardCharsets.UTF_8;
#Test
public void testLocalMp() {
Path testInputFile = null, testOutputFile = null;
try {
testInputFile = prepareInputFile();
LocalFileItem lfi = new LocalFileItem(testInputFile);
CommonsMultipartFile cmf = new CommonsMultipartFile(lfi);
System.out.println("Empty: " + cmf.isEmpty());
testOutputFile = testInputFile.getParent().resolve("testMpOutput.txt");
cmf.transferTo(testOutputFile.toFile());
System.out.println("Size: " + cmf.getSize());
printOutput(testOutputFile);
} catch (Exception e) {
e.printStackTrace();
fail();
} finally {
deleteSilent(testInputFile, testOutputFile);
}
}
#Test
public void testMp() {
Path testInputFile = null, testOutputFile = null;
try {
testInputFile = prepareInputFile();
DiskFileItem di = new DiskFileItem("file", "text/plain", false, testInputFile.getFileName().toString(),
(int) Files.size(testInputFile), testInputFile.getParent().toFile());
try (OutputStream out = di.getOutputStream();
InputStream in = Files.newInputStream(testInputFile)) {
IOUtils.copy(in, out);
}
CommonsMultipartFile cmf = new CommonsMultipartFile(di);
System.out.println("Size: " + cmf.getSize());
testOutputFile = testInputFile.getParent().resolve("testMpOutput.txt");
cmf.transferTo(testOutputFile.toFile());
printOutput(testOutputFile);
} catch (Exception e) {
e.printStackTrace();
fail();
} finally {
deleteSilent(testInputFile, testOutputFile);
}
}
private Path prepareInputFile() throws IOException {
Path tmpDir = Paths.get(System.getProperty("java.io.tmpdir"));
Path testInputFile = tmpDir.resolve("testMpinput.txt");
try (OutputStream out = Files.newOutputStream(testInputFile)){
out.write("Just a test.".getBytes(CS));
}
return testInputFile;
}
private void printOutput(Path p) throws IOException {
byte[] outBytes = Files.readAllBytes(p);
System.out.println("Output: " + new String(outBytes, CS));
}
private void deleteSilent(Path... paths) {
for (Path p : paths) {
try { if (p != null) p.toFile().delete(); } catch (Exception ignored) {}
}
}
}
The custom LocalFileItem class, YMMV!
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.UnsupportedEncodingException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileItemHeaders;
public class LocalFileItem implements FileItem {
private static final long serialVersionUID = 2467880290855097332L;
private final Path localFile;
public LocalFileItem(Path localFile) {
this.localFile = localFile;
}
#Override
public void write(File file) throws Exception {
Files.move(localFile, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
}
#Override
public long getSize() {
// Spring's CommonsMultipartFile caches the file size and uses it to determine availability.
long size = -1L;
try {
size = Files.size(localFile);
} catch (IOException ignored) {}
return size;
}
#Override
public void delete() {
localFile.toFile().delete();
}
/* *** properties and unsupported methods *** */
private FileItemHeaders headers;
private String contentType;
private String fieldName;
private boolean formField;
#Override
public FileItemHeaders getHeaders() {
return headers;
}
#Override
public void setHeaders(FileItemHeaders headers) {
this.headers = headers;
}
#Override
public InputStream getInputStream() throws IOException {
throw new IOException("Only method write(File) is supported.");
}
public void setContentType(String contentType) {
this.contentType = contentType;
}
#Override
public String getContentType() {
return contentType;
}
#Override
public String getName() {
return localFile.getFileName().toString();
}
#Override
public boolean isInMemory() {
return false;
}
#Override
public byte[] get() {
throw new RuntimeException("Only method write(File) is supported.");
}
#Override
public String getString(String encoding)
throws UnsupportedEncodingException {
throw new RuntimeException("Only method write(File) is supported.");
}
#Override
public String getString() {
throw new RuntimeException("Only method write(File) is supported.");
}
#Override
public String getFieldName() {
return fieldName;
}
#Override
public void setFieldName(String name) {
this.fieldName = name;
}
#Override
public boolean isFormField() {
return formField;
}
#Override
public void setFormField(boolean state) {
this.formField = state;
}
#Override
public OutputStream getOutputStream() throws IOException {
throw new IOException("Only method write(File) is supported.");
}
}
I am trying to append text to a text file on the Google Drive. But when I write, it whole file is overwritten. Why can't I just add the text in the end of the file?
DriveFile file = Drive.DriveApi.getFile(mGoogleApiClient, id);
file.open(mGoogleApiClient, DriveFile.MODE_WRITE_ONLY, null).setResultCallback(new ResultCallback<DriveApi.DriveContentsResult>() {
#Override
public void onResult(DriveApi.DriveContentsResult driveContentsResult) {
msg.Log("ContentsOpenedCallBack");
if (!driveContentsResult.getStatus().isSuccess()) {
Log.i("Tag", "On Connected Error");
return;
}
final DriveContents driveContents = driveContentsResult.getDriveContents();
try {
msg.Log("onWrite");
OutputStream outputStream = driveContents.getOutputStream();
Writer writer = new OutputStreamWriter(outputStream);
writer.append(et.getText().toString());
writer.close();
driveContents.commit(mGoogleApiClient, null);
} catch (IOException e) {
e.printStackTrace();
}
}
});
Finally I've found the answer to append the text on the drive document.
DriveContents contents = driveContentsResult.getDriveContents();
try {
String input = et.getText().toString();
ParcelFileDescriptor parcelFileDescriptor = contents.getParcelFileDescriptor();
FileInputStream fileInputStream = new FileInputStream(parcelFileDescriptor
.getFileDescriptor());
// Read to the end of the file.
fileInputStream.read(new byte[fileInputStream.available()]);
// Append to the file.
FileOutputStream fileOutputStream = new FileOutputStream(parcelFileDescriptor
.getFileDescriptor());
Writer writer = new OutputStreamWriter(fileOutputStream);
writer.write("\n"+input);
writer.close();
driveContentsResult.getDriveContents().commit(mGoogleApiClient, null);
} catch (IOException e) {
e.printStackTrace();
}
SO
The reason is that commit's default resolution strategy is to overwrite existing files. Check the API docs and see if there is a way to append changes.
For anyone facing this problem in 2017 :
Google has some methods to append data Here's a link!
Though copying the method from google didn't worked entirely for me , so here is the class which would append data : ( Please note this is a modified version of this code link )
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.Writer;
import android.content.Context;
import android.content.SharedPreferences;
import android.os.Bundle;
import android.os.ParcelFileDescriptor;
import android.preference.PreferenceManager;
import android.util.Log;
import com.google.android.gms.common.api.Result;
import com.google.android.gms.common.api.ResultCallback;
import com.google.android.gms.drive.Drive;
import com.google.android.gms.drive.DriveApi.DriveContentsResult;
import com.google.android.gms.drive.DriveApi.DriveIdResult;
import com.google.android.gms.drive.DriveContents;
import com.google.android.gms.drive.DriveFile;
import com.google.android.gms.drive.DriveId;
/**
* An activity to illustrate how to edit contents of a Drive file.
*/
public class EditContentsActivity extends BaseDemoActivity {
private static final String TAG = "EditContentsActivity";
#Override
public void onConnected(Bundle connectionHint) {
super.onConnected(connectionHint);
final ResultCallback<DriveIdResult> idCallback = new ResultCallback<DriveIdResult>() {
#Override
public void onResult(DriveIdResult result) {
if (!result.getStatus().isSuccess()) {
showMessage("Cannot find DriveId. Are you authorized to view this file?");
return;
}
DriveId driveId = result.getDriveId();
DriveFile file = driveId.asDriveFile();
new EditContentsAsyncTask(EditContentsActivity.this).execute(file);
}
};
SharedPreferences sp= PreferenceManager.getDefaultSharedPreferences(EditContentsActivity.this);
Drive.DriveApi.fetchDriveId(getGoogleApiClient(), EXISTING_FILE_ID)
.setResultCallback(idCallback);
}
public class EditContentsAsyncTask extends ApiClientAsyncTask<DriveFile, Void, Boolean> {
public EditContentsAsyncTask(Context context) {
super(context);
}
#Override
protected Boolean doInBackgroundConnected(DriveFile... args) {
DriveFile file = args[0];
SharedPreferences sp=PreferenceManager.getDefaultSharedPreferences(EditContentsActivity.this);
System.out.println("0"+sp.getString("drive_id","1"));
DriveContentsResult driveContentsResult=file.open(getGoogleApiClient(), DriveFile.MODE_READ_WRITE, null).await();
System.out.println("1");
if (!driveContentsResult.getStatus().isSuccess()) {
return false;
}
DriveContents driveContents = driveContentsResult.getDriveContents();
try {
System.out.println("2");
ParcelFileDescriptor parcelFileDescriptor = driveContents.getParcelFileDescriptor();
FileInputStream fileInputStream = new FileInputStream(parcelFileDescriptor
.getFileDescriptor());
// Read to the end of the file.
fileInputStream.read(new byte[fileInputStream.available()]);
System.out.println("3");
// Append to the file.
FileOutputStream fileOutputStream = new FileOutputStream(parcelFileDescriptor
.getFileDescriptor());
Writer writer = new OutputStreamWriter(fileOutputStream);
writer.write("hello world");
writer.close();
System.out.println("4");
driveContents.commit(getGoogleApiClient(), null).await();
return true;
} catch (IOException e) {
e.printStackTrace();
}
return false;
};
#Override
protected void onPostExecute(Boolean result) {
if (!result) {
showMessage("Error while editing contents");
return;
}
showMessage("Successfully edited contents");
}
}
}
Existing_File_id is the resource id. Here is one link if you need resource id a link
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class Test6 implements Runnable {
private File file;
private int totalNumberOfFiles = 0;
private static int nextFile = -1;
private static ArrayList<String> allFilesArrayList = new ArrayList<String>();
private static ExecutorService executorService = null;
public Test6(File file) {
this.file = file;
}
private String readFileToString(String fileAddress) {
FileInputStream stream = null;
MappedByteBuffer bb = null;
String stringFromFile = "";
try {
stream = new FileInputStream(new File(fileAddress));
FileChannel fc = stream.getChannel();
bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
/* Instead of using default, pass in a decoder. */
stringFromFile = Charset.defaultCharset().decode(bb).toString();
} catch (IOException e) {
System.out.println("readFileToString IOException");
e.printStackTrace();
} finally {
try {
stream.close();
} catch (IOException e) {
System.out.println("readFileToString IOException");
e.printStackTrace();
}
}
return stringFromFile;
}
private void toFile(String message, String fileName) {
try {
FileWriter fstream = new FileWriter("C:/Users/Nomi/Desktop/Workspace2/Test6/TestWritten/" + fileName);
System.out.println("printing to file: ".concat(fileName));
BufferedWriter out = new BufferedWriter(fstream);
out.write(message);
out.close();
} catch (Exception e) {
System.out.println("toFile() Exception");
System.err.println("Error: " + e.getMessage());
}
}
// private void listFilesForFolder(final File fileOrFolder) {
// String temp = "";
// if (fileOrFolder.isDirectory()) {
// for (final File fileEntry : fileOrFolder.listFiles()) {
// if (fileEntry.isFile()) {
// temp = fileEntry.getName();
// toFile(readFileToString(temp), "Copy".concat(temp));
// }
// }
// }
// if (fileOrFolder.isFile()) {
// temp = fileOrFolder.getName();
// toFile(readFileToString(temp), "Copy".concat(temp));
// }
// }
public void getAllFilesInArrayList(final File fileOrFolder) {
String temp = "";
System.out.println("getAllFilesInArrayList fileOrFolder.getAbsolutePath()" + fileOrFolder.getAbsolutePath());
if (fileOrFolder.isDirectory()) {
for (final File fileEntry : fileOrFolder.listFiles()) {
if (fileEntry.isFile()) {
temp = fileEntry.getAbsolutePath();
allFilesArrayList.add(temp);
}
}
}
if (fileOrFolder.isFile()) {
temp = fileOrFolder.getAbsolutePath();
allFilesArrayList.add(temp);
}
totalNumberOfFiles = allFilesArrayList.size();
for (int i = 0; i < allFilesArrayList.size(); i++) {
System.out.println("getAllFilesInArrayList path: " + allFilesArrayList.get(i));
}
}
public synchronized String getNextFile() {
nextFile++;
if (nextFile < allFilesArrayList.size()) {
// File tempFile = new File(allFilesArrayList.get(nextFile));
return allFilesArrayList.get(nextFile);
} else {
return null;
}
}
#Override
public void run() {
getAllFilesInArrayList(file);
executorService = Executors.newFixedThreadPool(allFilesArrayList.size());
while(nextFile < totalNumberOfFiles)
{
String tempGetFile = getNextFile();
File tempFile = new File(allFilesArrayList.get(nextFile));
toFile(readFileToString(tempFile.getAbsolutePath()), "Copy".concat(tempFile.getName()));
}
}
public static void main(String[] args) {
Test6 test6 = new Test6(new File("C:/Users/Nomi/Desktop/Workspace2/Test6/Test Files/"));
Thread thread = new Thread(test6);
thread.start();
// executorService.execute(test6);
// test6.listFilesForFolder(new File("C:/Users/Nomi/Desktop/Workspace2/Test6/"));
}
}
The programs' doing what's expected. It goes into the folder, grabs a file, reads it into a string and then writes the contents to a new file.
I would like to do this multi threaded. If the folder has N number of files, I need N number of threads. Also I would like to use executor framework if possible. I'm thinking that there can be a method along this line:
public synchronized void getAllFilesInArrayList() {
return nextFile;
}
So each new thread could pick the next file.
Thank you for your help.
Error:
Exception in thread "Thread-0" java.lang.IllegalArgumentException
at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:589)
at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:480)
at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:59)
at Test6.run(Test6.java:112)
at java.lang.Thread.run(Thread.java:662)
Firstly, your approach to the problem will result in more synchronization and race condition worries than seems necessary. A simple strategy to keep your threads from racing would be this:
1) Have a dispatcher thread read all the file names in your directory.
2) For each file, have the dispatcher thread spawn a worker thread and hand off the file reference
3) Have the worker thread process the file
4) Make sure you have some sane naming convention for your output file names so that you don't get threads overwriting each other.
As for using an executor, a ThreadPoolExecutor would probably work well. Go take a look at the javadoc: http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html