Develop Reference

Unable to save file to disk "No space left on device"

I've a java program that writes files to a directory on Linux VM. After writing 4.9 million files, it is failing with the following error:
java.io.FileNotFoundException: /home/user/test/6BA30639CA0A2772AA0217312B3E847E2399E9A25F50F9960D6A670F4F2533EF.blob.lock (No space left on device)
at java.io.RandomAccessFile.open0(Native Method)
at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:124)
at com.xyz.azure.AsyncADLSHashFileUploader.SaveToLocalStore(AsyncADLSHashFileUploader.java:231)
My logic is:
public String SaveToLocalStore(byte[] bytes, String subfolder) throws Exception {
MessageDigest md = MessageDigest.getInstance("SHA-256");
md.update(bytes);
byte[] sha256 = md.digest();
String sha256str = bytesToHex(sha256);
Path tmpDir = Paths.get(LocalRootDirectory.toString(), subfolder);
Path tmpFile = Paths.get(tmpDir.toString(), sha256str + ".tmp");
Path localFile = Paths.get(LocalRootDirectory.toString(), subfolder, sha256str + ".blob");
String dstFile = getDestFileFromSrcFile(localFile.toString());
//noinspection ResultOfMethodCallIgnored
tmpDir.toFile().mkdirs();
//We can make a safe assumption that if the .blob file is present, that means that it has been fully written to
if (!Files.exists(localFile)) {
try (
RandomAccessFile randomAccessFile = new RandomAccessFile(localFile + ".lock", "rw");
FileChannel fc = randomAccessFile.getChannel();
FileLock fileLock = fc.tryLock()) {
//if other thread/process is already handling this file... no point of us doing anything
if (fileLock != null) {
//local file is already there, no need to write it again
try (FileOutputStream fos = new FileOutputStream(tmpFile.toString())) {
fos.write(bytes);
fos.flush();
fos.close();
Files.move(tmpFile, localFile, REPLACE_EXISTING);
if(statisticsEnabled) {
synchronized (StatsLock) {
StatsSavedHashes.add(localFile.toString());
}
}
} catch (Exception e) {
LOGGER.error("Failed to create local temp file: " + tmpFile + ": " + e, e);
//cleanup temp if it exists
Files.deleteIfExists(tmpFile);
throw e;
}
}
} catch (OverlappingFileLockException e) {
//other thread is handling it already, so just carry one as if(fileLock == null) was at play
return dstFile;
}
catch (Exception e) {
LOGGER.error("Error while saving local file: " + localFile + ": " + e, e);
throw e;
}
}
return dstFile;
}
I can see that there is more than 80% disk space available.
I've also checked for available inodes on the file system.
I don't know where might be the problem
Can someone please help me with this regard?

Writing to an External File on Android --- File Doesn't Register, But Java Can Read

I'm trying to write to an external txt (or csv) file for Android. I can run an app, close it, and run it again, and readData() will read back to my log what I've stored. However, the dirFile (file directory) appears nowhere within my Android files (even if I connect it to a computer and search).
Something interesting, though: if I clear my log (similar to a list of print statements shown within Eclipse) and disconnect my phone from my computer, then reconnect it, the log reappears with everything I've ever written to my file (even if I later overwrote it)...yet the app isn't even running!
Here is my code. Please help me understand why I cannot find my file!
(Note: I've tried appending a "myFile.txt" extension to the directory, but it just causes an EISDIR exception.)
public void writeData(String dirName){
try
{
File root = new File(getExternalFilesDir(null), dirName);
// Writes to file
//
// The "true" argument allows the file to be appended. Without this argument (just root),
// the file will be overwritten (even though we later call append) rather than appended to.
FileWriter writer = new FileWriter(root, true);
writer.append("Append This Text\n");
writer.flush();
writer.close();
// Checks if we actually wrote to file by reading it back in (appears in Log)
//readData(dirName);
}
catch(Exception e)
{
Log.v("2222", "2222 ERROR: " + e.getMessage());
}
}
If you're interested, here's the function I wrote to read in the data:
public void readData(String dirName){
try
{
File root = new File(getExternalFilesDir(null), dirName);
// Checks to see if we are actually writing to file by reading in the file
BufferedReader reader = new BufferedReader(new FileReader(root));
try {
String s = reader.readLine();
while (s != null) {
Log.v("2222", "2222 READ: " + s);
s = reader.readLine();
}
}
catch(Exception e) {
Log.v("2222", "2222 ERROR: " + e.getMessage());
}
finally {
reader.close();
}
}
catch(Exception e) {
Log.v("2222", "2222 ERROR: " + e.getMessage());
}
}
Thanks!

even if I connect it to a computer and search
if I clear my log (similar to a list of print statements shown within Eclipse) and disconnect my phone from my computer, then reconnect it, the log reappears with everything I've ever written to my file (even if I later overwrote it).
What you are seeing on your computer is what is indexed by MediaStore, and possibly a subset of those, depending upon whether your computer caches information it gets from the device in terms of "directory" contents.
To help ensure that MediaStore indexes your file promptly:
Use a FileOutputStream (optionally wrapped in an OutputStreamWriter), not a FileWriter
Call flush(), getFD().sync(), and close() on the FileOutputStream, instead of calling flush() and close() on the FileWriter (sync() will ensure the bytes are written to disk before continuing)
Use MediaScannerConnection and scanFile() to tell MediaStore to index your file
You can then use whatever sort of "reload" or "refresh" or whatever option is in your desktop OS's file manager, and your file should show up.
This blog post has more on all of this.

public void create(){
folder = new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_MOVIES),"video");
boolean success = true;
if (!folder.exists()) {
success=folder.mkdirs();
}
if (success) {
readfile();
} else {
System.out.println("failed");
}
}
The above code will be used to crete the directory in th emobile at desired path
private void readfile() {
// TODO Auto-generated method stub
AssetManager assetManager = getResources().getAssets();
String[] files = null;
try {
files = assetManager.list("clipart");
} catch (Exception e) {
Log.e("read clipart ERROR", e.toString());
e.printStackTrace();
}
for(String filename : files) {
System.out.println("File name => "+filename);
InputStream in = null;
OutputStream out = null;
try {
in = assetManager.open("clipart/" + filename);
out = new FileOutputStream(folder + "/" + filename);
copyFile(in, out);
in.close();
in = null;
out.flush();
out.close();
out = null;
} catch(Exception e) {
Log.e("copy clipart ERROR", e.toString());
e.printStackTrace();
}
}}private void copyFile(InputStream in, OutputStream out) throws IOException {
byte[] buffer = new byte[1024];
int read;
while((read = in.read(buffer)) != -1){
out.write(buffer, 0, read);
}}
this is my code used to write file in internal memory from the assets folder in project. This code can read all type(extension) of file from asset folder to mobile.
Don't forget to add permission in manifest file
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
and call the above function by
readfile();//this call the function to read and write the file
I hope this may help you.
Thank you.

Issue creating a file in android

I am trying to create a file inside a directory using the following code:
ContextWrapper cw = new ContextWrapper(getApplicationContext());
File directory = cw.getDir("themes", Context.MODE_WORLD_WRITEABLE);
Log.d("Create File", "Directory path"+directory.getAbsolutePath());
File new_file =new File(directory.getAbsolutePath() + File.separator + "new_file.png");
Log.d("Create File", "File exists?"+new_file.exists());
When I check the file system of emulator from eclipse DDMS, I can see a directory "app_themes" created. But inside that I cannot see the "new_file.png" . Log says that new_file does not exist. Can someone please let me know what the issue is?
Regards,
Anees

Try this,
File new_file =new File(directory.getAbsolutePath() + File.separator + "new_file.png");
try
{
new_file.createNewFile();
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
Log.d("Create File", "File exists?"+new_file.exists());
But be sure,
public boolean createNewFile ()
Creates a new, empty file on the file system according to the path information stored in this file. This method returns true if it creates a file, false if the file already existed. Note that it returns false even if the file is not a file (because it's a directory, say).

Creating a File instance doesn't necessarily mean that file exists. You have to write something into the file to create it physically.
File directory = ...
File file = new File(directory, "new_file.png");
Log.d("Create File", "File exists? " + file.exists()); // false
byte[] content = ...
FileOutputStream out = null;
try {
out = new FileOutputStream(file);
out.write(content);
out.flush(); // will create the file physically.
} catch (IOException e) {
Log.w("Create File", "Failed to write into " + file.getName());
} finally {
if (out != null) {
try {
out.close();
} catch (IOException e) {
}
}
}
Or, if you want to create an empty file, you could just call
file.createNewFile();

Creating a File object doesn't mean the file will be created. You could call new_file.createNewFile() if you wanted to create an empty file. Or you could write something to it.

Why AccessDeniedException is raising when using just created folder?

I'm creating simple object serialization, and creation of BufferedOutputStream is raising an exception AccessDeniedException. Here is the code:
Path filePath = Paths.get("c:\\temp\\");
File xmlFile = new File("c:\\temp\\");
boolean success = xmlFile.mkdirs();
if (!success && ! xmlFile.exists() ) {
// Directory creation failed
System.out.println("Failed to create a file: " + filePath);
}
try (
ObjectOutputStream objectOut = new ObjectOutputStream(
new BufferedOutputStream(Files.newOutputStream(filePath, StandardOpenOption.WRITE)))){
// Write three objects to the fi le
objectOut.writeObject(solarSystem); // Write object
System.out.println("Serialized: " + solarSystem);
} catch(IOException e) {
e.printStackTrace();
}
But directory is empty and if it doesn't not exist, it's created...

I'll repeat my comment here: you seem to try to write to a directory not to a file. Try changing filePath to a file instead.

How to deal with corrupted files that were created but IOException occured?

Could you please suggest how to deal with these situations ? I understand that in the second example, it is very rare that it would happen on unix, is it ? If access rights are alright. Also the file wouldn't be even created. I don't understand why the IOException is there, either it is created or not, why do we have to bother with IOException ?
But in the first example, there will be a corrupted zombie file. Now if you tell the user to upload it again, the same thing may happen. If you can't do that, and the inputstream has no marker. You loose your data ? I really don't like how this is done in Java, I hope the new IO in Java 7 is better
Is it usual to delete it
public void inputStreamToFile(InputStream in, File file) throws SystemException {
OutputStream out;
try {
out = new FileOutputStream(file);
} catch (FileNotFoundException e) {
throw new SystemException("Temporary file created : " + file.getAbsolutePath() + " but not found to be populated", e);
}
boolean fileCorrupted = false;
int read = 0;
byte[] bytes = new byte[1024];
try {
while ((read = in.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
} catch (IOException e) {
fileCorrupted = true;
logger.fatal("IO went wrong for file : " + file.getAbsolutePath(), e);
} finally {
IOUtils.closeQuietly(in);
IOUtils.closeQuietly(out);
if(fileCorrupted) {
???
}
}
}
public File createTempFile(String fileId, String ext, String root) throws SystemException {
String fileName = fileId + "." + ext;
File dir = new File(root);
if (!dir.exists()) {
if (!dir.mkdirs())
throw new SystemException("Directory " + dir.getAbsolutePath() + " already exists most probably");
}
File file = new File(dir, fileName);
boolean fileCreated = false;
boolean fileCorrupted = false;
try {
fileCreated = file.createNewFile();
} catch (IOException e) {
fileCorrupted = true;
logger.error("Temp file " + file.getAbsolutePath() + " creation fail", e);
} finally {
if (fileCreated)
return file;
else if (!fileCreated && !fileCorrupted)
throw new SystemException("File " + file.getAbsolutePath() + " already exists most probably");
else if (!fileCreated && fileCorrupted) {
}
}
}

I really don't like how this is done in Java, I hope the new IO in Java 7 is better
I'm not sure how Java is different than any other programming language/environment in the way you are using it:
a client sends some data to your over the wire
as you read it, you write it to a local file
Regardless of the language/tools/environment, it's possible for the connection to be interrupted or lost, for the client to go away, for the disk to die, or for any other error to occur. I/O errors can occur in any and all environments.
What you can do in this situation is highly dependent on the situation and the error that occured. For example, is the data structured in some way where you could ask the user to resume uploading from record 1000, for example? However, there is no single solution that fits all here.