I would like to write a file that comes from over a network, so I don't know the size of the file that's comming in. Sometimes the disk on the file server might get filled up and I would like return a message to my client notifying them of this error. I couldn't find any documentation on being able to catch this type of i/o error. FileChannel streams bytes from memory to disk, so it may not be trivial to detect this. Is the saving happening asynchronously? Is it possible to detect disk full?
// Create a new file to write to
RandomAccessFile mFile = new RandomAccessFile(this.mFilePath, "rw");
FileChannel mFileChannel = this.mFile.getChannel();
// wrappedBuffer has my file in it
ByteBuffer wrappedBuffer = ByteBuffer.wrap(fileBuffer);
while(wrappedBuffer.hasRemaining()) {
bytesWritten += this.mFileChannel.write(wrappedBuffer, this.mBytesProcessed);
}
I figured in the File class, we can do something like this:
// if there is less than 1 mb left on disk
new File(this.mFilePath, "r").getUsableSpace() < 1024;
But if there a way to throw an except if this.mFileChannel.write() fails because the disk is full?
Even if it's not recommended to parse the error message you could do something like this :
import java.io.IOException;
import java.nio.file.FileStore;
import java.nio.file.FileSystemException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.regex.Pattern;
public class SmallDisk {
final static String SMALL_DISK_PATH = "/Volumes/smallDisk";
final static Pattern NO_SPACE_LEFT = Pattern.compile(": No space left on device$");
public static void main(String[] args) throws NoSpaceException {
Path p = Paths.get(SMALL_DISK_PATH);
FileStore fs = null;
try {
fs = Files.getFileStore(p);
System.out.println(fs.getUsableSpace());
Path newFile = Paths.get(SMALL_DISK_PATH + "/newFile");
Files.createFile(newFile);
} catch (FileSystemException e) {
//We catch the "No space left on device" from the FileSystemException and propagate it
if(NO_SPACE_LEFT.matcher(e.getMessage()).find()){
throw new NoSpaceException("Not enough space");
}
//Propagate exception or deal with it here
} catch (IOException e) {
//Propagate exception or deal with it here
}
}
public static class NoSpaceException extends IOException{
public NoSpaceException(String message) {
super(message);
}
}
}
Another way but it doesn't guaranty that you won't have exception is to use the FileStore to check that you enough space before you write (not enough if you are working with a shared folder or multi threads software)
Related
I'm aware this question might be a duplicate in some sense but first hear me out.
I tried to create a code where i can create gitignore file with contents and for some reason i always end up having a file with txt extension and without name. Can someone explain this behavior and why?
Example Code:
System.out.println(fileDir+"\\"+".gitignore");
FileOutputStream outputStream = new FileOutputStream(fileDir+"\\"+".gitignore",false);
byte[] strToBytes = fileContent.getBytes();
outputStream.write(strToBytes);
outputStream.close();
You can use java.nio for it. See the following example:
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
public class StackoverflowMain {
public static void main(String[] args) {
// create the values for a folder and the file name as Strings
String folder = "Y:\\our\\destination\\folder"; // <-- CHANGE THIS ONE TO YOUR FOLDER
String gitignore = ".gitignore";
// create Paths from the Strings, the gitignorePath is the full path for the file
Path folderPath = Paths.get(folder);
Path gitignorPath = folderPath.resolve(gitignore);
// create some content to be written to .gitignore
List<String> lines = new ArrayList<>();
lines.add("# folders to be ignored");
lines.add("**/logs");
lines.add("**/classpath");
try {
// write the file along with its content
Files.write(gitignorPath, lines);
} catch (IOException e) {
e.printStackTrace();
}
}
}
It creates the file on my Windows 10 machine without any problems. You need Java 7 or higher for it.
So, I'm working on a program that allows you to import animations in the form of JSON files into Minecraft, and, when working on a completely different part of the program, my import code stopped working.
I'm using eclipse, and this is how my import code looks:
package com.github.sam54123.mc_animation.utils;
import java.io.InputStream;
public class FileHandle
{
public static InputStream inputStreamFromFile(String path)
{
try
{
InputStream inputStream = FileHandle.class.getResourceAsStream(path);
return inputStream;
}
catch(Exception e)
{
e.printStackTrace();
}
return null;
}
}
new file
package com.github.sam54123.mc_animation.utils;
import java.io.File;
import java.io.InputStream;
import java.util.Scanner;
import org.json.JSONObject;
public class JSONUtils
{
public static String getJSONStringFromFile(String path)
{
// Open file
Scanner scanner;
try
{
InputStream in = FileHandle.inputStreamFromFile(path);
scanner = new Scanner(in);
// Get JSON as string without spaces or newlines
String json = scanner.useDelimiter("\\Z").next();
// Close file
scanner.close();
return json;
}
catch(Exception e)
{
System.out.println(e.getStackTrace());
return null;
}
}
public static JSONObject getJSONObjectFromFile(String path)
{
File file = new File(path);
if (!file.exists())
{
System.out.println("Invalid Path");
return null;
}
String string = getJSONStringFromFile(path);
return new JSONObject(string);
}
}
And I proceed to do some more fancy pampering of the file later on. This used to work reliably, until I made this in a completely different and un-related class:
String command = getCommand(object);
if (command != null && command.length() > 0)
{
commands.add(new AnimCommand(command, i));
}
And then it started throwing this error:
[Ljava.lang.StackTraceElement;#7852e922
Exception in thread "main" java.lang.NullPointerException
at java.io.StringReader.<init>(Unknown Source)
at org.json.JSONTokener.<init>(JSONTokener.java:94)
at org.json.JSONObject.<init>(JSONObject.java:406)
at com.github.sam54123.mc_animation.utils.JSONUtils.getJSONObjectFromFile(JSONUtils.java:47)
at com.github.sam54123.mc_animation.system.Animation.<init>(Animation.java:20)
at com.github.sam54123.mc_animation.testing.Tester.main(Tester.java:13)
I've double checked that the file hasn't changed, and I tried deleting that section of code, restarting Eclipse, the whole deal, and nothing seems to fix it. The code is even able to recognize that the file is valid using the File class, but nothing seems to change. Does anyone have some insight on how this might be fixed? Here is the rest of my code: https://github.com/Sam54123/mc-animation/
EDIT
Okay, I've just done some more debugging, and it looks like it's the
return new JSONObject(string);
on line 47 of the second file that's crashing. No idea why, as the risky stuff of reading a file off disk is okay.
EDIT 2
It looks looks like it's failing because
InputStream in = FileHandle.inputStreamFromFile(path);
is returning null, which makes sense because of the try catch statement
InputStream inputStream = FileHandle.class.getResourceAsStream(path);
is in. Why that's failing beats me though, because the validity of the file is verified elsewhere in the code. It also used to work, and I haven't changed anything about the layout of the files.
EDIT 3
Interesting, a couple System.out.printlns reveal the catch is not actually getting activated, and therefore getResourceAsStream() must actually be returning null. I've confirmed this by printing it out before I return it.
I'm using Watcher in JDK7 which relies on inotify events. If the file is on a NFS, I want my program to fallback and use polling instead. Is there a way to detect if a file is on a remote drive (other than using Runtime.exec and parsing the mount table)? I'm only concerned with Linux compatibility for now.
I suppose one option is to use both inotify and polling when the program starts, but then disable the polling thread if an inotify event for my file is created.
You should be able to get relatively reliable info about the underlying file system type with FileStore.type().
It will definitely tell you if it's an NFS, or CIFS, not sure about other network mount types.
However I have no info about how reliable it is, #hoaz's suggestion to check if events are coming through might be a good idea.
I had the same problem. I have solved it by creating a new thread in de main class and touching the files periodically so a new change event gets fired.
The sample polls the dir for every 10 seconds does a touch.
Here a sample of the code:
package com.ardevco.files;
import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.FileTime;
import java.util.ArrayList;
import java.util.List;
public class Touch implements Runnable {
private Path touchPath;
public Touch(Path touchPath) {
this.touchPath = touchPath;
this.checkPath = checkPath;
}
public static void touch(Path file) throws IOException {
long timestamp = System.currentTimeMillis();
touch(file, timestamp);
}
public static void touch(Path file, long timestamp) throws IOException {
if (Files.exists(file)) {
FileTime ft = FileTime.fromMillis(timestamp);
Files.setLastModifiedTime(file, ft);
}
}
List<Path> listFiles(Path path) throws IOException {
final List<Path> files = new ArrayList<>();
try (DirectoryStream<Path> stream = Files.newDirectoryStream(path)) {
for (Path entry : stream) {
if (Files.isDirectory(entry)) {
files.addAll(listFiles(entry));
}
files.add(entry);
}
}
return files;
}
#Override
public void run() {
while (true) {
try {
for (Path path : listFiles(touchPath)) {
touch(path);
}
} catch (IOException e) {
System.out.println("Exception: " + e);
}
try {
Thread.sleep(10000L);
} catch (InterruptedException e) {
System.out.println("Exception: " + e);
}
}
}
}
This code attempts to resize images in a directory called "imgs". Unfortunately for some reason when I uncomment the listFiles(..) loop ImageIO.read(sourceImageFile) will return null. Yet processing the same file straightaway outside the loop (res("imgs/foto_3.jpg")) works. So apparently, this loop is preventing the files from being read. Solutions?
import java.awt.image.*;
import java.io.*;
import javax.imageio.*;
import static org.imgscalr.Scalr.*;
public class App {
public static void main(String[] args) throws IOException {
// for (File sourceImageFile : new File("imgs").listFiles()) {
// res("imgs/"+sourceImageFile.getName());
// }
res("imgs/foto_3.jpg");
}
public static void res(String arg) throws IOException {
File sourceImageFile = new File(arg);
BufferedImage img = ImageIO.read(sourceImageFile);
BufferedImage thumbnail = resize(img, 500);
thumbnail.createGraphics().drawImage(thumbnail, 0, 0, null);
ImageIO.write(thumbnail, "jpg", new File("resized/" + sourceImageFile.getName()));
}
}
To reproduce the problem you can download the Maven project.
Can you change res to accept a File object rather than a String? Then you could write the following, which is a lot nicer:
for (File sourceImageFile : new File("imgs").listFiles()) {
res(sourceImageFile);
}
As to your original question, try adding some tracing statements or using a debugger to find what exactly gets passed to res.
I am not sure why the file listing iteration would mess with the thumbnail generation, but you mentioned privately to me that you were using imgscalr and were curious what the correct code to batch-process a directory would look like, so I wrote up this example code for you.
The code below will process any directory (imgs is hard-coded to stay consistent with your example code) and write it out to any other directory (resized is used to stay consistent -- feel free to change either directory)
import static org.imgscalr.Scalr.*;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
public class ScaleDirExample {
public static void main(String[] args) throws IOException {
File[] images = new File("imgs").listFiles();
for (File f : images) {
System.out.println("Processing: " + f.getName() + "...");
if (f.isDirectory()) {
System.out.println("\tSkipping, file is a directory...");
continue;
} else
process(f);
}
}
private static void process(File file) throws IOException {
// Load image.
BufferedImage image = ImageIO.read(file);
// Resize image.
image = resize(image, 500);
// Save the resized image as the thumbnail.
saveThumbnail(file, image);
}
private static void saveThumbnail(File originalFile, BufferedImage thumbnail)
throws IOException {
String filename = originalFile.getName();
// Determine file extension.
String fileExt = filename.substring(filename.lastIndexOf('.') + 1);
// Save the thumbnail to the resized dir.
ImageIO.write(thumbnail, fileExt, new File("resized/" + filename));
}
}
The call to resize(image, 500) can be modified to match any of the Scalr.resize methods - you can pass filters or improve quality if you want.
Example output from my test setup looks like:
Processing: flower-dog-gimp.jpg...
Processing: flower-dog.jpg...
Processing: logoOXdaComida.png...
Processing: mr-t-thumbnail.jpg...
Processing: mr-t.jpg...
Processing: problem-trans-peter-griffin.png...
Hope that helps!
OK, I figured it out. now it's working.
You (or whoever created the downloadable project u pasted here) are using Mac OS, and it automatically creates a .DS_Store file.
When you try to pass it to the res method, it doesn't know how to handle a non-image file and acts as null.
public static void main(String[] args) throws IOException {
for (File sourceImageFile : new File("imgs").listFiles()) {
if (sourceImageFile.getName().endsWith(".jpg"))
res(sourceImageFile.getAbsolutePath());
}
}
This is the modified void main method. it works, u can refine the if statement for more in depth filtering of wrong files.
Also I changed the argument given to the res method. it looks better now, as getAbsoulutePath returns the name and the path.
Let me know how it worked
One obvious way to debug this to output the specific sourceImageFile that ImageIO.read(..) is returning null on. I suspect its because listFiles will give you a list of all files and directories that are in the image directory. listFiles javadoc. You can prevent that by using a FileFilter that makes sure that listFiles only returns files and additionally files of the right type.
An example of a file filter that only returns file is below:
import java.io.FileFilter
class RegularFilesOnlyFileFilter implements FileFilter {
public boolean accept(File pathName) {
return pathName.isFile();
}
}
The way to use this filter is this - new File("imgs").listFiles(new RegularFilesOnlyFilesFilter())
I've changed res method:
public static void res(File arg) throws IOException {
if (arg.contains(".DS_Store")) {
return;
}
A mac-issue (or should filter non-image files, as was suggested)!
I use this test to convert txt to pdf :
package convert.pdf;
//getResourceAsStream(String name) : Returns an input stream for reading the specified resource.
//toByteArray : Get the contents of an InputStream as a byte[].
import java.io.FileOutputStream;
import java.io.IOException;
import org.apache.commons.io.IOUtils;
import convert.pdf.txt.TextConversion;
public class TestConversion {
private static byte[] readFilesInBytes(String file) throws IOException {
return IOUtils.toByteArray(TestConversion.class.getResourceAsStream(file));
}
private static void writeFilesInBytes(byte[] file, String name) throws IOException {
IOUtils.write(file, new FileOutputStream(name));
}
//just change the extensions and test conversions
public static void main(String args[]) throws IOException {
ConversionToPDF algorithm = new TextConversion();
byte[] file = readFilesInBytes("/convert/pdf/text.txt");
byte[] pdf = algorithm.convertDocument(file);
writeFilesInBytes(pdf, "text.pdf");
}
}
Problem:
Exception in thread "main" java.lang.NullPointerException
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1025)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:999)
at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:218)
at convert.pdf.TestConversion.readFilesInBytes(TestConversion.java:17)
at convert.pdf.TestConversion.main(TestConversion.java:28)
I use the debugger, and the problem seems to be located here :
private static byte[] readFilesInBytes(String file) throws IOException {
return IOUtils.toByteArray(TestConversion.class.getResourceAsStream(file));
}
What is my problem?
Sounds like the resource probably doesn't exist with that name.
Are you aware that Class.getResourceAsStream() finds a resource relative to that class's package, whereas ClassLoader.getResourceAsStream() doesn't? You can use a leading forward slash in Class.getResourceAsStream() to mimic this, so
Foo.class.getResourceAsStream("/bar.png")
is roughly equivalent to
Foo.class.getClassLoader().getResourceAsStream("bar.png")
Is this actually a file (i.e. a specific file on the normal file system) that you're trying to load? If so, using FileInputStream would be a better bet. Use Class.getResourceAsStream() if it's a resource bundled in a jar file or in the classpath in some other way; use FileInputStream if it's an arbitrary file which could be anywhere in the file system.
EDIT: Another thing to be careful of, which has caused me problems before now - if this has worked on your dev box which happens to be Windows, and is now failing on a production server which happens to be Unix, check the case of the filename. The fact that different file systems handle case-sensitivity differently can be a pain...
Are you checking to see if the file exists before you pass it to readFilesInBytes()? Note that Class.getResourceAsStream() returns null if the file cannot be found. You probably want to do:
private static byte[] readFilesInBytes(String file) throws IOException {
File testFile = new File(file);
if (!testFile.exists()) {
throw new FileNotFoundException("File " + file + " does not exist");
}
return IOUtils.toByteArray(TestConversion.class.getResourceAsStream(file));
}
or better yet:
private static byte[] readFilesInBytes(String file) throws IOException {
InputStream stream = TestConversion.class.getResourceAsStream(file);
if (stream == null) {
throw new FileNotFoundException("readFilesInBytes: File " + file
+ " does not exist");
}
return IOUtils.toByteArray(stream);
}
This class reads a TXT file in the classpath and uses TextConversion to convert to PDF, then save the pdf in the file system.
Here TextConversion code :
package convert.pdf.txt;
//Conversion to PDF from text using iText.
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import convert.pdf.ConversionToPDF;
import convert.pdf.ConvertDocumentException;
import com.lowagie.text.Document;
import com.lowagie.text.DocumentException;
import com.lowagie.text.Font;
import com.lowagie.text.Paragraph;
import com.lowagie.text.pdf.PdfWriter;
public class TextConversion implements ConversionToPDF {
public byte[] convertDocument(byte[] documents) throws ConvertDocumentException {
try {
return this.convertInternal(documents);
} catch (DocumentException e) {
throw new ConvertDocumentException(e);
} catch (IOException e) {
throw new ConvertDocumentException(e);
}
}
private byte[] convertInternal(byte[] documents) throws DocumentException, IOException {
Document document = new Document();
ByteArrayOutputStream pdfResultBytes = new ByteArrayOutputStream();
PdfWriter.getInstance(document, pdfResultBytes);
document.open();
BufferedReader reader = new BufferedReader( new InputStreamReader( new ByteArrayInputStream(documents) ) );
String line = "";
while ((line = reader.readLine()) != null) {
if ("".equals(line.trim())) {
line = "\n"; //white line
}
Font fonteDefault = new Font(Font.COURIER, 10);
Paragraph paragraph = new Paragraph(line, fonteDefault);
document.add(paragraph);
}
reader.close();
document.close();
return pdfResultBytes.toByteArray();
}
}
And here the code to ConversionToPDF :
package convert.pdf;
// Interface implemented by the conversion algorithms.
public interface ConversionToPDF {
public byte[] convertDocument(byte[] documentToConvert) throws ConvertDocumentException;
}
I think the problem come from my file system (devbox on windows and server is Unix).
I will try to modify my classpath.
This problem may be caused by calling methods on test.txt, which can be a folder shortcut. In other words, you're calling a method on a file that doesn't exist, resulting in a NullPointerException.