Given a Java function which uses a stream:
List<String> function(List<String> input) {
Stream<String> a = input.parallelStream();
Stream<String> b = a.map(c -> c.toString());
return b.collect(Collectors.toList());
}
Now, I want to control whether the mapping is performed by a parameter. Eclipse/the compiler gives me a warning: Resource leak: 'a' is never closed
List<String> function(List<String> input, boolean doMap) {
Stream<String> a = input.parallelStream(); // Resource leak: 'a' is never closed
Stream<String> b = doMap ? a.map(c -> c.toString()) : a;
return b.collect(Collectors.toList());
}
I can work around this with a try-with-resources statement:
List<String> function(List<String> input, boolean doMap) {
try (Stream<String> a = input.parallelStream()) {
Stream<String> b = doMap ? a.map(c -> c.toString()) : a;
return b.collect(Collectors.toList());
}
}
My question is: Why can there be resource leaks at all when I work with streams instead of for-loops? Why is there a resource leak if I only optionally add the mapping step? (Why isn’t there a resource leak in the first version of the function?) Is it “dangerous” to compose processing streams conditionally? What am I missing?
I summarize #Nikolas answer again in other words: There is no problem in Java, but the “problem” (if you want to call it that at all) is the variable. If a variable is declared to a type that implements AutoCloseable (and Stream does), Eclipse apparently reports a warning here that it is not being closed, if it does not find a call to close().
Since this error is Eclipse-related, it is likely that other checking tools will not fail on this and does not need to be “fixed” to pass such checks.
To my understanding, this shows a basic problem of Java, namely that it does not release the object on a variable as soon as it is no longer needed, but at some random point. With resources, that fails. Therefore, resources have to be tracked manually by the developer, where to close them, and manually to be closed. The Java runtime would (hypothetically) have to implement an approach like C++’s std::auto_ptr, then this would not be necessary and if the last reference to the resource was deleted, this could be closed. But it's not like java "thinks".
Related
Consider this piece of code:
private static ArrayList<Level> levels = new ArrayList<>();
static {
try (Stream<Path> paths = Files.walk(Paths.get("levels"))) {
paths
.filter(Files::isRegularFile)
.forEach(levels.add(new Level(file))); //CAN'T DO!!
} catch (IOException e) {
e.printStackTrace();
}
}
I think the code pretty much says what I'm trying to do. I found this piece of code somewhere and I tried to apply it, to create Level objects from folder with files such as level1.txt, level2.txt, etc. The Level class takes a File as argument.
The forEach method gives out a SyntaxError.
What are the "::" in the filter method?
Why is the path, fallowed by a new line and 2 methods? Never seen such a thing before.
How can I make this work properly?
The following explains well what is :: (double colon) operator since Java 8
Code can be run written on several lines, but this is same as the following. Your calling the method filter then calling forEach on the object returned by filter
paths.filter(Files::isRegularFile).forEach(levels.add(new Level(file)));
Get it work, you need to define file variable this is done with lambda functions
paths.filter(Files::isRegularFile).forEach(file -> levels.add(new Level(file)));
NOT POSSIBLE because walks() throws IOException
In case you don't need the try/catch you can use Collectors to directly build the list
private static ArrayList<Level> levels = Files.walk(Paths.get("levels"))
.filter(Files::isRegularFile)
.map(file -> new Level(file))
.collect(Collectors.toList());
How do i delete a file after serving it over http,
Files.TemporaryFile file = null;
try {
file = new Files.TemporaryFile(f);
return ok().sendFile(file.file());
} catch (IllegalArgumentException e) {
return badRequest(Json.newObject().put("message", e.getMessage()));
} finally {
file.clean();
}
with this code, the file gets deleted before it is served. i receive an empty file on the client.
Play framework in version 2.8 should support onClose argument in sendFile method also in Java (so far it seems to be supported only in Scala version).
In older versions (I have tried only on 2.7.x) you may apply the same approach like in the fix for 2.8, so:
public Result doSomething() {
final File fileToReturn = ....;
final Source<ByteString, CompletionStage<IOResult>> source = FileIO.fromFile(fileToReturn);
return Results.ok().streamed(wrap(source, () -> fileToReturn.delete()), Optional.of(fileToReturn.length()), Optional.of("content type, e.g. application/zip"));
}
private Source<ByteString, CompletionStage<IOResult>> wrap(final Source<ByteString, CompletionStage<IOResult>> source, final Runnable handler) {
return source.mapMaterializedValue(
action -> action.whenCompleteAsync((ioResult, exception) -> handler.run())
);
}
From reading the JavaFileUpload documentation for 2.6.x, it sounds like you don't need that finally block to clean up the file afterwards. Since you are using a TemporaryFile, garbage collection should take care of deleting the resource:
...the idea behind TemporaryFile is that it’s only in scope at completion and should be moved out of the temporary file system as soon as possible. Any temporary files that are not moved are deleted [by the garbage collector].
The same section goes on to describe that there is the potential that the file will not get garbage collection causing Denial Of Service issues. If you find that the files are not getting removed, then you can use the TemporaryFilesReaper:
However, under certain conditions, garbage collection does not occur in a timely fashion. As such, there’s also a play.api.libs.Files.TemporaryFileReaper that can be enabled to delete temporary files on a scheduled basis using the Akka scheduler, distinct from the garbage collection method.
I am not forcing all the project, but you can use a Scala for only this controller, then you can use onClose parameter of the sendFile method. The only attention - that parameter is not workable in all versions, it looks like in 2.5 there was an issue so it was not triggered (was not work: https://github.com/playframework/playframework/issues/6351).
Another way - you can use Akka streams, like here: https://www.playframework.com/documentation/2.6.x/JavaStream#Chunked-responses.
Disclaimer: I work on a non-traditional project, so don't be shocked if some assumptions seem absurd.
Context
I wish to create a stream reader for integers, strings, and the other common types in Scala, but to start with I focus only on integers. Also note that I'm not interesting in handling exception at the moment -- I'll deal with them in due time and this will be reflected in the API and in the meantime I can make the huge assumption that failures won't occur..
The API should be relatively simple, but due to the nature of the project I'm working on, I can't rely on some feature of Scala and the API needs to look something like this (slightly simplified for the purpose of this question):
object FileInputStream {
def open(filename: String): FileInputStream =
new FileInputStream(
try {
// Check whether the stream can be opened or not
val out = new java.io.FileReader(filename)
out.close()
Some[String](filename)
} catch {
case _: Throwable => None[String]
}
)
}
case class FileInputStream(var filename: Option[String]) {
def close: Boolean = {
filename = None[String]
true // This implementation never fails
}
def isOpen: Boolean = filename.isDefined
def readInt: Int = nativeReadInt
private def nativeReadInt: Int = {
??? // TODO
}
}
object StdIn {
def readInt: Int = nativeReadInt
private def nativeReadInt: Int = {
??? // TODO
}
}
Please also note that I cannot rely on additional fields in this class, with the exception of Int variables. This (probably) implies that the stream has to be opened and closed for every operations. Hence, it goes without saying that the implementation will not be efficient, but this is not an issue.
The Question
My goal is to implement the two nativeReadInt methods such that the input stream gets consumed by only one integer if one is available straight away. However, if the input doesn't start (w.r.t. the last read operation) with an integer then nothing should be read and a fixed value can be returned, say -1.
I've explored several high level Java and Scala standard APIs, but none seemed to offer a way to re-open a stream to a given position trivially. My hope is to avoid implementing low level parsing based solely on java.io.InputStream and its read() and skip(n) methods.
Additionally, to let the user read from the standard input stream, I need to avoid using scala.io.StdIn.readInt() method because it reads "an entire line of the default input", therefore trashing some potential data.
Are you aware of a Java or Scala API that could do the trick here?
Thank you
This question already has answers here:
Value does not update in while loop unless printed out [duplicate]
(4 answers)
Closed 8 years ago.
I have the following piece of code
public String ls() throws IOException, InterruptedException {
String pwd = getPWD();
String inodePath = "Upload/Files" + pwd + "inode";
// Request the inode file first
Peer.setFileReceivedCheck(false);
Inode i = new Inode();
i.requestInode(pwd + "inode");
boolean fileCheck = Peer.getFileReceivedCheck();
System.out.println(fileCheck);
while (fileCheck == false) {
System.out.println(); // DOES NOT WORK IF THIS LINE IS REMOVED!!
fileCheck = Peer.getFileReceivedCheck();
}
System.out.println(fileCheck);
return i.readFromInode(inodePath);
}
In the above java method, the fileCheck variable keeps track of when a file is downloaded from the network and becomes true when the file download completes; so that the functions returns when the file download is completed.
The weird problem which I am facing is that if I remove the above line (the one with just a println); the method is stuck in the while loop and does not return even though the file has been downloaded! Any idea as to why this is happening?
EDIT: This is not running in a separate thread. The file download code in another class (Peer.java) is running in a separate. Again, the getter for fileCheck is out of the thread.
Please, in the name of $DEITY, do not use hot polling. Stop. Right now. You are wasting billions of cycles, you will make the whole system less responsive, while you wait for something to come over the network. Use a callback - we are in 2014, it's not rocket science.
That said, repace the contents of the while loop with this:
Object o = new Object();
synchronized(o) {
fileCheck = Peer.getFileReceivedCheck();
}
What happens is that your check probably just returns a non-volatile field, and the compiler is free to optimize that away - unless your code hits a synchronization point.
Did you try adding a small timeout (100ms), instead of println()?
While messing around with the custom formatting options in Eclipse, in one of the sample pieces of code, I saw code as follows:
/**
* 'try-with-resources'
*/
class Example {
void foo() {
try (FileReader reader1 = new FileReader("file1"); FileReader reader2 = new FileReader("file2")) {
}
}
}
I've never seen try used like this and I've been coding in Java for 9 years! Does any one know why you would do this? What is a possible use-case / benefit of doing this?
An other pieces of code I saw, I thought was a very useful shorthand so I'm sharing it here as well, it's pretty obvious what it does:
/**
* 'multi-catch'
*/
class Example {
void foo() {
try {
} catch (IllegalArgumentException | NullPointerException | ClassCastException e) {
e.printStackTrace();
}
}
}
It was added in Java 7. It's called the try-with-resources statement.
/edit
Might as well throw this in here too. You can use the try-with-resources statement to manage Locks if you use a wrapper class like this:
public class CloseableLock implements Closeable {
private final Lock lock;
private CloseableLock(Lock l) {
lock = l;
}
public void close() {
lock.unlock();
}
public static CloseableLock lock(Lock l) {
l.lock();
return new CloseableLock(l);
}
}
try(CloseableLock l = CloseableLock.lock(lock)) { // acquire the lock
// do something
} // release the lock
However, since you have to declare a variable for every resource, the advantage of this is debatable.
This is Java 7's new try-with-resources statement: http://download.oracle.com/javase/7/docs/technotes/guides/language/try-with-resources.html
Those are changes introduced in JDK7.
First statement is a try-with-resources. I don't know exactly why they exist but exceptions are often caused by inputstreams etc, I guess it just improves readability. Edit: thanks to the other answerers, I read the javadoc and I now know that it will close all i/o streams that implement AutoCloseable, omitting the need for a finally block in a lot of situations
Second is a multi-catch, which is really handy when you have different exceptions that you handle in exactly the same way.
Same usage as using(Resource) in C Sharp,which means this resource will be automatic recycled when your program has leaven out of this code block.(Just my opinion)
The try-with-resources statement ensures that each resource is closed at the end of the statement. Any object that implements java.lang.AutoCloseable, which includes all objects which implement java.io.Closeable, can be used as a resource
The try-with-resources Statement
It's called try-with-resource. It's a way so as to not have to clean after yourself as the language will do it for you.
it was added in java 7. It is called try with resources. Try with resources statement feature was introduced in java 7 version. Try with resource statement is a try statement that declares one or more statements. A resource is an object that must be closed after the program is finished with it.
Before java 7 we use finally block to close the resources that we have used in our program. In finally block we have to close all the resources manually that we have used in our program.
For more information you can visit try with resources
That is called with a try with resources. in a try with resources, any kind of closable stream declared in the resources section will be closed after the try statement is done. So it pretty much is a
try{
InputStream is;
//Stuff
}finally{
is.close()
}
The try-with-resources statement is a try statement that declares one or more resources. A resource is an object that must be closed after the program is finished with it. The try-with-resources statement ensures that each resource is closed at the end of the statement. Any object that implements java.lang.AutoCloseable, which includes all objects which implement java.io.Closeable, can be used as a resource.