Develop Reference

delete file after serving over HTTP in play framework - java

How do i delete a file after serving it over http,
Files.TemporaryFile file = null;
try {
file = new Files.TemporaryFile(f);
return ok().sendFile(file.file());
} catch (IllegalArgumentException e) {
return badRequest(Json.newObject().put("message", e.getMessage()));
} finally {
file.clean();
}
with this code, the file gets deleted before it is served. i receive an empty file on the client.

Play framework in version 2.8 should support onClose argument in sendFile method also in Java (so far it seems to be supported only in Scala version).
In older versions (I have tried only on 2.7.x) you may apply the same approach like in the fix for 2.8, so:
public Result doSomething() {
final File fileToReturn = ....;
final Source<ByteString, CompletionStage<IOResult>> source = FileIO.fromFile(fileToReturn);
return Results.ok().streamed(wrap(source, () -> fileToReturn.delete()), Optional.of(fileToReturn.length()), Optional.of("content type, e.g. application/zip"));
}
private Source<ByteString, CompletionStage<IOResult>> wrap(final Source<ByteString, CompletionStage<IOResult>> source, final Runnable handler) {
return source.mapMaterializedValue(
action -> action.whenCompleteAsync((ioResult, exception) -> handler.run())
);
}

From reading the JavaFileUpload documentation for 2.6.x, it sounds like you don't need that finally block to clean up the file afterwards. Since you are using a TemporaryFile, garbage collection should take care of deleting the resource:
...the idea behind TemporaryFile is that it’s only in scope at completion and should be moved out of the temporary file system as soon as possible. Any temporary files that are not moved are deleted [by the garbage collector].
The same section goes on to describe that there is the potential that the file will not get garbage collection causing Denial Of Service issues. If you find that the files are not getting removed, then you can use the TemporaryFilesReaper:
However, under certain conditions, garbage collection does not occur in a timely fashion. As such, there’s also a play.api.libs.Files.TemporaryFileReaper that can be enabled to delete temporary files on a scheduled basis using the Akka scheduler, distinct from the garbage collection method.

I am not forcing all the project, but you can use a Scala for only this controller, then you can use onClose parameter of the sendFile method. The only attention - that parameter is not workable in all versions, it looks like in 2.5 there was an issue so it was not triggered (was not work: https://github.com/playframework/playframework/issues/6351).
Another way - you can use Akka streams, like here: https://www.playframework.com/documentation/2.6.x/JavaStream#Chunked-responses.

My own Logging Handler for GAE/J (using appengine.api.log?)

I need to write my own logging handler on GAE/J. I have Android code that I'm trying to adapt such that it can be shared between GAE/J and Android. The GAE code I'm trying to write would allow the log statements in my existing code to work on GAE.
The docs say that I can just print to system.out and system.err, and it works, but badly. My logging shows up in the log viewer with too much extraneous text:
2013-03-08 19:37:11.355 [s~satethbreft22/1.365820955097965155].: [my_log_msg]
So, I started looking at the GAE log API. This looked hopeful initially: I can construct an AppLogLine and set the log records for a RequestLogs object.
However, there is no way to get the RequestLogs instance for the current request - the docs say so explicitly here:
Note: Currently, App Engine doesn't support the use of the request ID to directly look up the related logs.
I guess I could invent a new requestID and add log lines to that, but it is starting to look like this is just not meant to be?
Has anyone used this API to create their own log records, or otherwise managed to do their own logging to the log console.
Also, where can I find the source for GAE's java.util.logging? Is this public? I would like to see how that works if I can.
If what I'm trying to do is impossible then I will need to consider other options, e.g. writing my log output to a FusionTable.

I ended up just layering my logging code on top of GAE's java.util.logging. This feels non-optimal since it increases the complexity and overhead of my logging, but I guess this is what any 3rd logging framework for GAE must do (unless it is OK with the extra cruft that gets added when you just print to stdout).
Here is the crux of my code:
public int println(int priority, String msg) {
Throwable t = new Throwable();
StackTraceElement[] stackTrace = t.getStackTrace();
// Optional: translate from Android log levels to GAE log levels.
final Level[] levels = { Level.FINEST, Level.FINER, Level.FINE, Level.CONFIG,Level.INFO, Level.WARNING, Level.SEVERE, Level.SEVERE };
Level level = levels[priority];
LogRecord lr = new LogRecord(level, msg);
if (stackTrace.length > 2) { // should always be true
lr.setSourceClassName(stackTrace[2].getClassName());
lr.setSourceMethodName(stackTrace[2].getMethodName());
}
log.log(lr);
return 0;
}
Note that I use a stack depth of 2, but that # will depend on the 'depth' of your logging code.
I hope that Google will eventually support getting the current com.google.appengine.api.log.RequestLogs instance and inserting our own AppLogLine instances into it. (The API's are actually there to do that, but they explicitly don't support it, as above.)

Java: Is this proper behavior of singleton class used to save basic JDK log file?

I created the below wrapper for a singleton logger class so that I can log exceptions from any file and/or class. My inspiration comes from here: Logging to one file from multiple java files
I'm new to java and this is my first use of attempting to log exceptions to a file. While the following code does work, I noticed a few quirks I wanted to ask whether or not they were "normal behavior" or whether a workaround exists.
(1) Once the log file is created and has been written to with exceptions, if I edit the log file (e.g. remove some lines of text) then the log file is never written again from that moment forward. The web server needs to restart the domain associated with the web application before the log file is written to again. Is this normal? Ideally, I would like to retain only relevant errors in the log file, and delete the irrelevant ones, without needing to restart the web application's domain in the web server.
(2) If I delete the log file, then later an exception occurs in the java program(s), the log file does not get created again, and the exception is lost. Again, the web server needs to restart the domain associated with the web application before a new log file is created. Is this normal, or is there a workaround somehow by editing the below code? Ideally, I'd like to be able to delete the log file at any time and have the application create a new one.
Again, I'm new here so feel free to point out of the above intentions are bad design, etc.
------code follows------
Here's the file: LoggerWrapper.java
import java.io.OutputStream;
import java.io.FileOutputStream;
import java.io.PrintStream;
import java.util.logging.Logger;
import java.util.logging.Level;
import java.util.logging.FileHandler;
import java.util.logging.SimpleFormatter;
public class LoggerWrapper {
public static final Logger myLogger = Logger.getLogger("Test");
private static LoggerWrapper instance = null;
public static LoggerWrapper getInstance() {
if(instance == null) {
prepareLogger();
instance = new LoggerWrapper ();
}
return instance;
}
private static void prepareLogger() {
try {
FileHandler myFileHandler = new FileHandler("/path/to/myLogFile.log", true);
myFileHandler.setFormatter(new SimpleFormatter());
myLogger.addHandler(myFileHandler);
myLogger.setUseParentHandlers(false);
myLogger.setLevel(Level.ALL);
} catch (Exception e) {
...
}
}
}
To call the above code in a different file or class, issue these lines (for example):
LoggerWrapper loggerWrapper = LoggerWrapper.getInstance();
...
loggerWrapper.myLogger.log(Level.SEVERE, "some text here"+e.fillInStackTrace());

You don't need to go to all of this trouble with logging. The log handler (and all of the alternatives, like logback, log4j) or a wrapper framework like Commons Logging or slf4j will do almost all of this work for you.
The most widely accepted method of using logging is to use a wrapper system (slf4j is very popular) and then just include the log class as a private static attribute of each class. Then you can do all of the configuration either with a separate program, or you can have configuration files that will set everything up for you.
If you want to set up your logging system in code (not the way I would do it, but you can...), do it in a static initializer that you know will load relatively early. Once the logging system has it's information, it won't need to be configured again.

I'm assuming this is running on linux? When you open the file it's pointing to a reference to that file. When you delete the file the reference is gone, but since you saved the reference in a static variable, your code is now holding a file handle that points to nothing. Instead you should do something like
cat /dev/null > file
Which will just copy nothing over the file without changing the actual inode that the file points to.

Have you seen this ?
It may have to do with your editor/os. The poster from the linked question had a similar problem with log4j.
As the other answer states, to clear out the log file w/o disturbing the handler:
cat /dev/null > file.log

I would suggest you just don't edit the files! And don't delete them!
HAHAHAHA, that would be an ace solution, no?
You should really use the open source logging (i've always liked log4j - most IB use that, or slf4j) that has been suggested by Jonathan - it is the standard way to do logging, nobody really does logging like you're doing it. At least, not that I am aware.
Ok, that said - and I do not know linux at all, in any shape or form - it sounds like, as Rick suggests, whatever you are pointing to disappears when you edit/delete the file. So, in pseudo-code (because, and i am sorry for this - as i indicated, i use windows so cannot test):
public class LoggerWrapper {
*private* FileHandler myFileHandler;
*private* static final Logger myLogger = Logger.getLogger("Test");
private static LoggerWrapper instance = null;
/*
here check if the file you're using has been changed! If so, re-do the file setting
*/
public void log(Level level, String message){
//argh you are on your own here. I don't know how to *check* the file being used by the file handler...
//you know what? you can just do this (but it isn't too clean)
myLogger.removeFileHandler(myFileHandler);
myFileHandler = new FileHandler("/path/to/myLogFile.log", true);
myLogger.addHandler(myFileHandler);
myLogger.log(level,message);
}
private static void prepareLogger() {
try {
//don't shadow the myFileHandler by accident!
myFileHandler = new FileHandler("/path/to/myLogFile.log", true);
...
of course, that just resets the file handler every time you log. And you won't log with the command:
loggerWrapper.myLogger.log(Level.SEVERE, "some text here"+e.fillInStackTrace());
but with
loggerWrapper.log(Level.SEVERE, "some text here"+e.fillInStackTrace());
Or just use open source logging (log4j!) and be happy! There might be some way to check if the file is lost - oh! maybe check the isLoggable method? I don't know if that will pick up on changed files... i don't think it does though.
Also, you might need to setErrorManager on your fileHandler - from the looks of things, if the logger cannot, well, log, it sends something to the errorManager. I am unfamiliar with this though, and i might be speaking jibberish.

Is it possible to supply a new PropertiesConfiguration file at runtime?

Background:
I have a requirement that messages displayed to the user must vary both by language and by company division. Thus, I can't use out of the box resource bundles, so I'm essentially writing my own version of resource bundles using PropertiesConfiguration files.
In addition, I have a requirement that messages must be modifiable dynamically in production w/o doing restarts.
I'm loading up three different iterations of property files:
-basename_division.properties
-basename_2CharLanguageCode.properties
-basename.properties
These files exist in the classpath. This code is going into a tag library to be used by multiple portlets in a Portal.
I construct the possible .properties files, and then try to load each of them via the following:
PropertiesConfiguration configurationProperties;
try {
configurationProperties = new PropertiesConfiguration(propertyFileName);
configurationProperties.setReloadingStrategy(new FileChangedReloadingStrategy());
} catch (ConfigurationException e) {
/* This is ok -- it just means that the specific configuration file doesn't
exist right now, which will often be true. */
return(null);
}
If it did successfully locate a file, it saves the created PropertiesConfiguration into a hashmap for reuse, and then tries to find the key. (Unlike regular resource bundles, if it doesn't find the key, it then tries to find the more general file to see if the key exists in that file -- so that only override exceptions need to be put into language/division specific property files.)
The Problem:
If a file did not exist the first time it was checked, it throws the expected exception. However, if at a later time a file is then later dropped into the classpath and this code is then re-run, the exception is still thrown. Restarting the portal obviously clears the problem, but that's not useful to me -- I need to be able to allow them to drop new messages in place for language/companyDivision overrides w/o a restart. And I'm not that interested in creating blank files for all possible divisions, since there are quite a few divisions.
I'm assuming this is a classLoader issue, in that it determines that the file did not exist in the classpath the first time, and caches that result when trying to reload the same file. I'm not interested in doing anything too fancy w/ the classLoader. (I'd be the only one who would be able to understand/maintain that code.) The specific environment is WebSphere Portal.
Any ways around this or am I stuck?

My guess is that I am not sure if Apache's FileChangedReloadingStrategy also reports the events of ENTRY_CREATE on a file system directory.
If you're using Java 7, I propose to try the following. Simply, implement a new ReloadingStrategy using Java 7 WatchService. In this way, every time either a file is changed in your target directories or a new property file is placed there, you poll for the event and able to add the properties to your application.
If not on Java 7, maybe using a library such as JNotify would be a better solution to get the event of a new entry in a directory. But again, you need to implement the ReloadingStrategy.
UPDATE for Java 6:
PropertiesConfiguration configurationProperties;
try {
configurationProperties = new PropertiesConfiguration(propertyFileName);
configurationProperties.setReloadingStrategy(new FileChangedReloadingStrategy());
} catch (ConfigurationException e) {
JNotify.addWatch(propertyFileDirectory, JNotify.FILE_CREATED, false, new FileCreatedListener());
}
where
class FileCreatedListener implements JNotifyListener {
// other methods
public void fileCreated(int watchId, String rootPath, String fileName) {
configurationProperties = new PropertiesConfiguration(rootPath + "/" + fileName);
configurationProperties.setReloadingStrategy(new FileChangedReloadingStrategy());
// or any other business with configurationProperties
}
}

Audit Java: system to detect exceptions thrown / caught (aop?)

Due to checked exceptions, we can have some problems in production having all exceptions caught in the right place and logged correctly.
I wonder if there is some opensource tool to help with auditing these problems.
For example, is there some AOP tool that would intercept all exceptions thrown and see if they are re-thrown, wrapped or logged? This would help identify the bad catches.

If you've decided that you would like to take the AOP route, the Spring Framework provides an easy to use AOP framework. Essentially, like much of Spring, you would use a combination of a xml config file and some java code to define the AOP functionality you are looking for.
In your case, I believe you would be looking to define an 'After Throwing Advice', in which you would of course have access to the exception thrown.
A good place to start in terms of documentation is the AOP Chapter in the Spring docs:
http://static.springsource.org/spring/docs/2.5.x/reference/aop.html
Oh, and I believe all Spring projects are open source as well =)

I know the question asks for an open source solution. I don't know of one but if the option is there then DynaTrace does exactly what you want. Good luck on your search.

There are tools such as FindBugs, PMD and Checkstyle which can identify some common Exception handling issues. I'm never seen a tool that specifically analyses your exception handling, if anyone knows I'll be interested!

I had this exact question, and I attempted to write something myself, and due to AOP nested proxying and lack of ability to use instrumenation / weaving, I gave up and just did a wide find and replace
One of he tools I did find back then for was AppSight by BMC software, but it's high cost was an issue

IntelliJ's Inspector can check code for many problems as you write it:
http://www.jetbrains.com/idea/documentation/inspections.jsp
But your problem sounds like it's more about education than technology. You need to educate your team on what proper exception handling means, when it should be done, etc. Tools will help, but not putting them into the code is the first place is better.
We use Spring aspects for our production systems to do logging, tracing, performance calculations, etc. Before, after, and exception advice work wonders - they keep the code in one place and give declarative flexibility as to where they are applied.
Just one caution: aspects aren't free. They add cost to each method you apply them to, so don't just pile them on. Moderation in all things is the key.

I didn't though about that yet but one solution, if you do not need to detect exceptions thrown on production envirionment, is to attach to your Java application a custom debugger that can be triggered whenever an exception is raised.
This french blog article talk about how to do it:
http://blog.xebia.fr/2011/12/12/legacy-code-gestion-des-exceptions-avec-jpda/
Here is the code:
Run with debug:
Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n
Connect to the JVM:
public static VirtualMachine connect(String port) throws IOException, IllegalConnectorArgumentsException {
AttachingConnector connector = null;
VirtualMachineManager vmManager = Bootstrap.virtualMachineManager();
for (Connector aconnector : vmManager.allConnectors()) {
if ("com.sun.jdi.SocketAttach".equals(aconnector.name())) {
connector = (AttachingConnector) aconnector;
break;
}
}
Map<String, Connector.Argument> args = connector.defaultArguments();
Connector.Argument pidArgument = args.get("port");
pidArgument.setValue(port);
return connector.attach(args);
}
Create your breakpoints. Exemple:
public static void createExceptionBreakPoint(VirtualMachine vm) {
EventRequestManager erm = vm.eventRequestManager();
List<ReferenceType> referenceTypes = vm.classesByName("java.lang.Throwable");
for (ReferenceType refType : referenceTypes){
ExceptionRequest exceptionRequest = erm.createExceptionRequest(refType, true, true);
exceptionRequest.setEnabled(true);
}
}
And then handle the exceptions:
public static void handleExceptionEvent(ExceptionEvent exceptionEvent) throws Exception {
ObjectReference remoteException = exceptionEvent.exception();
ThreadReference thread = exceptionEvent.thread();
List<Value> paramList = new ArrayList<Value>(1);
paramList.add(dumpFileName);
//crer un printStream dans la JVM cible
ObjectReference printStreamRef = printStreamClassType.newInstance(thread, printStreamConstructor, paramList,
ObjectReference.INVOKE_SINGLE_THREADED);
ReferenceType remoteType = remoteException.referenceType();
Method printStackTrace = (Method) remoteType.methodsByName("printStackTrace").get(1);
paramList.clear();
paramList.add(printStreamRef);
remoteException.invokeMethod(thread, printStackTrace, paramList, ObjectReference.INVOKE_SINGLE_THREADED);
Scanner scanner = new Scanner(new File(dumpFileName.value()));
while (scanner.hasNextLine()){
System.out.println(scanner.nextLine());
}
}
A bit heavy but it works, now how to catch the exceptions that are logged and the others?