I'm working with JUL for my logging (no I can't change that). I've developed a simple wrapper that I pass in the parameters and it creates the FileHandler with the correct format every time so that I don't have to recreate the logging in every project.
My test app functions exactly as intended, but when I import the library into other projects I seem to be getting one (only one so far) unique error: Every single time, it adds a ".0" to the end of the log file.
It does this even when there is no conflicts and the Filehandler has been configured to append to the end an existing file (which it does fine). I've played with various file names, most recently I've been using the simple "mylog.log" and the log file still gets output as "mylog.log.0". I've checked and the fileHandler is being passed the correct file ("mylog.log"), but it isn't logging there.
This does not happen in my logging test, only in the project I actually want to use it in. Even when using the exact same parameters, I get different file names.
Is there some quirk about JUL that I'm missing? Code is very simple. Relevent code:
String logFilePath = directory+name; // directory and name are method arguments
Handler newFileHandler;
File dirFile = new File(directory);
if(!dirFile.exists())
{
dirFile.mkdirs();
}
newFileHandler = new FileHandler(logFilePath, true);
newFileHandler.setFormatter(myformatter);
//... etc
It does this even when there is no conflicts ....
Is that proven with evidence or an assumption? According to the FileHandler documentation:
If no "%g" field has been specified and the file count is greater than one, then the generation number will be added to the end of the generated filename, after a dot.
If there is a conflict and no "%u" field has been specified, it will be added at the end of the filename after a dot. (This will be after any automatically added generation number.)
Note that the use of unique ids to avoid conflicts is only guaranteed to work reliably when using a local disk file system.
A conflict can include opening more than one FileHandler to the same location. You need to verify each of these points. What helps is adding code to grab the RuntimeMXBean and then add a single log statement to record the calling ClassLoader, current thread, runtime name and the start time. The runtime name usually maps to a process id and a host name. Run the program and verify the contents of the file.
The code you have included helps but, you need to include details on how the application is launched and what is included in the logging.properties.
I eventually figured this out and forgot to post the cause.
Two things were at work:
Due to the environment I was in, the "rolling" logging was being activated by some back ground variables I wasn't aware of, hence why the ".0" was being added when it shouldn't have been, but I only saw it once I moved it out of testing and into the actual implementing project.
JUL is ridiculously inflexible in how it works. Really I can't speak poorly enough about it. Anyway, long story short, if rolling logging is enabled it will always append a file number such that the active log ends in ".0". Typically I've used APIs that only had the number on the secondary logs, while the current log would maintain the exact name you gave it - this caused me some trouble as JUL also has no "get current log file" method to get the name of active file, so I needed to create an ugly method that predicted what the name should be based on the parameters, and hope nothing went wrong. As an aside, you cannot change the format of the generation numbers (which also caused me some issue as it was preferred to have the files number 01, 02, ... 10, 11 rather than 0,1,2,...10).
Related
I understand how the Java FileHandler rolls to the next log file when a particular size is reached. What I want is a little different. I want to use the FileHandler to use the log file with the oldest last written time in the sequence when the program starts.
For example if I have specified to use 5 log files:
mylog.0.log, mylog.1.log...mylog.4.log
If the program last updated mylog.2.log then the next time I start the program I want it to start logging to mylog.3.log.
The problem I am trying to solve is when a user executes the program and something happens they typically restart the program and if mylog.0.log is available it will always use it and not go to mylog.1.log. I lose the information from the previous execution of the program.
Per the documentation for the java.util.logging.FileHandler:
Successively older files are named by adding "0", "1", "2", etc. into the base filename.
Seems to imply that the order will always be the opposite of what you want.
Looks like your only option is to implement a config class to generate the file name you want and pass it to the FileHandler.
You can manually rollover a log file but not in the order your want.
I'm working on a project which, in part, displays all the files in a directory in a JTable, including sub-directories. Users can double-click the sub-directories to update the table with that new directory's content. However, I've run into a problem.
My lists of files are generated with file.listFiles(), which pulls up everything: hidden files, locked files, OS files, the whole kit and caboodle, and I don't have access to all of them. For example, I don't have permission to read/write in "C:\Users\user\Cookies\" or "C:\ProgramData\ApplicationData\". That's ok though, this isn't a question about getting access to these. Instead, I don't want the program to display a directory it can't open. However, the directories I don't have access to and the directories I do are behaving almost exactly the same, which is making it very difficult to filter them out.
The only difference in behavior I've found is if I call listFiles() on a locked directory, it returns null.
Here's the block of code I'm using as a filter:
for(File file : folder.listFiles())
if(!(file.isDirectory() && file.listFiles() == null))
strings.add(file.getName());
Where 'folder' is the directory I'm looking inside and 'strings' is a list of names of the files in that directory. The idea is a file only gets loaded into the list if it's a file or directory I'm allowed to edit. The filtering aspect works, but there are some directories which contain hundreds of sub-directories, each of which contains hundreds more files, and since listFiles() is O(n), this isn't a feasible solution (list() isn't any better either).
However,
file.isHidden() returns false
canWrite()/canRead()/canExecute() return true
getPath() returns the same as getAbsolutePath() and getCanonicalPath()
createNewFile() returns false for everything, even directories I know are ok. Plus, that's a solution I'd really like to avoid even if that worked.
Is there some method or implementation I just don't know to help me see if this directory is accessible without needing to parse through all of its contents?
(I'm running Windows 7 Professional and I'm using Eclipse Mars 4.5.2, and all instances of File are java.io.File).
The problem you have is that you are dealing with File. By all accounts, in 2016, and, in fact, since 2011 (when Java 7 came out), it has been superseded by JSR 203.
Now, what is JSR 203? It is a totally new API to deal with anything file systems and file system objects; and it extend the definition of a "file system" to include what you find on your local machine (the so called "default filesystem" by the JDK) and other file systems which you may use.
Sample page on how to use it: here
Among the many advantages of this API is that it grants access to metadata which you could not access before; for instance, you specifically mention the case, in a comment, that you want to know which files Windows considers as "system files".
This is how you can do it:
// get the path
final Path path = Paths.get(...);
// get the attributes
final DosAttributes attrs = Files.readAttributes(path, DosFileAttributes.class);
// Is this file a "system file"?
final boolean isSystem = attrs.isSystem();
Now, what is Paths.get()? As mentioned previously, the API gives you access to more than one filesystem at a time; a class called FileSystems gives access to all file systems visible by the JDK (including creating new filesystems), and the default file system, which always exists, is given by FileSystems.getDefault().
A FileSystem instance also gives you access to a Path using FileSystem#getPath.
Combine this and you get that those two are equivalent:
Paths.get(a, b, ...)
FileSystems.getDefault().getPath(a, b, ...)
About exceptions: File handles them very poorly. Just two examples:
File#createNewFile will return false if the file cannot be created;
File#listFiles will return null if the contents of the directory pointed by the File object cannot be read for whatever reason.
JSR 203 has none of these drawbacks, and does even more. Let us take the two equivalent methods:
File#createNewFile becomes Files#createFile;
File#listFiles becomes either of Files#newDirectoryStream (or derivatives; see javadoc) or (since Java 8) Files#list.
These methods, and others, have a fundamental difference in behaviour: in the event of a failure, they will throw an exception.
And what is more, you can differentiate what exception this is:
if it is a FileSystemException or derivative, the error is at the filesystem level (for instance, "access denied" is an AccessDeniedException);
if is is an IOException, then the problem is more fundamental.
This answer cannot contain each and every use case of JSR 203; this API is vast, very complete, although not without flaws, but it is infinitely better than what File has to offer in any case.
I faced the very same problem with paths like C://users/myuser/cookies.
I already used JSR203, so the above answer kind of didn't help me.
In my case the important attribute of those files was the hidden one.
I ended up using the FileSystemview, which excluded those files as I wanted.
File[] files = FileSystemView.getFileSystemView().getFiles(new File(strHomeDirectory), !showHidden);
I am facing this security flaw in my project at multiple places. I don't have any white-list to do a check at every occurrence of this flaw. I want to use ESAPI call to perform a basic blacklist check on the file name. I have read that we can use SafeFile object of ESAPI but cannot figure out how and where.
Below are a few options I came up with, Please let me know which one will work out?
ESAPI.validator().getValidInput() or ESAPI.validator().getValidFileName()
Blacklists are a no-win scenario. This can only protect you against known threats. Any code scanning tool you use here will continue to report the vulnerability... because a blacklist is a vulnerability. See this note from OWASP:
This strategy, also known as "negative" or "blacklist" validation is a
weak alternative to positive validation. Essentially, if you don't
expect to see characters such as %3f or JavaScript or similar, reject
strings containing them. This is a dangerous strategy, because the set
of possible bad data is potentially infinite. Adopting this strategy
means that you will have to maintain the list of "known bad"
characters and patterns forever, and you will by definition have
incomplete protection.
Also, character encoding and OS makes this a problem too. Let's say we accept an upload of a *.docx file. Here's the different corner-cases to consider, and this would be for every application in your portfolio.
Is the accepting application running on a linux platform or an NT platform? (File separators are \ in Windows and / in linux.)
a. spaces are also treated differently in file/directory paths across systems.
Does the application already account for URL-encoding?
Is the file being sent stored in a database or on the system itself?
Is the file you're receiving executable or not? For example, if I rename netcat.exe to foo.docx does your application actually check to see if the file being uploaded contains the magic numbers for an exe file?
I can go on. But I won't. I could write an encyclopedia.
If this is across multiple applications against your company's portfolio it is your ethical duty to state this clearly, and then your company needs to come up with an app/by/app whitelist.
As far as ESAPI is concerned, you would use Validator.getValidInput() with a regex that was an OR of all the files you wanted to reject, ie. in validation.properties you'd do something like: Validator.blackListsAreABadIdea=regex1|regex2|regex3|regex4
Note that the parsing penalty for blacklists is higher too... every input string will have to be run against EVERY regex in your blacklist, which as OWASP points out, can be infinite.
So again, the correct solution is to have every application team in your portfolio construct a whitelist for their application. If this is really impossible (and I doubt that) then you need to make sure that you've stated the risks cited here clearly to management and you refuse to proceed with the blacklist approach until you have written documentation that the company chooses to accept the risk. This will protect you from legal liability when the blacklist fails and you're taken to court.
[EDIT]
The method you're looking for was called HTTPUtilites.safeFileUpload() listed here as acceptance criteria but this was most likely never implemented due to the difficulties I posted above. Blacklists are extremely custom to the application. The best you'll get is a method HTTPUtilities.getFileUploads() which uses a list defined in ESAPI.properties under the key HttpUtilities.ApprovedUploadExtensions
However, the default version needs to be customized as I doubt you want your users uploading .class files and dll to your system.
Also note: This solution is a whitelist and NOT a blacklist.
The following code snippet works to get past the issue CWE ID 73, if the directory path is static and just the filename is externally controlled :
//'DIRECTORY_PATH' is the directory of the file
//'filename' variable holds the name of the file
//'myFile' variable holds reference to the file object
File dir = new File(DIRECTORY_PATH);
FileFilter fileFilter = new WildcardFileFilter(filename);
File[] files = dir.listFiles(fileFilter);
File myFile = null ;
if(files.length == 1 )
myFile = files[0];
I have batch java program. It will run every 1 hour.
I want to know what is the correct log4j appender for following cases:
1) creating a fresh log file every time when the job runs. meaning that log contains for that particular run only and should not contain old run data.
2) maintain same log file for that day. if next day , then create fresh file
Any help please.
RollingFileAppender allows you to define how often you want a fresh log to be created, if you add the append=false parameter, the log will get truncated every time it's opened,
log4j.appender.LOG=org.apache.log4j.RollingFileAppender
log4j.appender.LOG.File=/path/to/your/file
log4j.appender.LOG.layout=org.apache.log4j.PatternLayout
log4j.appender.LOG.MaxFileSize=10MB
log4j.appender.LOG.MaxBackupIndex=5
log4j.appender.LOG.append=false
You can try DailyRollingFileAppender too, which rolls every day by default, if none of those quite fit your needs you can always write your own Appender implementation by extending one of the standard implementations.
There are some restricted characters (and even full filenames, in Windows), for file and directory names. This other question covers them already.
Is there a way, in Java, to retrieve this list of forbidden characters, which would be depending on the system (a bit like retrieving the line breaker chars)? Or can I only put the list myself, checking for the system?
Edit: More background on my particular situation, aside from the general question.
I use a default name, coming from some data (no real control over their content), and this name is given to a JFileChooser, as a default file name to use (with setSelectedFile()). However, this one truncates anything prior to the last invalid character.
These default names occasionally end with dates in a "mm/dd/yy" format, which leaves only the "yy", in the default name, because "/" are forbidden. As such, checking for Exceptions is not really an option there, because the file itself is not even created yet.
Edit bis: Hmm, that makes me think, if JFileChooser is truncating the name, it probably has access to a list of such characters, can be interesting to check that further.
Edit ter: Ok, checking sources from JFileChooser shows something completely simple. For the text field, it uses file.getName(). It doesn't actually check for invalid characters, it's simply that it takes the "/" as a path separator, and keeps only the end, the "actual filename". Other forbidden characters actually go through.
When it comes to dealing with "forbidden" characters I'd rather be overcautious and ban/replace all "special" characters that may cause a problem on any filesystem.
Even if technically allowed, sometimes those characters can cause weirdness.
For example, we had an issue where the PDF files were being written (successfully) to a SAN, but when served up via a web server from that location some of the characters would cause issues when we were embedding the PDF in an HTML page that was being rendered in Firefox. It was fine if the PDF was accessed directly and it was fine in other browser. Some weird error with how Firefox and Adobe Reader interact.
Summary: "Special" characters in file names -> weird errors waiting to happen
Ultimately, the only way to be sure is to use a white-list.
Having certain "forbidden characters" is just one of many things that can go wrong when creating a file (others are access rights and file and path name lengths).
It doesn't really make sense to try and catch some of these early when there are others you can't catch until you actually try to create the file. Just handle the exceptions properly.
Have you tried using File.getCanonicalPath and comparing it to the original file name (or whatever is retrieved from getAbsolutePath)?
This will not give you the actual characters, but it may help you in determining whether this is a valid filename in the OS you're running on.
Have a look at this link for some info on how to get the OS the application is running on. Basically you need to use System.getProperty("os.name") and do an equals() or contains() to find out the operating system.
Something to be weary of though is that knowing the OS does not necessarily tell you the underlying file system being used, for example a Mac can read and write onto the FAT32 file system.
source: http://www.mkyong.com/java/how-to-detect-os-in-java-systemgetpropertyosname/