I am using a EMC Documentum content management system. I am trying to automate some file importing (moving files to CMS repository) with the help of their own API. (Dont panic by this next sentence, just read it, since its EMCs' own API that you may be unaware of) For that, I have to first create an object of type IDfFile and then pass that object to IDfImportNode.add() which performs importing.
What I want to do is to fetch the file at runtime from one server and immediately perform import operation. I want to do all this in-memory, without saving fetched files on the disk - since those files are confidential.
However the problem is that IdfFile(string) takes absolute path of the file to be imported. So the file has to exist on the disk physically, which will eventually leave traces of files on disk even after I delete files after import. I was guessing if it can take Stream object, but there is no such overload.
So I want to know if I can encrypt files before saving to disk or any other way out of it. Or that I request EMC people to provide suitable API method.
If the files are truly confidential, and you really want to go through so much trouble to keep them that way, you should have encrypted them in the first place.
The files leaving traces on disk should be the least of your worries, one would need physical access to the disk to perform deep sleuthing on it to find out what the files were. Much more realistic problems are attackers gaining access to the server and reading the files like that.
But to still answer your question:
Encryption could be a solution to a lot of things, but always consider if it's worth it.
You could always ask the "EMC people" for a solution to this, of course, especially if you have commercial support
for leaving traces of the file on the disk, it's easy: encrypt the disk. Take some solution like Truecrypt, encrypt the whole disk, mount it, run your program. Everything that is saved in the disk will be encrypted, but it will be transparent to your program (it will only read and write to a device, and won't have to bother if the file is stored encrypted or not).
The problems are:
if someone has access to your computer while it is doing that, he can read the files
if your computer is hacked somehow and can be accessed from the outside, he can read the files
The way to do this is to use the IDfSysObject.setContent() method. It is going to be more code, because you can't use the Import operation conveniences, but it should allow you to save a stream. Maybe something like this (but you would already have the stream from somewhere):
File pdfInput = new File("C:\\Rupinder\\MyFile.txt");
byte[] outBytes = new byte[(int)pdfInput.length()];
FileInputStream fileInputStream = new FileInputStream(pdfInput);
fileInputStream.read(outBytes);
ByteArrayOutputStream out = new ByteArrayOutputStream();
out.write(outBytes);
IDfSysObject sysObj = (IDfSysObject)session.newObject("c_pdf");
sysObj.setObjectName("testDoc");
sysObj.setContentType("crtext");
sysObj.setTitle("import operation");
sysObj.link("/Temp/Source Folder");
sysObj.setContent(out);
sysObj.save();
source: https://community.emc.com/message/98225
Related
There's a file I wanted to get into, but whenever I try to open it I get the message "The process cannot access the file because it is being used by another process".
Well, I want in! So, how can i do it?
I've been brainstorming a few ways to try, I'm hoping to get some input on other ways, or if my ideas wouldn't work for some reason that is not apparent to me.
Idea 1 The folder knows where the file is, it just won't open it. What if I create a program to read from the memory address of the file, copy it, then rebuild it somewhere else? I'm not sure if this has hope, because it relies on the file being the issue.
Idea 2 How does my process know that another process is using the file? If it's checking against all the other processes, maybe I can also figure out which process is using that file and pause it or end it.
Either of these ideas will probably take me weeks. Is anyone more creative and can think of another way; or more knowledgeable and eliminate an impractical idea?
In Windows, applications are allowed to obtain exclusive locks on files. When the process opens the file, one thing you specify is who else can access it while your process does (those are the .NET methods, but equivalents exist in other languages). Excel, for example, is notorious for getting an exclusive lock when you open a file. The way around it is usually to find the offending process and kill it to break the lock. Unlocker is the app that I'm most familiar with to accomplish this. If the process is a System process, however, you may not be able to kill it. You'd have to reboot to reset the lock.
Reading directly from another process's memory is unlikely to be reliable. The application may not have an in-memory copy, may not have a complete in memory copy, may not have a consistent in memory copy, and may not have an in memory copy that matches what's on disk (If they're editing the document, for example).
Your process knows that the file is locked because when it tries to open the file, it does so by asking the operating system for access to the file. The operating system responds saying, "Request denied. Another process has this file open and locked." The OS doesn't tell your process what process has the file open because trying to open a file doesn't include asking for who already has it open. Your process must ask the right question to get the answer you're looking for.
Windows makes you specify a sharing modes when opening a file. The sharing mode may prevent the file from being read, written, or deleted while you have it open. If you want to allow simultaneous read access you should include FILE_SHARE_READ in the dwShareMode parameter when you call CreateFile (http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx).
In other words, if you want to enable concurrent access to an open file you must modify the way the file is opened in the first place.
The portable standard libraries in C and Java don't offer a way to set the sharing mode when opening a file, but their usual implementations on windows set the sharing mode to READ+WRITE.
Is there any way in Java to write out to a temporary file securely?
As far as I can tell, the only way to create a temporary file (createTempFile) does't actually open it at the same time, so there's a race condition between file open & file write. Am I missing something? I couldn't find the C source code behind createFileExclusively(String) in UnixFileSystem.java, but I doubt it can really do anything since the file open occurs in the Java code after the temp file is created (unless it tries to do something with file locks?).
The problem
Between when the temporary file is created & you open it, a malicious attacker could unlink that temporary file & put malicious stuff there. For example, an attacker could create a named pipe to read sensitive data. Or similarly if you eventually copy the file by reading it, then the named pipe could just ignore everything written & supply malicious content to be read.
I remember reading of numerous examples of temporary file attacks in the past 10+ years that exploit the race condition between when the name appears in the namespace and when the file is actually opened.
Hopefully a mitigating factor is that Java set's the umask correctly so a less-privileged user can't read/write to the file and typically the /tmp directory restricts permissions properly so that you can't perform an unlink attack.
Of course if you pass a custom directory for the temporary file that's owned by a less-privileged user who's compromised, the user could do an unlink attack against you. Hell, with inotify, it's probably even easier to exploit the race condition than just a brute force loop that does a directory listing.
http://kurt.seifried.org/2012/03/14/creating-temporary-files-securely/
Java
use java.io.File.createTempFile() – some interesting info at http://www.veracode.com/blog/2009/01/how-boring-flaws-become-interesting/
for directories there is a helpful posting at How to create a temporary directory/folder in Java?
Java 7
for files use java.io.File.createTempFile()
for directories use createTempDirectory()
http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html
Since Java 7 we have OpenOption.
An object that configures how to open or create a file.
Objects of this type are used by methods such as newOutputStream, newByteChannel, FileChannel.open, and AsynchronousFileChannel.open when opening or creating a file.
Of particular interest is StandardOpenOptions.CREATE_NEW.
Create a new file, failing if the file already exists. The check for the existence of the file and the creation of the file if it does not exist is atomic with respect to other file system operations.
So, you can do something like this:
FileChannel mkstemp() {
Path path = Files.createTempFile(null, null);
Files.delete(path);
return FileChannel.open(path, WRITE, CREATE_NEW);
}
Implementing the same template behaviour is left as exercise to the reader.
Keep in mind that on many systems, just because a file doesn't have a name doesn't at all mean it's inaccessible. For example, on Linux open file descriptors are available in /proc/<pid>/fd/<fdno>. So you should make sure that your use of temporary files is secure even if someone knows / has a reference to the open file.
You might get a more useful answer if you specify exactly what classes of attacks you are trying to prevent.
Secure against other ordinary userid's? Yes, on any properly functioning multi-user system.
Secure against the your own userid or the superuser? No.
I have file reader object in Java.
When we load the file in, is entire file loaded or only file pointer will be loaded like in C. Because I have application (producer) wherein it keeps on writing to the file and other application (consumer) will start to read the file after few minutes, I want to read all file data which will be written by the application producer.
I tried to search but could not get the answer.
Best and canonical approach would be to use Java NIO memory mapped files. Use MappedByteBuffer.force() on each write to insure the consumer process flushes the dirty page files. Consumers (in other process if necessary) can map the same file in read mode.
A FileReader works a lot like a FileInputStream, which in turn works a lot like its C analogue -- when you first open a file, very little data will actually be loaded immediately. The file's contents generally aren't read in til you first read some of it, at which point the runtime reads enough to fill the request, or the buffer if you're using a BufferedReader (possibly plus enough to fill up a block, depending on the OS and JVM).
When i say it "works like its C analogue", though, i mean it. A FileReader is opened for reading, and a FileWriter is opened for writing. There could be issues with having a file open for reading and writing at the same time. Windows, in particular, isn't too fond of it. If you care about portability, you may want to find another way -- alphazero's answer sounds promising, but i've never tried it.
I don't understand what you mean when you wrote, "when we load the file in". Sample code would be helpful here. Please see How to Ask.
If you've called anything that sounds like "read" on your Reader then you've got data in memory. If this is text data (assuming you're using FileReader) then you're better off using a BufferedReader. Not only is it buffered, it's more convenient for text files.
If I have the answer all wrong then please post some code so I know what you're talking about. And welcome!
I am developing a Java Desktop Application. This app needs a configuration to be started. For this, I want to supply a defaultConfig.properties or defaultConfig.xml file with the application so that If user doesn't select any configuration, then the application will start with the help of defaultConfig file.
But I am afraid of my application crash if the user accidentally edit the defaultConfig file. So Is there any mechanism through which I can check before the start of the application that whether the config file has changed or not.
How other applications (out in the market) deal with this type of situation in which their application depends on a configuration file?
If the user edited the config file accidentally or intentionally, then the application won't run in future unless he re-installs the application.
I agree with David in that using a MD5 hash is a good and simple way to accomplish what you want.
Basically you would use the MD5 hashing code provided by the JDK (or somewhere else) to generate a hash-code based on the default data in Config.xml, and save that hash-code to a file (or hardcode it into the function that does the checking). Then each time your application starts load the hash-code that you saved to the file, and then load the Config.xml file and again generate a hash-code from it, compare the saved hash-code to the one generated from the loaded config file, if they are the same then the data has not changed, if they are different, then the data has been modified.
However as others are suggesting if the file should not be editable by the user then you should consider storing the configuration in a manner that the user can not easily edit. The easiest thing I can think of would be to wrap the Output Stream that you are using to write the Config.xml file in a GZIP Output Stream. Not only will this make it difficult for the user to edit the configuration file, but it will also cause the Config.xml file to take up less space.
I am not at all sure that this is a good approach but if you want to go ahead with this you can compute a hash of the configuration file (say md5) and recompute and compare every time the app starts.
Come to think of it, if the user is forbidden to edit a file why expose it? Stick it in a jar file for example, far away from the user's eyes.
If the default configuration is not supposed to be edited, perhaps you don't really want to store it in a file in the first place? Could you not store the default values of the configuration in the code directly?
Remove write permissions for the file. This way the user gets a warning before trying to change the file.
Add a hash or checksum and verify this before loading file
For added security, you can replace the simple hash with a cryptographic signature.
From I have found online so far there seems to be different approaches code wise. none appear to be a 100 hundred percent fix, ex:
The DirectoryWatcher implements
AbstractResourceWatcher to monitor a
specified directory.
Code found here twit88.com develop-a-java-file-watcher
one problem encountered was If I copy
a large file from a remote network
source to the local directory being
monitored, that file will still show
up in the directory listing, but
before the network copy has completed.
If I try to do almost anything non
trivial to the file at that moment
like move it to another directory or
open it for writing, an exception will
be thrown because really the file is
not yet completely there and the OS
still has a write lock on it.
found on the same site, further below.
How the program works It accepts a ResourceListener class, which is FileListener. If a change is detected in the program a onAdd, onChange, or onDelete event will be thrown and passing the file to.
will keep searching for more solutions.
Background:
Our software generates reports for customers in the usual suspect formats (HTML, PDF, etc.) and each report can contain charts and other graphics unique to that report. For PDFs everthing is held in one place - the PDF file itself. HTML is trickier as the report is basically the sum of more than 1 file. The files are available via HTTP through Tomcat.
Problem:
I really want to have a tidy environment and wrap the HTML reports into a single file. There's MTHML, Data URIs, several formats to consider. This excellent question posits that, given the lack of cross-broser support for these formats, ZIP is a neat solution. This is attractive to me as I can also offer the zip for download as a "HTML report you can email" option. (In the past users have complained about losing the graphics about when they set about emailling HTML reports)
The solution seems simple. A request comes in, I locate the appropriate zip, unpack it somewhere on the webserver, point the request at the new HTML file, and after a day or so tidy everything up again.
But something doesn't quite seem right about that. I've kind of got a gut feeling that it's not a good solution, that there's something intrisically wrong with it, or that maybe a better way exists that I can't see at the moment.
Can anyone suggest whether this is good or bad, and offer an alternative solution?
Edit for more background information!
The reports need to persist on the server. Our customers are users at sites, and the visibility of a single report could be as wide as everyone at the site. The creation process involves the user selecting the criteria for the report, and submitting it for creation to the server. Data is extracted from the database and a document built. A placeholder record goes into the database, and the documents themselves get stored on the fileserver somewhere. It's the 'documents on the fileserver' part that I'd like to be tidier - zipping also means less disk space used!. Once a report is created, it is available to anyone who can see it.
I would have thought the plan would be that the zip file ends up on the client rather than staying on the server.
Without knowing about your architecture, I would guess at an approach like this:
User requests report
Server displays report as HTML
User perhaps tweaks some parameters, repeats request
Server displays report as HTML (repeat until user is happy)
On each of the HTML reports, there's a "download as zip" link
User clicks on link
Server regenerates report, stores it in a zip file and serves it to the user
User saves zip file somewhere, emails it around etc - server isn't involved at all
This relies on being able to rerun the report to generate the zip file, of course. You could generate a zip file each time you generate some HTML, but that's wasteful if you don't need to do it, and requires clean-up etc.
Perhaps I've misunderstood you though... if this doesn't sound appropriate, could you update your question?
EDIT: Okay, having seen the update to your question, I'd be tempted to store the files for each report in a separate directory (e.g. using a GUID as the directory name). Many file systems support compression at the file system level, so "premature zipping" probably wouldn't save much disk space, and would make extracting individual files harder. Then if the user requests a zip, you just need to build the zip file at that point, probably just in memory, before serving it.
Once a report is created, it is
available to anyone who can see it.
that is quite telling - it means that the reports are sharable, and you also would like to "cache" reports so that it doesnt have to be regenerated.
one way to do this would be to work out a way to hash the parameters together, in such a way that different parameter combinations (that result in different a report) hash to different values. then, you can use those hash as a key into a large cache of reports stored in disk in zip (may be the name of the file is the hash?)
that way, every time someone requests a report, you hash the parameters, and check if that report was already generated, and serve that up, either as a zip download, or, you can unzip it, and serve up the html as per normal. If the report doesnt exist, generate it, and zip it, make sure to be able to identify it later on as being produced by these parameters (i.e., record the hash).
one thing to be careful of is that file system writes tends to be non-atomic, so if you are not careful, you will regenerate the report twice, which sucks, but luckily in your case, not too harmful. to avoid, you can use a single thread to do it (slower), or implement some kind of lock.
You dont need to physically create zip files on a file system. Theres nothing wrong with creating the zips in memory, stream it to the browser and let GC take care of releasing the memory taken by the temporary zip. This of course introduces problems as it could be potentially ineffecient to continnally recreate the zip each time a request is made. However judge these things according to your needs and so on.