Develop Reference

copy the contents of a local directory to a directory in hdfs

I have got a requirement that I should be able to copy the contents of a directory from local system to a directory on HDFS.
The condition is only the directory contents should be copied to the location I have specified, not the source directory itself. Using command copyfromlocal I can achieve this. But I need to use Java. There is this method copyFromLocalFile which should be used for making a copy from local file system, the problem is it copies the directory itself. Also tried using FileUtils.copy method, gives the same result as the copyFromLocalFile
As a test I tried to copy directory contents from a test directory to another directory , both on the local file system. I used FileUtils.copyDirectory. This works but I cannot use it for HDFS. I have seen many links to related to this same question but could not find any way.
Guys, could you please let me know if this is possible or not or is it some design flaw? If this is possible how can I proceed ?

Ya it's really hard to get things done with FileSystem api exactly the way you want. Insufficient documentation makes things even harder(Too many methods, little explanation). I faced the same problem an year ago.
The only solution I got is iterative :
void copyFilesToDir(String srcDirPath, FileSystem fs, String destDirPath)
throws IOException {
File dir = new File(srcDirPath);
for (File file : dir.listFiles()) {
fs.copyFromLocalFile(new Path(file.getPath()),
new Path(destDirPath, file.getName()));
}
}
I think the code needs little explanation.

Why fopen gets fail in native method due to permission issue from Android-NDK application?

see i have following code in one native call
errno = 0;
FILE *fp;
fp = fopen("jigar.txt","wb");
if(fp == NULL)
__android_log_print(ANDROID_LOG_ERROR, APPNAME, "FOPEN FAIL with %d",errno);
else
__android_log_print(ANDROID_LOG_ERROR, APPNAME, "FOPEN pass ");
which gets fail and shows
FOPEN FAIL with 30
now here 30 means it shows error
#define EROFS 30 /* Read-only file system */
In MainFest file on my application i have added this line
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" >
still i am getting this error..
How to resolve this issue?
Edit
thanks by specifies folder name it works But
In my case i have one Library which has such fopen() call where file names i can not give. It takes it default filename.
And i am using this Library in my ndk application so in this case how to solve this?

By default when you are running your App, it runs in its own context. fopen("jigar.txt","wb"); will try to open the file in current directory, which i think mostly would be /data/.. So you cant create files like that. Instead, if you want a folder inside /data/ you can call a function gteDir() and inside it you can create your own files. Ok this is all for general information.
Coming to your problem, as mentioned above you need to give absolute path to create a file in a different directory. This is the same case even in Linux.
For the Library thing, you can do two things.
1) Make changes in the library source code and compile it again using NDK. While making changes, give some string as the argument for fopen() which you will pass it while executing the Application.
2) remount the filesystem in which your library is creating the file, then do chmod 777 to the specific directory inside which your file is being created. Now execute the Application. It should work. But this is not generic. If you are doing it for some testing purpose then this solution is simplest and time saving...
If stuck somewhere, let me know..

You should specify folder where to write the "jigar.txt" file. Like this:
fp = fopen("/sdcard/figar.txt", "wb");

yuppi finally i got that
i have did this way
first change the current directory of my process from "/" to "/sdcard/"
chdir("/sdcard/");
and then it find path with respect to /sdcard/
and everything WORKING..!!!

Java IO issue when using SVN versioning system

I have a Java program that is supplied a directory name, gets a list of all the file in that directory using dirName.listFiles() and then iterates through every file parsing information from them.
The files would normally all just be normal text files, but I am using SVN and there seems to be a directory called .svn in my dirName directory which is causing my program to fail because .svn is a directory and not a text file.
Now, I could implement filters using a FileFilter object, but I would really only expect text files to be in that directory in the final program.
My question is: Is there a way round my issue without using a FileFilter? I also think that my program is ignoring the .svn directory in other programs that I've written, so I'm not sure why it's an issue now.
Thanks in advance.

You would have this issue with many version control systems (not just SVN) as some of them have files on disk that help identify where the working copy comes from (.svn for SVN, view.dat for clearcase). You really should just implement a FileFilter to exclude those, or use the ones from commons-io:
makeSvnAware
It's null safe, so if you give it null input, it simply returns an svn filter for you. If you give it another IOFileFilter (a subinterface of FileFilter) it simply returns one that does an AND between the existing filter and the svn filter.
FileFilter svnFilter = FileFilterUtils.makeSvnAware(null);

You could call isDirectory() on each object that listFiles() returns.

Two possible soulutions (at least):
FileFilter or FileNameFilter
isFile()
Look here: http://docs.oracle.com/javase/6/docs/api/java/io/File.html

Better than using java for file and directory search, i would prefer writing a jni program and use C's dirent.h and stat.h to differentiate between files. The jni program would be much faster.

If dirName is not the root directory of your working copy, you can upgrade to the latest version of svn. This doesn't have an .svn directory for every directory but only for the root.

Java .isDirectory() return true also on file without extension

I've noticed that checking if an instance of a File class is a Directory with the method isDirectory() this method return true also for file without an extension .
For example if i have a file name myfile without exetension isDirectory() method return true, is there another method that check if the file passed is really a directory ?
I'm trying on Windows ...

Works properly on my Windows7, JDK 6. Try upgrading your JRE/JDK and double-check if you don't have a directory with the same name. I used this snippet to check:
File file = new File("C:\\tomcat\\apache-tomcat-7.0.20\\LICENSE");
System.out.println(file.isDirectory()); // prints false

I've tried Windows 7 32 bit, 64 bit with JDK 6 and 7 and on both of those it works absolutely fine for me. Can't find any bug reports of this nature around either, what code are you using? I very much doubt this is a bug that's gone undiscovered up until now!

Perhaps the directory contains both a file with extension and a directory with the same name but without extension. Then configure the Windows file explorer to hide the extension, think that that is displayed is the real and full name of the file, put that into some test code ... Bingo.

Netbeans Built .jar doesn't work with class file inside

I had problems while finding the path of file(s) in Netbeans..
Problem is already solved (checked answer).
Today I noticed another problem: When project is finished,
I have to execute the generated .jar to launch the program, but it doesn't work because an error occurs: NullPointer (where to load a file) when accessing/openning jar outside Netbeans.
Is it possible to open a file with the class file in Java/Netbeans which works in Netbeans and even in any directory?
I've found already some threads about my problem in site but none was helpful.
Code:
File file = new File(URLDecoder.decode(this.getClass().getResource("file.xml").getFile(), "UTF-8"));

The problem you have is that File only refer to files on the filesystem, not files in jars.
If you want a more generic locator, use a URL which is what getResource provides. However, usually you don't need to know the location of the file, you just need its contents, in which case you can use getResourceAsInputStream()
This all assumes your class path is configured correctly.

Yes, you should be able to load a file anywhere on your file system that the java process has access to. You just need to have the path explicitly set in your getResource call.
For example:
File file = new File(URLDecoder.decode(this.getClass().getResource("C:\\foo\\bar\\file.xml").getFile(), "UTF-8"));