Not able to open HDFS file from mapper Hadoop - java

I searched a lot but failed to find a solution to this problem.
Actually the file I want to access is in HDFS, but not in input path (the path which was input to the map/reduce job). And I want to access it from mapper.
The hdfs path specified in the input path is perfectly accessible from mapper but the other hdfs files are not.
INside mapper:-
FileSystem FS1=FileSystem.get(conf);
Path path=new Path(""+FS1.getHomeDirectory());
FSDataInputStream fsdis=FS1.open(path);
RESULTS IN the following ERROR:
java.io.IOException : Cannot open filename /user/hadoop
Thanks in advance,
Harsh

I remember using this tutorial to get something similar working. You can give it a try, it has only a few difference tho what you've written but still it might help...
#Edit: ah and I just noticed (after reading the comments) that you are trying to open FS1.getHomeDirectory() and that is a directory. You should point out to a file not a directory, I think (you can check it out in the linked tutorial under "Reading data from a file").

can u try this once
try {
FileSystem fs = FileSystem.get (new Configuration ());
FileStatus[] status = fs.listStatus (new Path ("hdfs://jp.seka.com:9000/user/jeka/in"));
for (int i=0;i < status.length;i++) {
BufferedReader br = new BufferedReader (new InputStreamReader (fs.open (status[i].getPath())));
String line;
line = br.readLine();
while (line != null) {
System.out.println (line);
line=br.readLine ();
}
}
} catch (Exception e) {
System.out.println("File not found");
}

Related

JAVA - File I/O cant get file to read

I'm trying to get inventory.csv to load in Eclipse, and I'm not sure where I'm supposed to put the location of the file (Example: c:\\Users\\...) or if I even need it, considering it's in the same folder. I recieved an "unable to load inventory.csv." output. My driver finishes the rest of the program with no errors afterwards.
public int readInventory(Automobile[] inventory)
{
final String FILENAME = "inventory.csv";
int size = 0;
Scanner input = new Scanner("inventory.csv");
try
{
input = new Scanner(new File(FILENAME));
}
catch (FileNotFoundException e)
{
System.out.println("Unable to open file " + FILENAME + ".");
}
// ...
return size;
}
Here is the output I'm getting with no syntax errors.
Unable to open file inventory.csv.
considering it's in the same folder.
Same folder as what? inventory.csv was not present in the current working directory when you executed your program.
If you are trying to read a CSV file from a single Java program then directly keep your file in the same location where the class was created.
In eclipse, keep the CSV file directly under your project directory. As you can see, the Employee.xml file directly under JavaPrac project.enter image description here
You should try this
try{
FileReader fr=new FileReader(filename);
BufferedReader br =new BufferedReader(fr)
String lime=br.readLine();
br.close();
catch(Exeption e){}

Java FileNotFoundException although file is there

I set up the method to try/catch this error. My issue is that it catches the fileNotFoundException for Trivia.txt even when I explicitly created Trivia.txt in the same package. I can't figure out why the file is not being found. I did some looking around for the answer to my problem, and had no luck. Anyway, here's my code
public static void readFile(){
try{
File file = new File("Trivia.txt");
FileReader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
while((line = br.readLine()) != null){
System.out.println(line);
}
br.close();
}
catch(FileNotFoundException e){
System.out.println("file not found");
System.out.println();
}
catch(IOException e){
System.out.println("error reading file");
}
}
The code here is just a method of the TextHandler class that is called statically by WindowComp class (totally unrelated class). The package is mainPackage which holds the main() and WindowComp() and textHandler() alond with Triva.Txt
The way you open the file, it's supposed to be found in the current working directory, not in the subdirectory your source is found.
Try System.out.println(file.getCanonicalPath()) in order to find out where the code is expecting the file.
Try loading your file as a Resource, like this
URL fileURL = this.getClass().getResource("Trivia.txt");
File file = new File(fileURL.getPath());
This will load your file from the same package of the class who loads the resource.
You can also provide an absolute path for your file, using
URL fileURL = this.getClass().getResource("/my/package/to/Trivia.txt");
If the file is found somewhere away from the current package of the class, you can also provide an absolute path directly to the constructor:
File file = new File("path/to/file/Trivia.txt");
You can also use a different constructor such as the one indicated in this answer:
Java - creating new file, how do I specify the directory with a method?
For further information, there's always the documentation:
https://docs.oracle.com/javase/7/docs/api/java/io/File.html

AndroidStudio can't find CSV file. Tried Absolute File Path & Relative File Path

I've created a simple Android app that displays text to the user.
Now I'm attempting to implement a CSVReader to retrieve text from a CSV file. After hours of trying different things.
I finally implemented an open source CSVReader (At least it's not giving me any compile errors anymore).
Now when I run the app, it crashes and I get a "file not found" exception. Either I'm not placing my CSV file in the correct location, I'm not referencing the correct file path, or both.
I've tried using the absolute file path (ex. starting with my C:/Users/Tim/AndroidStudioProjects/...).
I've also tried using a relative path starting with the Assets folder (ex. "Assets/SOCIAL_POSTS.csv") and nothing I've tried has worked.
I've tried looking for similar Questions on Stack Overflow, and I've tried several variations of file paths and nothing has worked so far. What do you think I should try next?
Here is a link to this project on GitHub.
The code pointing to the CSV file is under
app > src > main > java > com > example > tim > inspiredquestions
The CSV file is called SOCIAL_POSTS.csv and it is under
Assets > SOCIAL_POSTS.csv
Final note: I've used StackOverflow for debugging help for a year now, but this is the first question I've asked.
Thank-you for your patience! I'm trying to be as self-reliant as I can, but I'm going on a limb and asking for help. I'm sure this problem has a simple answer I'm overlooking.
I finally figured out the answer. I added
final CSVReader csvReader = new CSVReader(new InputStreamReader(c.getAssets().open("SOCIAL_POSTS.csv") to my load function in Game.java after looking at the answer to this question. Previously I had put my CSV file in an assets folder under app/src/main but I didn't know how to access the file properly until reviewing that SO post.
How I use Android Studio for files is to position my files in the raw folder, then push them to the /data/data/yourapplication/files directory. I do this through Tools/Android/Android Device Monitor and navigate to the /data/data/yourapplication/files directory and select an icon in the upper right that says "Push a file onto the device", then navigate to the raw files in my app and select the csv file(s) I want placed in the /files directory.
From that point I aim my stream readers at that directory,
in my case, /data/data/app/com.android/example/darrell/MenuPlanner/files Directory. Files addressed here use the file extension and do not need the path specified:
public List<EntreeData> ReadEntreesFromFilesDir(Context inContext) {
this.mContext = inContext;
List<String> lines = new ArrayList<String>();
String[] separated;
try {
FileInputStream fis = mContext.openFileInput("entrees.csv");
InputStreamReader isr = new InputStreamReader(fis);
BufferedReader bufferedReader = new BufferedReader(isr);
while ((mLine = bufferedReader.readLine()) != null) {
if (mLine == null) break;
lines.add(mLine);
}
fis.close();
isr.close();
} catch (FileNotFoundException e) {
Log.d("FILE ERROR", e.getMessage() + " not found");
} catch (IOException ioe) {
Log.d("FILE ERROR", ioe.getMessage());
}
for (int x = 0;x<lines.size();x++){
separated = lines.get(x).split(",");
EntreeData thisEntree = new EntreeData();
thisEntree.setEntreeName(separated[0].trim());
thisEntree.setCategory(separated[1].trim());
thisEntree.setSubcategory(separated[2].trim());
thisEntree.setRecipe(separated[3].trim());
mEntreeList.add(thisEntree);
}
return mEntreeList;
}
As a fallback I read from /res/raw directory which is not writeable and I am assured of file integrety. Files here do not use an extension, so point your stream reader at a file without using the extension name.
Also use the asset manager to access these files:
public List<EntreeData> ReadEntreesFileFromRaw(Context inContext) {
this.mContext = inContext;
List<String> lines = new ArrayList<String>();
String[] separated;
AssetManager assetManager = mContext.getAssets();
try {
InputStream inputStream = mContext.getResources().openRawResource(R.raw.entrees);
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
int lineCount = 0;
while ((mLine = reader.readLine()) != null) {
separated = mLine.split(",");
EntreeData entreeEntry = new EntreeData();
if (mLine == null) break;
separated = mLine.split(",");
entreeEntry.setEntreeName(separated[0].trim());
entreeEntry.setCategory(separated[1].trim());
entreeEntry.setSubcategory(separated[2].trim());
entreeEntry.setRecipe(separated[3].trim());
mEntreeList.add(lineCount, entreeEntry);
lineCount++;
}
inputStream.close();
} catch (FileNotFoundException e) {
Log.d("FILE ERROR", e.getMessage() + " not found");
} catch (IOException ioe) {
Log.d("FILE ERROR", ioe.getMessage());
}
return mEntreeList;
}
Good luck and happy programming!

(Java) BufferedReader returning null when reading a file that isn't empty

Each file has one line with one letter. Why are both are returning null?
File saveFile = new File("saved.txt");
File pocFile = new File("playerOrComputer.txt");
if (!pocFile.exists()) {
pocFile.createNewFile();
}
if (!saveFile.exists()) {
saveFile.createNewFile();
}
BufferedReader brPoC = new BufferedReader(new FileReader(pocFile));
BufferedReader brSave = new BufferedReader(new FileReader(saveFile));
String savedChar = brSave.readLine();
brSave.close();
String playerOrComputerChar = brPoC.readLine();
brPoC.close();
System.out.println(savedChar);
System.out.println(playerOrComputerChar);
Try the while loop while reading using BufferedReader:
while ((savedChar = brSave.readLine()) != null) {
System.out.println(savedChar);
}
If your files Contains text it will definitely show:
Do not also leave the BufferedReader Open. Once you are done reading Close it:
if (brSave != null)br.close();
I suspect the below lines.
if (!pocFile.exists()) {
pocFile.createNewFile();
}
if (!saveFile.exists()) {
saveFile.createNewFile();
}
you have checked if not exists you are creating new file. I think it creates a new file.
Try giving complete location path of file File saveFile = new File("C:\\testing.txt");
And you can do this
if (!saveFile.exists()) {
System.out.println("File doesn't exits! Creating new file..");
saveFile.createNewFile();
}
Either try with the absolute path or if you are considering these files as part of your application, I mean if put in source folder, then you may use:
BufferedReader brPoC = new BufferedReader(new InputStreamReader(getClass().getResourceAsStream("saved.txt"));
BufferedReader brSave = new BufferedReader(new InputStreamReader(getClass().getResourceAsStream("playerOrComputer.txt"));
try giving the absolute path in the File constructor and in the absolute path replace \ by \\ in order to get rid of the escape sequence
Why are both are returning null?
With respect to you,both files contain some data.The IDE not able to find the files from location.
try getAbsolutePath() will return the exact path of your file.
saveFile.getAbsolutePath();
Its always better to null check while using readLine()

Java.io.FileNotFoundException using File object

File read = new File("Numbers.txt");
Scanner inputFile = new Scanner(read);
while(inputFile.hasNext())
{
sum = inputFile.nextDouble() + sum;
count++;
}
inputFile.close();//close the input file
I'm trying to read data out of the text file Numbers.txt and the following code compiles fine but I get the Java.io.FileNotFoundException error when the program runs. I've also tried entering the full file path but I might have done it wrong. Any ideas?
Make Sure your text file is in the folder with your java file
because you used the direct path .
and try this code check, if still not working .
BufferedReader read = new BufferedReader(new FileReader("yourTextFile.txt"));
String line = read.readLine();
while(line !=null)
{
System.out.println(line);
line=read.readLine();
}
}catch(Exception ex)
{System.out.println(ex.getMessage());}
Try adding
System.out.println("Full path is " + read.getCanonicalPath()
+ ", canRead=" + read.canRead()
+ ", exists=" + read.exists());
and then see whether the full path exists on your file system, and whether it is readable according to canRead.
If the file is a symlink, canRead might return true in that the symlink is resolvable even though the file to which the link points is unreadable. To deal properly with symlinks you really need to use the new java.nio.file APIs.

Categories

Resources