I have 5 input files in a folder, for each input file (1.in for example), I process it and produce a new output file (1.out) printing a integer in it.
How do I do this?
Right now this is what I have, I want to run different files like a2.in and print output in new file like a2.out (name them based on the input file names).
public static void main(String[] args) throws IOException {
BufferedReader br = new BufferedReader(new FileReader("a1.in"));
PrintWriter pw = new PrintWriter(new BufferedWriter(new FileWriter("a1.out")));
public int compareTo(Event a) {
return ans;
}
That depends on unit test and assertion library you are using, below is example for junit-jupiter akin junit5 and assertJ:
#ParameterizedTest
#CsvSource(delimiterString = ";", value = {"a1.in;a1.out", "a2.in;a2.out""})
void test(String input, String output) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (InputStream in = getStream(input); InputStream expected = getStream(output)) {
// test body reading from in and writing to baos
assertThat(new ByteArrayInputStream(baos.toByteArray()))
.hasSameContentAs(expected);
}
}
InputStream getStream(String resource) {
return getClass().getResourceAsStream("/" + resource);
}
UPD (Thanks Reinier for comments).
If your goal is to test #main method (i.e. you are writing CLI application, which reads and writes some files) and your #main method looks like:
public static void main(String[] args) throws IOException {
BufferedReader br = new BufferedReader(new FileReader(args[0]));
PrintWriter pw = new PrintWriter(new BufferedWriter(new FileWriter(args[2])));
// do something with br and pw
}
consider redesigning it to something like:
public static void main(String[] args) throws IOException {
main(Files.newInputStream(Paths.get(args[0])), Files.newOutputStream(Paths.get(args[1])));
}
public static void main(InputStream in, OutputStream out) throws IOException {
try (
BufferedReader br = new BufferedReader(new InputStreamReader(in));
PrintWriter pw = new PrintWriter(new BufferedWriter(new OutputStreamWriter(out);
) {
// do something with br and pw
pw.flush();
}
}
in that case the testability of #main method gets improved.
Related
I want to declare a static FileWriter and FileReader instance so I can use the same one throughout a large program. Eclipse tells me that an IOException must be thrown. How can I throw an exception for the declaration?
static FileWriter fileWriter = new FileWriter(file, true);
static BufferedWriter writer = new BufferedWriter(fileWriter);
static FileReader fileReader = new FileReader(file);
static BufferedReader reader = new BufferedReader(fileReader);
Thanks.
Try something like this:
static File file = new File("");
static FileWriter fileWriter;
static BufferedWriter writer;
static FileReader fileReader;
static BufferedReader reader;
static {
try {
fileWriter = new FileWriter(file, true);
writer = new BufferedWriter(fileWriter);
fileReader = new FileReader(file);
reader = new BufferedReader(fileReader);
} catch (final IOException e) {
throw new ExceptionInInitializerError(e.getMessage());
}
}
Just put a throws IOException after the method declaration.
i.e.: public static void main(String[] args) throws IOException { ... }
That's it.
Though it is not elegant to use it this way...
Put the initialization in a static block as follows:
public final class MyUtil {
static FileWriter fileWriter;
static BufferedWriter writer;
static FileReader fileReader;
static BufferedReader reader;
static {
try {
File file = new File("<pathToFile>");
fileWriter = new FileWriter(file, true);
writer = new BufferedWriter(fileWriter);
fileReader = new FileReader(file);
reader = new BufferedReader(fileReader);
}
catch (IOException e) {
// Handle the exception here
}
}
}
Note that this will kind of global state to a file writer/reader can cause issues in a multi-threaded environment, so you might want to make sure there is some synchronization to handle this.
I have following test program to read a file from HDFS.
public class FileReader {
public static final String NAMENODE_IP = "172.32.17.209";
public static final String FILE_PATH = "/notice.html";
public static void main(String[] args) throws MalformedURLException,
IOException {
String url = "hdfs://" + NAMENODE_IP + FILE_PATH;
InputStream is = new URL(url).openStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = br.readLine();
while(line != null) {
System.out.println(line);
line = br.readLine();
}
}
}
It is giving java.net.MalformedURLException
Exception in thread "main" java.net.MalformedURLException: unknown protocol: hdfs
at java.net.URL.<init>(URL.java:592)
at java.net.URL.<init>(URL.java:482)
at java.net.URL.<init>(URL.java:431)
at in.ksharma.hdfs.FileReader.main(FileReader.java:29)
Register Hadoop's Url handler. Standard Url handler won't know how to handle hdfs:// scheme.
Try this:
public static void main(String[] args) throws MalformedURLException,
IOException {
URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
String url = "hdfs://" + NAMENODE_IP + FILE_PATH;
InputStream is = new URL(url).openStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = br.readLine();
while(line != null) {
System.out.println(line);
line = br.readLine();
}
}
I get the same issue while writing a Java application for reading from hdfs on hadoop 2.6.
My solution is : Add
hadoop-2.X/share/hadoop/hdfs/hadoop-hdfs-2.X.jar to your classpath.
In our case we had to combine it with other answer:
https://stackoverflow.com/a/21118824/1549135
So firstly in our HDFS setup class (Scala code):
val hadoopConfig: Configuration = new Configuration()
hadoopConfig.set("fs.hdfs.impl", classOf[DistributedFileSystem].getName)
hadoopConfig.set("fs.file.impl", classOf[LocalFileSystem].getName)
And later, like in accepted answer:
https://stackoverflow.com/a/25971334/1549135
URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory)
Try(new URL(path))
Side note:
We already had:
"org.apache.hadoop" % "hadoop-hdfs" % "2.8.0" in our dependencies and it did not help.
so Im trying to write this class which is going to parse a file and read commands from it.
I want the ctor to just open the stream and do nothing else.
while I parse the file in other class methods.
But I'm getting a nullpointerexception when I try to read the file in the methods I made.
help will be apprecated :D
public class Parser {
private BufferedReader _input;
private String _command;
public Parser(String filename) throws FileNotFoundException {
FileInputStream fstream = new FileInputStream(filename);
BufferedReader _input = new BufferedReader(new InputStreamReader(fstream));
}
public boolean hasMoreCommands() throws IOException {
String line;
if ( (line = _input.readLine()) != null) {
return true;
} else {
_input.close();
return false;
}
}
public void advance() throws IOException {
String line;
do {
line = _input.readLine().trim();
} while (line.equals("") || line.substring(0,2).equals(COMMENT_SIGN));
String[] splittedLine = line.split(COMMENT_SIGN);
_command = splittedLine[0];
_command = _command.replace(" ", "");
}
my main for testing it + the exception trace
public static void main(String[] args) throws IOException {
Parser input = null;
input = new Parser("D:\\test.asm");
System.out.println( input.hasMoreCommands());
}
Exception in thread "main" java.lang.NullPointerException
at nand6.Parser.hasMoreCommands(Parser.java:40)
at nand6.Parser.main(Parser.java:116)
Have a look at this snippet of code
public Parser(String filename) throws FileNotFoundException {
FileInputStream fstream = new FileInputStream(_filename);
BufferedReader _input = new BufferedReader(new InputStreamReader(fstream));
}
Change
BufferedReader _input = new BufferedReader(new InputStreamReader(fstream));
to
_input = new BufferedReader(new InputStreamReader(fstream));
Your mistake : You are creating another local variable of type *BufferedReader _input* hence your class level variable is still null resulting in a NullPointerException
You are basically defining a new BufferedReader object called "_input" inside your constructor. You think that after calling the constructor you instantiate "_input" outside the constructor. But you are not, it refers to null. That's why you are getting NullPointerException. Just remove "BufferedReader" in front of "_input" inside your constructor, in order to refer to correct object.
What is the correct way to write from an OutputStream (url.openStream()) to a Writer (it can be a FileWriter or a Stringwriter) without loosing encoding/charset??
public static String parseURLContentIntoFile(final URL url, final File fileToFill) throws IOException{
return parseURLContentIntoOutputWriter(url, new OutputStreamWriter(FileUtils.openOutputStream(fileToFill),"UTF-8"));
}
public static String parseURLContentIntoString(final URL url) throws IOException{
final StringWriter output = new StringWriter(); // Or StringBuilderWriter
parseURLContentIntoOutputWriter(url, output);
return output.getBuffer().toString(); //Or output.getBuilder().toString()
}
private static String parseURLContentIntoOutputWriter(final URL url, final Writer writer) throws IOException{
InputStreamReader in = null;
BufferedWriter out = null;
try {
out = new BufferedWriter(writer);
in = new InputStreamReader(url.openStream(),"UTF-8");
for(String line : IOUtils.readLines(in)){ //Uses a buffer internally
(...VERY LONG parsing...)
if (!line.isEmpty()) IOUtils.write(line,out);
}
(...Exception handling and stream closing...)
}
Thanks!
I just started doing file I/O andim using an example from Murach's Se 6.
Here is my code. Am i missing something. I know the code further on has more but as this is an example this should work right?
//Import import java.io.*; for use with the File I/O Methods.
import java.io.*;
public class MainApp
{
public static void main(String[] args)
{
//Create a file object.
File productFile = new File("product.txt");
//Open a buffered output stream to allow write to file operations.
PrintWriter out = new PrintWriter(
new BufferedWriter(
new FileWriter(productFile)));
out.println("java\tMurach's Beginning Java 2\t$49.99");
out.close();
BufferedReader in = new BufferedReader(
new FileReader(productFile));
String line = in.readLine();
System.out.println(line);
out.close();
}
}
//Answer
by adding a throws exception to the end of where i initialised the main this code works. Even the txt file products.txt is in the class folder as expected.
//Import import java.io.*; for use with the File I/O Methods.
import java.io.*;
public class MainApp
{
public static void main(String[] args) throws Exception
{
//Create a file object.
File productFile = new File("product.txt");
//Open a buffered output stream to allow write to file operations.
PrintWriter out = new PrintWriter(
new BufferedWriter(
new FileWriter(productFile)));
out.println("java\tMurach's Beginning Java 2\t$49.99");
out.close();
BufferedReader in = new BufferedReader(
new FileReader(productFile));
String line = in.readLine();
System.out.println(line);
out.close();
}
}
The problem is that a number of the calls to the java.io package throw exceptions.
easy fix: add the following to your method signature
public static void main(String[] args) throws IOException
almost as easy fix: add try/catch/finally blocks.
public static void main(String[] args)
{
//Create a file object.
File productFile = new File("product.txt");
//Open a buffered output stream to allow write to file operations.
PrintWriter out = null;
try {
out = new PrintWriter(
new BufferedWriter(
new FileWriter(productFile)));
out.println("java\tMurach's Beginning Java 2\t$49.99");
}
catch(IOException ex) {
// todo exception handling
System.out.println("ERROR! " + ex);
}
finally {
out.close();
}
BufferedReader in = null;
try {
in = new BufferedReader(
new FileReader(productFile));
String line = in.readLine();
System.out.println(line);
}
catch (IOException ex) {
// todo more exception handling
System.out.println("ERROR! " + ex);
}
finally {
in.close();
}
}
edit: you know you are trying to call out.close() twice? The second should be a call to in.close()