I'm trying to webget some bz2 files from Wikipedia, I don't care whether they are save as bz2 or unpacked, since I can unzip them locally.
When I call:
public static void getZip(String theUrl, String filename) throws IOException {
URL gotoUrl = new URL(theUrl);
try (InputStreamReader isr = new InputStreamReader(new BZip2CompressorInputStream(gotoUrl.openStream())); BufferedReader in = new BufferedReader(isr)) {
StringBuffer sb = new StringBuffer();
String inputLine;
// grab the contents at the URL
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine + "\r\n");
}
// write it locally
Wget.createAFile(filename, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
I get a part of the unzipped file, never more than +- 883K.
When I don't use the BZip2CompressorInputStream, like:
public static void get(String theUrl, String filename) throws IOException {
try {
URL gotoUrl = new URL(theUrl);
InputStreamReader isr = new InputStreamReader(gotoUrl.openStream());
BufferedReader in = new BufferedReader(isr);
StringBuffer sb = new StringBuffer();
String inputLine;
// grab the contents at the URL
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine);// + "\r\n");
}
// write it locally
Statics.writeOut(filename, false, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
I get a file of which the size is the same as it suppose to (compared to the KB not B). But also a message that that the zipped file is damaged, also when using byte [] instead of readLine(), like:
public static void getBytes(String theUrl, String filename) throws IOException {
try {
char [] cc = new char[1024];
URL gotoUrl = new URL(theUrl);
InputStreamReader isr = new InputStreamReader(gotoUrl.openStream());
BufferedReader in = new BufferedReader(isr);
StringBuffer sb = new StringBuffer();
// grab the contents at the URL
int n = 0;
while (-1 != (n = in.read(cc))) {
sb.append(cc);// + "\r\n");
}
// write it locally
Statics.writeOut(filename, false, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
Finally, when I bzip2 the inputstream and outputstream, I get a valid bzip2 file, but of the size like the first one, using:
public static void getWriteForBZ2File(String urlIn, final String filename) throws CompressorException, IOException {
URL gotoUrl = new URL(urlIn);
try (final FileOutputStream out = new FileOutputStream(filename);
final BZip2CompressorOutputStream dataOutputStream = new BZip2CompressorOutputStream(out);
final BufferedInputStream bis = new BufferedInputStream(gotoUrl.openStream());
final CompressorInputStream input = new CompressorStreamFactory().createCompressorInputStream(bis);
final BufferedReader br2 = new BufferedReader(new InputStreamReader(input))) {
String line = null;
while ((line = br2.readLine()) != null) {
dataOutputStream.write(line.getBytes());
}
}
}
So, how do I get the entire bz2 file, in either bz2 format or unzipped?
A bz2 file contains bytes, not characters. You can't read it as if it contained characters, with a Reader.
Since all you want to do is download the file and save it locally, all you need is
Files.copy(gotoUrl.openStream(), Paths.get(fileName));
Related
Below I have the following code to read in a file and go through it line by line.. This is using java's BufferedReader class. That I am fine with.
String filename = "C:\\test.txt"
String line = null;
FileReader fileReader = new FileReader(filename);
BufferedReader bufferedReader = new BufferedReader(fileReader);
try {
while (((line = bufferedReader.readLine()) != null)) {
//do the following....
}
} catch (IOException) {
e.printStackTrace();
}
However I want to now start using InputStreamReader in Spring / Java. I have the below code written but I am unsure how I can step through my file line by line. Really confused over this part. Anyone have any ideas or know how this can be done?
String filepath= "C:\\test.txt"
File filename= new File(filepath);
try {
InputStream fileInputStream = new BOMInputStream(new fileInputStream(filename));
// now want to step through the file, line by line..
} catch (IOException) {
e.printStackTrace();
}
Thanks
This is how you can read your input file byte by byte using InputStreamReader.
char[] chars = new char[100];
try {
InputStream inputStream = new FileInputStream("C:\\test.txt");
InputStreamReader inputStreamReader = new InputStreamReader(inputStream,"UTF-8");
inputStreamReader.read(chars);
System.out.println(new String(chars).trim());
} catch (IOException e) {
e.printStackTrace();
}
Check this out -
String filename = "C:\\test.txt"
String line = null;
FileInputStream fileInputStream = new FileInputStream(filename);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(fileInputStream));
try {
while (((line = bufferedReader.readLine()) != null)) {
//do the following....
}
} catch (IOException) {
e.printStackTrace();
}
public static void main(String[] args) {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream("c:\\test.txt")))) {
reader.lines().forEach(line -> {
// do what you want with the line
});
} catch (IOException e) {
throw new RuntimeException(e);
}
}
At the moment i'm trying to save a response to the internal storage in the phone. Everything works fine up until i try and retrieve the data again. When i log out the retrieved data it only logs out one small section of the response and the rest isn't there. Ive tried deleting the file and calling it again just incase it was using an old one.
Saving Code
try {
String response = apiResponse.getRawResponse();
Log.e("Response", response);
FileOutputStream userInfo = openFileOutput("personal_profile", MODE_PRIVATE);
userInfo.write(response.getBytes());
userInfo.close();
} catch (Exception e) {
e.printStackTrace();
Retrieving Code
String response = "";
try {
FileInputStream fis = getActivity().openFileInput("personal_profile");
DataInputStream isr = new DataInputStream(fis);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(isr));
StringBuilder sb = new StringBuilder();
String line;
while ((line = bufferedReader.readLine()) != null) {
sb.append(line);
}
line = response;
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
Log.e("Saved File", response);
Any kind of suggestions would be great!
REASON
The problem was that the line variable is assigned again in every iteration
Try this:
String response = "";
try {
FileInputStream fis = getActivity().openFileInput("personal_profile");
DataInputStream isr = new DataInputStream(fis);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(isr));
StringBuilder sb = new StringBuilder();
String line;
while ((line = bufferedReader.readLine()) != null) {
sb.append(line);
}
line = response;
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
CHANGE LAST LINE
Log.e("Saved File", sb.toString());
Have you got this in your AndroidManifest.xml file?
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
Also, this link has everything you need to know about reading and writing files:
http://www.anddev.org/working_with_files-t115.html
Code::
String response = "";
try {
FileInputStream fis = getActivity().openFileInput("personal_profile");
DataInputStream isr = new DataInputStream(fis);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(isr));
StringBuilder sb = new StringBuilder();
String line;
while ((line = bufferedReader.readLine()) != null) {
sb.append(line);
}
line = response;
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
Log.e("Saved File", sb.toString());
I have looked around on how to do this and I keep finding different solutions, none of which has worked fine for me and I don't understand why. Does FileReader only work for local files? I tried a combination of scripts found on the site and it still doesn't quite work, it just throws an exception and leaves me with ERROR for the variable content. Here's the code I've been using unsuccessfully:
public String downloadfile(String link){
String content = "";
try {
URL url = new URL(link);
URLConnection conexion = url.openConnection();
conexion.connect();
InputStream is = url.openStream();
BufferedReader br = new BufferedReader( new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
sb.append(line);
}
content = sb.toString();
br.close();
is.close();
} catch (Exception e) {
content = "ERROR";
Log.e("ERROR DOWNLOADING",
"File not Found" + e.getMessage());
}
return content;
}
Use this as a downloader(provide a path to save your file(along with the extension) and the exact link of the text file)
public static void downloader(String fileName, String url) throws IOException {
File file = new File(fileName);
url = url.replace(" ", "%20");
URL website = new URL(url);
if (file.exists()) {
file.delete();
}
if (!file.exists()) {
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(fileName);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
fos.close();
}
}
Then call this function to read the text file
public static String[] read(String fileName) {
String result[] = null;
Vector v = new Vector(10, 2);
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader(fileName));
String tmp = "";
while ((tmp = br.readLine()) != null) {
v.add(tmp);
}
Iterator i = v.iterator();
result = new String[v.toArray().length];
int count = 0;
while (i.hasNext()) {
result[count++] = i.next().toString();
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
return (result);
}
And then finally the main method
public static void main(){
downloader("D:\\file.txt","http://www.abcd.com/textFile.txt");
String data[]=read("D:\\file.txt");
}
try this:
try {
// Create a URL for the desired page
URL url = new URL("mysite.com/thefile.txt");
// Read all the text returned by the server
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String str;
StringBuilder sb = new StringBuilder();
while ((str = in.readLine()) != null) {
// str is one line of text; readLine() strips the newline character(s)
sb.append(str );
}
in.close();
String serverTextAsString = sb.toString();
} catch (MalformedURLException e) {
} catch (IOException e) {
}
Ok, forgive my beginner-ness and please tell me how I can output my text from "before.txt" into a fresh new file called "after". Obviously I have altered the text along the way to make it lower-case and eliminate non alphabetic characters.
import java.io.*;
public class TextReader {
public void openFile() throws IOException {
try {
// Read in the file
BufferedReader br = new BufferedReader(
new FileReader(
new File("before.txt")));
String currentLine = br.readLine();
currentLine = currentLine.toLowerCase();
currentLine = currentLine.replaceAll("[A-Z]", "");
br.close(); // Close br to prevent resource leak
}
// Exception if the file is not in the path specified
catch (Exception e) {
System.out.println("Error: File not found");
}
}
public void writeFile() throws IOException {
BufferedWriter output = new BufferedWriter(new FileWriter("/WS3Ex3/after.txt"));
output.write("before.txt");
output.close();
}
}
What about this
public void openFile() throws IOException {
try {
// Read in the file
BufferedReader br = new BufferedReader(
new FileReader(
new File("before.txt")));
String currentLine = br.readLine();
currentLine = currentLine.toLowerCase();
currentLine = currentLine.replaceAll("[A-Z]", "");
br.close(); // Close br to prevent resource leak
writeFile(currentLine);
}
// Exception if the file is not in the path specified
catch (Exception e) {
System.out.println("Error: File not found");
}
}
public void writeFile(String text) throws IOException {
BufferedWriter output = new BufferedWriter(new FileWriter("/WS3Ex3/after.txt"));
output.write(text);
output.close();
}
}
Let me guess, is this a school assignment?
Try this:
public void ReadAndWrite() throws IOException {
try {
// Read in the file
BufferedWriter output = new BufferedWriter(new FileWriter("/WS3Ex3/after.txt"));
BufferedReader br = new BufferedReader(
new FileReader(
new File("before.txt")));
String currentLine;
while((currentLine = br.readLine()) != NULL){
currentLine = currentLine.toLowerCase();
currentLine = currentLine.replaceAll("[A-Z]", "");
output.write(currentLine);
}
br.close(); // Close br to prevent resource leak
output.close();
}
// Exception if the file is not in the path specified
catch (Exception e) {
System.out.println("Error: File not found");
}
}
I'd like to send a file contained in a ZIP archive unzipped to an external program for further decoding and to read the result back into Java.
ZipInputStream zis = new ZipInputStream(new FileInputStream(ZIPPATH));
Process decoder = new ProcessBuilder(DECODER).start();
???
BufferedReader br = new BufferedReader(new InputStreamReader(
decoder.getInputStream(),"us-ascii"));
for (String line = br.readLine(); line!=null; line = br.readLine()) {
...
}
What do I need to put into ??? to pipe the zis content to the decoder.getOutputStream()? I guess a dedicated thread is needed, as the decoder process might block when its output is not consumed.
Yes a thread is needed (or you wait/block until the copy is finished) for copying the InputStream to the OutputStream. Check the org.apache.commons.net.io.Util class for several helper methods to copy the data.
Ok, I got as far as following:
public class CopyStream extends Thread {
static final int BUFFERSIZE = 10 * 1024;
InputStream input; OutputStream output;
boolean closeInputOnExit, closeOutputOnExit, flushOutputOnWrite;
public IOException ex;
public CopyStream (InputStream input, boolean closeInputOnExit, OutputStream output, boolean closeOutputOnExit,
boolean flushOutputOnWrite) {
super("CopyStream");
this.input = input; this.closeInputOnExit = closeInputOnExit;
this.output = output; this.closeOutputOnExit = closeOutputOnExit;
this.flushOutputOnWrite = flushOutputOnWrite;
start();
}
public void run () {
try {
byte[] buffer = new byte[BUFFERSIZE];
for (int bytes = input.read(buffer); bytes>=0; bytes = input.read(buffer)) {
output.write(buffer,0,bytes);
if (flushOutputOnWrite) output.flush();
}
} catch (IOException ex) {
this.ex = ex;
} finally {
if (closeInputOnExit) {
try {
input.close();
} catch (IOException ex) {
if (this.ex==null) this.ex = ex;
}
}
if (closeOutputOnExit) {
try {
output.close();
} catch (IOException ex) {
if (this.ex==null) this.ex = ex;
}
}
}
}
}
Then the code would look as following:
ZipInputStream zis = new ZipInputStream(new FileInputStream(ZIPPATH));
for (ZipEntry ze = zis.getNextEntry(); ze!=null; ze = zis.getNextEntry()) {
Process decoder = new ProcessBuilder(EXTERNALPROCESSOR).start();
CopyStream cs1 = new CopyStream(is,false,decoder.getOutputStream(),true,true);
CopyStream cs2 = new CopyStream(decoder.getErrorStream(),true,System.err,false,true);
BufferedReader br = new BufferedReader(new InputStreamReader(decoder.getInputStream(),"us-ascii"));
ArrayList<String> lines = new ArrayList<String>();
for (String line = br.readLine(); line!=null; line = br.readLine()) {
lines.add(line);
}
if (decoder.exitValue()!=0) throw new IOException("Decoder exits with "+decoder.exitValue());
try {
cs1.join(100);
} catch (InterruptedException ex) {
throw new IOException(ex);
}
if (cs1.isAlive()) throw new IOException("cs1 not terminated");
if (cs1.ex!=null) throw cs1.ex;
try {
cs2.join(100);
} catch (InterruptedException ex) {
throw new IOException(ex);
}
if (cs2.isAlive()) throw new IOException("cs2 not terminated");
if (cs2.ex!=null) throw cs2.ex;
for (String line: lines) {
processline(line);
}
}
However, I find this a bit fragile. Isn't this a pattern for which some more robust implementation is around?