How to get the page source from specific link in Java? [closed] - java

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question appears to be off-topic because it lacks sufficient information to diagnose the problem. Describe your problem in more detail or include a minimal example in the question itself.
Closed 8 years ago.
Improve this question
Is it possible to get the inputStream of particular website content or it's page source into String?
For instance, I want to download the whole html tag from particular website into string or xml. Is it possible?

Yes of course you just have to do something like
public static void main(String[] args) {
URL url;
try {
// get URL content
url = new URL("http://www.mkyong.com");
URLConnection conn = url.openConnection();
// open the stream and put it into BufferedReader
BufferedReader br = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String inputLine;
//save to this filename
String fileName = "/users/mkyong/test.html";
File file = new File(fileName);
if (!file.exists()) {
file.createNewFile();
}
//use FileWriter to write file
FileWriter fw = new FileWriter(file.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);
while ((inputLine = br.readLine()) != null) {
bw.write(inputLine);
}
bw.close();
br.close();
System.out.println("Done");
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
CREDIT : mkyong

You may want to look at guava's CharStreams class.
CharStreams.toString(new InputStreamReader(..))
will save you from writing much boilerplate code.
Here is doc

Related

BufferedReader/Writer not working in Java [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Closed 6 years ago.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Improve this question
I'm trying to make two basic functions which will allow me to call them from other classes in order to return a HashMap from the 'getAllData()' and to write to the file in 'writeToFile()', without any luck. I've been tampering with it for a while now and just getting a multitude of strange errors.
Code:
static HashMap<Integer ,String> getAllData(Integer choice) throws Exception{
InputStream localInputStream = ClassLoader.getSystemClassLoader().getResourceAsStream("Shadow.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(localInputStream));
if (choice.equals(1)){
while ((!data.equals(null))){
data = br.readLine();
dataString = dataString+data;
}
writeToFile(dataString);
br.close();
}return name;
}
static void writeToFile(String data) throws IOException{;
File file = new File ("Shadow.txt");
FileWriter fileWriter = new FileWriter(file, true);
BufferedWriter bw = new BufferedWriter(fileWriter);
bw.write(data);
bw.newLine();
bw.flush();
bw.close();
}
With this current code, nothing happens. The file remains exactly how it is, although to me, the code should read everything from it, and then append it.
How can I fix this?
Thanks
This might help you:
static void getAllData(final Integer choice) throws Exception {
final BufferedReader br = new BufferedReader(new FileReader("Shadow.txt"));
String data = "";
final StringBuilder builder = new StringBuilder();
if(choice.equals(1)) {
while(data != null) {
data = br.readLine();
if(data != null) {
builder.append(data + "\n");
}
}
writeToFile(builder.toString());
br.close();
}
}
static void writeToFile(final String data) throws IOException {
final File file = new File("Shadow.txt");
final FileWriter fileWriter = new FileWriter(file, true);
final BufferedWriter bw = new BufferedWriter(fileWriter);
bw.write(data);
bw.flush();
bw.close();
}

why doesn't my file delete no matter why I do? [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 6 years ago.
Improve this question
public void removeLine(String s) throws IOException, FileNotFoundException{
File tempFile = new File("temp.txt");
FileInputStream reader = new FileInputStream(sharkFile);
Scanner scanner = new Scanner(reader);
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile, true));
String currentLine;
while(scanner.hasNextLine()){
currentLine = scanner.nextLine();
String trimmedLine = currentLine.trim();
System.out.println(trimmedLine);
trimmedLine.equals(sharkName);
if(trimmedLine.equals(sharkName)) continue;
writer.write(currentLine + System.getProperty("line.separator"));
}
scanner.close();
scanner = null;
reader.close();
writer.flush();
writer.close();
writer = null;
System.gc();
if(!sharkFile.delete()){
System.out.println("Could not delete file d");
return;
}
if(!tempFile.renameTo(sharkFile)){
System.out.println("Could not rename file");
return;
}
}
I've gone through numerous threads on stackoverflow and have implemented those changes but my file just won't delete. Appreciate the help.
The File API is notoriously weak on explaining why something fail, e.g. File.delete() simply returns a boolean, and value false cannot explain why.
Use the new Path API instead.
Also, please (PLEASE!) use try-with-resources.
Scanner is slow, so better to use BufferedReader, and for writing the lines back with newlines, use a PrintWriter.
Path sharkPath = sharkFile.toPath();
Path tempPath = Paths.get("temp.txt");
Charset cs = Charset.defaultCharset();
try (BufferedReader reader = Files.newBufferedReader(sharkPath, cs);
PrintWriter writer = new PrintWriter(Files.newBufferedWriter(tempPath, cs, StandardOpenOption.CREATE, StandardOpenOption.APPEND, StandardOpenOption.WRITE)))) {
for (String currentLine; (currentLine = reader.readLine()) != null; ) {
String trimmedLine = currentLine.trim();
System.out.println(trimmedLine);
if (! trimmedLine.equals(sharkName))
writer.println(currentLine);
}
}
Files.delete(sharkPath); // throws descriptive exception if cannot delete
Files.move(tempPath, sharkPath); // throws exception if cannot move
Use below code, rename file before delete, it's appearing that you are accessing file name after delete:
try { //rename file first
tempFile.renameTo(sharkFile);
} catch (Exception e) {
JOptionPane.showMessageDialog(null, "Unable to rename.");
}
try {
sharkFile.delete();
}
catch(Exception e) {
JOptionPane.showMessageDialog(null, "Unable to delete.");
}

Read directly from a URL and write in a file - Java [duplicate]

This question already has answers here:
How can I download and save a file from the Internet using Java?
(23 answers)
Closed 7 years ago.
I am reading the contents of a URL and write a file the problem is that I'm not able to write all the content in the file and do not know what I'm doing wrong.
My code,
try {
URL url = new URL(sourceUri);
URLConnection conn = url.openConnection();
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream()));
file.getParentFile().mkdirs();
file.createNewFile();
FileWriter fw = new FileWriter(file);
BufferedWriter bw = new BufferedWriter(fw);
while ((inputLine = br.readLine()) != null) {
bw.write(inputLine + System.getProperty("line.separator"));
}
br.close();
System.out.println("DONE");
}catch (IOException ioe){
ioe.printStackTrace();
}catch (Exception e){
e.printStackTrace();
}
return ontologies;
}
Please help
You are doing many things incorrectly.
First: you don't close all your resources; where is the writer to the file closed?
Second: you use new InputStreamReader(...) without specifying the encoding. What says that the encoding on the other end is the one of your JVM/OS combination?
Last but not least, and in fact, this is the most important, you should use java.nio.file. This is 2015 after all.
Simple solution:
final Path path = file.toPath(); // or rather use Path directly
Files.createDirectories(path.getParent());
try (
final InputStream in = conn.getInputStream();
) {
Files.copy(in, path);
}
Done, encoding independent, and all resources closed.
The problem is you're using a BufferedWriter and you don't close it. It has some content in his buffer that is not writing and you're missing.
Try flushing the buffer and closing the BufferedWriter:
bw.flush();
bw.close();
Include this two lines after before your br.close();.
Also you can read how BufferedWriter works here.
And I think you should close FileWriter, too, in order to unblock the file.
fw.close();
EDIT 1:
Closing the BufferedWriter will flush the buffer for you. You need only to close it.

How to download xml data of a website? [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I have searched about this. All I got was xml parsing/ Sax parser. I need a program that will download xml data.
I need this for my android application development. thanks
For example i have a website localhost:8080/folder/sample.html.. How do i get a .xml file from that?
Sorry if I'm not answering the question - but is it the website content, you want to download? If positive, these are similar questions where the solution may lie:
How to get a web page's source code from Java
Get source of website in java
How do I retrieve a URL from a web site using Java?
How do you Programmatically Download a Webpage in Java
A good library to do URL Query String manipulation in Java
try this code:
public String getXmlText(String urlXml) {
URL url;
InputStream is = null;
BufferedReader br;
String line;
String result = null;
try {
url = new URL(urlXml);
is = url.openStream();
br = new BufferedReader(new InputStreamReader(is));
while ((line = br.readLine()) != null) {
result = result + line + "\n";
}
} catch (Exception e) {
return "";
} finally {
try {
if (is != null) is.close();
} catch (IOException ioe) {}
}
return result;
}
Try this code
import java.net.*;
import java.io.*;
class Crawle {
public static void main(String ar[]) throws Exception {
URL url = new URL("http://www.foo.com/your_xml_file.xml");
InputStream io = url.openStream();
BufferedReader br = new BufferedReader(new InputStreamReader(io));
FileOutputStream fio = new FileOutputStream("file.xml");
PrintWriter pr = new PrintWriter(fio, true);
String data = "";
while ((data = br.readLine()) != null) {
pr.println(data);
}
}
}

Syntax error on token "try" please delete this token [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 9 years ago.
Improve this question
So I was trying to read a file, this is my piece of code;
try{
FileReader fr = new FileReader("mikuname.txt");
BufferedReader br = new BufferedReader(fr);
String str;
while ((str = br.readLine()) != null){
System.out.println(str + "\n");
}
br.close();
}
But when compiling I get an error saying "Syntax error on token "try" please delete this token"
Any idea how I can fix this?
Thanks!
You have inserted your try block out of the blue into a class. You may be coming from a scripting language background, where it is always legal to just write standalone action code. In Java such code must find itself either within a method or within an initializer. If you just want to run some code, then put it into the main method:
public class MyGuy {
public static void main(String[] args) {
... your try-block here ...
}
}
Note that your case is a perfect match for Java 7's try with resources:
try (FileReader fr = new FileReader("mikuname.txt");
BufferedReader br = new BufferedReader(fr))
{
... your actions ...
}
In this case, the try block must be accompanied by atleast one catch block or a finally block.
eg
try {
} catch(SomeException e){
} finally {
}
try always come with atleast one catch block
try{
FileReader fr = new FileReader("mikuname.txt");
BufferedReader br = new BufferedReader(fr);
String str;
while ((str = br.readLine()) != null){
System.out.println(str + "\n");
}
br.close();
}catch (Exception e){
//some exception information
}
check doc for more information about try..catch [link] http://docs.oracle.com/javase/tutorial/essential/exceptions/try.html
"Any idea how I can fix this?"
put catch block after try block. this would be one of the right solution:
catch(Exception e){}
you should have a catch block corresponding to try block as follows:
try {
...
catch(<exception>) {
}
For Java 1.7+, move the declaration of your reader into the resource block:
try (FileReader fr = new FileReader("mikuname.txt");) {
// rest of block same as yours
}
catch and/or finally block is not necessary when using resources.
It looks like you're trying to use the new try with resources in Java 7. To do that, you use () around your resources:
try (
FileReader fr = new FileReader("mikuname.txt");
BufferedReader br = new BufferedReader(fr);
) {
String str;
while ((str = br.readLine()) != null){
System.out.println(str + "\n");
}
// You don't need this, it's done by the try: br.close();
}
...and of course, you have to make sure you're using Java 7, and you have to use it in a proper context (within a method or within either an instance or static initializer).
As the compiler suggested, you should remove the try keyword.
But if you actually intended to catch the IOException that could be thrown, you should add a catch clause rather than delete the try keyword.
If you're using Java 7, you can clean up your code a bit more. Right now, if there is an error while you read the file, you don't close the file and your application may run out of file descriptors. In Java 7 the try-with-resources statement has a nice way of handling this:
try (BufferedReader br = new BufferedReader(new FileReader("mikuname.txt"))) {
String str;
while ((str = br.readLine()) != null) {
System.out.println(str + "\n");
}
} catch (IOException e) {
// Do something with the IO problem that occurred while reading the file
}

Categories

Resources