Read URL in Java - java

My goal with this program is to extract a website's content and output it to console. However, an exception gets thrown every time I run this code. I am wondering what I am doing wrong, and if anyone can point me in the right direction. Thank you ahead of time!
public class twikiripper {
public static URL url;
public static void main(String[] args) {
BufferedReader br = null;
try{
URL url = new URL("http://www.google.com");
}catch(MalformedURLException ex){}
try {
url.openConnection();
br = new BufferedReader(new InputStreamReader(url.openStream()));
String line;
StringBuilder sb = new StringBuilder();
while ((line = br.readLine()) != null) {
sb.append(line);
sb.append(System.lineSeparator());
}
System.out.println(sb);
}catch(Exception e){
System.out.println("Exception: "+e.toString());
}
}
My code is above. I was wondering, why am I always outputting Exception: java.lang.NullPointerException ? I seem to always throw this exception. I thought I was doing everything right.
What I am trying to do is display the output code from a website, that is all. Please help !

You have an unnecessary try-catch block in your code. Try this:
public static void main(String[] args) {
BufferedReader br = null;
try {
URL url = new URL("http://www.google.com");
url.openConnection();
br = new BufferedReader(new InputStreamReader(url.openStream()));
String line;
StringBuilder sb = new StringBuilder();
while ((line = br.readLine()) != null) {
sb.append(line);
sb.append(System.lineSeparator());
}
System.out.println(sb);
}catch(Exception e){
System.out.println("Exception: "+e.toString());
}
}
And also make sure that your are importing the correct classes.

The reason for null pointer exception is:
In your code, in url.openConnection(); variable url is local and its scope ends at the end of the its try block.
Try this:
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
public class twikiripper {
public static URL url;
public static void main(String[] args) {
BufferedReader br = null;
try{
url = new URL("http://www.google.com"); // I have changed this line
}catch(MalformedURLException ex){}
try {
url.openConnection();
br = new BufferedReader(new InputStreamReader(url.openStream()));
String line;
StringBuilder sb = new StringBuilder();
while ((line = br.readLine()) != null) {
sb.append(line);
sb.append(System.lineSeparator());
}
System.out.println(sb);
}catch(Exception e){
System.out.println("Exception: "+e.toString());
}
}
}

In your first try block a the local variable hides the field url. You've got two different variables with the same name. Change URL url = new URL("http://www.google.com"); to url = new URL("http://www.google.com"); or follow NiVeRs answer. – Eritrean 8 mins ago
Correct! Thank you.

below code will fullfill your requirement --
package com.subham.testing;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
public class Test13 {
public static URL url;
public static void main(String[] args) {
BufferedReader br = null;
try {
url = new URL("http://www.google.com");
} catch (MalformedURLException ex) {
System.out.println("came exception");
}
try {
url.openConnection();
br = new BufferedReader(new InputStreamReader(url.openStream()));
String line;
StringBuilder sb = new StringBuilder();
while ((line = br.readLine()) != null) {
sb.append(line);
sb.append(System.lineSeparator());
}
System.out.println(sb);
} catch (Exception e) {
System.out.println("Exception: " + e.toString());
}
}
}
You were creating new url object in first try block so in second try block was getting null as it was just decleared but not initialized.

Related

BufferedReader stuck at last input line without ending the program

I'm using BufferedReader to read data from System.in (a text file redirected context: < file.txt) then write it to the console. The problem is my program shows all lines except the last one and still works without doing any thing. If I manually end it it will write the final line.
This is my code:
public void updateText() throws IOException {
try {
InputStreamReader inputStreamReader = new InputStreamReader(System.in);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String inputLine;
while ((inputLine = bufferedReader.readLine()) != null) {
System.out.println(inputLine);
}
inputStreamReader.close();
bufferedReader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Here an alternative way of waiting on available data (Ref.: Java 11 - method BufferedReader.ready()):
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
class TestClass {
public static void main(String[] args) {
try (InputStreamReader inputStreamReader = new InputStreamReader(System.in);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader)) {
String line;
// wait until data available
while (!bufferedReader.ready()){
while ((line = bufferedReader.readLine()) != null) {
System.out.println(line);
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Example output:
Hello world
Hello world
Hello world
Hello world
If this only occurs in Eclipse, your issue is most likely a duplicate and bug 513713.
You don't need to read standard input line by line when you could simply transfer the data in one step, replacing the body of updateText():
System.in.transferTo(System.out);

URLConnectionReader produces UnknownHostException

Thanks in advance for every input!
I'm getting a little familiar with how to read data from websites with Java and have tried to do this by reading data using a URLConnectionReader.
Unfortunately I get an UnknownHostException when I test the whole thing in a Java online compiler (https://www.jdoodle.com/online-java-compiler/).
Have I forgotten any imports? I proceeded according to a tutorial.
Code: (designed for online-java-compiler jdoodle):
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public static void main(String[] args)
{
String output = getUrlContents("https://www.tradegate.de/orderbuch_umsaetze.php?isin=NO0010892359");
System.out.println(output);
}
private static String getUrlContents(String theUrl)
{
StringBuilder content = new StringBuilder();
try
{
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String line;
while ((line = bufferedReader.readLine()) != null)
{
content.append(line + "\n");
}
bufferedReader.close();
}
catch(Exception e)
{
e.printStackTrace();
}
return content.toString();
}
}
Error message:
java.net.UnknownHostException: www.tradegate.de
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:220)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.base/java.net.Socket.connect(Socket.java:591)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:285)
at java.base/sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:182)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:265)
at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:372)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1515)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250)
at URLConnectionReader.getUrlContents(URLConnectionReader.java:21)
at URLConnectionReader.main(URLConnectionReader.java:8)
I separated the classes as follows and your code works without any exceptions=>
class Mian:
public class Mian {
public static void main(String[] args) throws ClassNotFoundException {
URLConnectionReader urlcr = new URLConnectionReader();
String output =
urlcr.getUrlContents("https://www.tradegate.de/orderbuch_umsaetze.php?
isin=NO0010892359");
System.out.println(output);
}
}
and URLConnectionReader class:
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public String getUrlContents(String theUrl)
{
StringBuilder content = new StringBuilder();
try
{
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
BufferedReader bufferedReader = new BufferedReader(new
InputStreamReader(urlConnection.getInputStream()));
String line;
while ((line = bufferedReader.readLine()) != null)
{
content.append(line + "\n");
}
bufferedReader.close();
}
catch(Exception e)
{
e.printStackTrace();
}
return content.toString();
}
}

Reading text file into String array, then converting to ArrayList

How can I read a file into a array of String[] and then convert it into ArrayList?
I can't use an ArrayList right away because my type of list is not applicable for the arguments (String).
So my prof told me to put it into an array of String, then convert it.
I am stumped and cannot figure it out for the life of me as I am still very new to java.
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* Created by tsenyurt on 06/04/15.
*/
public class ReadFile
{
public static void main(String[] args) {
List<String> strings = new ArrayList<>();
BufferedReader br = null;
try {
String sCurrentLine;
br = new BufferedReader(new FileReader("/Users/tsenyurt/Development/Projects/java/test/pom.xml"));
while ((sCurrentLine = br.readLine()) != null) {
System.out.println(sCurrentLine);
strings.add(sCurrentLine);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (br != null)br.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
}
there is a code that reads a file and create a ArrayList from it
http://www.mkyong.com/java/how-to-read-file-from-java-bufferedreader-example/
Well there are many ways to do that,
you can use this code if you want have a List of each word exist in your file
public static void main(String[] args) {
BufferedReader br = null;
StringBuffer sb = new StringBuffer();
List<String> list = new ArrayList<>();
try {
String sCurrentLine;
br = new BufferedReader(new FileReader(
"Your file path"));
while ((sCurrentLine = br.readLine()) != null) {
sb.append(sCurrentLine);
}
String[] words = sb.toString().split("\\s");
list = Arrays.asList(words);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (br != null)
br.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
for (String string : list) {
System.out.println(string);
}
}

How to open a txt file located on the web and read its contents to a string?

I have looked around on how to do this and I keep finding different solutions, none of which has worked fine for me and I don't understand why. Does FileReader only work for local files? I tried a combination of scripts found on the site and it still doesn't quite work, it just throws an exception and leaves me with ERROR for the variable content. Here's the code I've been using unsuccessfully:
public String downloadfile(String link){
String content = "";
try {
URL url = new URL(link);
URLConnection conexion = url.openConnection();
conexion.connect();
InputStream is = url.openStream();
BufferedReader br = new BufferedReader( new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
sb.append(line);
}
content = sb.toString();
br.close();
is.close();
} catch (Exception e) {
content = "ERROR";
Log.e("ERROR DOWNLOADING",
"File not Found" + e.getMessage());
}
return content;
}
Use this as a downloader(provide a path to save your file(along with the extension) and the exact link of the text file)
public static void downloader(String fileName, String url) throws IOException {
File file = new File(fileName);
url = url.replace(" ", "%20");
URL website = new URL(url);
if (file.exists()) {
file.delete();
}
if (!file.exists()) {
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(fileName);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
fos.close();
}
}
Then call this function to read the text file
public static String[] read(String fileName) {
String result[] = null;
Vector v = new Vector(10, 2);
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader(fileName));
String tmp = "";
while ((tmp = br.readLine()) != null) {
v.add(tmp);
}
Iterator i = v.iterator();
result = new String[v.toArray().length];
int count = 0;
while (i.hasNext()) {
result[count++] = i.next().toString();
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
return (result);
}
And then finally the main method
public static void main(){
downloader("D:\\file.txt","http://www.abcd.com/textFile.txt");
String data[]=read("D:\\file.txt");
}
try this:
try {
// Create a URL for the desired page
URL url = new URL("mysite.com/thefile.txt");
// Read all the text returned by the server
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String str;
StringBuilder sb = new StringBuilder();
while ((str = in.readLine()) != null) {
// str is one line of text; readLine() strips the newline character(s)
sb.append(str );
}
in.close();
String serverTextAsString = sb.toString();
} catch (MalformedURLException e) {
} catch (IOException e) {
}

How to handle problem with Network Connectivity in Java

I have a simple java code which gets html text from the input url:
try {
URL url = new URL("www.abc.com");
// Get the response
BufferedReader rd = new BufferedReader(new InputStreamReader(url.openStream()));
while ((line = rd.readLine()) != null) {
String code = code + line;
} catch (IOException e){}
I am using this code in an android project. Now the problem comes when there is no internet connectivity. The application just halts and later gives error.
Is there some way to break this after some fixed timeout, or even return some specific string after an exception is thrown. Can you please tell me how to do that??
Try this:
try
{
URL url = new URL("www.abc.com");
String newline = System.getProperty("line.separator");
InputStream is = url.openStream();
if (is != null)
{
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
StringBuilder contents = new StringBuilder();
while ((line = rd.readLine()) != null)
{
contents.append(line).append(newline);
}
}
else
{
System.out.println("input stream was null");
}
}
catch (Exception e)
{
e.printStackTrace();
}
An empty catch block is asking for trouble.
I don't know what the default timeout is for URL, and a quick look at the javadocs doesn't seem to reveal anything. So try using HttpURLConnection directly instead http://download.oracle.com/javase/1.5.0/docs/api/java/net/HttpURLConnection.html. This lets you set timeout values:
public static void main(String[] args) throws Exception {
URL url = new URL("http://www.google.com");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(5000); // 5 seconds
conn.setRequestMethod("GET");
conn.connect();
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
conn.disconnect();
}
You can also set a read time out as well, as well as specify behaviour re redirects and a few other things.
I think in addition to timeouts it could be also smart to check the Internet availability right before the requesting:
public class ConnectivityHelper {
public static boolean isAnyNetworkConnected(Context context) {
return isWiFiNetworkConnected(context) || isMobileNetworkConnected(context);
}
public static boolean isWiFiNetworkConnected(Context context) {
return getWiFiNetworkInfo(context).isConnected();
}
public static boolean isMobileNetworkConnected(Context context) {
return getMobileNetworkInfo(context).isConnected();
}
private static ConnectivityManager getConnectivityManager(Context context) {
return (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE);
}
}
UPDATE: For timeouts see an excellent kuester2000's reply here.
Just a general tip on working with Streams always close them when they are no longer needed. I just wanted to post that up as it seems that most people didn't take care of it in their examples.

Categories

Resources