Can't get html source code - java

I'm trying to get the HTML source code from a url and then pass it in to a string.
After I press the debug button it asks me if I want to open the debuger (the code compile fine.)
The debuger: ActivityThread.performLaunchActivity(ActivityThread$ActivityClientRecord, Intent) line: 2180
says e > cause > "java.lang.NullPointerException" and "detailMessage null".
So I understands that this is a NullPointerException but cant see where.
Amd the only thing the LogCat says is:
08-14 00:34:10.437: E/Trace(1784): error opening trace file: No such file or directory (2)
08-14 00:34:10.437: I/System.out(1784): Sending WAIT chunk
08-14 00:34:10.437: I/dalvikvm(1784): Debugger is active
08-14 00:34:10.644: I/System.out(1784): Debugger has connected
08-14 00:34:10.644: I/System.out(1784): waiting for debugger to settle...
08-14 00:34:11.867: I/System.out(1784): debugger has settled (1300)
08-14 00:34:13.428: D/dalvikvm(1784): threadid=1: still suspended after undo (sc=1 dc=1)
I don't really know what more info to give you. :/
The code I use for this is (this is the GetCode class)
public String html;
public void getSourceCode() throws ClientProtocolException, IOException {
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("http://www.novasoftware.se/webviewer/(S(pisjjgujku50by55lpbdl1a2))/design1.aspx?schoolid=18200&code=83310");
HttpResponse response = client.execute(request);
html = "";
InputStream in = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder str = new StringBuilder();
String line = null;
while((line = reader.readLine()) != null)
{
str.append(line);
}
in.close();
html = str.toString();
System.out.println("html");
}
Then in the main class I just try to print the code.
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
doStuff();
}
private void doStuff() {
try {
getcode.getSourceCode();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(getcode.html);
}

Firstly, you are still not providing enough information for an accurate diagnosis. You should supply:
a complete and accurate description of what is happening
the complete stacktrace for the exception, including the line numbers
all of the code that "connects the dots"; e.g. when and where is getSourceCode called.
Having said that, I can see two possible causes of NPE's in the (partial) code you have provided:
If getSourceCode() is not called, then the html variable could be null when the onCreate method is called.
If the Response object returned by client.execute(request) is an error response (i.e. the status is a 4xx or 5xx response code), then response.getEntity() will return null. I think this can also happen if the response has no content.

Related

JsonReader running in thread crashes app

My app is crashing unpredictably and I don't understand why. I'm using HttpURLConnection to retrieve a json file and I'm trying to use the JsonReader class to read from, and use that file. The trouble is that the application crashes after reading or doing anything to the instance of JsonReader.
I've read the input stream with scanner but that causes a similar problem. However when reading with the scanner I can print small amounts of json from the server. So the data from the server is probably making it into the JsonReader as well.
This is my code:
public class ConnectToServer implements Runnable {
private URL connectionUrl;
private MainActivity output;
public ConnectToServer(URL url, MainActivity mainActivity) {
connectionUrl = url;
output = mainActivity;
}
public void printError(String errorMessage){
output.output(errorMessage);
}
public InputStream GetInputStream(){
try {
HttpURLConnection connection = (HttpURLConnection)connectionUrl.openConnection();
InputStream inputStream = new BufferedInputStream(connection.getInputStream());
if (inputStream == null){
printError("Input stream is null");
}
return inputStream;
} catch (IOException ex){
printError("IO exception has been thrown");
} catch (Exception ex){
printError("Normal exception thrown: " + ex.getClass().getSimpleName());
}
return null;
}
#Override
public void run() {
InputStream inputStream = GetInputStream();
try {
if (inputStream == null){
printError("Input stream not initiated");
} else {
JsonReader reader = new JsonReader(new InputStreamReader(inputStream, "UTF-8"));
printError(reader.toString());
reader.close();
}
} catch (Exception ex){
printError("Exception while printing input stream: " + ex.getClass().getSimpleName());
}
printError("Thread finished");
}
This gets run on a thread created in the main UI thread.
public void connect(){
output("connect started");
boolean success = true;
try {
URL url = new URL("http://geonews.azurewebsites.net/api/Location");
serverConnection = new ConnectToServer(url,this);
Thread thread = new Thread(serverConnection);
thread.start();
} catch (MalformedURLException ex){
output("Messed up the URL");
success = false;
}
if (success){
output("Thread has been started");
} else {
output("exception was thrown while trying to run thread");
}
}
Does anybody know why this code would be causing my app to crash? Even if it gets to "Thread finished" it will crash soon after.
Btw I realise I should be using AsyncTask but I've already gone down this track and I'd rather get this going first.
Logcat:
06-14 22:45:00.058 4032-4052/com.tomsapps.thomas.jsonreadertestapp E/AndroidRuntime﹕ FATAL EXCEPTION: Thread-314
Process: com.tomsapps.thomas.jsonreadertestapp, PID: 4032
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:6247)
at android.view.ViewRootImpl.requestLayout(ViewRootImpl.java:867)
at android.view.View.requestLayout(View.java:17364)
at android.view.View.requestLayout(View.java:17364)
at android.view.View.requestLayout(View.java:17364)
at android.view.View.requestLayout(View.java:17364)
at android.view.View.requestLayout(View.java:17364)
at android.view.View.requestLayout(View.java:17364)
at android.widget.RelativeLayout.requestLayout(RelativeLayout.java:360)
at android.view.View.requestLayout(View.java:17364)
at android.widget.TextView.checkForResize(TextView.java:6798)
at android.widget.TextView.updateAfterEdit(TextView.java:7693)
at android.widget.TextView.handleTextChanged(TextView.java:7709)
at android.widget.TextView$ChangeWatcher.onTextChanged(TextView.java:9440)
at android.text.SpannableStringBuilder.sendTextChanged(SpannableStringBuilder.java:964)
at android.text.SpannableStringBuilder.replace(SpannableStringBuilder.java:515)
at android.text.SpannableStringBuilder.append(SpannableStringBuilder.java:272)
at android.text.SpannableStringBuilder.append(SpannableStringBuilder.java:33)
at android.widget.TextView.append(TextView.java:3616)
at android.widget.TextView.append(TextView.java:3603)
at com.tomsapps.thomas.jsonreadertestapp.MainActivity.output(MainActivity.java:63)
at com.tomsapps.thomas.jsonreadertestapp.ConnectToServer.printError(ConnectToServer.java:26)
at com.tomsapps.thomas.jsonreadertestapp.ConnectToServer.run(ConnectToServer.java:74)
at java.lang.Thread.run(Thread.java:818)
I believe your output method from MainActivity modifies a TextView, therefore triggering the exception.
A view hierarchy can only be changed from the "Main" thread, in this case, you might want to try looking into runOnUiThread , some resources on StackOverflow :
how to use runOnUiThread
Android runOnUiThread explanation
Android Java runOnUiThread()
Hope this helps, good luck :)
Edit :
You might also want to use android Log instead of using TextView to display the errors.

Java, HttpClient issue (android)

Past few hours I have been working on this without any luck.
public void OnClickEventOnButton(View view)
{
this.commonContext = new BasicHttpContext(); //defined in the class to acihive
HttpClient httpClient=new DefaultHttpClient();
HttpGet get=new HttpGet("http://somesite.com");
HttpResponse response=null;
try{
response=httpClient.execute(get);
if(response.getStatusLine().getStatusCode()==200)
{
}
}catch(ClientProtocolException e){
alert(e.toString(),"エラ");
}
catch (IOException e) {
alert("HTTPHelp : IOException : "+e,"エラ");
}
catch(NullPointerException e){
alert("Null pointer : "+e.getCause(), "sfasfd");
}
}
The line below where my exception is generating
response=httpClient.execute(get);
The error my device showing is Unfortunately, Data Retriver has stopped.
If you need any more information, please let me know. If you need any specific data, please guide me, just started learning, I will provide the data.
Just a guess, but do you
a) don't have the INTERNET permission added in your manifest?
- this means you should see a "IllegalStateException" in your logcat
b) you execute Network on the applications main thread?
- this means you should see a "NetworkOnMainthreadException in your logcat

Nullpointerexception java

First off, I'm an experienced programmer, but have very little familiarity with Java. I have about two years of experience with it, eight years ago.
I'm getting a NullPointerException in the following code:
public static void handle(HttpServletRequest request,HttpServletResponse response)throws IOException,ServletException {
Response gfexResponse = null;
try {
ActionFactory actionFactory = ActionFactory.getInstance();
String requestURL = request.getRequestURI();
String actionId = actionFactory.getActionId(requestURL);
IAction action = actionFactory.createAction(actionId);
ActionEvent event = new ActionEvent(request, 0, actionId);
gfexResponse = action.execute(event);
} catch (Exception ex) {
gfexResponse = new Response();
gfexResponse.setError(ex.getMessage());
gfexResponse.setOutcome(IViewConstants.ERROR);
} finally {
if(request.getParameter("loginId") != null){
request.setAttribute("loginId", request.getParameter("loginId"));
}
if(gfexResponse.getMessage()!= null){
request.setAttribute("message", gfexResponse.getMessage());
}
if(gfexResponse.getError()!= null){
request.setAttribute("error", gfexResponse.getError());
}
if (gfexResponse.getContentType() != null) {
response.setContentType(gfexResponse.getContentType());
OutputStream outputStream = response.getOutputStream();
outputStream.write(gfexResponse.getOutputData());
outputStream.flush();
outputStream.close();
}
if(gfexResponse.getOutcome() != null){
RequestDispatcher dispatcher = request.getRequestDispatcher(gfexResponse.getOutcome());
dispatcher.forward(request, response);
}
}
}
Here's the StackTrace:
[6/18/13 17:10:04:518 GMT] 00000023 ServletWrappe E SRVE0068E: Uncaught exception thrown in one of the service methods of the servlet: GfexServlet. Exception thrown : java.lang.NullPointerException
at com.svl.gfex.handlers.RequestHandler.handle(RequestHandler.java:44)
at com.svl.gfex.servlets.GfexServlet.processRequest(GfexServlet.java:43)
at com.svl.gfex.servlets.GfexServlet.doPost(GfexServlet.java:39)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:966)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:907)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:118)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:87)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:701)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:646)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:475)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:463)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3129)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:238)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:811)
at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1433)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:93)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:465)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:394)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:102)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:152)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:213)
at com.ibm.io.async.AbstractAsyncFuture.fireCompletionActions(AbstractAsyncFuture.java:195)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:194)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:741)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:863)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1510)
The stacktrace points to this line:
if(gfexResponse.getMessage()!= null){ <-------- this line
request.setAttribute("message", gfexResponse.getMessage());
}
This code was being maintained by an offshore contractor, but the company laid off all the contractors. For my sins, I was given the job of fixing it.
If anyone can help me figure out why I'm getting this error, I would appreciate it.
That error indicates that the gfexResponse object itself is null (i.e. action.execute(event) is returning null in the code above, and no exception is being thrown)
Your basic outline is this:
public static void handle(HttpServletRequest request,HttpServletResponse response)throws IOException,ServletException {
Response gfexResponse = null;
try {
//Try to get your gfexResponse
} catch (Exception ex) {
//Provide a default gfexResponse
} finally {
//Do some stuff with gfexResponse
}
}
This is bad practice: you're attempting to use exception handling for flow control. Further, you assume that the method you use to get gfexResponse will throw an exception if it fails, which clearly it does not. (Some simple debugging/tracing will reveal this directly.)
What you should be doing is the following:
public static void handle(HttpServletRequest request,HttpServletResponse response)throws IOException,ServletException {
Response gfexResponse = null;
try {
//Try to get your gfexResponse
//Make sure you got your response object and throw SomeAppropriateException if not
//Do some stuff with gfexResponse
} catch (SomeAppropriateException e) {
//properly handle this case
} catch (Exception ex) {
//properly handle the general case that something else failed (But you should try to be more specific)
} finally {
//remove any resources that might not be properly cleaned up if an exception is thrown.
}
}
Are you sure that gfexResponse is getting an actual value from action.execute(event); (in the try {})? I'd guess that action.execute(event); is returning null.
To solve the immediate pain - your action.execute(event); call is likely returning null. However, this can be mitigated in several ways:
Null checking, or
Turning the try-catch block into its own separate method call, to return Response.
From there, the finally block becomes the main focus of your method, and you can check for null without having to worry about finally.
Actually problem lies at the line gfexResponse = action.execute(event);
only chance in the below line to get NPE here is gfexResponse is null
if(gfexResponse.getMessage()!= null){ <-------- this line
change it to
if(gfexResponse!=null && gfexResponse.getMessage()!= null){ <-------- this line

nearest equivalent of a webpage reload in java

I am taking some data from a database via a servlet and a db handler java class and hosting it at a url. Since the database is changing I'm taking care only to host the changes rather than the entire db data.
I'm getting the required functionality by a browser i.e after every (manual) reload, I'm getting the data as required by me,
1. at the first page load, entire data gets displayed.
2. at subsequent reloads, I get either null data if there is no change in the database, or the appended rows if the database extends. (the database can only extend).
But then in a java program, I'm not getting the same functionality. The java program using HttpUrlConnection.
This is the code for the java client for servlet...
public class HTTPClient implements Runnable {
private CallbackInterface callbackinterface;
private URL url;
private HttpURLConnection http;
private InputStream response;
private String previousMessage = "";
public HTTPClient() {
try {
url = new URL("http://localhost:8080/RESTful-Server/index.jsp");
http = (HttpURLConnection) url.openConnection();
http.connect();
} catch (IOException e) {
}
}
#Override
public void run() {
while (true) {
try {
String currentmessage = "";
response = http.getInputStream();
if (http.getResponseCode() == HttpURLConnection.HTTP_OK) {
BufferedReader buffread = new BufferedReader(new InputStreamReader(response));
String line;
for (; (line = buffread.readLine()) != null;) {
currentmessage += line;
}
if ((!currentmessage.equals(previousMessage)
|| !previousMessage.equals(""))
&& !currentmessage.equals("")) {
//this.callbackinterface.event(currentmessage);\
System.out.println(currentmessage + "\t" + previousMessage);
}
previousMessage = currentmessage;
Thread.sleep(2500);
} else {
throw new IOException();
}
} catch (IOException | InterruptedException e) {
System.err.println("Exception" + e);
}
}
}
The shown class is a thread which read the connections every 2.5 s. If it gets something significant in the getline(), it will issue a callback to a worker method, which takes care of remaining things.
I am thinking the issues is because of the class variable conn, and that reload as in the browser is not getting replicated..
Any idea how to do this?
You're basically connecting (requesting) only once and trying to read the response multiple times, while it can be read only once. You basically need to create a new connection (request) everytime. You need to move the creation of the connection by url.openConnection() to inside the loop. The line http.connect() is by the way superfluous. You can safely omit it. The http.getInputStream() will already implicitly do it.
See also:
Using java.net.URLConnection to fire and handle HTTP requests

How to get external IP successfully

After reading: Getting the 'external' IP address in Java
code:
public static void main(String[] args) throws IOException
{
URL whatismyip = new URL("http://automation.whatismyip.com/n09230945.asp");
BufferedReader in = new BufferedReader(new InputStreamReader(whatismyip.openStream()));
String ip = in.readLine(); //you get the IP as a String
System.out.println(ip);
}
I thought I was a winner but I get the following error
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: http://automation.whatismyip.com/n09230945.asp
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at getIP.main(getIP.java:12)
I think this is because the server isnt responding quick enough, is there anyway to ensure that it will get the external ip?
EDIT: okay so its getting rejected, anyone else know of another site that can do the same function
public static void main(String[] args) throws IOException
{
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection();
String str = null;
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
str = reader.readLine();
System.out.println(str);
}
Before you run the following code take a look at this: http://www.whatismyip.com/faq/automation.asp
public static void main(String[] args) throws Exception {
URL whatismyip = new URL("http://automation.whatismyip.com/n09230945.asp");
URLConnection connection = whatismyip.openConnection();
connection.addRequestProperty("Protocol", "Http/1.1");
connection.addRequestProperty("Connection", "keep-alive");
connection.addRequestProperty("Keep-Alive", "1000");
connection.addRequestProperty("User-Agent", "Web-Agent");
BufferedReader in =
new BufferedReader(new InputStreamReader(connection.getInputStream()));
String ip = in.readLine(); //you get the IP as a String
System.out.println(ip);
}
While playing with Go I saw your question. I made a quick App on Google App Engine using Go:
Hit this URL:
http://agentgatech.appspot.com/
Java code:
new BufferedReader(new InputStreamReader(new URL('http://agentgatech.appspot.com').openStream())).readLine()
Go code for the app which you can copy and make your own app:
package hello
import (
"fmt"
"net/http"
)
func init() {
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, r.RemoteAddr)
}
Some servers has triggers that blocks access from "non-browsers". They understand that you are some kind of automatic app that can do a DOS attack. To avoid this, you can try to use a lib to access the resource and set the "browser" header.
wget works in this way:
wget -r -p -U Mozilla http://www.site.com/resource.html
Using Java, you can use the HttpClient lib and set the "User-Agent" header.
Look the topic 5 of "Things To Try" section.
Hope this can help you.
A 403 response indicates that the server is explicitly rejecting your request for some reason. Contact the operator of WhatIsMyIP for details.
We've set up CloudFlare and as designed they're challenging unfamiliar useragents. If you can set your UA to something common, you should be able to gain access.
You can use another web service like this; http://freegeoip.net/static/index.html
Using the Check IP address link on AWS worked for me.Please note that MalformedURLException,IOException are to be added as well
public String getPublicIpAddress() throws MalformedURLException,IOException {
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection();
String str = null;
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
str = reader.readLine();
return str;
}
This is how I do it with rxJava2 and Butterknife. You'll want to run the networking code in another thread because you'll get an exception for running network code on the main thread!
I use rxJava instead of AsyncTask because the rxJava cleans up nicely when the user moves on to the next UI before the thread is finished. (this is super useful for very busy UI's)
public class ConfigurationActivity extends AppCompatActivity {
// VIEWS
#BindView(R.id.externalip) TextInputEditText externalIp;//this could be TextView, etc.
// rxJava - note: I have this line in the base class - for demo purposes it's here
private CompositeDisposable compositeSubscription = new CompositeDisposable();
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.my_wonderful_layout);
ButterKnife.bind(this);
getExternalIpAsync();
}
// note: I have this code in the base class - for demo purposes it's here
#Override
protected void onStop() {
super.onStop();
clearRxSubscriptions();
}
// note: I have this code in the base class - for demo purposes it's here
protected void addRxSubscription(Disposable subscription) {
if (compositeSubscription != null) compositeSubscription.add(subscription);
}
// note: I have this code in the base class - for demo purposes it's here
private void clearRxSubscriptions() {
if (compositeSubscription != null) compositeSubscription.clear();
}
private void getExternalIpAsync() {
addRxSubscription(
Observable.just("")
.map(s -> getExternalIp())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe((String ip) -> {
if (ip != null) {
externalIp.setText(ip);
}
})
);
}
private String getExternalIp() {
String externIp = null;
try {
URL connection = new URL("http://checkip.amazonaws.com/");
URLConnection con = connection.openConnection(Proxy.NO_PROXY);
con.setConnectTimeout(1000);//low value for quicker result (otherwise takes about 20secs)
con.setReadTimeout(5000);
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
externIp = reader.readLine();
} catch (Exception e) {
e.printStackTrace();
}
return externIp;
}
}
UPDATE - I've found that URLConnection is really quite shit; it'll take a long time to get a result, not really time out very well, etc. The code below improves the situation with OKhttp
private String getExternalIp() {
String externIp = "no connection";
OkHttpClient client = new OkHttpClient();//should have this as a member variable
try {
String url = "http://checkip.amazonaws.com/";
Request request = new Request.Builder().url(url).build();
Response response = client.newCall(request).execute();
ResponseBody responseBody = response.body();
if (responseBody != null) externIp = responseBody.string();
} catch (IOException e) {
e.printStackTrace();
}
return externIp;
}

Categories

Resources