Jsoup can't open html page Status=-1 - java

While connecting to a URL i'm getting Below is the logcat error i'm getting
and below that my code to connect to url.
When i try to connect to base url http://www.apkmania.co/ it connects successfully
but when try to connect to this url it throws me an error.
07-31 20:47:20.150: I/System.out(14295): IOException: org.jsoup.HttpStatusException: HTTP error fetching URL. Status=404, URL=http://www.apkmania.co/2013/07/blood-sword-thd-v16-apk.html/
break;
Thread thread=new Thread(new Runnable(){
public void run(){
PrepareItem(webrss);
runOnUiThread(new Runnable(){
public void run() {
if(dialog.isShowing()){
try {
setDataToHandels();
} catch (NullPointerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
System.out.println("SET DATA TO HANDELS: " + e);
}
dialog.dismiss();
}
}
});
}
});
thread.start();
break;
private void PrepareItem(String url) {
System.out.println(url);
Document content= null;
try {
content = Jsoup.connect(url).userAgent("Mozilla").timeout(10*1000).get();
} catch (final IOException e) {
// TODO Auto-generated catch block
System.out.println("IOException: " + e.toString());
}
int i=0;
try {
Elements html1 = content.getElementById("main-wrapper").getElementsByTag("div").get(16).children();
html1.select("img").first().remove();
String[] main_content= new String[html1.size()];
i=0;
for (Element element_src : html1.select("div")) {
if (element_src.attr("dir").equals("ltr")) {
main_content[i] = element_src.toString();
i++;
}
}
for (int j = 0; j < main_content.length-2; j++) {
all_text = all_text+main_content[j];
}
all_images_ems = html1.select("img");
all_images_src = new String[all_images_ems.size()];
i=0;
for (Element img_src : all_images_ems) {
all_images_src[i] = img_src.attr("src");
i++;
}
anchor_link_ems = html1.select("a");
all_links = new String[anchor_link_ems.size()];
i=0;
for (Element anchor_links : anchor_link_ems) {
all_links[i] = anchor_links.attr("href");
i++;
}
} catch (NullPointerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Please help me in this regard.

There's a slash at the end of the url.
http://www.apkmania.co/2013/07/blood-sword-thd-v16-apk.html/
If you remove the slash at the end, it should work.
http://www.apkmania.co/2013/07/blood-sword-thd-v16-apk.html
Edit:
It's working for me like this.
String url = "http://www.apkmania.co/2013/07/blood-sword-thd-v16-apk.html";
Document content = null;
try {
content = Jsoup.connect(url).userAgent("Mozilla").timeout(10*1000).get();
} catch (final IOException e) {
// TODO Auto-generated catch block
System.out.println("IOException: " + e.toString());
}
System.out.println(content);
And I'm getting
<!DOCTYPE html PUBLIC "- ...

I solved the problem here is the code if anybody in future needs
String url = "http://www.apkmania.co/2013/07/blood-sword-thd-v16-apk.html"
URL url1;
InputStream in = null;
try {
url1 = new URL(url);
URLConnection urlConnection = url1.openConnection();
in = new BufferedInputStream(urlConnection.getInputStream());
} catch (MalformedURLException e2) {
// TODO Auto-generated catch block
e2.printStackTrace();
}
Document content= null;
try {
content = Jsoup.parse(in, "UTF-8", url);
} catch (IOException e1) {
// TODO Auto-generated catch block
System.out.println(e1.getMessage());
e1.printStackTrace();
}

Related

Java SocketException: Network is unreachable, but the URL works in the browser

I am trying to read some data via a GET request as
URL corpusDbUrl = null;
try {
corpusDbUrl = new URL("http://xxx.cc.ww.tt:1234/erwet/erherh/iouiiu");
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
URLConnection corpusDbConn = null;
try {
corpusDbConn = corpusDbUrl.openConnection();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
BufferedReader in = null;
try {
in = new BufferedReader(new InputStreamReader(corpusDbConn.getInputStream()));
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
String inputLine="";
try {
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
} catch (Exception e) { // TODO Auto-generated catch block
e.printStackTrace();
}
and i get
java.net.SocketException: Network is unreachable
but when i try the URL in browser, it works just fine. What am i doing wrong?

InvocationTargetException for ClassLoaders.callStaticFunction Java Eclipse

I have created a program to convert text to xml by using ReverseXSL API.
This program is to be executed by an application by calling static method (static int transformXSL).
I am able to execute and produce output with running from Eclipse. However, When I ran program (jar) by using application it stuck somewhere and I couldnt find anything.
Then, I debugged by "Debug as...-> Remote Java Application" in Eclipse from Application and found "InvocationTargetException" at ClassLoaders.callStaticFunction.
Below Static method is called by application.
public class MyTest4 {
public MyTest4()
{
}
public static int transformXSL(String defFile, String inputFile, String XSLFile, String OutputFile) {
System.out.println("Dheeraj's method is called");
// start time
FileWriter fw=null;
try {
fw = new FileWriter("D://Countime.txt");
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
BufferedWriter output=new BufferedWriter(fw);
DateFormat sd=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
Date dt= new Date();
System.out.println("Date is calculated");
try {
output.write("Start Time:"+sd.format(dt).toString());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println(sd.format(dt));
FileReader myDEFReader=null, myXSLReader=null;
TransformerFactory tf = TransformerFactory.newInstance();
Transformer t=null;
FileInputStream inStream = null;
ByteArrayOutputStream outStream = null;
// Step 1:
//instantiate a transformer with the specified DEF and XSLT
if (new File(defFile).canRead())
{
try {
myDEFReader = new FileReader(defFile);
System.out.println("Definition file is read");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
else myDEFReader = null;
if (new File(XSLFile).canRead())
try {
myXSLReader = new FileReader(XSLFile);
System.out.println("XSL file is read");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
else myXSLReader = null;
try {
t = tf.newTransformer(myDEFReader, myXSLReader);
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("Step 1: DEF AND XSLT Transformation completed");
// Step 2:
// Read Input data
try {
inStream = new FileInputStream(inputFile);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
outStream = new ByteArrayOutputStream();
System.out.println("Step 2: Reading Input file: completed");
// Step 3:
// Transform Input
try {
try (BufferedReader br = new BufferedReader(new FileReader("D://2.txt"))) {
String line = null;
while ((line = br.readLine()) != null) {
System.out.println("Content: "+line);
}
}
System.out.println("File: "+inputFile.toString());
System.out.println("\n content: \n"+ inStream.toString());
System.out.println("Calling Transform Function");
t.transform(inStream, outStream);
System.out.println("Transformation is called");
outStream.close();
try(OutputStream outputStream = new FileOutputStream(OutputFile)) {
outStream.writeTo(outputStream);
System.out.println("Outstream is generated; Output file is creating");
}
System.out.println(outStream.toString());
} catch (TransformerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ParserException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ParserConfigurationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (FactoryConfigurationError e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (TransformerFactoryConfigurationError e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (javax.xml.transform.TransformerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("output file is created");
// End time
Date dt2= new Date();
System.out.println(sd.format(dt2));
System.out.println("End time:"+dt2.toString());
try {
output.append("End Time:"+sd.format(dt2).toString());
output.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return 0;
}
}

Android really long execution time of HTTP request

i have a problem in my app. At start i will describe how it works. It send request to the server (which wasn't created by me), and in the response i am getting JSON string ( it's not so big average 10 records in JSONarray) there in one parameter there is URL to the picture which i download and save as a bitmap for every JSON object. To summarize i download a JSON which looks like:
{
"id":"125",
"description":"desc",
"type":"type",
"state":"state",
"date":"2012-09-22 10:40:46.0",
"lat":"52.321911",
"lng":"19.464111",
"author":"user",
"photo":"GetImage?size=small&id=0",
"comments":[
]
}
x 10 for example, and then i download from URL "photo" image for every object. Problem lies in the time of execution, it is really really long which it should't it is not big data. Here it is how i do this: AsyncClass that download image:
private class HttpGetImage extends AsyncTask<Object, Integer, Integer>
{
public boolean ready = false;
public boolean success = false;
#Override
protected Integer doInBackground(Object... obj) {
DefaultHttpClient client = new MyHttpClient(ViewEdit.this);
HttpGet get = new HttpGet(url+photoUrl);
HttpResponse getResponse = null;
try {
getResponse = client.execute(get);
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
}
HttpEntity responseEntity = getResponse.getEntity();
BufferedHttpEntity httpEntity = null;
try {
httpEntity = new BufferedHttpEntity(responseEntity);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
ready=true; return null;
}
InputStream imageStream = null;
try {
imageStream = httpEntity.getContent();
m_orders.get((Integer)obj[1])
.setOrderBitmap(BitmapFactory.decodeStream(imageStream));
success = true;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
} finally {
try {
imageStream.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
}
}
ready = true;
return null;
}
}
and AsyncClass for jsonString:
private class HttpGetNotifications extends AsyncTask<Double, Integer, Integer>
{
public boolean ready = false;
public boolean success = false;
#Override
protected Integer doInBackground(Double... params)
{
DefaultHttpClient client = new MyHttpClient(ViewEdit.this);
HttpGet get = new HttpGet(url + Double.toString(params[0]) + "&longitude=" + Double.toString(params[1]) + "&radius="+distance);
HttpResponse getResponse = null;
try {
getResponse = client.execute(get);
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
}
HttpEntity responseEntity = getResponse.getEntity();
String entityContents="";
try {
entityContents = EntityUtils.toString(responseEntity);
loadNotifications(entityContents);
success = true;
} catch (ParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
ready=true; return null;
}
ready = true;
return null;
}
public void loadNotifications(String jsonstring) throws JSONException
{
JSONObject jsonResponse = new JSONObject(jsonstring);
JSONArray notifi = jsonResponse.getJSONArray("notifications");
for (int i =0, count = notifi.length(); i <count; i++){
//storage of data
}
}
}
Maybe you guys have an idea how can i optimize that code to reduce a time of execution?
Put in some profiling to find what's taking the time. In each of your AsyncTasks:
private long time0, time1;
#Override protected void onPreExecute() {
time0 = System.currentTimeMillis();
}
#Override protected void onPostExecute(HttpResponse response) {
time1 = System.currentTimeMillis();
long deltaT = (time1 - time0);
Log.d(TAG, "Execute took "+deltaT+"ms");
}
And then go from there.

Client Handler Won't Start

Im trying to make a basic email system in Java. I have a server that the clients connect to and a gui for the clients. When the server picks up a new connection from a client it starts a new thread running that handles all the different operations. The problem is that the server accepts the new client but doesnt start a new client handler thread running. Any help?
public class Server {
public static Connection link;
public static void main(String[] args) throws IOException {
ServerSocket serverSocket = null;
final int PORT = 1234; // Define the sockets and ports and i/o
Socket client;
ClientHandler handler;
try {
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
link = DriverManager.getConnection("jdbc:odbc:emails", "", "");
System.out.println("Connected to Database . . . ");
} catch (ClassNotFoundException cnfEx) {
System.out.println("* Unable to load driver! *");
System.exit(1);
} catch (SQLException sqlEx) {
System.out.println("* Cannot connect to database! *");
System.exit(1);
}
try {
serverSocket = new ServerSocket(PORT); // Set the server socket
} catch (IOException ioEx) {
System.out.println("\nUnable to set up port!"); // If failed let the
// user know
System.exit(1); // Exit the system with error code 1
}
System.out.println("\nServer running...\n"); // Tell the user the server
// is running
do {
client = serverSocket.accept();
// Wait for client.
System.out.println(client);
handler = new ClientHandler(client);
System.out.println(handler);
System.out.println("\nNew client accepted.\n"); // Tell the user the
// server has accepted
// a client
System.out.println("RUNNING");
handler.start(); // Start the handler
} while (true); // Continuous loop
}
static class ClientHandler extends Thread {
/**
*
*/
private Socket client;
private ObjectInputStream input; // Define sockets, i/o and local array
// list
private ObjectOutputStream output;
public ClientHandler(Socket socket) throws IOException // Client handler
// constructor
{
client = socket;
input = new ObjectInputStream(socket.getInputStream());// Set client
// and i/o
// stream
output = new ObjectOutputStream(socket.getOutputStream());
}
public void run() {
String userCommand = null;
System.out.println("RUNNING CLIENT HANDLER");
boolean quit = false;
do {
try {
userCommand = (String) input.readObject();
} catch (IOException e2) {
// TODO Auto-generated catch block
e2.printStackTrace();
} catch (ClassNotFoundException e2) {
// TODO Auto-generated catch block
e2.printStackTrace();
} // Receive user command
if (userCommand.equals("SEND MESSAGE")) // If user command is to
// send a message
{
String username = null, recipient = null, message = null, insert, fileType = null;
try {
username = (String) input.readObject();
recipient = (String) input.readObject(); // Receive username,
// recipient and
// message
message = (String) input.readObject();
fileType = (String) input.readObject();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (ClassNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
insert = "INSERT INTO [emails] ([From], [To], [Message], [Attachment])"
+ " VALUES('"
+ username
+ "','"
+ recipient
+ "','"
+ message + "','" + fileType + "')";
Statement statement = null;
try {
statement = link.createStatement();
statement.executeUpdate(insert);
link.commit();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
getFile(input, fileType);
} catch (IOException ioEx) {
ioEx.printStackTrace();
} catch (ClassNotFoundException cnfEx) {
cnfEx.printStackTrace();
}
System.out.println(username // Tell the user a message has been
// sent
+ " sent message to " + recipient);
}
if (userCommand.equals("READ MESSAGE")) // If user command is to
// read a message
{
String username = null; // Define local variables
try {
username = (String) input.readObject();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (ClassNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} // Receive the username of the current user
Statement statement = null;
String insert = "SELECT [Email No],[From],[Message],[Attachment] FROM [emails] WHERE To = '"
+ username + "'";
ResultSet rs = null;
try {
statement = link.createStatement();
rs = statement.executeQuery(insert);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
while (rs.next()) {
String from = rs.getString("From");
String message = rs.getString("Message");
String attachment = rs.getString("Attachment");
int emailNo = rs.getInt("Email no");
output.writeObject(from);
output.writeObject(message); // Send the username the
// message is from
output.writeObject(attachment);
output.writeObject(emailNo);
output.flush(); // and the message and flush the output
// stream
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
output.writeObject("END");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} // Send an end message
try {
output.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} // Flush the output stream
System.out.println(username // Tell the user someone is reading
// their messages
+ " reading messages.");
}
if (userCommand.equals("QUIT")) // If the user command is quit
{
quit = true; // Set quit to true
}
if (userCommand.equals("DELETE MESSAGE")) // If the user command is
// delete a message
{
int valueToDelete = 0; // Define local variables
try {
valueToDelete = Integer.parseInt((String) input.readObject());
} catch (NumberFormatException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (ClassNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} // Receive the value to delete
Statement statement = null;
String delete = "DELETE FROM emails WHERE [Email No] = "
+ valueToDelete + "";
try {
statement = link.createStatement();
statement.executeUpdate(delete);
link.commit();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("Deleted message " + valueToDelete); // Tell
// the
// user a
// message
// has
// been
// deleted
}
} while (!quit); // Run while quit is false
System.out.println("Client Closed"); // Tell the user the client has // closed
}
private static void getFile(ObjectInputStream inStream, String fileType)
throws IOException, ClassNotFoundException {
byte[] byteArray = (byte[]) inStream.readObject();
FileOutputStream mediaStream = null;
if (fileType.equals("Text File"))
mediaStream = new FileOutputStream("file.txt");
else if (fileType.equals("Image"))
mediaStream = new FileOutputStream("file.gif");
else if (fileType.equals("Movie"))
mediaStream = new FileOutputStream("file.mpeg");
mediaStream.write(byteArray);
}
}
}
When you are instantiating the ObjectInputStream it will block until the client sends a sufficient header, thus you never get to actually run the thread... See also

Retry a connection on timeout in Java

I have a method (below) that pulls down and returns the source of a webpage as a String. It all works fine and dandy, but when the connection times out, the program throws an exception and exits. Is there a better method to do this that would allow it to try again on timeout, or is there a way to do it within this method?
public static String getPage(String theURL) {
URL url = null;
try {
url = new URL(theURL);
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
exitprint();
}
InputStream is = null;
try {
is = url.openStream();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
exitprint();
}
int ptr = 0;
StringBuffer buffer = new StringBuffer();
try {
while ((ptr = is.read()) != -1) {
buffer.append((char)ptr);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
exitprint();
}
return buffer.toString();
}
Here's a refactoring of your code that should retry the download N times. Haven't tested it though, but it should kick you off in the right direction.
public static String getPage(String theURL) {
URL url = null;
try {
url = new URL(theURL);
} catch (MalformedURLException e) {
e.printStackTrace();
exitprint();
}
for (int i = 0; i < N; i++) {
try {
InputStream is = url.openStream();
int ptr = 0;
StringBuffer buffer = new StringBuffer();
while ((ptr = is.read()) != -1)
buffer.append((char)ptr);
} catch (IOException e) {
continue;
}
return buffer.toString();
}
throw new SomeException("Failed to download after " + N + " attepmts");
}
I think AOP and Java annotations is a good option. I would recommend to use a read-made mechanism from jcabi-aspects:
#RetryOnFailure(attempts = 2, delay = 10)
public String load(URL url) {
return url.openConnection().getContent();
}
Write a wrapper function around it and allow the connect exception to propogate out. Then you can loop calling your existing function while you receive connect exception upto some max retries.
This is better than embedding a for loop in your existing function because it logically separates retry logic from mainline code. And it's easier to read and understand as a result.
Instead of
try {
is = url.openStream();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
exitprint();
}
you can try set longer timeout and you can still handle timeout exception by catching it
try {
URLConnection con= url.openConnection();
con.setConnectTimeout(5000);
con.setReadTimeout(50000);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
} catch (SocketTimeoutException e) {
//here you can still handle timeout like try again under certain conditions
}
You could put the whole thing in a while loop:
while (true) {
try {
...
} catch (IOException e) {
continue;
}
return buffer.toString();
}
The return statement will break you out of the loop. You might also want to keep track of the number of attempts and stop after 5-10, for politeness, but that's the basic shape of it.
Edit
The better version, based on comments:
int retries = 10;
for (int i = 0 ; i < retries ; i++) {
try {
...
} catch (IOException e) {
continue;
}
return buffer.toString();
}

Categories

Resources