I'm creating an app where I need a function to get plain text from a website. I am able to get the text and print it out on my PC just fine, but when I try running it on an Android device, the app won't start.
I believe it has something to do with throwing an IOException. I've been reading that I am not supposed to do that because I don't define the interface. Is there a way to get around this? Android Studio won't compile my code if I don't throw the exception.
The function:
public String getText(String site) throws IOException {
// Make a URL to the web page
URL url = new URL(site);
// Get the input stream through URL Connection
URLConnection con = url.openConnection();
InputStream is =con.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
// read each line and return the final text
String res = "";
String line = null;
while ((line = br.readLine()) != null) {
//System.out.println(line);
res += line;
}
return res;
}
And this is how Android Studio makes me run it in the onCreate method:
String text = null;
try {
text = getText("http://myWebsite.com");
} catch (IOException e) {
e.printStackTrace();
}
Toast.makeText(getApplicationContext(), text, Toast.LENGTH_LONG).show();
First, read your logcat - you should see your exception there with full stacktrace. Second, there is nothing wrong with catching IOException, but you must do something with it once cached - like inform user of problem in functionality - like no more space, etc.
And this is how Android Studio makes me run it in the onCreate method:
this is a problem, because your are getting data from your site on UI thread, you must do it from worker thread, ie. AsyncTask.
You can not do it in the main thread
try this
class MyTask extends AsyncTask<Void, Void, String>{
private String site;
MyTask(String site) {
this.site = site;
}
#Override
protected String doInBackground(Void... params) {
try {
URL url = new URL(site);
URLConnection con = url.openConnection();
InputStream is =con.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
// read each line and return the final text
String res = "";
String line = null;
while ((line = br.readLine()) != null) {
//System.out.println(line);
res += line;
}
return res;
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute(String s) {
super.onPostExecute(s);
if(s != null){
Toast.makeText(getApplicationContext(), s, Toast.LENGTH_LONG).show();
}
}
}
where to get a string is used as
new MyTask("http://myWebsite.com").execute()
Related
I am trying to download the HTML of page. After it downloads I try to Log it. Everything goes smoothly but the HTML stops at a certain point every time, even though it has a lot more HTML to show.
I tried using a different page, my page which just has some instructions for my Company and it worked perfectly. Is there a limit maybe? I tried it with urlconnection.connect(), and without it and there is no difference.
public class MainActivity extends AppCompatActivity {
public class DownloadHTML extends AsyncTask<String, Void, String>{
#Override
protected String doInBackground(String... urls) {
URL url;
String result = "";
HttpURLConnection urlConnection = null;
try {
url = new URL(urls[0]);
urlConnection = (HttpURLConnection)url.openConnection();
InputStream in = urlConnection.getInputStream();
InputStreamReader reader = new InputStreamReader(in);
int data = reader.read();
while (data!=-1){
char current = (char) data;
result += current;
data = reader.read();
}
return result;
} catch (Exception e) {
e.printStackTrace();
return "Fail";
}
}
}
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
String Result = "";
DownloadHTML task = new DownloadHTML();
try {
Result = task.execute("http://www.posh24.se/kandisar").get();
} catch (Exception e) {
e.printStackTrace();
}
Log.i("URL", Result);
}
}
Here is the splitting and it wont work.
try {
Result = task.execute("http://www.posh24.se/kandisar").get();
String[] splitStrings = Result.split("<div class=\"channelListEntry\">");
Pattern p = Pattern.compile("<img src=\"(.*?)\"");
Matcher m = p.matcher(splitStrings[0]);
while (m.find()){
CelebUrls.add(m.group(1));
}
p = Pattern.compile("alt=\"(.*?)\"");
m = p.matcher(splitStrings[0]);
while (m.find()){
CelebNames.add(m.group(1));
}
} catch (Exception e) {
e.printStackTrace();
}
Log.i("URL", Arrays.toString(CelebUrls.toArray()));
}
}
Modifing your method like this will give you the content of the html page in UTF-8 format.
(In this case its UTF-8 because the page is encoded like that, in doubt you can pass Charset.forName("utf-8") as second paramter to the constructor of InputStreamReader)
When testing you example implementation I only got some output with various unreadable characters.
Ignore the class and the method changes, I only made them to have a standalone example.
public class ParsingTest {
static String doInBackground(String address) {
URL url;
StringBuilder result = new StringBuilder(1000);
HttpURLConnection urlConnection = null;
try {
url = new URL(address);
urlConnection = (HttpURLConnection)url.openConnection();
InputStream in = urlConnection.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
String line = reader.readLine();
while (line != null){
result.append(line);
result.append("\n");
line = reader.readLine();
}
return result.toString();
} catch (Exception e) {
e.printStackTrace();
return "Fail";
}
}
public static void main(String[] args) {
String result = doInBackground("http://www.posh24.se/kandisar");
System.out.println(result);
}
}
If the only part that interest you are the images of the top100, you can just adjust the while loop to:
String line = reader.readLine();
while (line != null){
if (line.contains("<div class=\"channelListEntry\">")) {
reader.readLine();
reader.readLine();
line = reader.readLine().trim();
// At this points its probably easier to use a List<String> for the result instead
result.append(line);
result.append("\n");
}
line = reader.readLine();
}
This is only a simplied example based on the current design of the page, where the img comes 3 lines after the declaration of the div.
If you want to you can also just extract the url of the image and the alt description directly at this point. Instead of using complicated regex you could rely on the String#indexOf instead.
private static final String SRC = "src=\"";
private static final String ALT = "\" alt=\"";
private static final String END = "\"/>";
public static void extract(String image) {
int index1 = image.indexOf(SRC);
int index2 = image.indexOf(ALT);
int index3 = image.indexOf(END);
System.out.println(image);
System.out.println(image.substring(index1 + SRC.length(), index2));
System.out.println(image.substring(index2 + ALT.length(), index3));
}
Note that if you directly process the content from the page your app does not require the memory to store the full page.
I want to get an HTML from a web page, remove some tags from the code and display it using a TextView... But those HTMLs are too big to be temporaly stored into a String...
When I try this way:
String html = "myBigHTML";
myTextView.setText(fromHtml(html));
compiler says error: constant string too long
If I put the html into a .txt and try this way:
InputStream is = getAssets().open("html.txt");
tvTeste.setText(fromHtml(convertStreamToString(is)));
public static String convertStreamToString(InputStream is) throws Exception {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line).append("\n");
}
reader.close();
return sb.toString();
}
It works but the app gets soooo slow, almost freezes... And also, if I store it in a .txt I couldn't work with the tags...
.:: EDIT ::.
My onCreate() method as asked...
private TextView tvTeste;
private InputStream is;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.layout_frequencia);
getSupportActionBar().setDisplayHomeAsUpEnabled(true);
getSupportActionBar().setHomeButtonEnabled(true);
tvTeste = (TextView)findViewById(R.id.tvTeste);
try {
is = getAssets().open("html.txt");
} catch (IOException e) {
e.printStackTrace();
}
BufferedReader br = new BufferedReader(new InputStreamReader(is));
String strLine;
List<String> stringList = new ArrayList<>();
try {
while ((strLine = br.readLine()) != null) {
stringList.add(strLine);
}
} catch (Exception e) {
e.printStackTrace();
}
tvTeste.setText(fromHtml(TextUtils.join("",stringList)));
}
Let's try this: each line of HTML text is a String. Each String is inside a List of String.
So, some pseudocode:
List<String> stringList = new ArrayList<>();
while (htmlHandler.next()) {
stringList.add(fromHtml(htmlHandler.readLine()));
}
myTextView.setText(joinStringArray(stringList));
Where joinStringArray uses a StringBuilder to produce a single big String object.
Basically you shouldn't read the entire web page, but you should read it sequentially.
Another point to mark. You should avoid any time consuming process that blocks the activity. try the same using, for example an AsyncTask.
Please check https://developer.android.com/reference/android/os/AsyncTask.html
I was being a bit naive, thinking I can read webpages the same way as you do in Java, but apparently you need to use threads or AsyncTask. Which I have no prior experience in and are proving to be hassle.
I've read http://www.vogella.com/tutorials/AndroidBackgroundProcessing/article.html (3. point) which I semi understand but when I try to implement into my problem it all falls apart.
Here's my problem:
I need to read from an URL, it's a RSS feed.
Parse it all together
Insert it into ListView (arrayadapter)
This is my AsyncTask class, that I tried to recreate. I just want it to return a buffered reader, or the whole page but it's always null.
private class DownloadWebPageTask extends AsyncTask<String, Void, BufferedReader> {
#Override
protected BufferedReader doInBackground(String... urls) {
BufferedReader bs;
URL url = null;
try {
url = new URL(urls[0]);
} catch (MalformedURLException e) {
e.printStackTrace();
}
try {
bs = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"));
return bs;
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
protected void onPostExecute(BufferedReader result) {
br = result;
}
}
public RSS() throws IOException, ExecutionException, InterruptedException {
DownloadWebPageTask dl = new DownloadWebPageTask();
dl.execute(new String[] {"http://www.promet.si/dc/PROMET.ROADEVENTS.PP.RSS.SL"});
}
I'm sorry if this is a stupid question, but I don't fully understand posts explaining this and I have to solve this problem. I just need the page's content in any way shape or form. (Can be a String, BufferedReader)
#Override
protected String doInBackground(String... urls) throws IOException {
URLConnection connection = new URL(urls[0]).openConnection();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(connection.getInputStream()),"UTF-8");
StringBuilder stringBuilder = new StringBuilder();
String buffer;
while ((buffer = bufferedReader.readLine()) != null) {
stringBuilder.append(buffer);
}
return stringBuilder.toString();
}
Use the buffered reader to read line by line in the background thread, and make it return string.
I'm a android newbie.
I need to create some kind of preloader for the Activity. Right now I click the button say "Show company" and afterwards I go to the next activity where the data is loaded from serwer and shown inside. The problem is that (from what I understand) the activity is connecting with internet and until connections is done nothing is show. User must wait and then suddelny (after few seconds - varies) he gets new 100% ready page based on new activity.
The best for me would be sth like: create a loding animation that is shown until the activity is fully loaded. (that would solve problem everywhere)
Alternative would be: loading the new activity before connecting with the internet url. When its loaded it say sth default like "Loading data" until full text is downloaded from url that will replace the first text.
Here is the code I use to load text from URL.
try {
// Create a URL for the desired page
URL url = new URL(serwer_url);
// Read all the text returned by the server
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String str;
while ((str = in.readLine()) != null) {
// str is one line of text; readLine() strips the newline character(s)
Plain_str = Plain_str + str;
}
Log.i("Plain read str", Plain_str);
in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {}
//end of reading file
//here is the anchor to the text in activity
TextView MainText = (TextView)findViewById(R.id.TextMain);
MainText.setText(Html.fromHtml(Plain_str.toString()));
You can use an AsyncTask like this:
protected class Mytask extends AsyncTask<Void, Void, String>{
#Override
protected String doInBackground(Void... params) {
String Plain_str= null;
try {
// Create a URL for the desired page
URL url = new URL(serwer_url);
// Read all the text returned by the server
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String str;
while ((str = in.readLine()) != null) {
// str is one line of text; readLine() strips the newline character(s)
Plain_str = Plain_str + str;
}
Log.i("Plain read str", Plain_str);
in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {}
return Plain_str;
}
protected void onPostExecute(String str){
TextView MainText = (TextView)findViewById(R.id.TextMain);
MainText.setText(Html.fromHtml(str.toString()));
}
}
And then to execute the Task
new MyTask().execute();
I have a simple java code which gets html text from the input url:
try {
URL url = new URL("www.abc.com");
// Get the response
BufferedReader rd = new BufferedReader(new InputStreamReader(url.openStream()));
while ((line = rd.readLine()) != null) {
String code = code + line;
} catch (IOException e){}
I am using this code in an android project. Now the problem comes when there is no internet connectivity. The application just halts and later gives error.
Is there some way to break this after some fixed timeout, or even return some specific string after an exception is thrown. Can you please tell me how to do that??
Try this:
try
{
URL url = new URL("www.abc.com");
String newline = System.getProperty("line.separator");
InputStream is = url.openStream();
if (is != null)
{
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
StringBuilder contents = new StringBuilder();
while ((line = rd.readLine()) != null)
{
contents.append(line).append(newline);
}
}
else
{
System.out.println("input stream was null");
}
}
catch (Exception e)
{
e.printStackTrace();
}
An empty catch block is asking for trouble.
I don't know what the default timeout is for URL, and a quick look at the javadocs doesn't seem to reveal anything. So try using HttpURLConnection directly instead http://download.oracle.com/javase/1.5.0/docs/api/java/net/HttpURLConnection.html. This lets you set timeout values:
public static void main(String[] args) throws Exception {
URL url = new URL("http://www.google.com");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(5000); // 5 seconds
conn.setRequestMethod("GET");
conn.connect();
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
conn.disconnect();
}
You can also set a read time out as well, as well as specify behaviour re redirects and a few other things.
I think in addition to timeouts it could be also smart to check the Internet availability right before the requesting:
public class ConnectivityHelper {
public static boolean isAnyNetworkConnected(Context context) {
return isWiFiNetworkConnected(context) || isMobileNetworkConnected(context);
}
public static boolean isWiFiNetworkConnected(Context context) {
return getWiFiNetworkInfo(context).isConnected();
}
public static boolean isMobileNetworkConnected(Context context) {
return getMobileNetworkInfo(context).isConnected();
}
private static ConnectivityManager getConnectivityManager(Context context) {
return (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE);
}
}
UPDATE: For timeouts see an excellent kuester2000's reply here.
Just a general tip on working with Streams always close them when they are no longer needed. I just wanted to post that up as it seems that most people didn't take care of it in their examples.