Develop Reference

How to work with big HTML String?

I want to get an HTML from a web page, remove some tags from the code and display it using a TextView... But those HTMLs are too big to be temporaly stored into a String...
When I try this way:
String html = "myBigHTML";
myTextView.setText(fromHtml(html));
compiler says error: constant string too long
If I put the html into a .txt and try this way:
InputStream is = getAssets().open("html.txt");
tvTeste.setText(fromHtml(convertStreamToString(is)));
public static String convertStreamToString(InputStream is) throws Exception {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line).append("\n");
}
reader.close();
return sb.toString();
}
It works but the app gets soooo slow, almost freezes... And also, if I store it in a .txt I couldn't work with the tags...
.:: EDIT ::.
My onCreate() method as asked...
private TextView tvTeste;
private InputStream is;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.layout_frequencia);
getSupportActionBar().setDisplayHomeAsUpEnabled(true);
getSupportActionBar().setHomeButtonEnabled(true);
tvTeste = (TextView)findViewById(R.id.tvTeste);
try {
is = getAssets().open("html.txt");
} catch (IOException e) {
e.printStackTrace();
}
BufferedReader br = new BufferedReader(new InputStreamReader(is));
String strLine;
List<String> stringList = new ArrayList<>();
try {
while ((strLine = br.readLine()) != null) {
stringList.add(strLine);
}
} catch (Exception e) {
e.printStackTrace();
}
tvTeste.setText(fromHtml(TextUtils.join("",stringList)));
}

Let's try this: each line of HTML text is a String. Each String is inside a List of String.
So, some pseudocode:
List<String> stringList = new ArrayList<>();
while (htmlHandler.next()) {
stringList.add(fromHtml(htmlHandler.readLine()));
}
myTextView.setText(joinStringArray(stringList));
Where joinStringArray uses a StringBuilder to produce a single big String object.
Basically you shouldn't read the entire web page, but you should read it sequentially.

Another point to mark. You should avoid any time consuming process that blocks the activity. try the same using, for example an AsyncTask.
Please check https://developer.android.com/reference/android/os/AsyncTask.html

Java Wget Bz2 file

I'm trying to webget some bz2 files from Wikipedia, I don't care whether they are save as bz2 or unpacked, since I can unzip them locally.
When I call:
public static void getZip(String theUrl, String filename) throws IOException {
URL gotoUrl = new URL(theUrl);
try (InputStreamReader isr = new InputStreamReader(new BZip2CompressorInputStream(gotoUrl.openStream())); BufferedReader in = new BufferedReader(isr)) {
StringBuffer sb = new StringBuffer();
String inputLine;
// grab the contents at the URL
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine + "\r\n");
}
// write it locally
Wget.createAFile(filename, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
I get a part of the unzipped file, never more than +- 883K.
When I don't use the BZip2CompressorInputStream, like:
public static void get(String theUrl, String filename) throws IOException {
try {
URL gotoUrl = new URL(theUrl);
InputStreamReader isr = new InputStreamReader(gotoUrl.openStream());
BufferedReader in = new BufferedReader(isr);
StringBuffer sb = new StringBuffer();
String inputLine;
// grab the contents at the URL
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine);// + "\r\n");
}
// write it locally
Statics.writeOut(filename, false, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
I get a file of which the size is the same as it suppose to (compared to the KB not B). But also a message that that the zipped file is damaged, also when using byte [] instead of readLine(), like:
public static void getBytes(String theUrl, String filename) throws IOException {
try {
char [] cc = new char[1024];
URL gotoUrl = new URL(theUrl);
InputStreamReader isr = new InputStreamReader(gotoUrl.openStream());
BufferedReader in = new BufferedReader(isr);
StringBuffer sb = new StringBuffer();
// grab the contents at the URL
int n = 0;
while (-1 != (n = in.read(cc))) {
sb.append(cc);// + "\r\n");
}
// write it locally
Statics.writeOut(filename, false, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
Finally, when I bzip2 the inputstream and outputstream, I get a valid bzip2 file, but of the size like the first one, using:
public static void getWriteForBZ2File(String urlIn, final String filename) throws CompressorException, IOException {
URL gotoUrl = new URL(urlIn);
try (final FileOutputStream out = new FileOutputStream(filename);
final BZip2CompressorOutputStream dataOutputStream = new BZip2CompressorOutputStream(out);
final BufferedInputStream bis = new BufferedInputStream(gotoUrl.openStream());
final CompressorInputStream input = new CompressorStreamFactory().createCompressorInputStream(bis);
final BufferedReader br2 = new BufferedReader(new InputStreamReader(input))) {
String line = null;
while ((line = br2.readLine()) != null) {
dataOutputStream.write(line.getBytes());
}
}
}
So, how do I get the entire bz2 file, in either bz2 format or unzipped?

A bz2 file contains bytes, not characters. You can't read it as if it contained characters, with a Reader.
Since all you want to do is download the file and save it locally, all you need is
Files.copy(gotoUrl.openStream(), Paths.get(fileName));

Java getting id from csv file to combo box

I am have a program were there are two forms one for the consultant and for the customer. On the first form the user will enter the consultant details and his ID will be saved in a csv file and this works fine.
Consultant cons_save = new Consultant();
cons_save.setPersonfirstname(this.jTextField1.getText());
cons_save.setPersonlastname(this.jTextField2.getText());
cons_save.setPersonID(this.jTextField4.getText());
this.jTextField1.setText("");
this.jTextField2.setText("");
this.jTextField3.setText("");
cons_save.ConsultantID = cons_save.PersonID;
cons_save.setConsultantID(this.jTextField4.getText());
this.jTextField4.setText("");
try
{
BufferedWriter writer = new BufferedWriter(new FileWriter("E:\\ryan_assignment_sit2\\ConsID\\consID.csv", true));
writer.append(cons_save.ConsultantID);
writer.append(",");
writer.flush();
writer.close();
}
catch(IOException e)
{
e.printStackTrace();
}
cons_save.savecons();
After the id is saved the Id is called out from the csv to an array and this works fine as well.
public CreateCustomer() {
initComponents();
ArrayList<String> ConsIDList = new ArrayList<String>();
String csvFileToRead = "E:\\ryan_assignment_sit2\\ConsID\\consID.csv"; // Reads the CSV File.
BufferedReader br = null; // Creates a buffer reader.
String line = "";
String splitBy = ","; // Reader Delimiter
try {
br = new BufferedReader(new FileReader(csvFileToRead)); // Buffer Reader with file name to read.
Scanner reader = new Scanner(System.in);
while ((line = br.readLine()) != null) { //While there is a line to read.
reader = new Scanner(line);
reader.useDelimiter(splitBy);
while (reader.hasNext()) { // While there is a next value (token).
ConsIDList.add(reader.next());
}
}
} catch (FileNotFoundException exception) { // Exception Handler if the File is not Found.
exception.printStackTrace();
} catch (IOException exception) { // Input/Output exception
exception.printStackTrace();
} finally {
if (br != null) {
try {
br.close(); // Close the Scanner.
} catch (IOException exception) {
exception.printStackTrace();
}
}
Vector<String> vectorData = new Vector<String>(ConsIDList);
DefaultComboBoxModel<String> comboBoxModel = new DefaultComboBoxModel<>(vectorData);
this.jComboBox1.setModel(comboBoxModel);
}
}
The array is working fine but the combo box is not getting populated with the arraylist.

Cannot read data from file

I am trying to read values from CSV file which is present in package com.example.
But when i run code with the following syntax:
DataModel model = new FileDataModel(new File("Dataset.csv"));
It says:
java.io.FileNotFoundException:Dataset.csv
I have also tried using:
DataModel model = new FileDataModel(new File("/com/example/Dataset.csv"));
Still not working.
Any help would be helpful.
Thanks.

If this is the FileDataModel from org.apache.mahout.cf.taste.impl.model.file then it can't take an input stream and needs just a file. The problem is you can't assume the file is available to you that easily (see answer to this question).
It might be better to read the contents of the file and save it to a temp file, then pass that temp file to FileDataModel.
InputStream initStream = getClass().getClasLoader().getResourceAsStream("Dataset.csv");
//simplistic approach is to put all the contents of the file stream into memory at once
// but it would be smarter to buffer and do it in chunks
byte[] buffer = new byte[initStream.available()];
initStream.read(buffer);
//now save the file contents in memory to a temporary file on the disk
//choose your own temporary location - this one is typical for linux
String tempFilePath = "/tmp/Dataset.csv";
File tempFile = new File(tempFilePath);
OutputStream outStream = new FileOutputStream(tempFile);
outStream.write(buffer);
DataModel model = new FileDataModel(new File(tempFilePath));
...

public class ReadCVS {
public static void main(String[] args) {
ReadCVS obj = new ReadCVS();
obj.run();
}
public void run() {
String csvFile = "file path of csv";
BufferedReader br = null;
String line = "";
String cvsSplitBy = ",";
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
// Do stuff here
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
System.out.println("Done");
}
}

CSV file which is present in package com.example
You can use getResource() or getResourceAsStream() to access the resource from within the package. For example
InputStream is = getClass().getResourceAsStream("/com/example/Dataset.csv");//uses absolute (package root) path
BufferedReader br = new BufferedReader(new InputStreamReader(is));
//read from BufferedReader
(note exception handling and file closing are omitted above for brevity)

Reading InputStream to Arraylist

I have to read a dict.txt file which contains one string for line and add these to an arraylist.
I tried this:
public ArrayList<String> myDict = new ArrayList<String>();
InputStream is = (getResources().openRawResource(R.raw.dict));
BufferedReader r = new BufferedReader(new InputStreamReader(is));
try {
while (r.readLine() != null) {
myDict.add(r.readLine());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
but something wrong...

You are iterating twice in each loop
String line;
while ((line=r.readLine()) != null) {
myDict.add(line);
}

Using Apache IOUtils:
List<String> lines = IOUtils.readLines(inputStream, "UTF-8");