Develop Reference

Android Reading a large text efficiently in Java

My code is too slow
How can I make my code efficiently? Currently the code needs several minutes until the file was read, which is way too long. Can this be done faster? There is no stacktrace, because it works, but too slow.
Thanks!
The Problem Code:
private void list(){
String strLine2="";
wwwdf2 = new StringBuffer();
InputStream fis2 = this.getResources().openRawResource(R.raw.list);
BufferedReader br2 = new BufferedReader(new InputStreamReader(fis2));
if(fis2 != null) {
try {
LineNumberReader lnr = new LineNumberReader(br2);
String linenumber = String.valueOf(lnr);
int i=0;
while (i!=1) {
strLine2 = br2.readLine();
wwwdf2.append(strLine2 + "\n");
String contains = String.valueOf(wwwdf2);
if(contains.contains("itisdonecomplet")){
i++;
}
}
// Toast.makeText(getApplicationContext(), strLine2, Toast.LENGTH_LONG).show();
Toast.makeText(getApplicationContext(), wwwdf2, Toast.LENGTH_LONG).show();
} catch (IOException e) {
e.printStackTrace();
}
}
}

Use StringBuilder instead of StringBuffer.
StringBuffer is synchronized, and you don't need that.
Don't use String.valueOf, which builds a string, negating the value using a StringBuffer/Builder. You are building a string from the whole buffer, checking it, discarding the string, then constructing nearly the same string again.
Use if (wwwdf2.indexOf("itisdonecomplet") >= 0) instead, which avoids creating the string.
But this will still be reasonably slow, as although you would not be constructing a string and searching through it all, you are still doing the searching.
You can make this a lot faster by only searching the very end of the string. For example, you could use wwwdf2.indexOf("itisdonecomplet", Math.max(0, wwwdf2.length() - strLine2.length() - "itisdonecomplet".length())).
Although, as blackapps points out in a comment, you could simply check if strLine2 contains that string.
Don't use string concatenation inside a call to append: make two separate calls.
wwwdf2.append(strLine2);
wwwdf2.append("\n");
You don't check if you reach the end of the file. Check if strLine2 is null, and break the loop if it is.

My new Created code:(My test device is a Samsung S8)
private void list(){
String strLine2="";
wwwdf2 = new StringBuilder();
InputStream fis2 = this.getResources().openRawResource(R.raw.list);
BufferedReader br2 = new BufferedReader(new InputStreamReader(fis2));
if(fis2 != null) {
try {
LineNumberReader lnr = new LineNumberReader(br2);
String linenumber = String.valueOf(lnr);
int i=0;
while (i!=1) {
strLine2 = br2.readLine();
wwwdf2.append(strLine2);
wwwdf2.append("\n");
if (wwwdf2.indexOf("itisdonecomplet") >= 0){
i++;
}
}
// Toast.makeText(getApplicationContext(), strLine2, Toast.LENGTH_LONG).show();
Toast.makeText(getApplicationContext(), wwwdf2, Toast.LENGTH_LONG).show();
} catch (IOException e) {
e.printStackTrace();
}
}
}

Java reading HTTP response (using StringBuilder) much slower than in python

I'm calling a webservice that returns a large response, about 59 megabytes of data.
This is how I read it from Java:
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream(),"UTF-8"));
result = result.concat(this.getResponseText(in));
private String getResponseText(BufferedReader in) throws IOException {
StringBuilder response = new StringBuilder(Integer.MAX_VALUE/2);
System.out.println("Started reading");
String line = "";
while((line = in.readLine()) != null) {
response.append(line);
response.append("\n");
}
in.close();
System.out.println("Done");
String r = response.toString();
System.out.println("Built r");
return r;
}
In Windows Resource manager during the reading I can see a throughput of about 100000 Bytes per second.
However when I read exactly the same data from the same webservice in python, i.e.:
response = requests.request("POST", url, headers=headers, verify=False, json=json)
I can see throughput up to 700000 Bytes per second (about 7 times faster). And also the code is finished 7 times faster.
The question is - Am I missing something that can make the reads in Java faster? Is this way really the fastest way how I can read HTTP response in Java?
Update - even after I'm not reading, just going through the response, I'm still at at most 100000 bytes / seconds, so I believe that the bottleneck is somewhere in the way how Java reads:
private List<String> getResponseTextAsList(BufferedReader in) throws IOException {
System.out.println("Started reading");
List<String> l = new ArrayList<String>();
int i = 0;
long q = 0;
String line = "";
while((line = in.readLine()) != null) {
//l.add(line);
i++;
q = q+line.length();
}
in.close();
System.out.println("Done" + i + " " + q);
return l;
}

How to run tshark in Java to get packets in real-time?

I have a problem with running tshark in Java. It seems that packets arrive in bulk instead of truly real-time (as it happens when run from terminal).
I tried a few different approaches:
ArrayList<String> command = new ArrayList<String>();
command.add("C:\\Program Files\\Wireshark\\tshark.exe");
ProcessBuilder pb = new ProcessBuilder(command);
Process process = pb.start();
BufferedReader br = null;
try {
//tried different numbers for BufferedReader's last parameter
br = new BufferedReader(new InputStreamReader(process.getInputStream()), 1);
String line = null;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
} catch...
also tried using InputStream's available() method as seen in What does InputStream.available() do in Java?
I also tried NuProcess library with the following code:
NuProcessBuilder pb = new NuProcessBuilder(command);
ProcessHandler processHandler = new ProcessHandler();
pb.setProcessListener(processHandler);
NuProcess process = pb.start();
try {
process.waitFor(0, TimeUnit.SECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
private class ProcessHandler extends NuAbstractProcessHandler {
private NuProcess nuProcess;
#Override
public void onStart(NuProcess nuProcess) {
this.nuProcess = nuProcess;
}
#Override
public void onStdout(ByteBuffer buffer) {
if (buffer == null)
return;
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);
System.out.println(new String(bytes));
}
}
None of the methods work. Packets always arrive, as if buffered, only when about 50 were sniffed.
Do you have any idea why this may be happening and how to solve it? It's pretty frustrating. I spent a lot of time looking at similar questions at SO, but none of them helped.
Do you see any errors in my code? Is it working in your case?

As the tshark man page says:
−l Flush the standard output after the information for each packet is
printed. (This is not, strictly speaking, line‐buffered if −V was
specified; however, it is the same as line‐buffered if −V wasn’t
specified, as only one line is printed for each packet, and, as −l
is normally used when piping a live capture to a program or script,
so that output for a packet shows up as soon as the packet is seen
and dissected, it should work just as well as true line‐buffering.
We do this as a workaround for a deficiency in the Microsoft Visual
C++ C library.)
This may be useful when piping the output of TShark to another
program, as it means that the program to which the output is piped
will see the dissected data for a packet as soon as TShark sees the
packet and generates that output, rather than seeing it only when
the standard output buffer containing that data fills up.
Try running tshark with the -l command-line argument.

I ran some tests to see how much Buffering would be done by BufferedReader versus just using the input stream.
ProcessBuilder pb = new ProcessBuilder("ls", "-lR", "/");
System.out.println("pb.command() = " + pb.command());
Process p = pb.start();
byte ba[] = new byte[100];
InputStream is = p.getInputStream();
int bytecountsraw[] = new int[10000];
long timesraw[] = new long[10000];
long last_time = System.nanoTime();
for (int i = 0; i < timesraw.length; i++) {
int bytecount = is.read(ba);
long time = System.nanoTime();
timesraw[i] = time - last_time;
last_time = time;
bytecountsraw[i] = bytecount;
}
try (PrintWriter pw = new PrintWriter(new FileWriter("dataraw.csv"))) {
pw.println("bytecount,time");
for (int i = 0; i < timesraw.length; i++) {
pw.println(bytecountsraw[i] + "," + timesraw[i] * 1.0E-9);
}
} catch (Exception e) {
e.printStackTrace();
}
BufferedReader br = new BufferedReader(new InputStreamReader(is));
int bytecountsbuffered[] = new int[10000];
long timesbuffered[] = new long[10000];
last_time = System.nanoTime();
for (int i = 0; i < timesbuffered.length; i++) {
String str = br.readLine();
int bytecount = str.length();
long time = System.nanoTime();
timesbuffered[i] = time - last_time;
last_time = time;
bytecountsbuffered[i] = bytecount;
}
try (PrintWriter pw = new PrintWriter(new FileWriter("databuffered.csv"))) {
pw.println("bytecount,time");
for (int i = 0; i < timesbuffered.length; i++) {
pw.println(bytecountsbuffered[i] + "," + timesbuffered[i] * 1.0E-9);
}
} catch (Exception e) {
e.printStackTrace();
}
I tried to find a command that would just keep printing as fast as it could so that any delays would be due to the buffering and/or ProcessBuilder rather than in the command itself. Here is a plot of the result.
You can plot the csv files with excel although I used a Netbeans plugin called DebugPlot. There wasn't a great deal of difference between the raw and the buffered. Both were bursty with majority of reads taking less than a microsecond separated by peaks of 10 to 50 milliseconds. The scale of the plot is in nanoseconds so the top of 5E7 is 50 milliseconds or 0.05 seconds. If you test and get similar results perhaps it is the best process builder can do. If you get dramatically worse results with tshark than other commands, perhaps there is an option to tshark or the packets themselves are coming in bursts.

Force the read() method to read a minimum no of characters

I am trying to do telnet to a router with expect kind of implementation.
My code is as follows for the socket communication is as follows,
server = "my-server-ip-domain-here";
socket = new Socket();
socket.connect(new InetSocketAddress(server, 23), 10000);//Will wait for 10 seconds
socket.setKeepAlive(true);
socket.setSoTimeout(10000);
expectBuffer = new StringBuilder();
br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
pw = new PrintWriter(socket.getOutputStream(), true);
My send implementation is as follows,
public static void send(String cmd) {
pw.print(cmd + "\r");
pw.flush();
}
My expect implementation is as follows,
public static String expect(String expectString) {
try {
int c = 0;
char[] buf = new char[4096];
//Here c will return the no. of chars read
while ((c = br.read(buf)) != -1) {
String tmp = "";
//converting that char array to String
for (int i = 0; i < c; i++) {
//Printing that character
System.out.print(buf[i]);
tmp += buf[i];
}
expectBuffer.append(tmp).append(NEW_LINE);
if (expectBuffer.toString().contains(expectString)) {
break;
}
}
String expBuff = expectBuffer.toString();
expectBuffer.setLength(0);
// System.out.println(expBuff);
return expBuff;
} catch (Exception e) {
System.out.println(e);
return "";
}
}
The problem i am facing is the no. of characters read by BufferedReader each time.
i.e. Each time I have to send some commands to the router and that is also being read by BufferedReader.
For eg.
send ("commandABC");
expect(prompt);
send ("command-efg");
expect(prompt);
send ("commandHij");
expect(prompt);
Based on the commands I am sending, it will show some output. Whatever I am sending, that is also being read and unfortunately, it is getting printed in a separate manner.
Like as below.
com
mandABC
<command output here>
command-
efg
<command output here>
commandHij
<command output here>
As i pointed out above, only the commands, whichever I am sending are getting printed in separate manner.
I have checked the no. of char read at that time and found that it is ranging from 2-10.
That is why it is getting printed in that manner.
Is there anyway to restrict the read at least a minimum of 100 chars?
Thanks in advance.

If you want to wait until you've read a full line of text, try bf.readLine() (you'd need to make sure each command was terminated by '\n'
If you want to make sure you've read a certain number of characters (say 100) before continuing processing, use a loop:
char buffer[128];
for (int charsRead = 0; charsRead < 100; ) {
charsRead += bf.read(buffer, charsRead, (100 - charsRead));
}
Note the (verbosified) syntax for bf.read():
bf.read(buffer, offset_size, max_to_read)
Passing charsRead as the offset size means each block of chars read will be stored right after the previously read ones. Passing (100 - charsRead) as max_to_read limits the total chars you read.
Source: API ref http://docs.oracle.com/javase/6/docs/api/java/io/BufferedReader.html#read(char[],%20int,%20int

Ping value in java

I know that this question has been approached under different ways, but I have checked stackoverflow and I didn't found the answer I was looking for.
To make it simple : Is there a way to get the Time ping value to an IP server under Windows ?
I know how to check if some servers are reachable, but I would like to have precise values, like we can read on terminal.
Thank you for your help and understanding.

You can do something like this :
//The command to execute
String pingCmd = "ping " + ip + " -t";
//get the runtime to execute the command
Runtime runtime = Runtime.getRuntime();
Process process = runtime.exec(pingCmd);
//Gets the inputstream to read the output of the command
BufferedReader in = new BufferedReader(new InputStreamReader(process.getInputStream()));
//reads the outputs
String inputLine = in.readLine();
while ((inputLine != null)) {
if (inputLine.length() > 0) {
........
}
inputLine = in.readLine();
}
reference
UPDATE: As per your need
public class PingDemo {
public static void main(String[] args) {
String ip = "localhost";
String time = "";
//The command to execute
String pingCmd = "ping " + ip;
//get the runtime to execute the command
Runtime runtime = Runtime.getRuntime();
try {
Process process = runtime.exec(pingCmd);
//Gets the inputstream to read the output of the command
BufferedReader in = new BufferedReader(new InputStreamReader(process.getInputStream()));
//reads the outputs
String inputLine = in.readLine();
while ((inputLine != null)) {
if (inputLine.length() > 0 && inputLine.contains("time")) {
time = inputLine.substring(inputLine.indexOf("time"));
break;
}
inputLine = in.readLine();
}
System.out.println("time --> " + time);
} catch (Exception ex) {
System.out.println(ex);
}
}
}
Written in little haste.

You can invoke the ping command and read the output (as explained in the previous answer), or if you need a lower lever access (like you can do with RAW sockets), you can have a look at the jpcap java library.

As shown here, you'll want to make use of the Runtime class to shell out a ping. All that's required of you is to parse the input stream (possibly using regex to get the time ping value).

public static void main(String[] args) {
System.out.println("Enter the host to be pinged : ");
Scanner sc = new Scanner(in);
String str = sc.next();
System.out.println("Enter the no. of packets to be sent : ");
int packets = sc.nextByte();
String pingResult;
int count=0;
try{
String command = "ping -c "+ packets +" -w 10 " + str;
Process process = Runtime.getRuntime().exec(command);
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String inputLine;
while ((inputLine = reader.readLine()) != null) {
if(count==packets+4) {
pingResult = (inputLine.substring(inputLine.indexOf("=")));
pingResult = (pingResult.substring
(pingResult.indexOf("/")+1,pingResult.indexOf("/")+7));
System.out.println(pingResult + " ms");
}
count++;
}
in.close();
if(count==0)System.out.println("Wrong host entered.");
}catch(Exception e){
System.out.println("Exception caught: " + e.toString());
}
}
}
Output:
Enter the host to be pinged :
8.8.8.8
Enter the no. of packets to be sent :
5
31.406 ms
Process finished with exit code 0