Develop Reference

How to read a tab separated file and select few values from it using java

I have a tab separated file which looks like this
STID STNM TIME TMAX TMAXO TMIN TMINO TAVG TBAD DMAX DMAXO DMIN DMINO DAVG VDEF DBAD SMAX SMAXO SMIN SMINO SAVG SBAD BMAX BMAXO BMIN BMINO BAVG BBAD S5MX S5MXO S5MN S5MNO S5AV S5BD S25X S25XO S25N S25NO S25AV S25BD S60X S60XO S60N S60NO S60AV S60BD HMAX HMAXO HMIN HMINO HAVG HBAD PMAX PMAXO PMIN PMINO PAVG MSLP PBAD AMAX AMAXO ATOT ABAD PDIR PDFQ SDIR SDFQ IBAD WSMX WSMXO WSMN WSMNO WSPD WDEV WMAX WMAXO WBAD RAIN RNUM RMAX RBAD 9AVG 9BAD 2MAX 2MIN 2AVG 2DEV 2BAD HDEG CDEG HTMX HTMXO HTBAD WCMN WCMNO WCBAD
ACME 110 0 76.32 131 69.22 184 71.57 0 69.10 286 61.55 3 66.48 4.22 0 83.16 3 78.24 288 80.85 0 85.37 3 77.74 288 81.77 0 83.12 150 77.86 288 80.58 0 83.84 3 81.23 288 82.34 0 81.54 3 80.94 285 81.29 0 96.82 278 66.82 1 84.59 0 28.74 284 28.67 23 28.71 30.10 0 412.73 130 5.46 0 -996 -999 -996 -999 59 10.92 132 0.00 37 4.34 2.41 14.61 146 0 0.22 19 0.24 0 71.67 0 8.44 0.00 2.49 2.30 0 0.00 7.77 -996 999 288 -996 999 288
ADAX 1 0 73.99 96 68.61 21 71.32 0 70.91 169 62.77 1 68.22 2.58 0 87.15 3 82.99 288 84.83 0 88.32 3 79.54 288 83.59 0 85.06 3 81.84 288 83.31 0 88.48 3 85.21 288 86.61 0 -996 999 -996 999 -996 96 98.40 274 73.27 1 90.20 0 29.08 137 29.01 17 29.04 30.08 0 210.42 151 5.23 0 -996 -999 -996 -999 139 12.83 106 0.00 33 3.65 3.03 19.28 121 0 0.24 23 0.24 0 71.57 0 8.84 0.00 2.07 2.48 0 0.00 6.30 -996 999 288 -996 999 288
ALTU 2 0 75.51 107 68.74 168 71.63 0 70.43 279 64.56 125 67.48 3.50 0 80.60 3 77.88 288 78.91 0 79.11 3 75.96 288 77.08 0 79.97 3 77.23 288 78.41 0 81.95 3 79.57 288 80.55 0 -996 999 -996 999 -996 96 98.36 286 70.28 106 87.18 0 28.68 276 28.60 51 28.64 30.09 0 202.20 123 5.03 0 2 30.80 4 18.63 25 13.72 128 0.00 70 5.79 2.71 18.19 128 0 0.19 19 0.12 0 71.53 0 9.55 0.00 3.71 2.22 0 0.00 7.12 -996 999 288 -996 999 288
I am trying to read this file so that I can append some of the values from this file to another file.
But firstly I am unable to read the values of the column TMAX which is 4th in the columns
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;
public class first {
public static void main(String[] args)
{
// TODO Auto-generated method stub
String fileName="daily.txt";
File file = new File(fileName);
try{
Scanner inputStream = new Scanner(file);
while (inputStream.hasNext()){
String data = inputStream.next();
String[] values = data.split("\t");
System.out.println(values[4]);
}
inputStream.close();
}
catch(FileNotFoundException e){
e.printStackTrace();
}
}
}
When I use the above code the output looks like this
STID
STNM
TIME
TMAX
TMAXO
TMIN
TMINO
TAVG
TBAD
DMAX
DMAXO
DMIN
DMINO
DAVG
VDEF
DBAD
SMAX
SMAXO
SMIN
I want to get an output which displays the values of the specified column numbers.

You need to use nextLine instead of next to read the whole line. Also I ran your program and found out that your file is not truely splitted by tab that's why your split may not work. Fix these two things and then you are good to go.

Here's a sample method which achieves what you're looking for (not tested, but the concept is there). Essentially, you need to read by line, split the line into some sort of array or list, and then have 2d array. you can also replace split("\t") with splitting by white space in general.
public List<String> getByColumn(int col, File file)
{
List<ArrayList<String>> arrayOfArrays = null;
try {
FileInputStream fis = new FileInputStream(file);
InputStreamReader isr = new InputStreamReader(fis, "UTF-8");
BufferedReader br = new BufferedReader(isr);
String line;
arrayOfArrays = new ArrayList<ArrayList<String>>();
while ( ( line = br.readLine() ) != null )
{
ArrayList<String> list = new ArrayList<String>(Arrays.asList(line.split("\t")));
arrayOfArrays.add(list);
}
} catch (IOException e) {
e.printStackTrace();
}
ArrayList<String> output = new ArrayList<String>();
//can use a foreach loop, or the below method.
//for (ArrayList<String> l : arrayOfArrays)
//{
// output.add(l.get(col));
//}
for ( int i = 1; i < arrayOfArrays.size(); i++ )
{
output.add(arrayOfArrays.get(i).get(col));
}
return output;
}

You should really use univocity-parsers for that - it will be way faster than your String.split and also helps a lot selecting the columns you want:
//configure the parser
TsvParserSettings parserSettings = new TsvParserSettings();
parserSettings.selectFields("TMAX" /*and others*/);
//then parse
TsvParser parser = new TsvParser(parserSettings);
List<String[]> parsedRows = parser.parseAll(new File("daily.txt"), "UTF-8");
Hope this helps.
Disclaimer: I'm the author of this library. It's open source and free (Apache 2.0 license)

use this code to parse the line
String[] values = data.trim().replaceAll(" +", " ").split(" ");

TCP/IP client incorrectly reading inputstream byte array

I'm creating a Java Client program that sends a command to server and server sends back an acknowledgement and a response string.
The response is sent back in this manner
client -> server : cmd_string
server -> client : ack_msg(06)
server -> client : response_msg
Client code
public static void readStream(InputStream in) {
byte[] messageByte = new byte[20];// assuming mug size -need to
// know eact msg size ?
boolean end = false;
String dataString = "";
int bytesRead = 0;
try {
DataInputStream in1 = new DataInputStream(in);
// while ctr==2 todo 2 streams
int ctr = 0;
while (ctr < 2) {//counter 2 if ACK if NAK ctr=1 todo
bytesRead = in1.read(messageByte);
if (bytesRead > -1) {
ctr++;
}
dataString += new String(messageByte, 0, bytesRead);
System.out.println("\ninput byte arr "+ctr);
for (byte b : messageByte) {
char c=(char)b;
System.out.print(" "+b);
}
}
System.out.println("MESSAGE: " + dataString + "\n bytesread " + bytesRead + " msg length "
+ dataString.length() + "\n");
char[] chars = dataString.toCharArray();
ArrayList<String> hex=new ArrayList<>();
// int[] msg ;
for (int i = 0; i < chars.length; i++) {
int val = (int) chars[i];
System.out.print(" " + val);
hex.add(String.format("%04x", val));
}
System.out.println("\n"+hex);
} catch (Exception e) {
e.printStackTrace();
}
// ===
}
Output
client Socket created ..
response:
input byte arr 1
6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
input byte arr 2
2 -77 67 79 -77 48 -77 3 -116 0 0 0 0 0 0 0 0 0 0 0
MESSAGE: ##³CO³0³##
(where # is some not supported special character )
bytesread 9 msg length 10
dec: 6 2 179 67 79 179 48 179 3 338
hex: [0006, 0002, 00b3, 0043, 004f, 00b3, 0030, 00b3, 0003, 0152]
bytes: 2 -77 67 79 -77 48 -77 3 -116 0 0 0 0 0 0 0 0 0 0 0 (bytes recieved in 2nd packet)
connection closed
Problem: I'm reading the last value incorrect, I have verified using wireshark the server has sent back the response as 06 02 b3 43
4f b3 30 b3 03 8c
Some how I'm reading the last value in correctly. Is there some issue with the reading stream?
EDIT
Rest of the response is read correctly but the last character should be 8c But is read as 0152Hex
Response from server : 06 02 b3 43 4f b3 30 b3 03 8c
Read by program : [0006, 0002, 00b3, 0043, 004f, 00b3, 0030, 00b3, 0003, 0152]
issue with reading the last character
EDIT 2
Response is received as 2 packets/streams
packet 1 byte arr : 6 (ACK)
packet 2 byte arr: 2 -77 67 79 -77 48 -77 3 -116 (response)
complete response read by client
dec: 6 2 179 67 79 179 48 179 3 338
hex: [0006, 0002, 00b3, 0043, 004f, 00b3, 0030, 00b3, 0003, 0152]
Thanks

The problem in this question was a matter of signed variables versus unsigned variables. When you have a number in computer memory, it is represented by a bunch of bits, each of them 0 or 1. Bytes are generally 8 bits, shorts are 16 etc. In an unsigned number, 8 bits will get you from positive 0 to 255, but not to negative numbers.
This is where signed numbers come in. In a signed number, the first bit tells you whether the following bits represent a negative or positive value. So now you can use 8 bits to represent -128 to +127. (Notice that the positive range is halved, from 255 to 127, because you "sacrifice" half of your range to the negative numbers).
So now what happens if you convert signed to unsigned? Depending on how you do it, things can go wrong. In the problem above, the code char c=(char)b; was converting a signed byte to an unsigned char. The proper way to do this is to "make your byte unsigned" before converting it to a char. You can do that like this: char c=(char)(b&0xFF); more info on casting a byte here.
Essentially, you can just remember that except for char, all java numbers are signed, and all you need to do is paste the &0xFF to make it work for a byte, 0xFFFF to make it work for a short, etc.
The details about why this works are as follows. Calling & means a bitwise and, and 0xFF is hexadecimal for 255. 255 is above the limit of a signed byte (127), so the number b&0xFF gets upgraded to a short by java. However, the short signed bit is on bit 16, while the byte signed bit is on bit 8. So now the byte signed bit becomes a normal 'data' bit in the short, and so the sign of your byte is essentially discarded.
If you do a normal cast, java recognizes that doing direct bitconversion like above would mean that you lose the sign, and java expects you don't like that (at least, that is my assumption), so it preserves the sign for you. So if this isn't what you want, you explicitly tell java what to do.

Echoing photo as bytes in php to java app

Someone tried to display image as bytes so, Java app can see it? Again this problem. I found this tutorial:
$fp = fopen("binary-data.dat","rb");
while (!feof($fp)) {
// Read the file, in chunks of 16 byte
$data = fread($fp,16);
$arr = unpack("C*",$data);
foreach ($arr as $key => $value) {
echo " " . $value;
}
echo "\n";
}
This is how tutorial suggest me to output data. It outputs something like:
0 0 0 16 144 0 5 0 0 0 0 4 0 48 0 0
0 176 0 0 0 0 0 3 0 0 0 96 0 0 0 0
0 112 0 0 15 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Java coder tells that it's wrong. He needs something like:
�R�{�,-l�p6���S�%�����J�#���彇sf1ZVĭ��p��A;[�sG`�2��U.Dڳ=��e*�#���ޱ�1�9H�Ӱ5.gS�T�[s4�3ht.����bl�}��G'L���(3N��ҵU6�J�IJ����7+LH��f��n�y<�����]ă�S�`[S�hJ#Q���������iG$S.J��
�*�J�Sj�3�r�$6�>T�6^�߯�!�⊌�Q�*ڔ���e*VQam:t�� ������l��\.%����:���۰��;� K<9j\�2�$�U�[ڋ
�sHH��%L�.��-(��XFf��)b�"��-޶�XR9���V���}�)���vi�t7 ������)Eɥ�V%�-�:VnZ^es��VT��J�JTBI7�[��4�9zg⦾�LIM|KRM6#��rr�|�)c���\���q�R���`4
I tried to change the parameter of unpack on H (it was wrong too).I changed it to A. It printed something like he wants, but now he says that there is some newlines there. So now my code looks like:
$fp = fopen("/var/www/9292/inputPhoto1.jpg","rb");
$image="";
while (!feof($fp)) {
// Read the file, in chunks of 16 byte
$data = fread($fp,16);
$arr = unpack("A*",$data);
foreach ($arr as $key => $value) {
$image.=$value;
}
}
fclose($fp);
But he still says about some newlines. Anyone can explain me how to output bytes properly in php?

Try replacing your entire code block with
header('Content-Type: image/jpeg'); // Set the correct MIME type
readfile('/var/www/9292/inputPhoto1.jpg'); // Read & output the file directly from disk

Getting data from Scale serial port without <CR>

Im trying to read the weight from a scale using the serial port.
But i think i'm receiving data with and i don't know how to print it without the
My serial event:
public void serialEvent(SerialPortEvent event){
switch(event.getEventType()) {
case SerialPortEvent.OUTPUT_BUFFER_EMPTY:
outputBufferEmpty(event);
break;
case SerialPortEvent.DATA_AVAILABLE:
byte[] readBuffer = new byte[20];
try {
int numBytes = inputStream.read(readBuffer);
String str = new String (readBuffer, "UTF-8");
ok+=str;
//System.out.print(str);
System.out.println();
System.out.print(ok.replaceAll("\\s+", ""));
//System.out.print(str.replace("\n", "").replace("\r", ""));
} catch (IOException ex) {}
break;
}
}
I tried all.
But this is my output now:
0
0 0
0 0 0
0 0 0 .
0 0 0 . 0
0 0 0 . 0 5
0 0 0 . 0 5 0
0 0 0 . 0 5 0
And I want this:
000.050 KG
This is the protocol of my scale:
For more information my scale is: Mettler Toledo VIVA or an OHAUS RV Series

Why dont u try to make it directly to double?
import java.nio.ByteBuffer;
public static double toDouble(byte[] bytes) {
return ByteBuffer.wrap(bytes).getDouble();
}

I guess the answer is:
You created a string from the entire 20-byte byte array, rather than the portion that you just read.
Change the line that creates the string to:
String str = new String(readBuffer, 0, numBytes, "UTF-8")
The extra gaps in your output were probably caused by the null-bytes from the end of the array, although it's hard to know for sure without details on what type of output device you were using (not that important)

.txt file seems to be different when opened on my computer. It seems to be OK on any other mobile device

I've got a strange problem with my newly made .txt file.
My Android Program is writing a string to a .txt file.
Here's my method:
public void generateNoteOnSD(String sFileName, String sBody){
try
{
File root = new File(Environment.getExternalStorageDirectory(), "Grafy");
if (!root.exists()) {
root.mkdirs();
}
File gpxfile = new File(root, sFileName + ".txt");
FileWriter writer = new FileWriter(gpxfile);
writer.append(sBody);
writer.append("**************\n");
writer.flush();
writer.close();
Toast.makeText(this, "Saved", Toast.LENGTH_SHORT).show();
}
catch(IOException e)
{
e.printStackTrace();
}
}
Here's how i invoke my method:
String s = "0 1 0 0 1 0\n1 0 1 0 1 0\n0 1 0 1 0 0\n0 0 1 0 1 1\n1 1 0 1 0 0\n0 0 0 1 0 0\n";
generateNoteOnSD("test", s);
The problem is that my .txt file looks like this(when i open it on my mobile):
0 1 0 0 1 0
1 0 1 0 1 0
0 1 0 1 0 0
0 0 1 0 1 1
1 1 0 1 0 0
0 0 0 1 0 0
**********************
But when i open this file on my Computer the '\n' chars are gone... o.o and file looks like this:
0 1 0 0 1 01 0 1 0 1 00 1 0 1 0 00 0 1 0 1 11 1 0 1 0 00 0 0 1 0 0**********************
Where's the problem? :(

Line breaks are composed of "\r\n" (Carriage Return + Line Feed) in Windows operating systems, while in *NIX they are composed of only "\n". This is why it works in your mobile phone, which is likely to be running on the Linux kernel, while on your Windows machine it doesn't.
There is no real harm to saving newlines as "\r\n" instead of just "\n", this will display correctly on both Windows and Linux operating systems.
The Wikipedia article on Newlines provides good information about this matter.
Here is a handy list of text editors, along with their newline support. You can use an editor that supports UNIX newlines without making changes to the file.