Develop Reference

Is this an efficient or even correct way of parsing serial data?

To preface I am new to java (so take it ez on me lol) but here is what I am trying to do:
I have a GPS puck that I am reading via usb. The GPS puck uses some UI but I am trying to capture the data so I can use for my own application. I am reading in the packet from the serial buffer and trying to strip it down to just the payload and dissect the fields into a hashMap (not sure if this is the right application for it or not).
I want to add that the reason I care about efficiency in this case is because data will be getting processed through every fairly rapidly as the GPS puck is constantly getting Sat data (approx. every 200/300ms), so I am trying to avoid any bottlenecking if possible.
Here is the code that I have dished out, I know that in the code the buffer is not reading anything but in an effort to remove redundant code, just assume the data written in the comments is what we would be saying if we sys.out.
/*
Assume this is serial data received from GPS serial packet:
B5 62 01 02 1C AA AA AA AA 01 02 03 04 0A 0B 0C 0D B1 B2 B3 B4 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 01 02;
| | / | / |
| \ len \ / CRC
/ | \ /
/ | \ PAYLOAD /
Header | \ /
class & id \ /
*/
HashMap<String, String> payloadDict = new HashMap<String, String> (); // HashMap to write to
Bytes[] buffer; // serial buffer
String byteToHex = DataTypeConverter.parseHexBinary(buffer); // converts byte data to string in Hex
// Since the first few fields in the packet are static (header, class & id, and len),
// I strip the HEX string to only contain the payload
int payloadStartIndex = byteToHex.indexOf("C") + 1;
String payload = byteToHex.substring(payloadStartIndex, payloadStartIndex + 29);
/*
Assume this is the payload string after we parse:
AA AA AA AA 01 02 03 04 0A 0B 0C 0D B1 B2 B3 B4 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4
**added seperation every 2 hex values for readability,
but assume it's just one giant string
*/
// write to HashMap
payloadDict = NAV_POSLLH(payload);
/*
so to visualize what I would imagine being saved to the HashMap:
iTOW: AA AA AA AA
lon: 01 02 03 04
lat: 0A 0B 0C 0D
height: B1 B2 B3 B4
hMSL: C1 C2 C3 C4
hAcc: D1 D2 D3 D4
vAcc: E1 E2 E3 E4
*/
public Map<String, String> NAV_POSLLH(String dataArr) {
// Temp HashMap
HashMap<String, String> dict = new HashMap<String, String> ();
/*
this is the payload format.
payloadStructure are the names of the all the fields I want to parse
and hexOffset are the beginning of each field.
*/
String[] payloadStructure = new String[6] {"iTOW", "lon", "lat", "height", "hMSL", "hAcc", "vAcc"};
int[] hexOffset = new int[6] {0, 7, 11, 15, 20, 24};
// loop through each field
for(int i = 0; i < byteOffset.length; i++) {
String parse = "";
// I am not sure if this is the best way to handle out of index errors
// in java, but this is what I thought would work
if(byteOffset[i] == 24) {
parse = dataArr.substring(hexOffset[i]);
map.put(payloadStructure[i], parse);
return dict;
}
// parse the substring from the beginning of each hex offset
// to the next index -1
parse = dataArr.substring(byteOffset[i], hexOffset[i+1] -1);
map.put(payloadStructure[i], parse);
// finally return HashMap
return dict;
}
}
I am not sure if the comments explain the packet structure well enough or if it is confusing (because the format is janky), so I attached a screenshot of the packet layout I made.
So what I am trying to ask from all of this:
Based on what I am trying to accomplish, is this an efficient or even a correct way of implementing it? I am not sure if there is anything I am just obviously missing.
I am unable to compile or test any code right now so I am just trying to visualize what the output would be haha. Thanks so much for anyone's input!

I would make a Packet class and fill it using ByteBuffer. That class has great methods for parsing, including bulk get methods for various primitive sizes. You might consider using a ByteBuffer for the payload member of that class

Encoding issue for sending data via java socket?

I have the following code. I can do the reading all well and convert data into hex. The issue is when I send back String replyMessage = "7E81";, the device receives it as "3745". What is wrong? Is it due to my encoding, or do I need to do some conversion before I send back?
w = new BufferedWriter(new OutputStreamWriter(receivedSocketConn1.getOutputStream(),"ISO-8859-15")); //
r = new BufferedReader(new InputStreamReader(receivedSocketConn1.getInputStream(),"ISO-8859-15"));
int nextChar=0;
while ((nextChar=r.read()) != -1) {
StringBuilder sb = new StringBuilder();
sb.append(Integer.toHexString(nextChar));
if (sb.length() < 2) {
sb.insert(0, '0'); // pad with leading zero if needed
}
String hexChar = sb.toString();
System.out.println("\n\n hex value is "+Integer.toHexString(nextChar).toUpperCase()+" "+"Int value is:"+nextChar);
message = message+hexChar;
String messageID=message.substring(2,6);
System.out.println("messageId is :"+messageID);
if(messageID.equals("0100")){
String replyMessage = "7E81";
w.write(replyMessage+"\r\n");
w.flush();
}
}

Based on comments in chat:
the documentation say
Start Byte (1 Byte) 7e
Message ID (2 Byte) 01 00
Message Body Nature (2 Byte) 00 19
Phone no. of Device (6 Byte) 09 40 27 84 94 70
Message Serial number (2 Byte) 00 01
Message Body (N Byte) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 09 40 27 84 94 70 00
Check Code (1Byte) 19
End Byte (1Byte) 7E
so the start and termination is 7E
for outgoing
Start Byte (1 Byte) 7e
Message ID (2 Byte) 81 00
Message Body Nature (2 Byte) 00 13
Phone no. of Device (6 Byte) 09 40 27 84 94 70
Message Serial number (2 Byte) 00 01
Message Body (N Byte) 00 01 00 32 30 31 31 31 31 30 38 31 31 33 33 32 31 39 36
Check Code (1Byte) 9A
End Byte (1Byte) 7e
This means the protocol in question is a binary protocol, not a textual protocol that sends hex strings, like you thought. As such, your use of OutputStreamWriter, InputStreamReader, StringBuilder, toHexString(), etc are all completely wrong for this protocol.
Each message received and sent begins with a fixed 13-byte header, followed by a variable-length body (the header specifies the body length), and terminated by a fixed 2-byte footer.
With that in mind, try something more like this instead:
final protected static char[] hexArray = "0123456789ABCDEF".toCharArray();
public static String bytesToHex(byte[] bytes) {
char[] hexChars = new char[bytes.length * 2];
for ( int j = 0; j < bytes.length; j++ ) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
return new String(hexChars);
}
...
w = new DataOutputStream(new BufferedOutputStream(receivedSocketConn1.getOutputStream()));
r = new DataInputStream(new BufferedInputStream(receivedSocketConn1.getInputStream()));
...
if (r.readByte() != 0x7E) // start byte
{
// ah oh, something went wrong!!
receivedSocketConn1.close();
return;
}
int messageID = r.readUnsignedShort(); // message ID
int bodyLen = r.readUnsignedShort(); // message body nature (body length)
byte[] phoneNum = new byte[6];
r.readFully(phoneNum); // device phone number
int serialNum = r.readUnsignedShort(); // message serial number
byte[] messageBody = new byte[bodyLen]; // message body
r.readFully(messageBody);
byte checkCode = r.readByte(); // check code
if (r.readByte() != 0x7E) // end byte
{
// ah oh, something went wrong!!
receivedSocketConn1.close();
return;
}
// TODO: validate checkCode if needed...
System.out.println("messageId is : 0x" + Integer.toHexString(messageID));
System.out.println("phoneNum is : " + bytesToHex(phoneNum));
System.out.println("serialNum is : 0x" + Integer.toHexString(serialNum));
System.out.println("messageBody is : " + bytesToHex(messageBody));
// process message data as needed...
switch (messageID)
{
case 0x100:
{
// ...
byte[] replyBody = new byte[19];
replyBody[0] = 0x00;
replyBody[1] = 0x01;
replyBody[2] = 0x00;
replyBody[3] = 0x32;
// and so on...
checkCode = 0x9A; // calculate as needed...
w.writeByte(0x7e); // start byte
w.writeShort(0x8100); // message ID
w.writeShort(replyBody.length); // message body nature (body length)
w.write(phoneNum); // device phone number
w.writeShort(0x0001); // message serial number
w.write(replyBody); // message body
w.writeByte(checkCode); // check code
w.writeByte(0x7e); // end byte
break;
}
// other message IDs as needed...
}
w.flush();

Issue with getBytes() for accented charaters

I'm trying to convert a string with special characters like É into a string with UTF-8 encoding. I tried doing this:
String str = "MARIE-HÉLÈNE";
byte sByte[] = str.getBytes("UTF-8");
str = new String(sByte,"UTF-8");
The problem is, when I do "É".getBytes("UTF-8"), I get 63 which is interpreted as '?' when it's being converted to a new string. How can I fix this issue?
EDIT: I also noticed that this issue was not reproducible on Eclipse, probably because the text file encoding is usually set to UTF-8.
I tried doing byte[] str = "MARIE-HÉLÈNE".getBytes("UTF-8") in http://www.javarepl.com/console.html and got the result byte[] str = [77, 65, 82, 73, 69, 45, 72, 63, 76, 63, 78, 69]

This kind of error happens when information about the encoding of the source file is not given to the compiler (javac) properly. If the encoding of your source file is UTF-8, compile the file like the following.
javac -encoding UTF-8 E.java
The following is another example for the case where the encoding of the source file is UTF-16 Big Endian.
javac -encoding UTF-16BE E.java
I've already confirmed that the program below properly shows "0xC3 0x89". So, there is no problem in your code.
public class E
{
public static void main(String[] args) throws Exception
{
byte[] bytes = "É".getBytes("UTF-8");
for (int i = 0; i < bytes.length; ++i)
{
System.out.format("0x%02X ", (byte)(bytes[i]));
}
System.out.println();
}
}

"É".getBytes("UTF-8") returns a byte[] of 2 bytes: c3 89.
"MARIE-HÉLÈNE" is 4d 41 52 49 45 2d 48 c3 89 4c c3 88 4e 45.
4d 41 52 49 45 2d 48 c3 89 4c c3 88 4e 45
M A R I E - H É L È N E
Converting the bytes back using new String(bytes,"UTF-8") will restore the original string.

RSA Decryption error

I am implementing RSA decryption. I have tried lot more things regarding RSA decryption and i have develop below code snipet.
public class AsymmetricEncryptionWithRSA {
public void decryptdata1(){
String PRIVATE_RSA_KEY_PKCS8 =
"-----BEGIN PRIVATE KEY-----\n" +
"<RSAKeyValue><Modulus>rotP8p9dJpRM/D7HHW9wsRWJEBVdZolRR0PuYGp/Mjae2gElTZSBQ44ifOvVquI95ECDa2ypIrEz1k/mKRtHPmsSMqmySWL9CVsEZAc/zNfYjIKb4BDaFiBaPEsboebIEuaN3NFt2bbl/RB45TCycYylgDjCdFFASRbHs0jX4cM=</Modulus><Exponent>AQAB</Exponent></RSAKeyValue>\n" +
"-----END PRIVATE KEY-----\n";
String key = PRIVATE_RSA_KEY_PKCS8
.replace("-----BEGIN PRIVATE KEY-----\n", "")
.replace("\n-----END PRIVATE KEY-----\n", "");
System.out.println("key : "+key);
PKCS8EncodedKeySpec keySpec =
new PKCS8EncodedKeySpec(DatatypeConverter.parseBase64Binary(key));
try {
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
PrivateKey privateKey = keyFactory.generatePrivate(keySpec);
Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA1AndMGF1Padding");
cipher.init(Cipher.DECRYPT_MODE, privateKey);
String s = "kjZzSUPJBY4rM23P7t47W3RYPTqhEARCqOcgRfxxBSc3O2mxQZKE/HKDN9OzW5pc3K37rcd+Dc/d55uNJcJUWwmZL0vVgrQIBkQdSkrzcpXV9JD80Qgy8sCuHoqMU49U6I34I43+NUAFmQDkApHbMA8LGV6KLtoyIyUSpDiuW8Q=";
byte[] ciphertextBytes = Base64.decode(s.getBytes());
byte[] decryptedData = cipher.doFinal(ciphertextBytes);
String decryptedString = new String(decryptedData);
System.out.println("decrypted (plaintext) = " + decryptedString);
}catch(Exception e){
e.printStackTrace();
}
}
public static void main(String[] args) {
AsymmetricEncryptionWithRSA ae = new AsymmetricEncryptionWithRSA();
ae.decryptdata1();
}
}
But the above code ran into the following error:
java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException: IOException : DerInputStream.getLength(): lengthTag=11, too big.
at sun.security.rsa.RSAKeyFactory.engineGeneratePrivate(Unknown Source)
at java.security.KeyFactory.generatePrivate(Unknown Source)
at decrypttest.AsymmetricEncryptionWithRSA.decryptdata1(AsymmetricEncryptionWithRSA.java:189)
at decrypttest.AsymmetricEncryptionWithRSA.main(AsymmetricEncryptionWithRSA.java:210)
Caused by: java.security.InvalidKeyException: IOException : DerInputStream.getLength(): lengthTag=11, too big.
at sun.security.pkcs.PKCS8Key.decode(Unknown Source)
at sun.security.pkcs.PKCS8Key.decode(Unknown Source)
at sun.security.rsa.RSAPrivateCrtKeyImpl.<init>(Unknown Source)
at sun.security.rsa.RSAPrivateCrtKeyImpl.newKey(Unknown Source)
at sun.security.rsa.RSAKeyFactory.generatePrivate(Unknown Source)
... 4 more

Crypto is frequently misunderstood, and thus I've decided to post a negative answer here. It's not very exciting, but if somebody finds themselves in scenario similar to the OPs they should probably be told why it can't work as early as possible in the process.
There are two possibilities when presented with a problem like this.
OP understands the problem correctly and they've been instructed to decrypt something with the public key.
or
OP does not completely understand their task but have been given a public key and some data and they need to make some progress with it.
For #1, it's simple: You can't decrypt RSA-encrypted data using a public key. It turns out the OP has only an RSA public key here, provided in an XML format. There's no way to reformat, massage, or otherwise fix up the public key to make it into the private key needed to decrypt the message. If this were doable then RSA would be trivially weak.
Now for #2, we have to step back and think of what might be true. We often find question askers here on stack overflow referring to "encrypting with the private key" in a context that inevitably implies that they really have an instance of an RSA signature. Rather than encryption, for clarity this should be called signing. An RSA signature is generated using the private key, and then everyone can verify the signature using the public key.
Now to verify the signature you also need the data that was signed, though there are some uncommon RSA signature schemes that allow for "message recovery". Since we have no data in this question, we have to try to go through the steps of verifying an RSA signature until we can make no more progress.
So we convert the XML data into an RSA modulus and public exponent. They are:
modulus
n = 122569097529914546498069291857337865246611656055620489010618854241554260593577609267786261511762266943671570032800601669006596562960007716726461943599993657820822102674379936038822313418546881231271467615708966752462196756466881703206510681054434799028982952649926637620598591790584184591371562810823451075011
and public exponent
e = 65537
The next step is to convert the signature string s into an integer. Using base64 decoding and the treating the result byte as big-endian integer I get:
sig = 102674047248928745858859342743181557926563805477311756100001889942342552706930945260621737497972422939132498975770606253415531107828422537100810225687174208681096157106683665614198008538708912251973616331643319025696474841958268570965587969021816350210845541576380037736081469197294740984924898949880343321540
And now I compute compute plain = sige mod n and get:
plain = 18281761513910757613247127600811463135393884768551021193363621715044011507027712128790112174797966270576132751506458211850250035395641481824697193105741034925363331241318624392861667609810000005509535564907655043671685464003456777077702725291701153287408411076758261166861253557068704213449855094306290667395
and finally convert plain to bytes using big endian to get:
92 36 73 49 43 c9 05 8e 2b 33 6d cf ee de 3b 5b 74 58 3d 3a a1 10 04 42 a8 e7 20 45 fc 71 05 27 37 3b 69 b1 41 92 84 fc 72 83 37 d3 b3 5b 9a 5c dc ad fb ad c7 7e 0d cf dd e7 9b 8d 25 c2 54 5b 09 99 2f 4b d5 82 b4 08 06 44 1d 4a 4a f3 72 95 d5 f4 90 fc d1 08 32 f2 c0 ae 1e 8a 8c 53 8f 54 e8 8d f8 23 8d fe 35 40 05 99 00 e4 02 91 db 30 0f 0b 19 5e 8a 2e da 32 23 25 12 a4 38 ae 5b c4
And ... nothing. It looks random to me, I see no patterns at all, nothing that looks like any possible version of PKCS#1 padding including PSS, nothing that looks like a 'message recovery' format.
So I can't solve this, none of the obvious things to try yield anything. It may be that the OP has been given an RSA-encrypted message but only given a public key, or maybe this is one of those RSA capture-the-flag exercises for which there is some weakness that allows for factoring the modulus, or maybe the public key is wrong. Or maybe I've made a mistake.

X509 serial number using java

I need to get some data from X509 certificate.
If I open a certificate file in windows, its showing its serial number in this format.
ex. 39 65 70 eb d8 9f 28 20 4e c2 a0 6b 98 48 31 0d
The same data I am trying to obtain using java. After get it loaded, I use
x509.getSerialNumber();
and result is : 76292708057987193002565060032465481997
So what is the difference between both of these ? I want the result as upper one.

Windows shows the hexadecimal representation of the serial number, whereas Java returns a BigInteger result from X509Certificate.getSerialNumber().
To display the BigInteger as a hexadecimal value, just call toString(16).
BigInteger bi = new BigInteger("76292708057987193002565060032465481997");
System.out.println(bi.toString(16));
Will output:
396570ebd89f28204ec2a06b9848310d

The first one is hexadecimal value of the certificate.
The other one is decimal.
Now it depends on how you convert the initial certificate bytearray for printing it out.
Lets say this is your certificate:
byte[] cert = { (byte) 0xFD, (byte) 0xB1, (byte) 0xDD, ..., (byte) 0x00 };
BigInteger certVal = new BigInteger(cert);
System.out.println("And result is (hex): " + certVal.toString(16));