Develop Reference

Decoding array of hexadecimal bytes to a specific codepage brings a wrong result when encoding afterwards

I created a simple app which looks like this:
String stringValue = new String(new byte[] { 0x00, 0x00, 0x00, 0x25 }, "273");
byte[] valueEncoded = Arrays.copyOfRange(stringValue.getBytes("273"), 0, 4);
int finalResult = ByteBuffer.wrap(valueEncoded).getInt();
System.out.println("Result: "+finalResult);
I expect the result to be 37, however the result is 21. How come? Am I missing something? Or is my method not the way it is supposed to be and therefore this error pops up?
I tried many other numbers and all seem to work fine...
As you can see I am using codepage 273 (IBM).

Looks to me like linefeed 0x25 got mapped to newline 0x15.
I assume that it went like this:
bytes to string: EBCDIC 0x25 -> UTF-16 0x000A
string to bytes: UTF-16 0x000A -> EBCDIC 0x15.
Why that's preferred, I can't guess. Well, actually I can guess (but it's only a guess). 0x000A is the standard line terminator on many systems, but it's generally output as "move to beginning of next line". Converting to IBM 273 is 'probably' because the text is destined for output on a device that uses that code page, and perhaps such devices want NL rather than LF for starting a new line.
Hypothesis validated:
class D {
public static void main(String... a) throws Exception {
String lf = "\n";
byte[] b = lf.getBytes("273");
System.out.println((int)b[0]);
}
}
Output is 21.

Trouble comparing Java strings (of different encoding)

I'm writing EXIF metadata to a JPEG using Apache Commons Imaging (Sanselan), and, at least in the 0.97 release of Sanselan, there were some bugs related to charset/encoding. The EXIF 2.2 standard requires that the encoding of fields of type UNDEFINED be prefixed with an 8-byte ASCII "signature", describing the encoding of the following content. The field/tag I'm writing to is the UserComment EXIF tag.
Windows expects the content to be encoded in UTF16, so the bytes written to the JPEG must contain a combination of (single byte) ASCII characters, followed by (double byte) Unicode characters. Furthermore, although UserComment doesn't seem to require it, I notice that often the content is "null-padded" to even length.
Here's the code I'm using to create and write the tag:
String textToSet = "Test";
byte[] ASCIIMarker = new byte[]{ 0x55, 0x4E, 0x49, 0x43, 0x4F, 0x44, 0x45, 0x00 }; // spells out "UNICODE"
byte[] comment = textToSet.getBytes("UnicodeLittle");
// pad with \0 if (total) length is odd (or is \0 byte automatically added by arraycopy?)
int pad = (ASCIIMarker.length + comment.length) % 2;
byte[] bytesComment = new byte[ASCIIMarker.length + comment.length + pad];
System.arraycopy(ASCIIMarker, 0, bytesComment, 0, ASCIIMarker.length);
System.arraycopy(comment, 0, bytesComment, ASCIIMarker.length, comment.length);
if (pad > 0) bytesComment[bytesComment.length-1] = 0x00;
TiffOutputField exif_comment = new TiffOutputField(TiffConstants.EXIF_TAG_USER_COMMENT,
TiffFieldTypeConstants.FIELD_TYPE_UNDEFINED, bytesComment.length - pad, bytesComment);
Then when I read the tag back from the JPEG, I do the following:
String textRead;
TiffField field = jpegMetadata.findEXIFValue(TiffConstants.EXIF_TAG_USER_COMMENT);
if (field != null) {
textRead= new String(field.getByteArrayValue(), "UnicodeLittle");
}
What confuses me is this: The bytes written to the JPEG are prefixed with 8 ASCII bytes, which obviously need to be "stripped off" in order to compare what was written to what was read:
if (textRead != null) {
if (textToSet.equals(textRead)) { // expecting this to FAIL
print "Equal";
} else {
print "Not equal";
if (textToSet.equals(textRead.substring(5))) { // this works
print "Equal after all...";
}
}
}
But why substring(5), as opposed to... substring(8)? If it was 4, I might think that 4 double byte (UTF-16) symbols total 8 bytes, but it only works if I strip off 5 bytes. Is this an indication that I'm not creating the payload (byte array bytesComment) properly?
PS! I will update to Apache Commons Imaging RC 1.0, which came out in 2016 and hopefully has fixed these bugs, but I'd still like to understand why this works once I've gotten this far with 0.97 :-)

Writing NDEF data to NTAG216 tag using low-level NFC communication methods

I have implemented the code to interact with an NTAG216 by means of low-level communication methods (following NTAG212 Mifare Ultralight with Authentication and the datasheet of NTAG216).
What I have achieved so far:
Set password write protection on NTAG216 if not set or if new/blank tag.
If password is already set, authenticate using PWD_AUTH and comparing PACK.
Read data.
Write/overwrite data.
What I haven't been able to do so far:
Detect NDEF messages that I write to the tag in other apps. In other words I'm able to write to the tag using writePage() method and also read tags using readPage(). However, on time of writing pages I convert an NdefMessage into a byte array which can be read as well as written. However, this NDEF message is not detected in other applications.
What do I need to do in order to be able to detect NDEF messages I write from other applications?

NTAG216 is an NFC Forum Type 2 tag. Consequently, you have to follow the NFC Forum Type 2 Tag Operation specification when writing data to this type of NFC tag.
Therefore, you will need to follow a few rules for a tag to be discoverable as NDEF tag (Type 2 tag):
First, the capability container (located in block 3) needs to be configured.
Byte 0 must be set to the "magic" value 0xE1.
Byte 1 must be set to 0x10 to indicate mapping version 1.0.
Byte 2 must be set to 0x6D to indicate the memory size of NTAG216.
Byte 3 can be set to 0x00 to indicate read/write access to the NDEF data or to 0x0F to indicate read-only access (note that these are permissions on the application layer).
So you can write the capability container as:
byte[] response = nfc.transceive(new byte[] {
(byte)0xA2, // WRITE
(byte)3, // block address
(byte)0xE1, (byte)0x10, (byte)0x6D, (byte)0x00
});
NTAG216 already ships with a properly configured capability container so there is no need to do this by hand. Also note that block 3 is one-time-programmable, which means that bits can only be set to one but cannot be cleared to zero again. So if you did already overwrite the capability container with a different value, the tag can most likely no longer be used as an NDEF tag.
The data must be written to the data blocks starting at block 4. NDEF messages must be wrapped into an NDEF message TLV (tag-length-value) structure. The tag for this TLV is 0x03. The length can be either in one-byte format (for NDEF messages with a length between 0 and 254 bytes) or in three-byte format (for NDEF messages with a length of 255 or more bytes). The data of this TLV block is the actual NDEF message (that you can obtain from ndefMessage.toByteArray()).
For example, for the NDEF message D1 01 0D 55 01 65 78 61 6D 70 6C 65 2E 63 6F 6D 2F (which is a URI record with the URL "http://www.example.com/"), you would get the following TLV structure:
03 11 D1010D55016578616D706C652E636F6D2F
If you have alonger NDEF message (e.g. one with 259 bytes), you would use the three-byte length format:
03 FF0103 D101FF5501...
Further, you should mark the end of the data on the tag with a Terminator TLV (tag 0xFE, no length and data fields):
FE
You could then write this data to the tag as:
byte[] ndefMessage = new byte[] {
(byte)0xD1, (byte)0x01, (byte)0x0D, (byte)0x55, (byte)0x01, (byte)0x65, (byte)0x78, (byte)0x61, (byte)0x6D, (byte)0x70, (byte)0x6C, (byte)0x65, (byte)0x2E, (byte)0x63, (byte)0x6F, (byte)0x6D, (byte)0x2F
};
// wrap into TLV structure
byte[] tlvEncodedData = null;
if (ndefMessage.length < 255) {
tlvEncodedData = new byte[ndefMessage.length + 3];
tlvEncodedData[0] = (byte)0x03; // NDEF TLV tag
tlvEncodedData[1] = (byte)(ndefMessage.length & 0x0FF); // NDEF TLV length (1 byte)
System.arraycopy(ndefMessage, 0, tlvEncodedData, 2, ndefMessage.length);
tlvEncodedData[2 + ndefMessage.length] = (byte)0xFE; // Terminator TLV tag
} else {
tlvEncodedData = new byte[ndefMessage.length + 5];
tlvEncodedData[0] = (byte)0x03; // NDEF TLV tag
tlvEncodedData[1] = (byte)0xFF; // NDEF TLV length (3 byte, marker)
tlvEncodedData[2] = (byte)((ndefMessage.length >>> 8) & 0x0FF); // NDEF TLV length (3 byte, hi)
tlvEncodedData[3] = (byte)(ndefMessage.length & 0x0FF); // NDEF TLV length (3 byte, lo)
System.arraycopy(ndefMessage, 0, tlvEncodedData, 4, ndefMessage.length);
tlvEncodedData[4 + ndefMessage.length] = (byte)0xFE; // Terminator TLV tag
}
// fill up with zeros to block boundary:
tlvEncodedData = Arrays.copyOf(tlvEncodedData, (tlvEncodedData.length / 4 + 1) * 4);
for (int i = 0; i < tlvEncodedData.length; i += 4) {
byte[] command = new byte[] {
(byte)0xA2, // WRITE
(byte)((4 + i / 4) & 0x0FF), // block address
0, 0, 0, 0
};
System.arraycopy(tlvEncodedData, i, command, 2, 4);
byte[] response = nfc.transceive(command);
}
Finally, be aware that you can't use the tag as NDEF tag if you have read-password set on the NDEF data area since the NFC Forum Type 2 Tag Operation specification requires the tag to be freely readable.

Java Convert 7bit Charset Octets to Readable String (From PDU SMS)

I'm receiving SMS from GSM modem in PDU format; the TP-User-Data is "C8329BFD06DDDF72363904"
and what I get is: "�2����r69", while the sent sms is "Hello World!".
Here is my java code:
private String fromPDUText(String PDUSMSText) {
String endoding = PDUSMSText.substring(0, 2);
PDUSMSText = PDUSMSText.substring(18);
byte bs[] = new byte[PDUSMSText.length() / 2];
for(int i = 0; i < PDUSMSText.length(); i += 2) {
bs[i / 2] = (byte) Integer.parseInt(PDUSMSText.substring(i, i + 2), 16);
}
try {
String out = new String(bs, "ASCII");
} catch(UnsupportedEncodingException e) {
e.printStackTrace();
return "";
} finally {
return out;
}
}

The input is packed in 7-bits per character, which means that every 8 bytes encode 9 characters. Constructing a parser for this format can be a fun exercise or a frustrating experience, depending on how you take it. You are probably better off using a library, and a quick Google search reveals several code examples.

This is how 7Bit characters are packed:
Encoding-Decoding-7-bit-User-Data-for-SMS-PDU-PDU

Personally I find it easiest to attack this kind of problem by viewing it as having a pipe where you feed 8 bits in one end and retrieve 7 bits in the other. As long as there is at least 7 bits in the pipe you read from it. When there are less than 7 bits you need to add some more so you write 8 new bits to it. So what you need is:
A pipe that can hold at least 14 bits (but why be cheap? Go with a 32-bit int!).
A counter keeping track of how many bits are in the pipe at any given moment.
The algorithm in pseudo code is as follows:
pipe = 0;
bitCount = 0;
while(hasMoreData())
{
pipe |= readByte() << bitCount;
bitCount += 8;
while(bitCount >= 7)
{
writeByte(pipe & 0x7F);
pipe >>= 7;
bitCount -= 7;
}
}

Extract hexadecimal values from a percent encoded URL

Let's say for example i have URL containing the following percent encoded character : %80
It is obviously not an ascii character.
How would it be possible to convert this value to the corresponding hex string in Java.
i tried the following with no luck.Result should be 80.
public static void main(String[] args) {
System.out.print(byteArrayToHexString(URLDecoder.decode("%80","UTF-8").getBytes()));
}
public static String byteArrayToHexString(byte[] bytes)
{
StringBuffer buffer = new StringBuffer();
for(int i=0; i<bytes.length; i++)
{
if(((int)bytes[i] & 0xff) < 0x10)
buffer.append("0");
buffer.append(Long.toString((int) bytes[i] & 0xff, 16));
}
return buffer.toString();
}

The best way to deal with this is to parse the url using either java.net.URL or java.net.URI, and then use the relevant getters to extract the components that you require. These will take care of decoding any %-encoded portions in the appropriate fashion.
The problem with your current idea is that %80 does not represent "80", or 80. Rather it represents a byte that further needs to be interpreted in the context of the character encoding of the URL. And if the encoding is UTF-8, then the %80 needs to be followed by one or two more %-encoded bytes ... otherwise this is a malformed UTF-8 character representation.

I don't really see what you are trying. However, I'll give it a try.
When you have got this String: "%80" and you want to got the string "80", you can use this:
String str = "%80";
String hex = str.substring(1); // Cut off the '%'
If you are trying to extract the value 0x80 (which is 128 in decimal) out of it:
String str = "%80";
String hex = str.substring(1); // Cut off the '%'
int value = Integer.parseInt(hex, 16);
If you are trying to convert an int to its hexadecimal representation use this:
String hexRepresenation = Integer.toString(value, 16);