Put byte array to JSON and vice versa - java

Is it possible to put a byte[] (byte array) to JSON?
if so, how can I do that in java? then read that JSON and convert that field again to byte[]?

Here is a good example of base64 encoding byte arrays. It gets more complicated when you throw unicode characters in the mix to send things like PDF documents. After encoding a byte array the encoded string can be used as a JSON property value.
Apache commons offers good utilities:
byte[] bytes = getByteArr();
String base64String = Base64.encodeBase64String(bytes);
byte[] backToBytes = Base64.decodeBase64(base64String);
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Base64_encoding_and_decoding
Java server side example:
public String getUnsecureContentBase64(String url)
throws ClientProtocolException, IOException {
//getUnsecureContent will generate some byte[]
byte[] result = getUnsecureContent(url);
// use apache org.apache.commons.codec.binary.Base64
// if you're sending back as a http request result you may have to
// org.apache.commons.httpclient.util.URIUtil.encodeQuery
return Base64.encodeBase64String(result);
}
JavaScript decode:
//decode URL encoding if encoded before returning result
var uriEncodedString = decodeURIComponent(response);
var byteArr = base64DecToArr(uriEncodedString);
//from mozilla
function b64ToUint6 (nChr) {
return nChr > 64 && nChr < 91 ?
nChr - 65
: nChr > 96 && nChr < 123 ?
nChr - 71
: nChr > 47 && nChr < 58 ?
nChr + 4
: nChr === 43 ?
62
: nChr === 47 ?
63
:
0;
}
function base64DecToArr (sBase64, nBlocksSize) {
var
sB64Enc = sBase64.replace(/[^A-Za-z0-9\+\/]/g, ""), nInLen = sB64Enc.length,
nOutLen = nBlocksSize ? Math.ceil((nInLen * 3 + 1 >> 2) / nBlocksSize) * nBlocksSize : nInLen * 3 + 1 >> 2, taBytes = new Uint8Array(nOutLen);
for (var nMod3, nMod4, nUint24 = 0, nOutIdx = 0, nInIdx = 0; nInIdx < nInLen; nInIdx++) {
nMod4 = nInIdx & 3;
nUint24 |= b64ToUint6(sB64Enc.charCodeAt(nInIdx)) << 18 - 6 * nMod4;
if (nMod4 === 3 || nInLen - nInIdx === 1) {
for (nMod3 = 0; nMod3 < 3 && nOutIdx < nOutLen; nMod3++, nOutIdx++) {
taBytes[nOutIdx] = nUint24 >>> (16 >>> nMod3 & 24) & 255;
}
nUint24 = 0;
}
}
return taBytes;
}

The typical way to send binary in json is to base64 encode it.
Java provides different ways to Base64 encode and decode a byte[]. One of these is DatatypeConverter.
Very simply
byte[] originalBytes = new byte[] { 1, 2, 3, 4, 5};
String base64Encoded = DatatypeConverter.printBase64Binary(originalBytes);
byte[] base64Decoded = DatatypeConverter.parseBase64Binary(base64Encoded);
You'll have to make this conversion depending on the json parser/generator library you use.

Amazingly now org.json now lets you put a byte[] object directly into a json and it remains readable. you can even send the resulting object over a websocket and it will be readable on the other side. but i am not sure yet if the size of the resulting object is bigger or smaller than if you were converting your byte array to base64, it would certainly be neat if it was smaller.
It seems to be incredibly hard to measure how much space such a json object takes up in java. if your json consists merely of strings it is easily achievable by simply stringifying it but with a bytearray inside it i fear it is not as straightforward.
stringifying our json in java replaces my bytearray for a 10 character string that looks like an id. doing the same in node.js replaces our byte[] for an unquoted value reading <Buffered Array: f0 ff ff ...> the length of the latter indicates a size increase of ~300% as would be expected

In line with #Qwertie's suggestion, but going further on the lazy side, you could just pretend that each byte is a ISO-8859-1 character. For the uninitiated, ISO-8859-1 is a single-byte encoding that matches the first 256 code points of Unicode.
So #Ash's answer is actually redeemable with a charset:
byte[] args2 = getByteArry();
String byteStr = new String(args2, Charset.forName("ISO-8859-1"));
This encoding has the same readability as BAIS, with the advantage that it is processed faster than either BAIS or base64 as less branching is required. It might look like the JSON parser is doing a bit more, but it's fine because dealing with non-ASCII by escaping or by UTF-8 is part of a JSON parser's job anyways. It could map better to some formats like MessagePack with a profile.
Space-wise however, it is usually a loss, as nobody would be using UTF-16 for JSON. With UTF-8 each non-ASCII byte would occupy 2 bytes, while BAIS uses (2+4n + r?(r+1):0) bytes for every run of 3n+r such bytes (r is the remainder).

If your byte array may contain runs of ASCII characters that you'd like to be able to see, you might prefer BAIS (Byte Array In String) format instead of Base64. The nice thing about BAIS is that if all the bytes happen to be ASCII, they are converted 1-to-1 to a string (e.g. byte array {65,66,67} becomes simply "ABC") Also, BAIS often gives you a smaller file size than Base64 (this isn't guaranteed).
After converting the byte array to a BAIS string, write it to JSON like you would any other string.
Here is a Java class (ported from the original C#) that converts byte arrays to string and back.
import java.io.*;
import java.lang.*;
import java.util.*;
public class ByteArrayInString
{
// Encodes a byte array to a string with BAIS encoding, which
// preserves runs of ASCII characters unchanged.
//
// For simplicity, this method's base-64 encoding always encodes groups of
// three bytes if possible (as four characters). This decision may
// unfortunately cut off the beginning of some ASCII runs.
public static String convert(byte[] bytes) { return convert(bytes, true); }
public static String convert(byte[] bytes, boolean allowControlChars)
{
StringBuilder sb = new StringBuilder();
int i = 0;
int b;
while (i < bytes.length)
{
b = get(bytes,i++);
if (isAscii(b, allowControlChars))
sb.append((char)b);
else {
sb.append('\b');
// Do binary encoding in groups of 3 bytes
for (;; b = get(bytes,i++)) {
int accum = b;
if (i < bytes.length) {
b = get(bytes,i++);
accum = (accum << 8) | b;
if (i < bytes.length) {
b = get(bytes,i++);
accum = (accum << 8) | b;
sb.append(encodeBase64Digit(accum >> 18));
sb.append(encodeBase64Digit(accum >> 12));
sb.append(encodeBase64Digit(accum >> 6));
sb.append(encodeBase64Digit(accum));
if (i >= bytes.length)
break;
} else {
sb.append(encodeBase64Digit(accum >> 10));
sb.append(encodeBase64Digit(accum >> 4));
sb.append(encodeBase64Digit(accum << 2));
break;
}
} else {
sb.append(encodeBase64Digit(accum >> 2));
sb.append(encodeBase64Digit(accum << 4));
break;
}
if (isAscii(get(bytes,i), allowControlChars) &&
(i+1 >= bytes.length || isAscii(get(bytes,i), allowControlChars)) &&
(i+2 >= bytes.length || isAscii(get(bytes,i), allowControlChars))) {
sb.append('!'); // return to ASCII mode
break;
}
}
}
}
return sb.toString();
}
// Decodes a BAIS string back to a byte array.
public static byte[] convert(String s)
{
byte[] b;
try {
b = s.getBytes("UTF8");
} catch(UnsupportedEncodingException e) {
throw new RuntimeException(e.getMessage());
}
for (int i = 0; i < b.length - 1; ++i) {
if (b[i] == '\b') {
int iOut = i++;
for (;;) {
int cur;
if (i >= b.length || ((cur = get(b, i)) < 63 || cur > 126))
throw new RuntimeException("String cannot be interpreted as a BAIS array");
int digit = (cur - 64) & 63;
int zeros = 16 - 6; // number of 0 bits on right side of accum
int accum = digit << zeros;
while (++i < b.length)
{
if ((cur = get(b, i)) < 63 || cur > 126)
break;
digit = (cur - 64) & 63;
zeros -= 6;
accum |= digit << zeros;
if (zeros <= 8)
{
b[iOut++] = (byte)(accum >> 8);
accum <<= 8;
zeros += 8;
}
}
if ((accum & 0xFF00) != 0 || (i < b.length && b[i] != '!'))
throw new RuntimeException("String cannot be interpreted as BAIS array");
i++;
// Start taking bytes verbatim
while (i < b.length && b[i] != '\b')
b[iOut++] = b[i++];
if (i >= b.length)
return Arrays.copyOfRange(b, 0, iOut);
i++;
}
}
}
return b;
}
static int get(byte[] bytes, int i) { return ((int)bytes[i]) & 0xFF; }
public static int decodeBase64Digit(char digit)
{ return digit >= 63 && digit <= 126 ? (digit - 64) & 63 : -1; }
public static char encodeBase64Digit(int digit)
{ return (char)((digit + 1 & 63) + 63); }
static boolean isAscii(int b, boolean allowControlChars)
{ return b < 127 && (b >= 32 || (allowControlChars && b != '\b')); }
}
See also: C# unit tests.

what about simply this:
byte[] args2 = getByteArry();
String byteStr = new String(args2);

Related

how to find byte index in string in java

I have string and there is 0x80 in it. string presentation is : serialno� and hex presentation is 73 65 72 69 61 6C 6E 6F 80. I want to remove 0x80 from string without convert string to hex string. is it possible in java ? I tried lastIndexOf(0x80). but it returns -1.
my code is (also you can find on https://ideone.com/3p8wKT) :
public static void main(String[] args) {
String hexValue = "73657269616C6E6F80";
String binValue = hexStringToBin(hexValue);
System.out.println("binValue : " + binValue);
int index = binValue.lastIndexOf(0x80);
System.out.println("index : " + index);
}
public static String hexStringToBin(String s) {
int len = s.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4) + Character.digit(s.charAt(i + 1), 16));
}
return new String(data);
}
Change your hex string method to map directly to characters.
char[] data = new char[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (char) ((Character.digit(s.charAt(i), 16) << 4) + Character.digit(s.charAt(i + 1), 16));
}
return new String(data);
The exchange between a String and byte[] requires an encoding.
Your hex string seems to be a string representation of bytes/characters. It would appear you had an original String -> converted it to your hex string, but we don't know the encoding.
If you want to say that each pair of characters maps to the corresponding character, eg "80" -> char c = 0x80; Then you can achieve that by using a char[], which doesn't get encoded/decoded when creating a string.
If you use a byte[] (as you have done in your example), then it will get decoded and invalid characters get mapped to 0xFFFD, which is unicode replacement character.
It's because you converted � symbol to hex incorrectly (0x80). 1 symbol in UTF-8 can take 1 byte or more. In your case � symbol takes 2 bytes and have the following representation 65533 or 0xFFFD. So, if you replace your code with
int index = variable.lastIndexOf(0xFFFD);
//index will be 8
all will work fine.
Code snippet to proof my words:
String variable = "serialno�";
for (char c : variable.toCharArray())
System.out.print(((int)c)+ " ");
// 115 101 114 105 97 108 110 111 65533
UPDATE
You've made a mistake in hexStringToBin function. Replace it with
public static String hexStringToBin(String s) {
int len = s.length();
char[] data = new char[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (char) ((Character.digit(s.charAt(i), 16) << 4) + Character.digit(s.charAt(i + 1), 16));
}
return new String(data);
}
and all will work fine.
This is working for me :
int hex = 0x6F;
System.out.println("serialno€".lastIndexOf((char)hex));
Output :
7

How can i credit or debit more than 1 byte to/from card?

I am newbie applets and i used from this link: working with Java Card
Wallet for creating an Wallet project.
I before could credit card amount by this command : 80 30 00 00 01 1A 00.
I now want add '5000' to the present amount. As you know 5000 in hex equals
with '1388' that is 2 byte. So i must send 2 byte data 13 and 88 to the card.
I create bellow command and sent it to card but i get '67 00 Wrong lenght' as
response.
80 30 00 00 02 13 88 00
How can i credit or debit more than 1 byte to/from card?
You'll have to change the code of the Applet you're pointing to of course:
if ((numBytes != 1) || (byteRead != 1)) {
ISOException.throwIt(ISO7816.SW_WRONG_LENGTH); // constant with value 0x6700
}
So you must make sure that it allows for 2 bytes to be send, then you can use the Util.getShort method to convert to the bytes to a 16 bit signed value (using big endian two complement notation, as usual).
Replace the creadit() method, with this one. But remember that you must use two byte value for crediting you walled henceforth. (even for values less than 255 or 0xFF. i.e. you must use 0x00FF to debit you wallet with 255$ )
private void credit(APDU apdu) {
// access authentication
if (!pin.isValidated()) {
ISOException.throwIt(SW_PIN_VERIFICATION_REQUIRED);
}
byte[] buffer = apdu.getBuffer();
// Lc byte denotes the number of bytes in the
// data field of the command APDU
byte numBytes = buffer[ISO7816.OFFSET_LC];
// indicate that this APDU has incoming data
// and receive data starting from the offset
// ISO7816.OFFSET_CDATA following the 5 header
// bytes.
byte byteRead = (byte) (apdu.setIncomingAndReceive());
// it is an error if the number of data bytes
// read does not match the number in Lc byte
if ((numBytes != 2) || (byteRead != 2)) {
ISOException.throwIt(ISO7816.SW_WRONG_LENGTH);
}
// get the creditBytes
byte[] creditBytes = new byte[2];
creditBytes[0]=buffer[ISO7816.OFFSET_CDATA];
creditBytes[1]=buffer[ISO7816.OFFSET_CDATA+1];
// convert 2 byte of creatBytes to a single short value.
short creditAmount = Util.getShort(creditBytes,(short)0);
// check the credit amount
if ((creditAmount > MAX_TRANSACTION_AMOUNT) || (creditAmount < 0)) {
ISOException.throwIt(SW_INVALID_TRANSACTION_AMOUNT);
}
// check the new balance
if ((short) (balance + creditAmount) > MAX_BALANCE) {
ISOException.throwIt(SW_EXCEED_MAXIMUM_BALANCE);
}
// credit the amount
balance = (short) (balance + creditAmount);
}
I propose using BCD addition and BCD subtraction, as follow:
Each byte represent two BCD, e.g. 0x99 represent 99 instead of 153.
All data included in the addition and subtraction shall have the same length, e.g. 6 bytes will represents 12 digits. This should cover most cases, but if you need more, simply change your constant.
Your applet performs loop through the bytes to do the addition or subtraction. Encode and decode operation from BCD to the value and vice versa are needed before and after the operation.
Here is sample for the implementation. It is not tested yet, but should give you idea of how it works:
public class BCD {
public static final short NUMBER_OF_BYTES = 6;
static void add(byte[] augend, byte[] addend, byte[] result) {
byte carry = 0;
short temp = 0;
for (short i = (short) (NUMBER_OF_BYTES - 1); i >= 0; i--) {
temp = (short) (decode(augend[i]) + decode(addend[i]) + carry);
carry = (byte) ((temp > 100) ? 1 : 0);
result[i] = encode((byte) temp);
}
if (carry == 1) {
// TODO: result more than maximum
// you can set all digits to 9 or throw exception
}
}
static void subtract(byte[] minuend, byte[] subtrahend, byte[] result) {
byte borrow = 0;
short temp = 0;
for (short i = (short) (NUMBER_OF_BYTES - 1); i >= 0; i--) {
temp = (short) (100 + decode(minuend[i]) - decode(subtrahend[i]) - borrow);
borrow = (byte) ((temp < 100) ? 1 : 0);
result[i] = encode((byte) temp);
}
if (borrow == 1) {
// TODO: subtrahend > minuend,
// you can set all digits to 0 or throw exception
}
}
static byte encode(byte value) {
value %= 100; // only convert two digits, ignore borrow/carry
return (byte) (((value / 10) << 4) | (value % 10));
}
static byte decode(byte bcdByte) {
byte highNibble = (byte) ((bcdByte >> 4) & 0x0F);
byte lowNibble = (byte) (bcdByte & 0x0F);
if ((highNibble > 9) || (lowNibble > 9)) {
// found 'A' to 'F' character which should be invalid
// you can change this line, e.g. throwing exception
return 0;
}
return (byte) ((highNibble * 10) + lowNibble);
}
}

Base64.encode(bytearray, Base64.DEFAULT) result of this method will add 10 at last item

When i encode my byte array using Base64.encode(bytearray, Base64.DEFAULT) so result of this method will add 10 at last item in the resulting byte array and when i converting this resulting byte array into string than 10 will convert into \n(line feed) at the end
please let me know why the \n will append at the end
below is the code that will convert the string into byte array
int inLength = hexValue.length();
int i, o = 0;
long outByte = 0;
byte[] outBytes = new byte[(inLength / 2)];
for (i = 0; i < inLength; i++) {
char c = hexValue.charAt(i);
int value = -1;
if (c >= '0' && c <= '9')
value = (c - '0');
else if (c >= 'A' && c <= 'F')
value = 10 + (c - 'A');
else if (c >= 'a' && c <= 'f')
value = 10 + (c - 'a');
if (value >= 0) {
if (i % 2 == 1) {
outBytes[o++] = (byte) ((outByte << 4) | value);
outByte = 0;
} else {
outByte = value;
}
} else {
if (o != 0)
break;
}
}
return outBytes;
I can't comment right now so i had to leave a reply.
Is this code written by you? or Someone else?
For Base64 encoding, byte are processed in blocks of 3 bytes at a time. If the length of the array is not a multiple of three then either one or two '0' zero bytes is appended to make full block.
And why are you writing your own logic where there are some API available to do the work for you?
Update: I just run the code again with help of API.
Input String: 51b034267f00000144495444
And
Corresponding Base64 encoded String: NTFiMDM0MjY3ZjAwMDAwMTQ0NDk1NDQ0
Simply put, 3 bytes will result in 4 plain text encoded bytes.

Bit manipulation C source in Java

I try to calculate the checksum of a Sega Genesis rom file in Java. For this i want to port a code snipped from C into Java:
static uint16 getchecksum(uint8 *rom, int length)
{
int i;
uint16 checksum = 0;
for (i = 0; i < length; i += 2)
{
checksum += ((rom[i] << 8) + rom[i + 1]);
}
return checksum;
}
I understand what the code does. It sums all 16bit numbers (combined from two 8 bit ones). But what i didn't understand is what's happening with the overflow of the uint16 and how this transfers to Java code?
Edit:
This code seems to work, thanks:
int calculatedChecksum = 0;
int bufferi1=0;
int bufferi2=0;
bs = new BufferedInputStream(new FileInputStream(this.file));
bufferi1 = bs.read();
bufferi2 = bs.read();
while(bufferi1 != -1 && bufferi2 != -1){
calculatedChecksum += (bufferi1*256 + bufferi2);
calculatedChecksum = calculatedChecksum % 0x10000;
bufferi1 = bs.read();
bufferi2 = bs.read();
}
Simply put, the overflow is lost.
A more correct approach (imho) is to use uint32 for summation, and then you have the sum in the lower 16 bits, and the overflow in the upper 16 bits.
static int checksum(final InputStream in) throws IOException {
short v = 0;
int c;
while ((c = in.read()) >= 0) {
v += (c << 8) | in.read();
}
return v & 0xffff;
}
This should work equivalently; by using & 0xffff, we get to treat the value in v as if it were unsigned the entire time, since arithmetic overflow is identical w.r.t. bits.
You want addition modulo 216, which you can simply spell out manually:
checksum = (checksum + ((rom[i] << 8) + rom[i + 1])) % 0x10000;
// ^^^^^^^^^

How do I convert a signed decimal value to a 32 bit little-endian binary string?

I need to convert a signed decimal number into a 32 bit little-endian binary value. Does anyone by any chance know of a built-in Java class or function that can do this? Or have built one to do this?
The data is a longtitude/latitude value like -78.3829. Thanks for any help.
If it helps at all, here's a class that I made that converts longs to binary Strings and binary Strings to longs:
public class toBinary {
public static void main(String[] args) {
System.out.println(decimalToBinary(16317));
System.out.println(binaryToDecimal("11111111111111111111111111111111111100101001"));
}
public static long binaryToDecimal(String bin) {
long result = 0;
int len = bin.length();
for(int i = 0; i < len; i++) {
result += Integer.parseInt(bin.charAt(i) + "") * Math.pow(2, len - i - 1);
}
return result;
}
public static String decimalToBinary(long num) {
String result = "";
while(true) {
result += num % 2;
if(num < 2)
break;
num = num / 2;
}
for(int i = result.length(); i < 32; i++)
result += "0";
result = reverse(result);
result = toLittleEndian(result);
return result;
}
public static String toLittleEndian(String str) {
String result = "";
result += str.substring(24);
result += str.substring(16, 24);
result += str.substring(8, 16);
result += str.substring(0, 8);
return result;
}
public static String reverse(String str) {
String result = "";
for(int i = str.length() - 1; i >= 0; i--)
result += str.charAt(i);
return result;
}
}
It doesn't take decimal values, but it could probably give you a bit of guidance.
The conversion is trivial once you know what the endianess means on binary level. The question is more what do you really want to do with it?
public static int flipEndianess(int i) {
return (i >>> 24) | // shift byte 3 to byte 0
((i >> 8) & 0xFF00) | // shift byte 2 to byte 1
(i << 24) | // shift byte 0 to byte 3
((i & 0xFF00) << 8); // shift byte 1 to byte 2
}
This little method will swap around the bytes in an int to switch between little/big endian order (the conversion is symetric). Now you have a little endian int. But what would you do with that in Java?
More likely you need to write the data to a stream or something, then its only a question in which order you write the bytes out:
// write int to stream so bytes are little endian in the stream
// OutputStream out = ...
out.write(i);
out.write(i >> 8);
out.write(i >> 16);
out.write(i >> 24);
(For big endian you would just order the lines from bottom to top...)

Categories

Resources