Develop Reference

how to find byte index in string in java

I have string and there is 0x80 in it. string presentation is : serialno� and hex presentation is 73 65 72 69 61 6C 6E 6F 80. I want to remove 0x80 from string without convert string to hex string. is it possible in java ? I tried lastIndexOf(0x80). but it returns -1.
my code is (also you can find on https://ideone.com/3p8wKT) :
public static void main(String[] args) {
String hexValue = "73657269616C6E6F80";
String binValue = hexStringToBin(hexValue);
System.out.println("binValue : " + binValue);
int index = binValue.lastIndexOf(0x80);
System.out.println("index : " + index);
}
public static String hexStringToBin(String s) {
int len = s.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4) + Character.digit(s.charAt(i + 1), 16));
}
return new String(data);
}

Change your hex string method to map directly to characters.
char[] data = new char[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (char) ((Character.digit(s.charAt(i), 16) << 4) + Character.digit(s.charAt(i + 1), 16));
}
return new String(data);
The exchange between a String and byte[] requires an encoding.
Your hex string seems to be a string representation of bytes/characters. It would appear you had an original String -> converted it to your hex string, but we don't know the encoding.
If you want to say that each pair of characters maps to the corresponding character, eg "80" -> char c = 0x80; Then you can achieve that by using a char[], which doesn't get encoded/decoded when creating a string.
If you use a byte[] (as you have done in your example), then it will get decoded and invalid characters get mapped to 0xFFFD, which is unicode replacement character.

It's because you converted � symbol to hex incorrectly (0x80). 1 symbol in UTF-8 can take 1 byte or more. In your case � symbol takes 2 bytes and have the following representation 65533 or 0xFFFD. So, if you replace your code with
int index = variable.lastIndexOf(0xFFFD);
//index will be 8
all will work fine.
Code snippet to proof my words:
String variable = "serialno�";
for (char c : variable.toCharArray())
System.out.print(((int)c)+ " ");
// 115 101 114 105 97 108 110 111 65533
UPDATE
You've made a mistake in hexStringToBin function. Replace it with
public static String hexStringToBin(String s) {
int len = s.length();
char[] data = new char[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (char) ((Character.digit(s.charAt(i), 16) << 4) + Character.digit(s.charAt(i + 1), 16));
}
return new String(data);
}
and all will work fine.

This is working for me :
int hex = 0x6F;
System.out.println("serialno€".lastIndexOf((char)hex));
Output :
7

Android: byte array to decimal string

I have a byte array and I have to print it in a textview, in two differents format :
hex string
decimal string
For hex string I use this function (found on stackoverflow) :
final protected static char[] hexArray = "0123456789ABCDEF".toCharArray();
public static String bytesToHex(byte[] bytes) {
char[] hexChars = new char[bytes.length * 2];
for ( int j = 0; j < bytes.length; j++ ) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
return new String(hexChars);
}
while to convert byte array to decimal string I use this :
BigInteger bi = new BigInteger(value);
// Format to decimal
String s = bi.toString();
For the byte to hex conversion I am secure that it works correctly, but for the byte-to-decimal string conversion I am no much secure..
Are there better methods ?
EDIT : my desidered output is a decimal value for each byte

This will print out the byte array as decimals, with leading zeros (as your hex output does) and a space after each number:
final protected static char[] decimalArray = "0123456789".toCharArray();
public static String bytesToDecimal(byte[] bytes) {
char[] decimalChars = new char[bytes.length * 4];
for ( int j = 0; j < bytes.length; j++ ) {
int v = bytes[j] & 0xFF;
decimalChars[j * 4] = decimalArray[v / 100];
decimalChars[j * 4 + 1] = decimalArray[(v / 10) % 10];
decimalChars[j * 4 + 2] = decimalArray[v % 10];
decimalChars[j * 4 + 3] = ' ';
}
return new String(decimalChars);
}
I've changed the base from 16 to 10, increased the maximum number of characters from 2 to 3 corresponding to max values of FF for base 16 and 255 for decimal. Modulo 10 is used instead of a binary bitmask for masking individual digits because bitmasks only work with powers of 2 like 16.

Put byte array to JSON and vice versa

Is it possible to put a byte[] (byte array) to JSON?
if so, how can I do that in java? then read that JSON and convert that field again to byte[]?

Here is a good example of base64 encoding byte arrays. It gets more complicated when you throw unicode characters in the mix to send things like PDF documents. After encoding a byte array the encoded string can be used as a JSON property value.
Apache commons offers good utilities:
byte[] bytes = getByteArr();
String base64String = Base64.encodeBase64String(bytes);
byte[] backToBytes = Base64.decodeBase64(base64String);
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Base64_encoding_and_decoding
Java server side example:
public String getUnsecureContentBase64(String url)
throws ClientProtocolException, IOException {
//getUnsecureContent will generate some byte[]
byte[] result = getUnsecureContent(url);
// use apache org.apache.commons.codec.binary.Base64
// if you're sending back as a http request result you may have to
// org.apache.commons.httpclient.util.URIUtil.encodeQuery
return Base64.encodeBase64String(result);
}
JavaScript decode:
//decode URL encoding if encoded before returning result
var uriEncodedString = decodeURIComponent(response);
var byteArr = base64DecToArr(uriEncodedString);
//from mozilla
function b64ToUint6 (nChr) {
return nChr > 64 && nChr < 91 ?
nChr - 65
: nChr > 96 && nChr < 123 ?
nChr - 71
: nChr > 47 && nChr < 58 ?
nChr + 4
: nChr === 43 ?
62
: nChr === 47 ?
63
:
0;
}
function base64DecToArr (sBase64, nBlocksSize) {
var
sB64Enc = sBase64.replace(/[^A-Za-z0-9\+\/]/g, ""), nInLen = sB64Enc.length,
nOutLen = nBlocksSize ? Math.ceil((nInLen * 3 + 1 >> 2) / nBlocksSize) * nBlocksSize : nInLen * 3 + 1 >> 2, taBytes = new Uint8Array(nOutLen);
for (var nMod3, nMod4, nUint24 = 0, nOutIdx = 0, nInIdx = 0; nInIdx < nInLen; nInIdx++) {
nMod4 = nInIdx & 3;
nUint24 |= b64ToUint6(sB64Enc.charCodeAt(nInIdx)) << 18 - 6 * nMod4;
if (nMod4 === 3 || nInLen - nInIdx === 1) {
for (nMod3 = 0; nMod3 < 3 && nOutIdx < nOutLen; nMod3++, nOutIdx++) {
taBytes[nOutIdx] = nUint24 >>> (16 >>> nMod3 & 24) & 255;
}
nUint24 = 0;
}
}
return taBytes;
}

The typical way to send binary in json is to base64 encode it.
Java provides different ways to Base64 encode and decode a byte[]. One of these is DatatypeConverter.
Very simply
byte[] originalBytes = new byte[] { 1, 2, 3, 4, 5};
String base64Encoded = DatatypeConverter.printBase64Binary(originalBytes);
byte[] base64Decoded = DatatypeConverter.parseBase64Binary(base64Encoded);
You'll have to make this conversion depending on the json parser/generator library you use.

Amazingly now org.json now lets you put a byte[] object directly into a json and it remains readable. you can even send the resulting object over a websocket and it will be readable on the other side. but i am not sure yet if the size of the resulting object is bigger or smaller than if you were converting your byte array to base64, it would certainly be neat if it was smaller.
It seems to be incredibly hard to measure how much space such a json object takes up in java. if your json consists merely of strings it is easily achievable by simply stringifying it but with a bytearray inside it i fear it is not as straightforward.
stringifying our json in java replaces my bytearray for a 10 character string that looks like an id. doing the same in node.js replaces our byte[] for an unquoted value reading <Buffered Array: f0 ff ff ...> the length of the latter indicates a size increase of ~300% as would be expected

In line with #Qwertie's suggestion, but going further on the lazy side, you could just pretend that each byte is a ISO-8859-1 character. For the uninitiated, ISO-8859-1 is a single-byte encoding that matches the first 256 code points of Unicode.
So #Ash's answer is actually redeemable with a charset:
byte[] args2 = getByteArry();
String byteStr = new String(args2, Charset.forName("ISO-8859-1"));
This encoding has the same readability as BAIS, with the advantage that it is processed faster than either BAIS or base64 as less branching is required. It might look like the JSON parser is doing a bit more, but it's fine because dealing with non-ASCII by escaping or by UTF-8 is part of a JSON parser's job anyways. It could map better to some formats like MessagePack with a profile.
Space-wise however, it is usually a loss, as nobody would be using UTF-16 for JSON. With UTF-8 each non-ASCII byte would occupy 2 bytes, while BAIS uses (2+4n + r?(r+1):0) bytes for every run of 3n+r such bytes (r is the remainder).

If your byte array may contain runs of ASCII characters that you'd like to be able to see, you might prefer BAIS (Byte Array In String) format instead of Base64. The nice thing about BAIS is that if all the bytes happen to be ASCII, they are converted 1-to-1 to a string (e.g. byte array {65,66,67} becomes simply "ABC") Also, BAIS often gives you a smaller file size than Base64 (this isn't guaranteed).
After converting the byte array to a BAIS string, write it to JSON like you would any other string.
Here is a Java class (ported from the original C#) that converts byte arrays to string and back.
import java.io.*;
import java.lang.*;
import java.util.*;
public class ByteArrayInString
{
// Encodes a byte array to a string with BAIS encoding, which
// preserves runs of ASCII characters unchanged.
//
// For simplicity, this method's base-64 encoding always encodes groups of
// three bytes if possible (as four characters). This decision may
// unfortunately cut off the beginning of some ASCII runs.
public static String convert(byte[] bytes) { return convert(bytes, true); }
public static String convert(byte[] bytes, boolean allowControlChars)
{
StringBuilder sb = new StringBuilder();
int i = 0;
int b;
while (i < bytes.length)
{
b = get(bytes,i++);
if (isAscii(b, allowControlChars))
sb.append((char)b);
else {
sb.append('\b');
// Do binary encoding in groups of 3 bytes
for (;; b = get(bytes,i++)) {
int accum = b;
if (i < bytes.length) {
b = get(bytes,i++);
accum = (accum << 8) | b;
if (i < bytes.length) {
b = get(bytes,i++);
accum = (accum << 8) | b;
sb.append(encodeBase64Digit(accum >> 18));
sb.append(encodeBase64Digit(accum >> 12));
sb.append(encodeBase64Digit(accum >> 6));
sb.append(encodeBase64Digit(accum));
if (i >= bytes.length)
break;
} else {
sb.append(encodeBase64Digit(accum >> 10));
sb.append(encodeBase64Digit(accum >> 4));
sb.append(encodeBase64Digit(accum << 2));
break;
}
} else {
sb.append(encodeBase64Digit(accum >> 2));
sb.append(encodeBase64Digit(accum << 4));
break;
}
if (isAscii(get(bytes,i), allowControlChars) &&
(i+1 >= bytes.length || isAscii(get(bytes,i), allowControlChars)) &&
(i+2 >= bytes.length || isAscii(get(bytes,i), allowControlChars))) {
sb.append('!'); // return to ASCII mode
break;
}
}
}
}
return sb.toString();
}
// Decodes a BAIS string back to a byte array.
public static byte[] convert(String s)
{
byte[] b;
try {
b = s.getBytes("UTF8");
} catch(UnsupportedEncodingException e) {
throw new RuntimeException(e.getMessage());
}
for (int i = 0; i < b.length - 1; ++i) {
if (b[i] == '\b') {
int iOut = i++;
for (;;) {
int cur;
if (i >= b.length || ((cur = get(b, i)) < 63 || cur > 126))
throw new RuntimeException("String cannot be interpreted as a BAIS array");
int digit = (cur - 64) & 63;
int zeros = 16 - 6; // number of 0 bits on right side of accum
int accum = digit << zeros;
while (++i < b.length)
{
if ((cur = get(b, i)) < 63 || cur > 126)
break;
digit = (cur - 64) & 63;
zeros -= 6;
accum |= digit << zeros;
if (zeros <= 8)
{
b[iOut++] = (byte)(accum >> 8);
accum <<= 8;
zeros += 8;
}
}
if ((accum & 0xFF00) != 0 || (i < b.length && b[i] != '!'))
throw new RuntimeException("String cannot be interpreted as BAIS array");
i++;
// Start taking bytes verbatim
while (i < b.length && b[i] != '\b')
b[iOut++] = b[i++];
if (i >= b.length)
return Arrays.copyOfRange(b, 0, iOut);
i++;
}
}
}
return b;
}
static int get(byte[] bytes, int i) { return ((int)bytes[i]) & 0xFF; }
public static int decodeBase64Digit(char digit)
{ return digit >= 63 && digit <= 126 ? (digit - 64) & 63 : -1; }
public static char encodeBase64Digit(int digit)
{ return (char)((digit + 1 & 63) + 63); }
static boolean isAscii(int b, boolean allowControlChars)
{ return b < 127 && (b >= 32 || (allowControlChars && b != '\b')); }
}
See also: C# unit tests.

what about simply this:
byte[] args2 = getByteArry();
String byteStr = new String(args2);

How do I convert a signed decimal value to a 32 bit little-endian binary string?

I need to convert a signed decimal number into a 32 bit little-endian binary value. Does anyone by any chance know of a built-in Java class or function that can do this? Or have built one to do this?
The data is a longtitude/latitude value like -78.3829. Thanks for any help.

If it helps at all, here's a class that I made that converts longs to binary Strings and binary Strings to longs:
public class toBinary {
public static void main(String[] args) {
System.out.println(decimalToBinary(16317));
System.out.println(binaryToDecimal("11111111111111111111111111111111111100101001"));
}
public static long binaryToDecimal(String bin) {
long result = 0;
int len = bin.length();
for(int i = 0; i < len; i++) {
result += Integer.parseInt(bin.charAt(i) + "") * Math.pow(2, len - i - 1);
}
return result;
}
public static String decimalToBinary(long num) {
String result = "";
while(true) {
result += num % 2;
if(num < 2)
break;
num = num / 2;
}
for(int i = result.length(); i < 32; i++)
result += "0";
result = reverse(result);
result = toLittleEndian(result);
return result;
}
public static String toLittleEndian(String str) {
String result = "";
result += str.substring(24);
result += str.substring(16, 24);
result += str.substring(8, 16);
result += str.substring(0, 8);
return result;
}
public static String reverse(String str) {
String result = "";
for(int i = str.length() - 1; i >= 0; i--)
result += str.charAt(i);
return result;
}
}
It doesn't take decimal values, but it could probably give you a bit of guidance.

The conversion is trivial once you know what the endianess means on binary level. The question is more what do you really want to do with it?
public static int flipEndianess(int i) {
return (i >>> 24) | // shift byte 3 to byte 0
((i >> 8) & 0xFF00) | // shift byte 2 to byte 1
(i << 24) | // shift byte 0 to byte 3
((i & 0xFF00) << 8); // shift byte 1 to byte 2
}
This little method will swap around the bytes in an int to switch between little/big endian order (the conversion is symetric). Now you have a little endian int. But what would you do with that in Java?
More likely you need to write the data to a stream or something, then its only a question in which order you write the bytes out:
// write int to stream so bytes are little endian in the stream
// OutputStream out = ...
out.write(i);
out.write(i >> 8);
out.write(i >> 16);
out.write(i >> 24);
(For big endian you would just order the lines from bottom to top...)

How do I divide a string of 64bits into four 16bits in Java?

How can I split the 64 bit input into 16 bits?

String s = someStringOfBits;
String s0 = someStringOfbits.substring(0, 16);
String s1 = someStringOfbits.substring(16, 32);
String s2 = someStringOfbits.substring(32, 48);
String s3 = someStringOfbits.substring(48, 64);
This is assuming you actually have a string that looks similar to this:
1010101010101010101010101010101010101010101010101010101010101010

Using a ByteBuffer:
ByteBuffer buf = ByteBuffer.allocate(Long.SIZE / Byte.SIZE);
buf.asLongBuffer().put(0, value);
short a0 = buf.asShortBuffer().get(0);
short a1 = buf.asShortBuffer().get(1);
short a2 = buf.asShortBuffer().get(2);
short a3 = buf.asShortBuffer().get(3);

Here's an inappropriately generic way of doing it :):
public class MyClass
{
static public long[] splitIntoNBits(long value, int numBitsPerChunk){
long[] retVal = null;
if(numBitsPerChunk == 64){
retVal = new long[1];
retVal[0] = value;
return retVal;
}
if(numBitsPerChunk <= 0 || numBitsPerChunk > 64){
return null;
}
long mask = (1 << numBitsPerChunk) - 1;
int numFullChunks = (byte) (64 / numBitsPerChunk);
int numBitsInLastChunk = (byte) (64 - numFullChunks * numBitsPerChunk);
int numTotalChunks = numFullChunks;
if(numBitsInLastChunk > 0){
numTotalChunks++;
}
retVal = new long[numTotalChunks];
for(int i = 0; i < numTotalChunks; i++){
retVal[i] = value & mask;
value >>= numBitsPerChunk;
}
// clean up the last chunk
if(numBitsInLastChunk > 0){
mask = (1 << numBitsInLastChunk) - 1;
retVal[retVal.length - 1] &= mask;
}
return retVal;
}
public static void main(String[] args)
{
long myvalue = ((long) 0x12345678) | (((long) 0xABCDEF99) << 32);
long[] bitFields = splitIntoNBits(myvalue, 16);
for(int i=0; i < bitFields.length; i++){
System.out.printf("Field %d: %x\r\n", i, bitFields[i]);
}
}
}
produces output:
Field 0: 5678
Field 1: 1234
Field 2: ef99
Field 3: abcd
and for a bitsPerField of 7 it produces:
Field 0: 78
Field 1: 2c
Field 2: 51
Field 3: 11
Field 4: 11
Field 5: 73
Field 6: 7b
Field 7: 66
Field 8: 2b
Field 9: 1
Isn't that fun!?

given that value contains your 64bit number
char a1 = (char) (value & 0xffff);
char a2 = (char) ((value>>16) & 0xffff);
char a3 = (char) ((value>>32) & 0xffff);
char a4 = (char) ((value>>48) & 0xffff);
edit: added casts as suggested by commenter