A trading partner has asked me to send an HMAC SHA1 hash as lowercase heaxits. The only reference I can find to them is in relation to PHP. I can do the hashing in .NET and Java but how do I output "lowercase hexits" with them? Lowercase hexits doesn't appear to be equivalent to Base64.
Ah! I love simplicity. Here's the solution.
Public Shared Function Encrypt(ByVal plainText As String, ByVal preSharedKey As String) As String
Dim preSharedKeyBytes() As Byte = Encoding.UTF8.GetBytes(preSharedKey)
Dim plainTextBytes As Byte() = Encoding.UTF8.GetBytes(plainText)
Dim hmac = New HMACSHA1(preSharedKeyBytes)
Dim cipherTextBytes As Byte() = hmac.ComputeHash(plainTextBytes)
Dim strTemp As New StringBuilder(cipherTextBytes.Length * 2)
For Each b As Byte In cipherTextBytes
strTemp.Append(Conversion.Hex(b).PadLeft(2, "0"c).ToLower)
Next
Dim cipherText As String = strTemp.ToString
Return cipherText
End Function
This is compatible with the PHP hash_hmac function with FALSE in the raw_output parameter.
For lowercase hex digits (hexits) use:
public static String toHex(byte[] bytes) {
BigInteger bi = new BigInteger(1, bytes);
return String.format("%0" + (bytes.length << 1) + "x", bi);
}
From related question:
In Java, how do I convert a byte array to a string of hex digits while keeping leading zeros?
Here's a C# translation of sedge's solution:
private static String toHex(byte[] cipherTextBytes)
{
var strTemp = new StringBuilder(cipherTextBytes.Length * 2);
foreach(Byte b in cipherTextBytes)
{
strTemp.Append(Microsoft.VisualBasic.Conversion.Hex(b).PadLeft(2, '0').ToLower());
}
String cipherText = strTemp.ToString();
return cipherText;
}
Related
I have a string in which the SMS text is encrypted in USC2 format received from a GSM modem
I'm trying to convert it to UTF16 but it doesn't work. Please tell me what am I doing wrong
public class USC {
public static void main(String[] args) {
String hex = "0412044B0020043F043E043B044C043704430435044204350441044C002004420430044004380444043D044B043C0020043F043B0430043D043E043C0020002204110438002B002200200441002000300033002E00310032002E0032003000320031002E002004230442043E0447043D04380442044C002004430441043B043E04320438044F";
byte[] v = hex.getBytes(StandardCharsets.UTF_16BE);
String str = new String(v);
System.out.println(str);
}
}
On the online decoder through the service https://dencode.com/ works fine
Try the following:
BigInteger bi = new BigInteger(hex, 16);
byte[] a = bi.toByteArray();
System.out.println(new String(a, Charset.forName("UTF-16")));
String hex = "0412044B0020043F043E043B044C043704430435044204350441044C002004420430044004380444043D044B043C0020043F043B0430043D043E043C0020002204110438002B002200200441002000300033002E00310032002E0032003000320031002E002004230442043E0447043D04380442044C002004430441043B043E04320438044F";
int n = hex.length/4;
char[] chars = new char[n];
for (int i = 0; i < n; ++i) {
chars[i] = Integer.parseInt(hex.substring(4*i, 4*i+4), 15) & 0xFFFF);
}
String str = new String(chars);
System.out.println(str);
4 hex chars form one UCS-2 big endian char. Same size as java char (2 bytes).
UTF-16 is superior to UCS-2 which forms a fixed-size subset. So from UCS-2 to UTF-16 needs no special treatment, only whether the 2 bytes of a char are big endian or little endian.
With JDK17 or above you could also make use of HexFormat class:
String str = new String(HexFormat.of().parseHex(hex), StandardCharsets.UTF_16BE);
I have come across a legacy piece of code encoding byte array to hex string which is in production and have never caused an issue.
This piece of code is used as:
We encrypt a user password. The encryptor returns a byte[].
We convert the byte[] to Hex String using this encoder code and then use that String representation in our properties file and so on.
However, yesterday we have hit a password, whose encrypted byte[] version is getting encoded incorrectly.
import java.math.BigInteger;
import java.util.HashMap;
import org.apache.commons.codec.DecoderException;
import org.apache.commons.codec.binary.Hex;
public class ByteArrayToHexEncoder {
public static void main(String[] args) throws DecoderException {
String hexString = "a0d21588c0a2c2fc68dc859197fc78cd"; // correct hex representation
// equivalent byte array: this is the byte array returned by the encryptor
byte[] byteArray = Hex.decodeHex(hexString.toCharArray());
// legacy encoder
System.out.println("Legacy code encodes as: " + encodeHexBytesWithPadding(byteArray));
// commons-codec encoder
System.out.println("Commons codec encode as: " + new String(Hex.encodeHex(byteArray)));
}
private static final String PADDING_ZEROS =
"0000000000000000000000000000000000000000000000000000000000000";
private static final HashMap<Integer, Character> MAP_OF_HEX = new HashMap<>();
static {
MAP_OF_HEX.put(0, '0');
MAP_OF_HEX.put(1, '1');
MAP_OF_HEX.put(2, '2');
MAP_OF_HEX.put(3, '3');
MAP_OF_HEX.put(4, '4');
MAP_OF_HEX.put(5, '5');
MAP_OF_HEX.put(6, '6');
MAP_OF_HEX.put(7, '7');
MAP_OF_HEX.put(8, '8');
MAP_OF_HEX.put(9, '9');
MAP_OF_HEX.put(10, 'a');
MAP_OF_HEX.put(11, 'b');
MAP_OF_HEX.put(12, 'c');
MAP_OF_HEX.put(13, 'd');
MAP_OF_HEX.put(14, 'e');
MAP_OF_HEX.put(15, 'f');
}
public static String encodeHexBytesWithPadding(byte[] inputByteArray) {
String encodedValue = encodeHexBytes(inputByteArray);
int expectedSize = inputByteArray.length * 2;
if (encodedValue.length() < expectedSize) {
int zerosToPad = expectedSize - encodedValue.length();
encodedValue = PADDING_ZEROS.substring(0, zerosToPad) + encodedValue;
}
return encodedValue;
}
public static String encodeHexBytes(byte[] inputByteArray) {
String encodedValue;
if (inputByteArray[0] < 0) {
// Something is wrong here! Don't know what!
byte oldValue = inputByteArray[0];
inputByteArray[0] = (byte) (oldValue & 0x0F);
int nibble = (oldValue >> 4) & 0x0F;
encodedValue = new BigInteger(inputByteArray).toString(16);
inputByteArray[0] = oldValue;
encodedValue = MAP_OF_HEX.get(nibble) + encodedValue;
} else {
encodedValue = new BigInteger(inputByteArray).toString(16);
}
return encodedValue;
}
}
The legacy code outputs the encoded value as: 0ad21588c0a2c2fc68dc859197fc78cd while the correct expected value should be: a0d21588c0a2c2fc68dc859197fc78cd.
I am trying to understand what's wrong with the encoder and need some help understanding.
BigInteger(byte[]) constructor is there to handle two's complement representation of a number where the most significant bit also denotes the sign. The Hex common-codec simply translates each byte into a hex representation, there is no special meaning to the most significant bit.
Your legacy code in the if (inputByteArray[0] < 0) branch attempts to modify the first byte in the byte[] input probably to work around the representation of negative numbers in the two-complement's form e.g. -1 being represented as ff. Unfortunately this is implemented incorrectly in your legacy code:
String input = "a000000001";
byte[] bytes = Hex.decodeHex(input.toCharArray());
System.out.println(encodeHexBytesWithPadding(bytes));
System.out.println(Hex.encodeHexString(bytes));
will print
00000000a1
a000000001
showing that the legacy code values are completely wrong.
There is not much to salvage here IMO, instead use Hex.encodeHexString() instead or check other options discussed in this question.
I try to convert BigInteger to a string using Base64 and check the reversibility :
This case works fine :
Check A1 : "999999999" => "O5rJ/w=="
Check B1 : "O5rJ/w==" => "999999999"
but this case doesn't :
Check B2 : "//////" => "4294967295"
Check A2 : "4294967295" => "/////w" instead of "//////"
I'm using Java 8 and org.apache.commons.codec.binary.Base64
#Test
public void testDecimalToBase64()
{
final String encodedMaxDec = new String(Base64.encodeInteger(new BigInteger("999999999")), StandardCharsets.UTF_8);
Assert.assertEquals("Check A1", "O5rJ/w==", encodedMaxDec);
final String encodedMaxB64 = new String(Base64.encodeInteger(new BigInteger("4294967295")), StandardCharsets.UTF_8);
Assert.assertEquals("Check A2", "//////", encodedMaxB64);
}
#Test
public void testBase64ToDecimal()
{
final BigInteger decodedMaxDec = Base64.decodeInteger(new String("O5rJ/w==").getBytes(StandardCharsets.UTF_8));
Assert.assertEquals("Check B1", "999999999", decodedMaxDec.toString());
final BigInteger decodedMaxB64 = Base64.decodeInteger(new String("//////").getBytes(StandardCharsets.UTF_8));
Assert.assertEquals("Check B2", "4294967295", decodedMaxB64.toString());
}
If anyone has an idea, thank you for helping me ;-)
The base64 encoding of a BitInteger's bytes when representing 4294967295 isn't //////.
I'm assuming that because every integer in Java is signed the BigInteger sticks an additional byte on the front to indicate whether the number is positive or negative.
Therefore this code...
byte[] bigIntegerBytes = BigInteger.valueOf(4294967295l).toByteArray();
.. returns a 5 byte array and you have the following bytes:
000000000 11111111 11111111 11111111 1111111
If you base64 encode that you get AP////8=
If what you want to do is base64 encode a BigInteger representing 4294967295 and then decode it back again, this should do it.
Base64.Encoder encoder = Base64.getEncoder();
Base64.Decoder decoder = Base64.getDecoder();
byte[] bigIntegerBytes = BigInteger.valueOf(4294967295l).toByteArray();
String base64EncodedBigIntegerBytes = encoder.encodeToString(bigIntegerBytes);
System.out.println(base64EncodedBigIntegerBytes);
byte[] decodedBigIntegerBytes = decoder.decode(base64EncodedBigIntegerBytes);
BigInteger decodedBigInteger = new BigInteger(decodedBigIntegerBytes);
System.out.println(decodedBigInteger);
This is the PHP code I have.
function decrypt($s_input, $s_key, $s_iv) {
$s_decrypted = pack("H*" , $s_input); // Hex to binary
$s_decrypted = mcrypt_decrypt (MCRYPT_3DES, $s_key, $s_decrypted, MCRYPT_MODE_CBC, $s_iv); // 3des decryption
return $s_decrypted;
}
echo encrypt('c37551bb77f741d0bcdc16497b4f97b1','123456781234567812345678','12345678' );
what it basically does is to decrypt a 3des encrypted string (first it convert the hex string to binary using pack function and then does the actual decryption).
This perfectly works in PHP-4 and prints the "Hello World" message.
However, if I run the equivalent java code (jdk 1.6), it prints garbage output as - ¬ªmjV=7xl_ÓÄ^›*?.
Can someone help to troubleshoot this? Why Java is not properly decrypting the hex string.
private static String decrypt(String inputStr, String keyStr, String ivStr) throws Exception {
IvParameterSpec iv = new IvParameterSpec(ivStr.getBytes());
SecretKeySpec key = new SecretKeySpec(keyStr.getBytes(), "DESede");
inputStr = hexToString(inputStr, 2);
Cipher cipher = Cipher.getInstance("DESede/CBC/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, key, iv);
byte[] decrypted = cipher.doFinal(inputStr.getBytes());
return new String(decrypted);
}
private static String hexToString(String input, int groupLength) {
StringBuilder sb = new StringBuilder(input.length() / groupLength);
for (int i = 0; i < input.length() - groupLength + 1; i += groupLength) {
String hex = input.substring(i, i + groupLength);
sb.append((char) Integer.parseInt(hex, 16));
}
return sb.toString();
}
public static void main(String[] args) throws Exception {
String decryptSignature = decrypt("c37551bb77f741d0bcdc16497b4f97b1", "123456781234567812345678", "12345678");
System.out.println(decryptSignature);
}
There are a few things you should check. You might find Encryption using AES-128 in Java to be of some assistance. There could be issues with differences between how you are handling keys in the PHP and Java code. Calling getBytes() on a String in Java without an encoding is almost always a bad idea. Plus the padding used could be a problem. From what I've seen PHP pads with null characters by default, which does not correspond to NoPadding in Java. Finally, the hexToString method should return a byte[] instead of a String. Add the result of calling Integer.parseInt(hex, 16) into an array:
byte[] results = new byte[input.length() / groupLength];
...
//inside the loop
results[i / groupLength] = (byte) Integer.parseInt(hex, 16);
...
return results;
How can I convert so called "php unicode"(link to php unicode) to normal character via Java? Example \xEF\xBC\xA1 -> A. Are there any embedded methods in jdk or should I use regex for this conversion?
You first need to get the bytes out of the string into a byte-array without changing them and then decode the byte-array as a UTF-8 string.
The simplest way to get the string into a byte array is to encode it using ISO-8859-1 which map every character with a unicode value less than 256 to a byte with the same value (or the equivalent negative)
String phpUnicode = "\u00EF\u00BC\u00A1"
byte[] bytes = phpUnicode.getBytes("ISO-8859-1"); // maps to bytes with the same ordinal value
String javaString = new String(bytes, "UTF-8");
System.out.println(javaString);
Edit
The above converts the UTF-8 to the Unicode character. If you then want to convert it to a reasonable ASCII equivalent, there's no standard way of doing that: but see this question
Edit
I assumed that you had a string containing characters that had the same ordinal value as the UTF-8 sequence but you indicate that your string literally contains the escape sequence, as in:
String phpUnicode = "\\xEF\\xBC\\xA1";
The JDK doesn't have any built-in methods to convert Strings like this so you'll need to use your own regex. Since we ultimately want to convert a utf-8 byte-sequence into a String, we need to set up a byte-array, using maybe:
Pattern oneChar = Pattern.compile("\\\\x([0-9A-F]{2})|(.)", Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
Matcher matcher = oneChar.matcher(phpUnicode);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
while (matcher.find()) {
int ch;
if (matcher.group(1) == null) {
ch = matcher.group(2).charAt(0);
}
else {
ch = Integer.parseInt(matcher.group(1), 16);
}
bytes.write((int) ch);
}
String javaString = new String(bytes.toByteArray(), "UTF-8");
System.out.println(javaString);
This will generate a UTF-8 stream by converting \xAB sequences. This UTF-8 stream is then converted to a Java string. It's important to note that any character that is not part of an escape sequence will be converted to a byte equivalent to to the low-order 8 bites of the unicode character. This works fine for ascii but can cause transcoding problems for non-ascii characters.
#McDowell:
The sequence:
String phpUnicode = "\u00EF\u00BC\u00A1"
byte[] bytes = phpUnicode.getBytes("ISO-8859-1");
creates a byte array containing as many bytes as the original string has characters and for each character with a unicode value below 256, the same numeric value is stored in the byte-array.
The character FULLWIDTH LATIN CAPITAL LETTER A (U+FF41) is not present in the original String so the fact that it is not in ISO-8859-1 is irrelevant.
I know that transcoding bugs can occur when you convert characters to bytes that's why I said that ISO-8859-1 would only "map every character with a unicode value less than 256 to a byte with the same value"
The character in question is U+FF21 (FULLWIDTH LATIN CAPITAL LETTER A). The PHP form (\xEF\xBC\xA1) is a UTF-8 encoded octet sequence.
In order to decode this sequence to a Java String (which is always UTF-16), you would use the following code:
// \xEF\xBC\xA1
byte[] utf8 = { (byte) 0xEF, (byte) 0xBC, (byte) 0xA1 };
String utf16 = new String(utf8, Charset.forName("UTF-8"));
// print the char as hex
for(char ch : utf16.toCharArray()) {
System.out.format("%02x%n", (int) ch);
}
If you want to decode the data from a string literal you could use code of this form:
public static void main(String[] args) {
String utf16 = transformString("This is \\xEF\\xBC\\xA1 string");
for (char ch : utf16.toCharArray()) {
System.out.format("%s %02x%n", ch, (int) ch);
}
}
private static final Pattern SEQ
= Pattern.compile("(\\\\x\\p{Alnum}\\p{Alnum})+");
private static String transformString(String encoded) {
StringBuilder decoded = new StringBuilder();
Matcher matcher = SEQ.matcher(encoded);
int last = 0;
while (matcher.find()) {
decoded.append(encoded.substring(last, matcher.start()));
byte[] utf8 = toByteArray(encoded.substring(matcher.start(), matcher.end()));
decoded.append(new String(utf8, Charset.forName("UTF-8")));
last = matcher.end();
}
return decoded.append(encoded.substring(last, encoded.length())).toString();
}
private static byte[] toByteArray(String hexSequence) {
byte[] utf8 = new byte[hexSequence.length() / 4];
for (int i = 0; i < utf8.length; i++) {
int offset = i * 4;
String hex = hexSequence.substring(offset + 2, offset + 4);
utf8[i] = (byte) Integer.parseInt(hex, 16);
}
return utf8;
}