Develop Reference

Convert UCS-2 HEX в UTF16 BE java

I have a string in which the SMS text is encrypted in USC2 format received from a GSM modem
I'm trying to convert it to UTF16 but it doesn't work. Please tell me what am I doing wrong
public class USC {
public static void main(String[] args) {
String hex = "0412044B0020043F043E043B044C043704430435044204350441044C002004420430044004380444043D044B043C0020043F043B0430043D043E043C0020002204110438002B002200200441002000300033002E00310032002E0032003000320031002E002004230442043E0447043D04380442044C002004430441043B043E04320438044F";
byte[] v = hex.getBytes(StandardCharsets.UTF_16BE);
String str = new String(v);
System.out.println(str);
}
}
On the online decoder through the service https://dencode.com/ works fine

Try the following:
BigInteger bi = new BigInteger(hex, 16);
byte[] a = bi.toByteArray();
System.out.println(new String(a, Charset.forName("UTF-16")));

String hex = "0412044B0020043F043E043B044C043704430435044204350441044C002004420430044004380444043D044B043C0020043F043B0430043D043E043C0020002204110438002B002200200441002000300033002E00310032002E0032003000320031002E002004230442043E0447043D04380442044C002004430441043B043E04320438044F";
int n = hex.length/4;
char[] chars = new char[n];
for (int i = 0; i < n; ++i) {
chars[i] = Integer.parseInt(hex.substring(4*i, 4*i+4), 15) & 0xFFFF);
}
String str = new String(chars);
System.out.println(str);
4 hex chars form one UCS-2 big endian char. Same size as java char (2 bytes).
UTF-16 is superior to UCS-2 which forms a fixed-size subset. So from UCS-2 to UTF-16 needs no special treatment, only whether the 2 bytes of a char are big endian or little endian.

With JDK17 or above you could also make use of HexFormat class:
String str = new String(HexFormat.of().parseHex(hex), StandardCharsets.UTF_16BE);

SHA512 in C # and Java are different

There is such code on C # and java, sha512 in them differs, whether it is possible to make somehow that the result sha512 was identical? I understand the problem in BaseConverter, analog Base64 in Java? Tried
Base64.getEncoder().encodeToString(str);
But I get an error because of getEncoder(). Do I need a library for this?
Code in C#:
public string Hash(string str)
{
string resultStr = String.Empty;
byte[] data = new UTF8Encoding().GetBytes(str);
byte[] result;
SHA512 shaM = new SHA512Managed();
result = shaM.ComputeHash(data);
resultStr = ReverseString(BitConverter.ToString(result).ToLower().Replace("-", String.Empty));
return resultStr.Substring(5, 25);
}
public static string ReverseString(string s)
{
char[] charArray = s.ToCharArray();
Array.Reverse(charArray);
return new string(charArray);
}
Code in Java:
public String Hash(String str) {
try {
MessageDigest digest = MessageDigest.getInstance("SHA-512");
digest.update(str.getBytes("UTF-16LE"));
byte messageDigest[] = digest.digest();
StringBuffer hexString = new StringBuffer();
for (int i = 0; i < messageDigest.length; i++) {
String h = Integer.toHexString(0xFF & messageDigest[i]);
while (h.length() < 2)
h = "0" + h;
hexString.append(h);
}
result = hexString.toString().toLowerCase();
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
return ReverseString(result).substring(5, 25);
}
public static String ReverseString(String s)
{
return new StringBuilder(s).reverse().toString();
}

You're hashing different data - in Java you're converting the string to UTF-16:
digest.update(str.getBytes("UTF-16LE"));
In C# you're using UTF-8:
byte[] data = new UTF8Encoding().GetBytes(str);
(I'm not sure why you're creating a new UTF8Encoding rather than using Encoding.UTF8, admittedly.)
With different input, you will get different hashes.
In general, the way to diagnose problems like this is to compare the data at every step of the transformation, whether that's through logging or debugging. In this case you have four transformations:
Message string to message bytes
Message bytes to hash bytes
Hash bytes to hash string (hex)
Reversed hash string (hex)
Next time, check the output of each step, and you'll work out where the problem is.
(It's not obvious why you'd want to reverse the hex output anyway, but that's a different matter.)

The problem was in the input line for hashing (the string was the salt without the rest of the data, the rest of the data was empty because there was an error in the definition of EditText (EditText returned an empty string) and also fixed the encoding in Java for UTF-8.

SHA-1 hashing on Java and C#

I'm trying to validate the content of an XML node with SHA-1 , basically, we generate an SHA-1 hash with the content of that node and both sides (client C# and server Java) should have exactly the same hash.
The problem is , I have checked with a diff tool the content of both texts and there is not any difference. But I'm getting a different hash than the client.
C# hash : 60-53-58-69-29-EB-53-BD-85-31-79-28-A0-F9-42-B6-DE-1B-A6-0A
Java hash: E79D7E6F2A6F5D776447714D896D4C3A0CBC793
The way the client (C#) is generating the hash is this:
try
{
Byte[] stream = null;
using (System.Security.Cryptography.SHA1CryptoServiceProvider shaProvider = new System.Security.Cryptography.SHA1CryptoServiceProvider())
{
stream = shaProvider.ComputeHash(System.Text.Encoding.UTF8.GetBytes(text));
if (stream == null)
{
hash = "Error";
}
else
{
hash = System.BitConverter.ToString(stream);
}
}
}
catch (Exception error)
{
hash = string.Format("Error SHA-1: {0}", error);
}
return hash;
and this is how the server (Java) is generating the hash:
byte[] key = content.getBytes();
MessageDigest md = MessageDigest.getInstance("SHA1");
byte[] hash = md.digest(key);
String result = "";
for (byte b : hash) {
result += Integer.toHexString(b & 255);
}
return result.toUpperCase();
can someone help me ? .. thanks :)
UPDATE:
In order to check what's going on I have checked other ways to get a SHA1 hash in C# and I found this:
/// <summary>
/// Compute hash for string encoded as UTF8
/// </summary>
/// <param name="s">String to be hashed</param>
/// <returns>40-character hex string</returns>
public static string SHA1HashStringForUTF8String(string s)
{
byte[] bytes = Encoding.UTF8.GetBytes(s);
using (var sha1 = SHA1.Create())
{
byte[] hashBytes = sha1.ComputeHash(bytes);
return System.BitConverter.ToString(hashBytes).Replace("-",string.Empty);
}
}
This code gives this output:
E79D07E6F2A6F5D776447714D896D4C3A0CBC793
AND !! I just noticed that Python is giving the same output (sorry, I should double checked this)
So this is the deal
Using this provider: System.Security.Cryptography.SHA1CryptoServiceProvider shaProvider = new System.Security.Cryptography.SHA1CryptoServiceProvider()
Is giving a completly different output on three different machines ..
Using the above method in C# gives the same result as python does, also, for some reason Java is giving a sightly different output:
E79D7E6F2A6F5D776447714D896D4C3A0CBC793
Ideas?, is java the problem? the byte to hex method on java is the problem? there is another alternative?

Try using this as your hashing in C#:
static string Hash(string input)
{
using (SHA1Managed sha1 = new SHA1Managed())
{
var hash = sha1.ComputeHash(Encoding.UTF8.GetBytes(input));
var sb = new StringBuilder(hash.Length * 2);
foreach (byte b in hash)
{
// can be "x2" if you want lowercase
sb.Append(b.ToString("x2"));
}
return sb.ToString();
}
}
Hash("test"); //a94a8fe5ccb19ba61c4c0873d391e987982fbbd3
And then use this as your Java hashing:
private static String convertToHex(byte[] data) {
StringBuilder buf = new StringBuilder();
for (byte b : data) {
int halfbyte = (b >>> 4) & 0x0F;
int two_halfs = 0;
do {
buf.append((0 <= halfbyte) && (halfbyte <= 9) ? (char) ('0' + halfbyte) : (char) ('a' + (halfbyte - 10)));
halfbyte = b & 0x0F;
} while (two_halfs++ < 1);
}
return buf.toString();
}
public static String SHA1(String text) throws NoSuchAlgorithmException, UnsupportedEncodingException {
MessageDigest md = MessageDigest.getInstance("SHA-1");
byte[] textBytes = text.getBytes("iso-8859-1");
md.update(textBytes, 0, textBytes.length);
byte[] sha1hash = md.digest();
return convertToHex(sha1hash);
}
SHA1("test"); //a94a8fe5ccb19ba61c4c0873d391e987982fbbd3
Note you need the following imports:
import java.io.UnsupportedEncodingException; import
java.security.MessageDigest; import
java.security.NoSuchAlgorithmException;
Throws declarations are option, adjust to best fit your code!

Your problem is that you're not hashing the same bytes in both API.
If you choose to modify java's version, it should look like this:
byte[] key = content.getBytes("UTF8");
[...]
If you choose to modify c#' version, it should look like this:
stream = shaProvider.ComputeHash(System.Text.Encoding.UTF16.GetBytes(text));
[...]
Either way, both api should get the key's bytes through the same encoding.

Need Java equvalent for 3DES decryption of PHP code

This is the PHP code I have.
function decrypt($s_input, $s_key, $s_iv) {
$s_decrypted = pack("H*" , $s_input); // Hex to binary
$s_decrypted = mcrypt_decrypt (MCRYPT_3DES, $s_key, $s_decrypted, MCRYPT_MODE_CBC, $s_iv); // 3des decryption
return $s_decrypted;
}
echo encrypt('c37551bb77f741d0bcdc16497b4f97b1','123456781234567812345678','12345678' );
what it basically does is to decrypt a 3des encrypted string (first it convert the hex string to binary using pack function and then does the actual decryption).
This perfectly works in PHP-4 and prints the "Hello World" message.
However, if I run the equivalent java code (jdk 1.6), it prints garbage output as - ¬ªmjV=7xl_ÓÄ^›*?.
Can someone help to troubleshoot this? Why Java is not properly decrypting the hex string.
private static String decrypt(String inputStr, String keyStr, String ivStr) throws Exception {
IvParameterSpec iv = new IvParameterSpec(ivStr.getBytes());
SecretKeySpec key = new SecretKeySpec(keyStr.getBytes(), "DESede");
inputStr = hexToString(inputStr, 2);
Cipher cipher = Cipher.getInstance("DESede/CBC/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, key, iv);
byte[] decrypted = cipher.doFinal(inputStr.getBytes());
return new String(decrypted);
}
private static String hexToString(String input, int groupLength) {
StringBuilder sb = new StringBuilder(input.length() / groupLength);
for (int i = 0; i < input.length() - groupLength + 1; i += groupLength) {
String hex = input.substring(i, i + groupLength);
sb.append((char) Integer.parseInt(hex, 16));
}
return sb.toString();
}
public static void main(String[] args) throws Exception {
String decryptSignature = decrypt("c37551bb77f741d0bcdc16497b4f97b1", "123456781234567812345678", "12345678");
System.out.println(decryptSignature);
}

There are a few things you should check. You might find Encryption using AES-128 in Java to be of some assistance. There could be issues with differences between how you are handling keys in the PHP and Java code. Calling getBytes() on a String in Java without an encoding is almost always a bad idea. Plus the padding used could be a problem. From what I've seen PHP pads with null characters by default, which does not correspond to NoPadding in Java. Finally, the hexToString method should return a byte[] instead of a String. Add the result of calling Integer.parseInt(hex, 16) into an array:
byte[] results = new byte[input.length() / groupLength];
...
//inside the loop
results[i / groupLength] = (byte) Integer.parseInt(hex, 16);
...
return results;

Convert "php unicode" to character

How can I convert so called "php unicode"(link to php unicode) to normal character via Java? Example \xEF\xBC\xA1 -> A. Are there any embedded methods in jdk or should I use regex for this conversion?

You first need to get the bytes out of the string into a byte-array without changing them and then decode the byte-array as a UTF-8 string.
The simplest way to get the string into a byte array is to encode it using ISO-8859-1 which map every character with a unicode value less than 256 to a byte with the same value (or the equivalent negative)
String phpUnicode = "\u00EF\u00BC\u00A1"
byte[] bytes = phpUnicode.getBytes("ISO-8859-1"); // maps to bytes with the same ordinal value
String javaString = new String(bytes, "UTF-8");
System.out.println(javaString);
Edit
The above converts the UTF-8 to the Unicode character. If you then want to convert it to a reasonable ASCII equivalent, there's no standard way of doing that: but see this question
Edit
I assumed that you had a string containing characters that had the same ordinal value as the UTF-8 sequence but you indicate that your string literally contains the escape sequence, as in:
String phpUnicode = "\\xEF\\xBC\\xA1";
The JDK doesn't have any built-in methods to convert Strings like this so you'll need to use your own regex. Since we ultimately want to convert a utf-8 byte-sequence into a String, we need to set up a byte-array, using maybe:
Pattern oneChar = Pattern.compile("\\\\x([0-9A-F]{2})|(.)", Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
Matcher matcher = oneChar.matcher(phpUnicode);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
while (matcher.find()) {
int ch;
if (matcher.group(1) == null) {
ch = matcher.group(2).charAt(0);
}
else {
ch = Integer.parseInt(matcher.group(1), 16);
}
bytes.write((int) ch);
}
String javaString = new String(bytes.toByteArray(), "UTF-8");
System.out.println(javaString);
This will generate a UTF-8 stream by converting \xAB sequences. This UTF-8 stream is then converted to a Java string. It's important to note that any character that is not part of an escape sequence will be converted to a byte equivalent to to the low-order 8 bites of the unicode character. This works fine for ascii but can cause transcoding problems for non-ascii characters.
#McDowell:
The sequence:
String phpUnicode = "\u00EF\u00BC\u00A1"
byte[] bytes = phpUnicode.getBytes("ISO-8859-1");
creates a byte array containing as many bytes as the original string has characters and for each character with a unicode value below 256, the same numeric value is stored in the byte-array.
The character FULLWIDTH LATIN CAPITAL LETTER A (U+FF41) is not present in the original String so the fact that it is not in ISO-8859-1 is irrelevant.
I know that transcoding bugs can occur when you convert characters to bytes that's why I said that ISO-8859-1 would only "map every character with a unicode value less than 256 to a byte with the same value"

The character in question is U+FF21 (FULLWIDTH LATIN CAPITAL LETTER A). The PHP form (\xEF\xBC\xA1) is a UTF-8 encoded octet sequence.
In order to decode this sequence to a Java String (which is always UTF-16), you would use the following code:
// \xEF\xBC\xA1
byte[] utf8 = { (byte) 0xEF, (byte) 0xBC, (byte) 0xA1 };
String utf16 = new String(utf8, Charset.forName("UTF-8"));
// print the char as hex
for(char ch : utf16.toCharArray()) {
System.out.format("%02x%n", (int) ch);
}
If you want to decode the data from a string literal you could use code of this form:
public static void main(String[] args) {
String utf16 = transformString("This is \\xEF\\xBC\\xA1 string");
for (char ch : utf16.toCharArray()) {
System.out.format("%s %02x%n", ch, (int) ch);
}
}
private static final Pattern SEQ
= Pattern.compile("(\\\\x\\p{Alnum}\\p{Alnum})+");
private static String transformString(String encoded) {
StringBuilder decoded = new StringBuilder();
Matcher matcher = SEQ.matcher(encoded);
int last = 0;
while (matcher.find()) {
decoded.append(encoded.substring(last, matcher.start()));
byte[] utf8 = toByteArray(encoded.substring(matcher.start(), matcher.end()));
decoded.append(new String(utf8, Charset.forName("UTF-8")));
last = matcher.end();
}
return decoded.append(encoded.substring(last, encoded.length())).toString();
}
private static byte[] toByteArray(String hexSequence) {
byte[] utf8 = new byte[hexSequence.length() / 4];
for (int i = 0; i < utf8.length; i++) {
int offset = i * 4;
String hex = hexSequence.substring(offset + 2, offset + 4);
utf8[i] = (byte) Integer.parseInt(hex, 16);
}
return utf8;
}