Develop Reference

How to Convert UTF-16 Surrogate Decimal to UNICODE in Java

I have some string data like
&#55357 ;&#56842 ;
These are surrogate pairs in UTF 16 in decimal format.
How can I convert them to Unicode Code Points in Java, so that my client can understand the Unicode decimal html entity without the surrogate pair?
Example: &#128522 ; - Get this response for the above string

Assuming you already parsed the string to get the 2 numbers, just create a String from those two char values:
String s = new String(new char[] { 55357, 56842 });
System.out.println(s);
Output
😊
To get the code point of that:
s.codePointAt(0) // returns 128522
You don't have to create a string though:
Character.toCodePoint((char) 55357, (char) 56842) // returns 128522

Convert a string of bits to unicode character in java

I'm trying to convert a string of bits into Unicode characters in java. Problem is that I only get chines signs etc.
String bits = "01010011011011100110000101110010"
Anyone know how to do this?

Values <= 32bits
Use Integer.parseInt to parse the binary string, then convert it to byte array (using ByteBuffer) and finally convert byte array to String:
String bits = "01010011011011100110000101110010"
new String(
ByteBuffer.allocate(4).putInt(
Integer.parseInt(bits, 2)
).array(),
StandardCharsets.UTF_8
);
Values > 32bits
For arbitrary large bits String you can use also BigInteger:
new String(
new BigInteger(bits, 2).toByteArray(),
StandardCharsets.UTF_8
);
Result
Snar

How to Convert ASCII to Hex

How can I convert ASCII values to hexadecimal and binary values (not their string representation in ASCII)? For example, how can I convert the decimal value 26 to 0x1A?
So far, I've tried converting using the following steps (see below for actual code):
Converting value to bytes
Converting each byte to int
Converting each int to hex, via String.toString(intValue, radix)
Note: I did ask a related question about writing hex values to a file.
Clojure code:
(apply str
(for [byte (.getBytes value)]
(.replace (format "%2s" (Integer/toString (.intValue byte) 16)) " " "0")))))
Java code:
Byte[] bytes = "26".getBytes();
for (Byte data : bytes) {
System.out.print(String.format("%2s", Integer.toString(data.intValue(), 16)).replace(" ", "0"));
}
System.out.print("\n");

Hexadecimal, decimal, and binary integers are not different things -- there's only one underlying representation of an integer. The one thing you said you're trying to avoid -- "the ASCII string representation" -- is the only thing that's different. The variable is always the same, it's just how you print it that's different.
Now, it's not 100% clear to me what you're trying to do. But given the above, the path is clear: if you've got a String, convert it to an int by parsing (i.e., using Integer.parseInt()). Then if you want it printed in some format, it's easy to print that int as whatever you want using, for example, printf format specifiers.

If you actually want hexadecimal strings, then (format "%02X" n) is much simpler than the hoops you jump through in your first try. If you don't, then just write the integer values to a file directly without trying to convert them to a string.
Something like (.write out (read-string string-representing-a-number)) should be sufficient.

Here are your three steps rolled up into one line of clojure:
(apply str (map #(format "0x%02X " %) (.getBytes (str 42))))
convert to bytes (.getBytes (str 42))
no actual need for step 2
convert each byte to a string of characters representing it in hex
or you can make it look more like your steps with the "thread last" macro
(->> (str 42) ; start with your value
(.getBytes) ; put it in an array of bytes
(map #(format "0x%02X " %)) ; make hex string representation
(apply str)) ; optionally wrap it up in a string

static String decimalToHex(String decimal, int minLength) {
Long n = Long.parseLong(decimal, 10);
// Long.toHexString formats assuming n is unsigned.
String hex = Long.toHexString(Math.abs(n), 16);
StringBuilder sb = new StringBuilder(minLength);
if (n < 0) { sb.append('-'); }
int padding = minLength - hex.length - sb.length();
while (--padding >= 0) { sb.append('0'); }
return sb.append(hex).toString();
}

//get Unicode for char
char theChar = 'a';
//use this to go from i`enter code here`nt to Unicode or HEX or ASCII
int theValue = 26;
String hex = Integer.toHexString(theValue);
while (hex.length() < 4) {
hex = "0" + hex;
}
String unicode = "\\u" + (hex);
System.out.println(hex);

Java String.codePointAt returns unexpected value

If I use any ASCII characters from 33 to 127, the codePointAt method gives the correct decimal value, for example:
String s1 = new String("#");
int val = s1.codePointAt(0);
This returns 35 which is the correct value.
But if I try use ASCII characters from 128 to 255 (extended ASCII/ISO-8859-1), this method gives wrong value, for example:
String s1 = new String("ƒ") // Latin small letter f with hook
int val = s1.codePointAt(0);
This should return 159 as per this reference table, but instead returns 409, why is this?

But if I try use ASCII characters from 128 to 255
ASCII doesn't have values in this range. It only uses 7 bits.
Java chars are UTF-16 (and nothing else!). If you want to represent ASCII using Java, you need to use a byte array.
The codePointAt method returns the 32-bit codepoint. 16-bit chars can't contain the entire Unicode range, so some code points must be split across two chars (as per the encoding scheme for UTF-16). The codePointAt method helps resolve to chars code points.
I wrote a rough guide to encoding in Java here.

Java chars are not encoded in ISO-8859-1. They use UTF-16 which has the same values for 7bit ASCII characters (only values from 0-127).
To get the correct value for ISO-8859-1 you have to convert your string into a byte[] with String.getBytes("ISO-8859-1"); and look in the byte array.
Update
ISO-8859-1 is not the extended ASCII encoding, use String.getBytes("Cp437"); to get the correct values.

in Unicode
ƒ 0x0192 LATIN SMALL LETTER F WITH HOOK

String.codePointAt returns the Unicode-Codepoint at this specified index.
The Unicode-Codepoint of ƒ is 402, see
http://www.decodeunicode.org/de/u+0192/properties
So
System.out.println("ƒ".codePointAt(0));
printing 402 is correct.
If you are interested in the representation in other charsets, you can printout the bytes representaion of the character in other charsets via getBytes(String charsetName):
final String s = "ƒ";
for (final String csName : Charset.availableCharsets().keySet()) {
try {
final Charset cs = Charset.forName(csName);
final CharsetEncoder encode = cs.newEncoder();
if (encode.canEncode(s))
{
System.out.println(csName + ": " + Arrays.toString(s.getBytes(csName)));
}
} catch (final UnsupportedOperationException uoe) {
} catch (final UnsupportedEncodingException e) {
}
}

Java Encode SHA-1 Byte Array

I have an SHA-1 byte array that I would like to use in a GET request. I need to encode this. URLEncoder expects a string, and if I create a string of it and then encode it, it gets corrupt?
To clarify, this is kinda a follow up to another question of mine.
(Bitorrent Tracker Request) I can get the value as a hex string, but that is not recognized by the tracker. On the other hand, encoded answer mark provided return 200 OK.
So I need to convert the hex representation that I got:
9a81333c1b16e4a83c10f3052c1590aadf5e2e20
into encoded form
%9A%813%3C%1B%16%E4%A8%3C%10%F3%05%2C%15%90%AA%DF%5E.%20

Question was edited while I was responding, following is ADDITIONAL code and should work (with my hex conversion code):
//Inefficient, but functional, does not test if input is in hex charset, so somewhat unsafe
//NOT tested, but should be functional
public static String encodeURL(String hexString) throws Exception {
if(hexString==null || hexString.isEmpty()){
return "";
}
if(hexString.length()%2 != 0){
throw new Exception("String is not hex, length NOT divisible by 2: "+hexString);
}
int len = hexString.length();
char[] output = new char[len+len/2];
int i=0;
int j=0;
while(i<len){
output[j++]='%';
output[j++]=hexString.charAt(i++);
output[j++]=hexString.charAt(i++);
}
return new String(output);
}
You'll need to convert the raw bytes to hexadecimal characters or whatever URL-friendly encoding they are using. Base32 or Base64 encodings are possible, but straight hexadecimal characters is the most common. URLEncoder is not needed for this string, because it shouldn't contain any characters that would require URL Encoding to %NN format.
The below will convert bytes for a hash (SHA-1, MD5SUM, etc) to a hexadecimal string:
/** Lookup table: character for a half-byte */
static final char[] CHAR_FOR_BYTE = {'0','1','2','3','4','5','6','7','8','9','A','B','C','D','E','F'};
/** Encode byte data as a hex string... hex chars are UPPERCASE*/
public static String encode(byte[] data){
if(data == null || data.length==0){
return "";
}
char[] store = new char[data.length*2];
for(int i=0; i<data.length; i++){
final int val = (data[i]&0xFF);
final int charLoc=i<<1;
store[charLoc]=CHAR_FOR_BYTE[val>>>4];
store[charLoc+1]=CHAR_FOR_BYTE[val&0x0F];
}
return new String(store);
}
This code is fairly optimized and fast, and I am using it for my own SHA-1 byte encoding. Note that you may need to convert uppercase to lowercase with the String.toLowerCase() method, depending on which form the server accepts.

This depends on what the recipient of your request expects.
I would imagine it could be a hexadecimal representation of the bytes in your hash. A string would probably not be the best idea, because the hash array will most likely contain non-printable character values.
I'd iterate over the array and use Integer.toHexValue() to convert the bytes to hex.

SHA1 is in hex format [0-9a-f], there should be no need to URLEncode it.

Use Apache Commons-Codec for all your encoding/decoding needs (except ASN.1, which is a pain in the ass)