Develop Reference

How to convert a string UTF-8 to ANSI in java?

I have a string in UTF-8 format. I want to convert it to clean ANSI format. How to do that?

You could use a java function like this one here to convert from UTF-8 to ISO_8859_1 (which seems to be a subset of ANSI):
private static String convertFromUtf8ToIso(String s1) {
if(s1 == null) {
return null;
}
String s = new String(s1.getBytes(StandardCharsets.UTF_8));
byte[] b = s.getBytes(StandardCharsets.ISO_8859_1);
return new String(b, StandardCharsets.ISO_8859_1);
}
Here is a simple test:
String s1 = "your utf8 stringáçﬠ";
String res = convertFromUtf8ToIso(s1);
System.out.println(res);
This prints out:
your utf8 stringáç?
The ﬠ character gets lost because it cannot be represented with ISO_8859_1 (it has 3 bytes when encoded in UTF-8). ISO_8859_1 can represent á and ç.

You can do something like this:
new String("your utf8 string".getBytes(Charset.forName("utf-8")));
in this format 4 bytes of UTF8 converts to 8 bytes of ANSI

Converting UTF-8 to ANSI is not possible generally, because ANSI only has 128 characters (7 bits) and UTF-8 has up to 4 bytes. That's like converting long to int, you lose information in most cases.

byte[] to string and String to byte[] keeping information unchanged

I have the byte array of a string.
How to convert byte[] to String and String to byte[]
After compressing I get the compressed value in form of byte array.
byte[] value at System.out is
h?ö\
É·ë’YO¸p˜à¼Œ[䔽"$žQºÍCïηfØzöÛBi¤!<
How can I convert this byte array value to String keeping its value unchanged and then getting it back to byte array for decompressing?
tried buitin functions but they change the whole compressed byte array

try new String(byteArray)
and string.getBytes()

Use ISO-8859-1 when converting bytes to String and back
String s = new String(bytes, "ISO-8859-1");
bytes = s.getBytes("ISO-8859-1");

You need to specify the char set for your byte[] array while decoding from raw into string.
The String type contain such constructor String(bytes, Charset charset)

byte[] b=new byte[10];
b[0]=100;
b[1]=101;
b[2]=102;
b[5]=0;
b[6]=0;
String st=new String(b);
System.out.println(st);//def
b=st.getBytes();
for(int i=0;i<b.length;i++){
System.out.println(b[i]);
}

byte[] to String {100,25,28,-122,-26,94,-3,-26}

How can I convert this byte[] to String :
byte[] mytest = new byte[] {100,25,28,-122,-26,94,-3,-26};
i get this : "d��^�" when I use :
new String( mytest , "UTF-8" )
Here is code java for creation of key :
m_key = new javax.crypto.spec.SecretKeySpec(new byte[] {100,25,28,-122,-26,94,-3,-26}, "DES");
Thanks.

In order to decode the byte array into something like ASCII, you need to know its original encoding. Otherwise you would need to treat it as binary.
Note: Base64 is intended for transferring binary data across networks.
I would suggest Base64 encoding your byte array. Then in your PHP code decoding the Base64 string back into a UTF-8 string.
In Java, here's how to Base64 encode your byte array and then decode it back to UTF-8:
import org.apache.commons.codec.binary.Base64;
public class MyTest {
public static void main(String[] args) throws Exception {
byte[] byteArray = new byte[] {100,25,28,-122,-26,94,-3,-26};
System.out.println("To UTF-8 string: " + new String(byteArray, "UTF-8"));
byte[] base64 = Base64.encodeBase64(byteArray);
System.out.println("To Base64 string: " + new String(base64, "UTF-8"));
byte[] decoded = Base64.decodeBase64(base64);
System.out.println("Back to UTF-8 string: " + new String(decoded, "UTF-8"));
/* the decoded byte array is the same as the original byte array */
for (int i = 0; i < decoded.length; i++) {
assert byteArray[i] == decoded[i];
}
}
}
The output from the above code is:
To UTF-8 string: d��^�
To Base64 string: ZBkchuZe/eY=
Back to UTF-8 string: d��^�
So if you wanted to use the same binary data in your PHP code, cut and paste the Base64 string into your PHP code and decode it back to UTF-8. Something like this:
<?php
$str = 'ZBkchuZe/eY=';
$key = base64_decode($str);
echo $key;
?>
I don't code in PHP, but you should be able to decode Base64 using this method:
http://php.net/manual/en/function.base64-decode.php
The above code should echo back the original binary data as UTF-8 (albeit with funny characters). The point is that the funny-looking string in the $key variable is representing the same binary data you had in the Java byte array:
d��^�
You should be able to pass the $key variable into your PHP encryption method.

with the way you are doing it makes no sense imo. you are creating a new string with the byte[] as an argument. i dont think that function is suppose to parse. so what you end up with is a lot of junk. but a little bit of googling got me this: http://www.mkyong.com/java/how-do-convert-byte-array-to-string-in-java/

Would m_key.getEncoded() give you the desired result.
Javadocs - SecretKeySpec
If not, you have to identify the Key provider that was used for the encoding (which resulted in the byte array that you have now) and decode.

byte to string and vice versa

I need to convert a salt value randomly generated and store it in the database. To store it in the database I converted it to a string. Then for retrieving the original value, I convert it back to byte. But both value are not matching. I have tried "UTF-8","UTF-16", BASE64Encoder.
SecureRandom ranGen = new SecureRandom();
byte[] aesKey = new byte[16]; // 16 bytes = 128 bits
ranGen.nextBytes(aesKey);
System.out.println(aesKey);
String a=new String(aesKey,"UTF-16");
byte[] b=new byte[16];
b=a.getBytes("UTF-16");
System.out.println(b);
Outputs for the above code(Executed it 2 times):
[B#11563ff
[B#1581593
and
[B#170888e
[B#11563ff

You really ought to use Base64 for converting binary data to Strings. There are lots of free implementations available, for example the one found in Apache Commons Codec.
Also, it's really easy to use, for example:
For encoding:
import org.apache.commons.codec.binary.Base64;
...
byte[] abValue = {...}; // Your data to encode
Base64 base64 = new Base64();
String strEncodedData = base64.encodeToString(abValue).trim();
For decoding:
import org.apache.commons.codec.binary.Base64;
...
String strEncodedData = "..."; // Your previously encoded data
Base64 base64 = new Base64();
byte[] abValue = base64.decode(strValue);

As your code is written above, printing aesKey and then b, what you are actually printing is the output of the toString method for an array object, which is just the default Object toString method. So I don't see how you can expect them to be the same.
If you really want to check they are the same you should compare them byte by byte.
In terms of your actual question regarding storing a byte[] as a String in the DB, your best bet is to Base64 encode it. I would suggest using the Apache Commons Codec library for this. See the user guide.
EDIT:
Using the BASE64Encode and BASE64Decoder you have referred to, the code would be like this:
SecureRandom ranGen = new SecureRandom();
byte[] aesKey = new byte[16]; // 16 bytes = 128 bits
ranGen.nextBytes(aesKey);
String a = new BASE64Encoder().encode(aesKey);
System.out.println(a);
byte[] b = new BASE64Decoder().decodeBuffer(a);
System.out.println(new BASE64Encoder().encode(b));
for (int i = 0; i < aesKey.length; i++) {
System.out.println(aesKey[i] + " " + b[i]);
}
Here, I have also looped through the bytes individually, to show that they are indeed equal.

Convert "php unicode" to character

How can I convert so called "php unicode"(link to php unicode) to normal character via Java? Example \xEF\xBC\xA1 -> A. Are there any embedded methods in jdk or should I use regex for this conversion?

You first need to get the bytes out of the string into a byte-array without changing them and then decode the byte-array as a UTF-8 string.
The simplest way to get the string into a byte array is to encode it using ISO-8859-1 which map every character with a unicode value less than 256 to a byte with the same value (or the equivalent negative)
String phpUnicode = "\u00EF\u00BC\u00A1"
byte[] bytes = phpUnicode.getBytes("ISO-8859-1"); // maps to bytes with the same ordinal value
String javaString = new String(bytes, "UTF-8");
System.out.println(javaString);
Edit
The above converts the UTF-8 to the Unicode character. If you then want to convert it to a reasonable ASCII equivalent, there's no standard way of doing that: but see this question
Edit
I assumed that you had a string containing characters that had the same ordinal value as the UTF-8 sequence but you indicate that your string literally contains the escape sequence, as in:
String phpUnicode = "\\xEF\\xBC\\xA1";
The JDK doesn't have any built-in methods to convert Strings like this so you'll need to use your own regex. Since we ultimately want to convert a utf-8 byte-sequence into a String, we need to set up a byte-array, using maybe:
Pattern oneChar = Pattern.compile("\\\\x([0-9A-F]{2})|(.)", Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
Matcher matcher = oneChar.matcher(phpUnicode);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
while (matcher.find()) {
int ch;
if (matcher.group(1) == null) {
ch = matcher.group(2).charAt(0);
}
else {
ch = Integer.parseInt(matcher.group(1), 16);
}
bytes.write((int) ch);
}
String javaString = new String(bytes.toByteArray(), "UTF-8");
System.out.println(javaString);
This will generate a UTF-8 stream by converting \xAB sequences. This UTF-8 stream is then converted to a Java string. It's important to note that any character that is not part of an escape sequence will be converted to a byte equivalent to to the low-order 8 bites of the unicode character. This works fine for ascii but can cause transcoding problems for non-ascii characters.
#McDowell:
The sequence:
String phpUnicode = "\u00EF\u00BC\u00A1"
byte[] bytes = phpUnicode.getBytes("ISO-8859-1");
creates a byte array containing as many bytes as the original string has characters and for each character with a unicode value below 256, the same numeric value is stored in the byte-array.
The character FULLWIDTH LATIN CAPITAL LETTER A (U+FF41) is not present in the original String so the fact that it is not in ISO-8859-1 is irrelevant.
I know that transcoding bugs can occur when you convert characters to bytes that's why I said that ISO-8859-1 would only "map every character with a unicode value less than 256 to a byte with the same value"

The character in question is U+FF21 (FULLWIDTH LATIN CAPITAL LETTER A). The PHP form (\xEF\xBC\xA1) is a UTF-8 encoded octet sequence.
In order to decode this sequence to a Java String (which is always UTF-16), you would use the following code:
// \xEF\xBC\xA1
byte[] utf8 = { (byte) 0xEF, (byte) 0xBC, (byte) 0xA1 };
String utf16 = new String(utf8, Charset.forName("UTF-8"));
// print the char as hex
for(char ch : utf16.toCharArray()) {
System.out.format("%02x%n", (int) ch);
}
If you want to decode the data from a string literal you could use code of this form:
public static void main(String[] args) {
String utf16 = transformString("This is \\xEF\\xBC\\xA1 string");
for (char ch : utf16.toCharArray()) {
System.out.format("%s %02x%n", ch, (int) ch);
}
}
private static final Pattern SEQ
= Pattern.compile("(\\\\x\\p{Alnum}\\p{Alnum})+");
private static String transformString(String encoded) {
StringBuilder decoded = new StringBuilder();
Matcher matcher = SEQ.matcher(encoded);
int last = 0;
while (matcher.find()) {
decoded.append(encoded.substring(last, matcher.start()));
byte[] utf8 = toByteArray(encoded.substring(matcher.start(), matcher.end()));
decoded.append(new String(utf8, Charset.forName("UTF-8")));
last = matcher.end();
}
return decoded.append(encoded.substring(last, encoded.length())).toString();
}
private static byte[] toByteArray(String hexSequence) {
byte[] utf8 = new byte[hexSequence.length() / 4];
for (int i = 0; i < utf8.length; i++) {
int offset = i * 4;
String hex = hexSequence.substring(offset + 2, offset + 4);
utf8[i] = (byte) Integer.parseInt(hex, 16);
}
return utf8;
}