Related
When using RFC7748 test vectors for elliptic curve diffie hellman in java, I cannot get expected shared secret key. I am able to do so in other languages. I am using openjdk 11 with default Sun security provider. I found official tests which use these test vectors. But I cannot get expected result even if I copy-paste and run them. For instance, here is test that uses these same vectors which will fail if I copy-paste and run locally. It uses some utility functions which are from here, which I also copied. I know I must be doing something wrong but I cannot figure out what exactly. Here is my code:
public class main {
public static BigInteger hexStringToBigInteger(boolean clearHighBit, String str) {
BigInteger result = BigInteger.ZERO;
for (int i = 0; i < str.length() / 2; i++) {
int curVal = Character.digit(str.charAt(2 * i), 16);
curVal <<= 4;
curVal += Character.digit(str.charAt(2 * i + 1), 16);
if (clearHighBit && i == str.length() / 2 - 1) {
curVal &= 0x7F;
result = result.add(BigInteger.valueOf(curVal).shiftLeft(8 * i));
}
}
return result;
}
public static byte[] hexStringToByteArray(String str) {
byte[] result = new byte[str.length() / 2];
for (int i = 0; i < result.length; i++) {
result[i] = (byte) Character.digit(str.charAt(2 * i), 16);
result[i] <<= 4;
result[i] += Character.digit(str.charAt(2 * i + 1), 16);
}
return result;
}
public static String byteArrayToHexString(byte[] arr) {
StringBuilder result = new StringBuilder();
for (byte curVal : arr) {
result.append(Character.forDigit(curVal >> 4 & 0xF, 16));
result.append(Character.forDigit(curVal & 0xF, 16));
}
return result.toString();
}
private static void runDiffieHellmanTest(String curveName, String a_pri,
String b_pub, String result) throws Exception {
NamedParameterSpec paramSpec = new NamedParameterSpec(curveName);
KeyFactory kf = KeyFactory.getInstance("XDH");
KeySpec privateSpec = new XECPrivateKeySpec(paramSpec, hexStringToByteArray(a_pri));
PrivateKey privateKey = kf.generatePrivate(privateSpec);
boolean clearHighBit = curveName.equals("X25519");
KeySpec publicSpec = new XECPublicKeySpec(paramSpec, hexStringToBigInteger(clearHighBit, b_pub));
PublicKey publicKey = kf.generatePublic(publicSpec);
byte[] encodedPrivateKey = privateKey.getEncoded();
System.out.println("Encoded private: " + byteArrayToHexString(encodedPrivateKey));
byte[] encodedPublicKey = publicKey.getEncoded();
System.out.println("Encoded public: " + byteArrayToHexString(encodedPublicKey));
KeyAgreement ka = KeyAgreement.getInstance("XDH");
ka.init(privateKey);
ka.doPhase(publicKey, true);
byte[] sharedSecret = ka.generateSecret();
byte[] expectedResult = hexStringToByteArray(result);
if (!Arrays.equals(sharedSecret, expectedResult)) {
throw new RuntimeException("fail: expected=" + result + ", actual="
+ byteArrayToHexString(sharedSecret));
}
}
public static void main(String[] args) throws Exception {
runDiffieHellmanTest(
"X25519",
"77076d0a7318a57d3c16c17251b26645df4c2f87ebc0992ab177fba51db92c2a",
"de9edb7d7b7dc1b4d35b61c2ece435373f8343c85b78674dadfc7e146f882b4f",
"4a5d9d5ba4ce2de1728e3bf480350f25e07e21c947d19e3376f09b3c1e161742");
}
}
You have incorrectly swapped two lines in hexStringToBigInteger:
if (clearHighBit && i == str.length() / 2 - 1) {
curVal &= 0x7F;
result = result.add(BigInteger.valueOf(curVal).shiftLeft(8 * i));
}
should instead be:
if (clearHighBit && i == str.length() / 2 - 1) {
curVal &= 0x7F;
}
result = result.add(BigInteger.valueOf(curVal).shiftLeft(8 * i));
I could not establish shared secret between Go and Java applications, so I tried to debug what was the cause of that which led me into reading RFC7748 and digging through source code for X25519 key exchange in Java. So for people who want to perform X25519 key exchange between Java and some other non-Java application - here is main takeaway. Java already expects input public key to be BigInteger instead of byte array. Some other languages may return public key as a byte array in big-endian format. Due to RFC7748 specification, X coordinate of a point on elliptic curve (which is your public key byte array) must be in little-endian format. So you only have to reverse input public key byte array to make it little-endian, before feeding in to BigInteger.
I am looking for a way to convert a long string (from a dump), that represents hex values into a byte array.
I couldn't have phrased it better than the person that posted the same question here.
But to keep it original, I'll phrase it my own way: suppose I have a string "00A0BF" that I would like interpreted as the
byte[] {0x00,0xA0,0xBf}
what should I do?
I am a Java novice and ended up using BigInteger and watching out for leading hex zeros. But I think it is ugly and I am sure I am missing something simple.
Update (2021) - Java 17 now includes java.util.HexFormat (only took 25 years):
HexFormat.of().parseHex(s)
For older versions of Java:
Here's a solution that I think is better than any posted so far:
/* s must be an even-length string. */
public static byte[] hexStringToByteArray(String s) {
int len = s.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+ Character.digit(s.charAt(i+1), 16));
}
return data;
}
Reasons why it is an improvement:
Safe with leading zeros (unlike BigInteger) and with negative byte values (unlike Byte.parseByte)
Doesn't convert the String into a char[], or create StringBuilder and String objects for every single byte.
No library dependencies that may not be available
Feel free to add argument checking via assert or exceptions if the argument is not known to be safe.
One-liners:
import javax.xml.bind.DatatypeConverter;
public static String toHexString(byte[] array) {
return DatatypeConverter.printHexBinary(array);
}
public static byte[] toByteArray(String s) {
return DatatypeConverter.parseHexBinary(s);
}
Warnings:
in Java 9 Jigsaw this is no longer part of the (default) java.se root
set so it will result in a ClassNotFoundException unless you specify
--add-modules java.se.ee (thanks to #eckes)
Not available on Android (thanks to Fabian for noting that), but you can just take the source code if your system lacks javax.xml for some reason. Thanks to #Bert Regelink for extracting the source.
The Hex class in commons-codec should do that for you.
http://commons.apache.org/codec/
import org.apache.commons.codec.binary.Hex;
...
byte[] decoded = Hex.decodeHex("00A0BF");
// 0x00 0xA0 0xBF
You can now use BaseEncoding in guava to accomplish this.
BaseEncoding.base16().decode(string);
To reverse it use
BaseEncoding.base16().encode(bytes);
Actually, I think the BigInteger is solution is very nice:
new BigInteger("00A0BF", 16).toByteArray();
Edit: Not safe for leading zeros, as noted by the poster.
One-liners:
import javax.xml.bind.DatatypeConverter;
public static String toHexString(byte[] array) {
return DatatypeConverter.printHexBinary(array);
}
public static byte[] toByteArray(String s) {
return DatatypeConverter.parseHexBinary(s);
}
For those of you interested in the actual code behind the One-liners from FractalizeR (I needed that since javax.xml.bind is not available for Android (by default)), this comes from com.sun.xml.internal.bind.DatatypeConverterImpl.java :
public byte[] parseHexBinary(String s) {
final int len = s.length();
// "111" is not a valid hex encoding.
if( len%2 != 0 )
throw new IllegalArgumentException("hexBinary needs to be even-length: "+s);
byte[] out = new byte[len/2];
for( int i=0; i<len; i+=2 ) {
int h = hexToBin(s.charAt(i ));
int l = hexToBin(s.charAt(i+1));
if( h==-1 || l==-1 )
throw new IllegalArgumentException("contains illegal character for hexBinary: "+s);
out[i/2] = (byte)(h*16+l);
}
return out;
}
private static int hexToBin( char ch ) {
if( '0'<=ch && ch<='9' ) return ch-'0';
if( 'A'<=ch && ch<='F' ) return ch-'A'+10;
if( 'a'<=ch && ch<='f' ) return ch-'a'+10;
return -1;
}
private static final char[] hexCode = "0123456789ABCDEF".toCharArray();
public String printHexBinary(byte[] data) {
StringBuilder r = new StringBuilder(data.length*2);
for ( byte b : data) {
r.append(hexCode[(b >> 4) & 0xF]);
r.append(hexCode[(b & 0xF)]);
}
return r.toString();
}
The HexBinaryAdapter provides the ability to marshal and unmarshal between String and byte[].
import javax.xml.bind.annotation.adapters.HexBinaryAdapter;
public byte[] hexToBytes(String hexString) {
HexBinaryAdapter adapter = new HexBinaryAdapter();
byte[] bytes = adapter.unmarshal(hexString);
return bytes;
}
That's just an example I typed in...I actually just use it as is and don't need to make a separate method for using it.
Here is a method that actually works (based on several previous semi-correct answers):
private static byte[] fromHexString(final String encoded) {
if ((encoded.length() % 2) != 0)
throw new IllegalArgumentException("Input string must contain an even number of characters");
final byte result[] = new byte[encoded.length()/2];
final char enc[] = encoded.toCharArray();
for (int i = 0; i < enc.length; i += 2) {
StringBuilder curr = new StringBuilder(2);
curr.append(enc[i]).append(enc[i + 1]);
result[i/2] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}
The only possible issue that I can see is if the input string is extremely long; calling toCharArray() makes a copy of the string's internal array.
EDIT: Oh, and by the way, bytes are signed in Java, so your input string converts to [0, -96, -65] instead of [0, 160, 191]. But you probably knew that already.
In android ,if you are working with hex, you can try okio.
simple usage:
byte[] bytes = ByteString.decodeHex("c000060000").toByteArray();
and result will be
[-64, 0, 6, 0, 0]
The BigInteger() Method from java.math is very Slow and not recommandable.
Integer.parseInt(HEXString, 16)
can cause problems with some characters without
converting to Digit / Integer
a Well Working method:
Integer.decode("0xXX") .byteValue()
Function:
public static byte[] HexStringToByteArray(String s) {
byte data[] = new byte[s.length()/2];
for(int i=0;i < s.length();i+=2) {
data[i/2] = (Integer.decode("0x"+s.charAt(i)+s.charAt(i+1))).byteValue();
}
return data;
}
Have Fun, Good Luck
EDIT: as pointed out by #mmyers, this method doesn't work on input that contains substrings corresponding to bytes with the high bit set ("80" - "FF"). The explanation is at Bug ID: 6259307 Byte.parseByte not working as advertised in the SDK Documentation.
public static final byte[] fromHexString(final String s) {
byte[] arr = new byte[s.length()/2];
for ( int start = 0; start < s.length(); start += 2 )
{
String thisByte = s.substring(start, start+2);
arr[start/2] = Byte.parseByte(thisByte, 16);
}
return arr;
}
For what it's worth, here's another version which supports odd length strings, without resorting to string concatenation.
public static byte[] hexStringToByteArray(String input) {
int len = input.length();
if (len == 0) {
return new byte[] {};
}
byte[] data;
int startIdx;
if (len % 2 != 0) {
data = new byte[(len / 2) + 1];
data[0] = (byte) Character.digit(input.charAt(0), 16);
startIdx = 1;
} else {
data = new byte[len / 2];
startIdx = 0;
}
for (int i = startIdx; i < len; i += 2) {
data[(i + 1) / 2] = (byte) ((Character.digit(input.charAt(i), 16) << 4)
+ Character.digit(input.charAt(i+1), 16));
}
return data;
}
I like the Character.digit solution, but here is how I solved it
public byte[] hex2ByteArray( String hexString ) {
String hexVal = "0123456789ABCDEF";
byte[] out = new byte[hexString.length() / 2];
int n = hexString.length();
for( int i = 0; i < n; i += 2 ) {
//make a bit representation in an int of the hex value
int hn = hexVal.indexOf( hexString.charAt( i ) );
int ln = hexVal.indexOf( hexString.charAt( i + 1 ) );
//now just shift the high order nibble and add them together
out[i/2] = (byte)( ( hn << 4 ) | ln );
}
return out;
}
I've always used a method like
public static final byte[] fromHexString(final String s) {
String[] v = s.split(" ");
byte[] arr = new byte[v.length];
int i = 0;
for(String val: v) {
arr[i++] = Integer.decode("0x" + val).byteValue();
}
return arr;
}
this method splits on space delimited hex values but it wouldn't be hard to make it split the string on any other criteria such as into groupings of two characters.
The Code presented by Bert Regelink simply does not work.
Try the following:
import javax.xml.bind.DatatypeConverter;
import java.io.*;
public class Test
{
#Test
public void testObjectStreams( ) throws IOException, ClassNotFoundException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
String stringTest = "TEST";
oos.writeObject( stringTest );
oos.close();
baos.close();
byte[] bytes = baos.toByteArray();
String hexString = DatatypeConverter.printHexBinary( bytes);
byte[] reconvertedBytes = DatatypeConverter.parseHexBinary(hexString);
assertArrayEquals( bytes, reconvertedBytes );
ByteArrayInputStream bais = new ByteArrayInputStream(reconvertedBytes);
ObjectInputStream ois = new ObjectInputStream(bais);
String readString = (String) ois.readObject();
assertEquals( stringTest, readString);
}
}
I found Kernel Panic to have the solution most useful to me, but ran into problems if the hex string was an odd number. solved it this way:
boolean isOdd(int value)
{
return (value & 0x01) !=0;
}
private int hexToByte(byte[] out, int value)
{
String hexVal = "0123456789ABCDEF";
String hexValL = "0123456789abcdef";
String st = Integer.toHexString(value);
int len = st.length();
if (isOdd(len))
{
len+=1; // need length to be an even number.
st = ("0" + st); // make it an even number of chars
}
out[0]=(byte)(len/2);
for (int i =0;i<len;i+=2)
{
int hh = hexVal.indexOf(st.charAt(i));
if (hh == -1) hh = hexValL.indexOf(st.charAt(i));
int lh = hexVal.indexOf(st.charAt(i+1));
if (lh == -1) lh = hexValL.indexOf(st.charAt(i+1));
out[(i/2)+1] = (byte)((hh << 4)|lh);
}
return (len/2)+1;
}
I am adding a number of hex numbers to an array, so i pass the reference to the array I am using, and the int I need converted and returning the relative position of the next hex number. So the final byte array has [0] number of hex pairs, [1...] hex pairs, then the number of pairs...
Based on the op voted solution, the following should be a bit more efficient:
public static byte [] hexStringToByteArray (final String s) {
if (s == null || (s.length () % 2) == 1)
throw new IllegalArgumentException ();
final char [] chars = s.toCharArray ();
final int len = chars.length;
final byte [] data = new byte [len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit (chars[i], 16) << 4) + Character.digit (chars[i + 1], 16));
}
return data;
}
Because: the initial conversion to a char array spares the length checks in charAt
If you have a preference for Java 8 streams as your coding style then this can be achieved using just JDK primitives.
String hex = "0001027f80fdfeff";
byte[] converted = IntStream.range(0, hex.length() / 2)
.map(i -> Character.digit(hex.charAt(i * 2), 16) << 4 | Character.digit(hex.charAt((i * 2) + 1), 16))
.collect(ByteArrayOutputStream::new,
ByteArrayOutputStream::write,
(s1, s2) -> s1.write(s2.toByteArray(), 0, s2.size()))
.toByteArray();
The , 0, s2.size() parameters in the collector concatenate function can be omitted if you don't mind catching IOException.
If your needs are more than just the occasional conversion then you can use HexUtils.
Example:
byte[] byteArray = Hex.hexStrToBytes("00A0BF");
This is the most simple case. Your input may contain delimiters (think MAC addresses, certificate thumbprints, etc), your input may be streaming, etc. In such cases it gets easier to justify to pull in an external library like HexUtils, however small.
With JDK 17 the HexFormat class will fulfill most needs and the need for something like HexUtils is greatly diminished. However, HexUtils can still be used for things like converting very large amounts to/from hex (streaming) or pretty printing hex (think wire dumps) which the JDK HexFormat class cannot do.
(full disclosure: I'm the author of HexUtils)
public static byte[] hex2ba(String sHex) throws Hex2baException {
if (1==sHex.length()%2) {
throw(new Hex2baException("Hex string need even number of chars"));
}
byte[] ba = new byte[sHex.length()/2];
for (int i=0;i<sHex.length()/2;i++) {
ba[i] = (Integer.decode(
"0x"+sHex.substring(i*2, (i+1)*2))).byteValue();
}
return ba;
}
My formal solution:
/**
* Decodes a hexadecimally encoded binary string.
* <p>
* Note that this function does <em>NOT</em> convert a hexadecimal number to a
* binary number.
*
* #param hex Hexadecimal representation of data.
* #return The byte[] representation of the given data.
* #throws NumberFormatException If the hexadecimal input string is of odd
* length or invalid hexadecimal string.
*/
public static byte[] hex2bin(String hex) throws NumberFormatException {
if (hex.length() % 2 > 0) {
throw new NumberFormatException("Hexadecimal input string must have an even length.");
}
byte[] r = new byte[hex.length() / 2];
for (int i = hex.length(); i > 0;) {
r[i / 2 - 1] = (byte) (digit(hex.charAt(--i)) | (digit(hex.charAt(--i)) << 4));
}
return r;
}
private static int digit(char ch) {
int r = Character.digit(ch, 16);
if (r < 0) {
throw new NumberFormatException("Invalid hexadecimal string: " + ch);
}
return r;
}
Is like the PHP hex2bin() Function but in Java style.
Example:
String data = new String(hex2bin("6578616d706c65206865782064617461"));
// data value: "example hex data"
Late to the party, but I have amalgamated the answer above by DaveL into a class with the reverse action - just in case it helps.
public final class HexString {
private static final char[] digits = "0123456789ABCDEF".toCharArray();
private HexString() {}
public static final String fromBytes(final byte[] bytes) {
final StringBuilder buf = new StringBuilder();
for (int i = 0; i < bytes.length; i++) {
buf.append(HexString.digits[(bytes[i] >> 4) & 0x0f]);
buf.append(HexString.digits[bytes[i] & 0x0f]);
}
return buf.toString();
}
public static final byte[] toByteArray(final String hexString) {
if ((hexString.length() % 2) != 0) {
throw new IllegalArgumentException("Input string must contain an even number of characters");
}
final int len = hexString.length();
final byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(hexString.charAt(i), 16) << 4)
+ Character.digit(hexString.charAt(i + 1), 16));
}
return data;
}
}
And JUnit test class:
public class TestHexString {
#Test
public void test() {
String[] tests = {"0FA1056D73", "", "00", "0123456789ABCDEF", "FFFFFFFF"};
for (int i = 0; i < tests.length; i++) {
String in = tests[i];
byte[] bytes = HexString.toByteArray(in);
String out = HexString.fromBytes(bytes);
System.out.println(in); //DEBUG
System.out.println(out); //DEBUG
Assert.assertEquals(in, out);
}
}
}
I know this is a very old thread, but still like to add my penny worth.
If I really need to code up a simple hex string to binary converter, I'd like to do it as follows.
public static byte[] hexToBinary(String s){
/*
* skipped any input validation code
*/
byte[] data = new byte[s.length()/2];
for( int i=0, j=0;
i<s.length() && j<data.length;
i+=2, j++)
{
data[j] = (byte)Integer.parseInt(s.substring(i, i+2), 16);
}
return data;
}
I think will do it for you. I cobbled it together from a similar function that returned the data as a string:
private static byte[] decode(String encoded) {
byte result[] = new byte[encoded/2];
char enc[] = encoded.toUpperCase().toCharArray();
StringBuffer curr;
for (int i = 0; i < enc.length; i += 2) {
curr = new StringBuffer("");
curr.append(String.valueOf(enc[i]));
curr.append(String.valueOf(enc[i + 1]));
result[i] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}
For Me this was the solution, HEX="FF01" then split to FF(255) and 01(01)
private static byte[] BytesEncode(String encoded) {
//System.out.println(encoded.length());
byte result[] = new byte[encoded.length() / 2];
char enc[] = encoded.toUpperCase().toCharArray();
String curr = "";
for (int i = 0; i < encoded.length(); i=i+2) {
curr = encoded.substring(i,i+2);
System.out.println(curr);
if(i==0){
result[i]=((byte) Integer.parseInt(curr, 16));
}else{
result[i/2]=((byte) Integer.parseInt(curr, 16));
}
}
return result;
}
I am looking for a way to convert a long string (from a dump), that represents hex values into a byte array.
I couldn't have phrased it better than the person that posted the same question here.
But to keep it original, I'll phrase it my own way: suppose I have a string "00A0BF" that I would like interpreted as the
byte[] {0x00,0xA0,0xBf}
what should I do?
I am a Java novice and ended up using BigInteger and watching out for leading hex zeros. But I think it is ugly and I am sure I am missing something simple.
Update (2021) - Java 17 now includes java.util.HexFormat (only took 25 years):
HexFormat.of().parseHex(s)
For older versions of Java:
Here's a solution that I think is better than any posted so far:
/* s must be an even-length string. */
public static byte[] hexStringToByteArray(String s) {
int len = s.length();
byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+ Character.digit(s.charAt(i+1), 16));
}
return data;
}
Reasons why it is an improvement:
Safe with leading zeros (unlike BigInteger) and with negative byte values (unlike Byte.parseByte)
Doesn't convert the String into a char[], or create StringBuilder and String objects for every single byte.
No library dependencies that may not be available
Feel free to add argument checking via assert or exceptions if the argument is not known to be safe.
One-liners:
import javax.xml.bind.DatatypeConverter;
public static String toHexString(byte[] array) {
return DatatypeConverter.printHexBinary(array);
}
public static byte[] toByteArray(String s) {
return DatatypeConverter.parseHexBinary(s);
}
Warnings:
in Java 9 Jigsaw this is no longer part of the (default) java.se root
set so it will result in a ClassNotFoundException unless you specify
--add-modules java.se.ee (thanks to #eckes)
Not available on Android (thanks to Fabian for noting that), but you can just take the source code if your system lacks javax.xml for some reason. Thanks to #Bert Regelink for extracting the source.
The Hex class in commons-codec should do that for you.
http://commons.apache.org/codec/
import org.apache.commons.codec.binary.Hex;
...
byte[] decoded = Hex.decodeHex("00A0BF");
// 0x00 0xA0 0xBF
You can now use BaseEncoding in guava to accomplish this.
BaseEncoding.base16().decode(string);
To reverse it use
BaseEncoding.base16().encode(bytes);
Actually, I think the BigInteger is solution is very nice:
new BigInteger("00A0BF", 16).toByteArray();
Edit: Not safe for leading zeros, as noted by the poster.
One-liners:
import javax.xml.bind.DatatypeConverter;
public static String toHexString(byte[] array) {
return DatatypeConverter.printHexBinary(array);
}
public static byte[] toByteArray(String s) {
return DatatypeConverter.parseHexBinary(s);
}
For those of you interested in the actual code behind the One-liners from FractalizeR (I needed that since javax.xml.bind is not available for Android (by default)), this comes from com.sun.xml.internal.bind.DatatypeConverterImpl.java :
public byte[] parseHexBinary(String s) {
final int len = s.length();
// "111" is not a valid hex encoding.
if( len%2 != 0 )
throw new IllegalArgumentException("hexBinary needs to be even-length: "+s);
byte[] out = new byte[len/2];
for( int i=0; i<len; i+=2 ) {
int h = hexToBin(s.charAt(i ));
int l = hexToBin(s.charAt(i+1));
if( h==-1 || l==-1 )
throw new IllegalArgumentException("contains illegal character for hexBinary: "+s);
out[i/2] = (byte)(h*16+l);
}
return out;
}
private static int hexToBin( char ch ) {
if( '0'<=ch && ch<='9' ) return ch-'0';
if( 'A'<=ch && ch<='F' ) return ch-'A'+10;
if( 'a'<=ch && ch<='f' ) return ch-'a'+10;
return -1;
}
private static final char[] hexCode = "0123456789ABCDEF".toCharArray();
public String printHexBinary(byte[] data) {
StringBuilder r = new StringBuilder(data.length*2);
for ( byte b : data) {
r.append(hexCode[(b >> 4) & 0xF]);
r.append(hexCode[(b & 0xF)]);
}
return r.toString();
}
The HexBinaryAdapter provides the ability to marshal and unmarshal between String and byte[].
import javax.xml.bind.annotation.adapters.HexBinaryAdapter;
public byte[] hexToBytes(String hexString) {
HexBinaryAdapter adapter = new HexBinaryAdapter();
byte[] bytes = adapter.unmarshal(hexString);
return bytes;
}
That's just an example I typed in...I actually just use it as is and don't need to make a separate method for using it.
Here is a method that actually works (based on several previous semi-correct answers):
private static byte[] fromHexString(final String encoded) {
if ((encoded.length() % 2) != 0)
throw new IllegalArgumentException("Input string must contain an even number of characters");
final byte result[] = new byte[encoded.length()/2];
final char enc[] = encoded.toCharArray();
for (int i = 0; i < enc.length; i += 2) {
StringBuilder curr = new StringBuilder(2);
curr.append(enc[i]).append(enc[i + 1]);
result[i/2] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}
The only possible issue that I can see is if the input string is extremely long; calling toCharArray() makes a copy of the string's internal array.
EDIT: Oh, and by the way, bytes are signed in Java, so your input string converts to [0, -96, -65] instead of [0, 160, 191]. But you probably knew that already.
In android ,if you are working with hex, you can try okio.
simple usage:
byte[] bytes = ByteString.decodeHex("c000060000").toByteArray();
and result will be
[-64, 0, 6, 0, 0]
The BigInteger() Method from java.math is very Slow and not recommandable.
Integer.parseInt(HEXString, 16)
can cause problems with some characters without
converting to Digit / Integer
a Well Working method:
Integer.decode("0xXX") .byteValue()
Function:
public static byte[] HexStringToByteArray(String s) {
byte data[] = new byte[s.length()/2];
for(int i=0;i < s.length();i+=2) {
data[i/2] = (Integer.decode("0x"+s.charAt(i)+s.charAt(i+1))).byteValue();
}
return data;
}
Have Fun, Good Luck
EDIT: as pointed out by #mmyers, this method doesn't work on input that contains substrings corresponding to bytes with the high bit set ("80" - "FF"). The explanation is at Bug ID: 6259307 Byte.parseByte not working as advertised in the SDK Documentation.
public static final byte[] fromHexString(final String s) {
byte[] arr = new byte[s.length()/2];
for ( int start = 0; start < s.length(); start += 2 )
{
String thisByte = s.substring(start, start+2);
arr[start/2] = Byte.parseByte(thisByte, 16);
}
return arr;
}
For what it's worth, here's another version which supports odd length strings, without resorting to string concatenation.
public static byte[] hexStringToByteArray(String input) {
int len = input.length();
if (len == 0) {
return new byte[] {};
}
byte[] data;
int startIdx;
if (len % 2 != 0) {
data = new byte[(len / 2) + 1];
data[0] = (byte) Character.digit(input.charAt(0), 16);
startIdx = 1;
} else {
data = new byte[len / 2];
startIdx = 0;
}
for (int i = startIdx; i < len; i += 2) {
data[(i + 1) / 2] = (byte) ((Character.digit(input.charAt(i), 16) << 4)
+ Character.digit(input.charAt(i+1), 16));
}
return data;
}
I like the Character.digit solution, but here is how I solved it
public byte[] hex2ByteArray( String hexString ) {
String hexVal = "0123456789ABCDEF";
byte[] out = new byte[hexString.length() / 2];
int n = hexString.length();
for( int i = 0; i < n; i += 2 ) {
//make a bit representation in an int of the hex value
int hn = hexVal.indexOf( hexString.charAt( i ) );
int ln = hexVal.indexOf( hexString.charAt( i + 1 ) );
//now just shift the high order nibble and add them together
out[i/2] = (byte)( ( hn << 4 ) | ln );
}
return out;
}
I've always used a method like
public static final byte[] fromHexString(final String s) {
String[] v = s.split(" ");
byte[] arr = new byte[v.length];
int i = 0;
for(String val: v) {
arr[i++] = Integer.decode("0x" + val).byteValue();
}
return arr;
}
this method splits on space delimited hex values but it wouldn't be hard to make it split the string on any other criteria such as into groupings of two characters.
The Code presented by Bert Regelink simply does not work.
Try the following:
import javax.xml.bind.DatatypeConverter;
import java.io.*;
public class Test
{
#Test
public void testObjectStreams( ) throws IOException, ClassNotFoundException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
String stringTest = "TEST";
oos.writeObject( stringTest );
oos.close();
baos.close();
byte[] bytes = baos.toByteArray();
String hexString = DatatypeConverter.printHexBinary( bytes);
byte[] reconvertedBytes = DatatypeConverter.parseHexBinary(hexString);
assertArrayEquals( bytes, reconvertedBytes );
ByteArrayInputStream bais = new ByteArrayInputStream(reconvertedBytes);
ObjectInputStream ois = new ObjectInputStream(bais);
String readString = (String) ois.readObject();
assertEquals( stringTest, readString);
}
}
I found Kernel Panic to have the solution most useful to me, but ran into problems if the hex string was an odd number. solved it this way:
boolean isOdd(int value)
{
return (value & 0x01) !=0;
}
private int hexToByte(byte[] out, int value)
{
String hexVal = "0123456789ABCDEF";
String hexValL = "0123456789abcdef";
String st = Integer.toHexString(value);
int len = st.length();
if (isOdd(len))
{
len+=1; // need length to be an even number.
st = ("0" + st); // make it an even number of chars
}
out[0]=(byte)(len/2);
for (int i =0;i<len;i+=2)
{
int hh = hexVal.indexOf(st.charAt(i));
if (hh == -1) hh = hexValL.indexOf(st.charAt(i));
int lh = hexVal.indexOf(st.charAt(i+1));
if (lh == -1) lh = hexValL.indexOf(st.charAt(i+1));
out[(i/2)+1] = (byte)((hh << 4)|lh);
}
return (len/2)+1;
}
I am adding a number of hex numbers to an array, so i pass the reference to the array I am using, and the int I need converted and returning the relative position of the next hex number. So the final byte array has [0] number of hex pairs, [1...] hex pairs, then the number of pairs...
Based on the op voted solution, the following should be a bit more efficient:
public static byte [] hexStringToByteArray (final String s) {
if (s == null || (s.length () % 2) == 1)
throw new IllegalArgumentException ();
final char [] chars = s.toCharArray ();
final int len = chars.length;
final byte [] data = new byte [len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit (chars[i], 16) << 4) + Character.digit (chars[i + 1], 16));
}
return data;
}
Because: the initial conversion to a char array spares the length checks in charAt
If you have a preference for Java 8 streams as your coding style then this can be achieved using just JDK primitives.
String hex = "0001027f80fdfeff";
byte[] converted = IntStream.range(0, hex.length() / 2)
.map(i -> Character.digit(hex.charAt(i * 2), 16) << 4 | Character.digit(hex.charAt((i * 2) + 1), 16))
.collect(ByteArrayOutputStream::new,
ByteArrayOutputStream::write,
(s1, s2) -> s1.write(s2.toByteArray(), 0, s2.size()))
.toByteArray();
The , 0, s2.size() parameters in the collector concatenate function can be omitted if you don't mind catching IOException.
If your needs are more than just the occasional conversion then you can use HexUtils.
Example:
byte[] byteArray = Hex.hexStrToBytes("00A0BF");
This is the most simple case. Your input may contain delimiters (think MAC addresses, certificate thumbprints, etc), your input may be streaming, etc. In such cases it gets easier to justify to pull in an external library like HexUtils, however small.
With JDK 17 the HexFormat class will fulfill most needs and the need for something like HexUtils is greatly diminished. However, HexUtils can still be used for things like converting very large amounts to/from hex (streaming) or pretty printing hex (think wire dumps) which the JDK HexFormat class cannot do.
(full disclosure: I'm the author of HexUtils)
public static byte[] hex2ba(String sHex) throws Hex2baException {
if (1==sHex.length()%2) {
throw(new Hex2baException("Hex string need even number of chars"));
}
byte[] ba = new byte[sHex.length()/2];
for (int i=0;i<sHex.length()/2;i++) {
ba[i] = (Integer.decode(
"0x"+sHex.substring(i*2, (i+1)*2))).byteValue();
}
return ba;
}
My formal solution:
/**
* Decodes a hexadecimally encoded binary string.
* <p>
* Note that this function does <em>NOT</em> convert a hexadecimal number to a
* binary number.
*
* #param hex Hexadecimal representation of data.
* #return The byte[] representation of the given data.
* #throws NumberFormatException If the hexadecimal input string is of odd
* length or invalid hexadecimal string.
*/
public static byte[] hex2bin(String hex) throws NumberFormatException {
if (hex.length() % 2 > 0) {
throw new NumberFormatException("Hexadecimal input string must have an even length.");
}
byte[] r = new byte[hex.length() / 2];
for (int i = hex.length(); i > 0;) {
r[i / 2 - 1] = (byte) (digit(hex.charAt(--i)) | (digit(hex.charAt(--i)) << 4));
}
return r;
}
private static int digit(char ch) {
int r = Character.digit(ch, 16);
if (r < 0) {
throw new NumberFormatException("Invalid hexadecimal string: " + ch);
}
return r;
}
Is like the PHP hex2bin() Function but in Java style.
Example:
String data = new String(hex2bin("6578616d706c65206865782064617461"));
// data value: "example hex data"
Late to the party, but I have amalgamated the answer above by DaveL into a class with the reverse action - just in case it helps.
public final class HexString {
private static final char[] digits = "0123456789ABCDEF".toCharArray();
private HexString() {}
public static final String fromBytes(final byte[] bytes) {
final StringBuilder buf = new StringBuilder();
for (int i = 0; i < bytes.length; i++) {
buf.append(HexString.digits[(bytes[i] >> 4) & 0x0f]);
buf.append(HexString.digits[bytes[i] & 0x0f]);
}
return buf.toString();
}
public static final byte[] toByteArray(final String hexString) {
if ((hexString.length() % 2) != 0) {
throw new IllegalArgumentException("Input string must contain an even number of characters");
}
final int len = hexString.length();
final byte[] data = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
data[i / 2] = (byte) ((Character.digit(hexString.charAt(i), 16) << 4)
+ Character.digit(hexString.charAt(i + 1), 16));
}
return data;
}
}
And JUnit test class:
public class TestHexString {
#Test
public void test() {
String[] tests = {"0FA1056D73", "", "00", "0123456789ABCDEF", "FFFFFFFF"};
for (int i = 0; i < tests.length; i++) {
String in = tests[i];
byte[] bytes = HexString.toByteArray(in);
String out = HexString.fromBytes(bytes);
System.out.println(in); //DEBUG
System.out.println(out); //DEBUG
Assert.assertEquals(in, out);
}
}
}
I know this is a very old thread, but still like to add my penny worth.
If I really need to code up a simple hex string to binary converter, I'd like to do it as follows.
public static byte[] hexToBinary(String s){
/*
* skipped any input validation code
*/
byte[] data = new byte[s.length()/2];
for( int i=0, j=0;
i<s.length() && j<data.length;
i+=2, j++)
{
data[j] = (byte)Integer.parseInt(s.substring(i, i+2), 16);
}
return data;
}
I think will do it for you. I cobbled it together from a similar function that returned the data as a string:
private static byte[] decode(String encoded) {
byte result[] = new byte[encoded/2];
char enc[] = encoded.toUpperCase().toCharArray();
StringBuffer curr;
for (int i = 0; i < enc.length; i += 2) {
curr = new StringBuffer("");
curr.append(String.valueOf(enc[i]));
curr.append(String.valueOf(enc[i + 1]));
result[i] = (byte) Integer.parseInt(curr.toString(), 16);
}
return result;
}
For Me this was the solution, HEX="FF01" then split to FF(255) and 01(01)
private static byte[] BytesEncode(String encoded) {
//System.out.println(encoded.length());
byte result[] = new byte[encoded.length() / 2];
char enc[] = encoded.toUpperCase().toCharArray();
String curr = "";
for (int i = 0; i < encoded.length(); i=i+2) {
curr = encoded.substring(i,i+2);
System.out.println(curr);
if(i==0){
result[i]=((byte) Integer.parseInt(curr, 16));
}else{
result[i/2]=((byte) Integer.parseInt(curr, 16));
}
}
return result;
}
What is the proper way to convert a byte [] to a Base64 string in Java? Better yet would be Grails / Groovy because it tells me that the encodeAsBase64() function is deprecated. The sun.misc.BASE64Encoder package isn't recommended for use and outputs a different size string on some Windows platforms.
The preferred way to do this in groovy is:
def encoded = "Hello World".bytes.encodeBase64().toString()
assert encoded == "SGVsbG8gV29ybGQ="
def decoded = new String("SGVsbG8gV29ybGQ=".decodeBase64())
assert decoded == "Hello World"
Apache Commons has many utilities:
Binary Package: http://commons.apache.org/codec/apidocs/org/apache/commons/codec/binary/Base64.html
Download:
http://commons.apache.org/codec/download_codec.cgi
You could use the open source Base64Coder library
import biz.source_code.base64Coder.Base64Coder
#Grab(group='biz.source_code', module='base64coder', version='2010-09-21')
String s1 = Base64Coder.encodeString("Hello world")
String s2 = Base64Coder.decodeString("SGVsbG8gd29ybGQ=")
Implement your own method like this :)
public class Coder {
private static final String base64code = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
public static String encodeAsBase64(String toEncode) {
return encodeAsBase64(toEncode.getBytes())
}
public static String encodeAsBase64(byte[] toEncode) {
int pos = 0;
int onhand = 0;
StringBuffer buffer = new StringBuffer();
for(byte b in toEncode) {
int read = b;
int m;
if(pos == 0) {
m = (read >> 2) & 63;
onhand = read & 3;
pos = 1;
} else if(pos == 1) {
m = (onhand << 4) + ((read >> 4) & 15);
onhand = read & 15;
pos = 2;
} else if(pos == 2) {
m = ((read >> 6) & 3) + (onhand << 2);
onhand = read & 63;
buffer.append(base64code.charAt(m));
m = onhand;
onhand = 0;
pos = 0;
}
buffer.append(base64code.charAt(m));
}
while(pos > 0 && pos < 4) {
pos++;
if(onhand == -1) {
buffer.append('=');
} else {
int m = pos == 2 ? onhand << 4 : (pos == 3 ? onhand << 2 : onhand);
onhand = -1;
buffer.append(base64code.charAt(m));
}
}
return buffer.toString()
}
}
(adding this to this thread in the hopes that somebody else will get a hit on this and doesn't have to waste his valuable time)
I got stymied today when I tried to add in my Grails 2.3.11/Groovy 2.1.9 application the output of
String src = render(
model: ...,
template: ...,
)
.encodeAsBase64()
as a data- attribute to a DOM element. But the atob() in the corresponding JavaScript, i.e. the code that decodes the Base64 string from the data attribute, kept complaining about illegal characters, while other decoders, e.g. base64 -d accepted the same Base64 string without problems.
The solution is to force the render() return value to a single string and then apply the Base64 encoding, i.e.
String src = render(
model: ...,
template: ...,
)
.toString()
.encodeAsBase64()
or (if you consider encodeAsBase64() as deprecated):
String src = render(
model: ...,
template: ...,
)
.toString()
.bytes
.encodeBase64() // add 'true' for chunked output
I've been experimenting with various bits of Java code trying to come up with something that will encode a string containing quotes, spaces and "exotic" Unicode characters and produce output that's identical to JavaScript's encodeURIComponent function.
My torture test string is: "A" B ± "
If I enter the following JavaScript statement in Firebug:
encodeURIComponent('"A" B ± "');
—Then I get:
"%22A%22%20B%20%C2%B1%20%22"
Here's my little test Java program:
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
public class EncodingTest
{
public static void main(String[] args) throws UnsupportedEncodingException
{
String s = "\"A\" B ± \"";
System.out.println("URLEncoder.encode returns "
+ URLEncoder.encode(s, "UTF-8"));
System.out.println("getBytes returns "
+ new String(s.getBytes("UTF-8"), "ISO-8859-1"));
}
}
—This program outputs:
URLEncoder.encode returns %22A%22+B+%C2%B1+%22
getBytes returns "A" B ± "
Close, but no cigar! What is the best way of encoding a UTF-8 string using Java so that it produces the same output as JavaScript's encodeURIComponent?
EDIT: I'm using Java 1.4 moving to Java 5 shortly.
This is the class I came up with in the end:
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.net.URLEncoder;
/**
* Utility class for JavaScript compatible UTF-8 encoding and decoding.
*
* #see http://stackoverflow.com/questions/607176/java-equivalent-to-javascripts-encodeuricomponent-that-produces-identical-output
* #author John Topley
*/
public class EncodingUtil
{
/**
* Decodes the passed UTF-8 String using an algorithm that's compatible with
* JavaScript's <code>decodeURIComponent</code> function. Returns
* <code>null</code> if the String is <code>null</code>.
*
* #param s The UTF-8 encoded String to be decoded
* #return the decoded String
*/
public static String decodeURIComponent(String s)
{
if (s == null)
{
return null;
}
String result = null;
try
{
result = URLDecoder.decode(s, "UTF-8");
}
// This exception should never occur.
catch (UnsupportedEncodingException e)
{
result = s;
}
return result;
}
/**
* Encodes the passed String as UTF-8 using an algorithm that's compatible
* with JavaScript's <code>encodeURIComponent</code> function. Returns
* <code>null</code> if the String is <code>null</code>.
*
* #param s The String to be encoded
* #return the encoded String
*/
public static String encodeURIComponent(String s)
{
String result = null;
try
{
result = URLEncoder.encode(s, "UTF-8")
.replaceAll("\\+", "%20")
.replaceAll("\\%21", "!")
.replaceAll("\\%27", "'")
.replaceAll("\\%28", "(")
.replaceAll("\\%29", ")")
.replaceAll("\\%7E", "~");
}
// This exception should never occur.
catch (UnsupportedEncodingException e)
{
result = s;
}
return result;
}
/**
* Private constructor to prevent this class from being instantiated.
*/
private EncodingUtil()
{
super();
}
}
Looking at the implementation differences, I see that:
MDC on encodeURIComponent():
literal characters (regex representation): [-a-zA-Z0-9._*~'()!]
Java 1.5.0 documentation on URLEncoder:
literal characters (regex representation): [-a-zA-Z0-9._*]
the space character " " is converted into a plus sign "+".
So basically, to get the desired result, use URLEncoder.encode(s, "UTF-8") and then do some post-processing:
replace all occurrences of "+" with "%20"
replace all occurrences of "%xx" representing any of [~'()!] back to their literal counter-parts
Using the javascript engine that is shipped with Java 6:
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
public class Wow
{
public static void main(String[] args) throws Exception
{
ScriptEngineManager factory = new ScriptEngineManager();
ScriptEngine engine = factory.getEngineByName("JavaScript");
engine.eval("print(encodeURIComponent('\"A\" B ± \"'))");
}
}
Output: %22A%22%20B%20%c2%b1%20%22
The case is different but it's closer to what you want.
I use java.net.URI#getRawPath(), e.g.
String s = "a+b c.html";
String fixed = new URI(null, null, s, null).getRawPath();
The value of fixed will be a+b%20c.html, which is what you want.
Post-processing the output of URLEncoder.encode() will obliterate any pluses that are supposed to be in the URI. For example
URLEncoder.encode("a+b c.html").replaceAll("\\+", "%20");
will give you a%20b%20c.html, which will be interpreted as a b c.html.
I came up with my own version of the encodeURIComponent, because the posted solution has one problem, if there was a + present in the String, which should be encoded, it will converted to a space.
So here is my class:
import java.io.UnsupportedEncodingException;
import java.util.BitSet;
public final class EscapeUtils
{
/** used for the encodeURIComponent function */
private static final BitSet dontNeedEncoding;
static
{
dontNeedEncoding = new BitSet(256);
// a-z
for (int i = 97; i <= 122; ++i)
{
dontNeedEncoding.set(i);
}
// A-Z
for (int i = 65; i <= 90; ++i)
{
dontNeedEncoding.set(i);
}
// 0-9
for (int i = 48; i <= 57; ++i)
{
dontNeedEncoding.set(i);
}
// '()*
for (int i = 39; i <= 42; ++i)
{
dontNeedEncoding.set(i);
}
dontNeedEncoding.set(33); // !
dontNeedEncoding.set(45); // -
dontNeedEncoding.set(46); // .
dontNeedEncoding.set(95); // _
dontNeedEncoding.set(126); // ~
}
/**
* A Utility class should not be instantiated.
*/
private EscapeUtils()
{
}
/**
* Escapes all characters except the following: alphabetic, decimal digits, - _ . ! ~ * ' ( )
*
* #param input
* A component of a URI
* #return the escaped URI component
*/
public static String encodeURIComponent(String input)
{
if (input == null)
{
return input;
}
StringBuilder filtered = new StringBuilder(input.length());
char c;
for (int i = 0; i < input.length(); ++i)
{
c = input.charAt(i);
if (dontNeedEncoding.get(c))
{
filtered.append(c);
}
else
{
final byte[] b = charToBytesUTF(c);
for (int j = 0; j < b.length; ++j)
{
filtered.append('%');
filtered.append("0123456789ABCDEF".charAt(b[j] >> 4 & 0xF));
filtered.append("0123456789ABCDEF".charAt(b[j] & 0xF));
}
}
}
return filtered.toString();
}
private static byte[] charToBytesUTF(char c)
{
try
{
return new String(new char[] { c }).getBytes("UTF-8");
}
catch (UnsupportedEncodingException e)
{
return new byte[] { (byte) c };
}
}
}
I came up with another implementation documented at, http://blog.sangupta.com/2010/05/encodeuricomponent-and.html. The implementation can also handle Unicode bytes.
This is a straightforward example Ravi Wallau's solution:
public String buildSafeURL(String partialURL, String documentName)
throws ScriptException {
ScriptEngineManager scriptEngineManager = new ScriptEngineManager();
ScriptEngine scriptEngine = scriptEngineManager
.getEngineByName("JavaScript");
String urlSafeDocumentName = String.valueOf(scriptEngine
.eval("encodeURIComponent('" + documentName + "')"));
String safeURL = partialURL + urlSafeDocumentName;
return safeURL;
}
public static void main(String[] args) {
EncodeURIComponentDemo demo = new EncodeURIComponentDemo();
String partialURL = "https://www.website.com/document/";
String documentName = "Tom & Jerry Manuscript.pdf";
try {
System.out.println(demo.buildSafeURL(partialURL, documentName));
} catch (ScriptException se) {
se.printStackTrace();
}
}
Output:
https://www.website.com/document/Tom%20%26%20Jerry%20Manuscript.pdf
It also answers the hanging question in the comments by Loren Shqipognja on how to pass a String variable to encodeURIComponent(). The method scriptEngine.eval() returns an Object, so it can converted to String via String.valueOf() among other methods.
I have found PercentEscaper class from google-http-java-client library, that can be used to implement encodeURIComponent quite easily.
PercentEscaper from google-http-java-client javadoc
google-http-java-client home
I have successfully used the java.net.URI class like so:
public static String uriEncode(String string) {
String result = string;
if (null != string) {
try {
String scheme = null;
String ssp = string;
int es = string.indexOf(':');
if (es > 0) {
scheme = string.substring(0, es);
ssp = string.substring(es + 1);
}
result = (new URI(scheme, ssp, null)).toString();
} catch (URISyntaxException usex) {
// ignore and use string that has syntax error
}
}
return result;
}
for me this worked:
import org.apache.http.client.utils.URIBuilder;
String encodedString = new URIBuilder()
.setParameter("i", stringToEncode)
.build()
.getRawQuery() // output: i=encodedString
.substring(2);
or with a different UriBuilder
import javax.ws.rs.core.UriBuilder;
String encodedString = UriBuilder.fromPath("")
.queryParam("i", stringToEncode)
.toString() // output: ?i=encodedString
.substring(3);
In my opinion using a standard library is a better idea rather than post processing manually. Also #Chris answer looked good, but it doesn't work for urls, like "http://a+b c.html"
Guava library has PercentEscaper:
Escaper percentEscaper = new PercentEscaper("-_.*", false);
"-_.*" are safe characters
false says PercentEscaper to escape space with '%20', not '+'
This is what I'm using:
private static final String HEX = "0123456789ABCDEF";
public static String encodeURIComponent(String str) {
if (str == null) return null;
byte[] bytes = str.getBytes(StandardCharsets.UTF_8);
StringBuilder builder = new StringBuilder(bytes.length);
for (byte c : bytes) {
if (c >= 'a' ? c <= 'z' || c == '~' :
c >= 'A' ? c <= 'Z' || c == '_' :
c >= '0' ? c <= '9' : c == '-' || c == '.')
builder.append((char)c);
else
builder.append('%')
.append(HEX.charAt(c >> 4 & 0xf))
.append(HEX.charAt(c & 0xf));
}
return builder.toString();
}
It goes beyond Javascript's by percent-encoding every character that is not an unreserved character according to RFC 3986.
This is the oposite conversion:
public static String decodeURIComponent(String str) {
if (str == null) return null;
int length = str.length();
byte[] bytes = new byte[length / 3];
StringBuilder builder = new StringBuilder(length);
for (int i = 0; i < length; ) {
char c = str.charAt(i);
if (c != '%') {
builder.append(c);
i += 1;
} else {
int j = 0;
do {
char h = str.charAt(i + 1);
char l = str.charAt(i + 2);
i += 3;
h -= '0';
if (h >= 10) {
h |= ' ';
h -= 'a' - '0';
if (h >= 6) throw new IllegalArgumentException();
h += 10;
}
l -= '0';
if (l >= 10) {
l |= ' ';
l -= 'a' - '0';
if (l >= 6) throw new IllegalArgumentException();
l += 10;
}
bytes[j++] = (byte)(h << 4 | l);
if (i >= length) break;
c = str.charAt(i);
} while (c == '%');
builder.append(new String(bytes, 0, j, UTF_8));
}
}
return builder.toString();
}
I used
String encodedUrl = new URI(null, url, null).toASCIIString();
to encode urls.
To add parameters after the existing ones in the url I use UriComponentsBuilder