Develop Reference

How to url encode data in Java

So Im trying to translate a working python code into Java. One of the steps required is to url encode the data. But when I encode the data in Java it looks different than the one in encoded in Python.
In one of the block of Python code theres this:
data = {'request-json': json}
print('Sending form data:', data)
data = urlencode(data)
data = data.encode('utf-8')
print('Sending data:', data)
The Output
Sending form data: {'request-json': '{"apikey": "xewpjipcpovwiiql"}'}
The output after being encoded
Sending data: b'request-json=%7B%22apikey%22%3A+%22xewpjipcpovwiiql%22%7D'
So this is what im trying to do in Java. As you can imagine Java is more involved. I used gson to convert to Json
Gson gson = new Gson();
API_Key key = new API_Key("xewpjipcpovwiiql");
String jsonInputString = gson.toJson(key);
Data data = new Data(key);
String request_form = gson.toJson(data);
System.out.println(request_form);
String urlencoded = URLEncoder.encode(request_form,StandardCharsets.UTF_8);
System.out.println(urlencoded);
The output:
Sending form data: {"request-json":{"apikey":"xewpjipcpovwiiql"}}
The output of the encoded string:
%7B%22requestjson%22%3A%7B%22apikey%22%3A%22xewpjipcpovwiiql%22%7D%7D
So they dont look the same so why are they coming differently ? How do I get the same python encoded String in Java ? I noticed in Python it used a combination of single and double quotes and in Java its only Double quotes so I dont know if that makes a difference.
Thank You!

On the Python side: The data.encode('utf-8') call is not necessary or at least the documentation describes with a different intention compared to this use https://docs.python.org/3/library/stdtypes.html#str.encode (and that's why there's a b' at the beggining).
The outer brackets are missing because it is interpreting request-json as the URL parameter name (it may be easier to understand if you add a second property at the json's top/first property level, you'll see you end with request-json=%7B%22apikey%22%3A+%22xewpjipcpovwiiql%22%7D&second-property=<second-property-value>).
On the Java side: the request_form is being completely interpreted as a single value to encode so you can put the encoded value as part of some parameter in a URL, as in: https://host:port?some-parameter-name=%7B%22requestjson%22%3A%7B%22apikey%22%3A%22xewpjipcpovwiiql%22%7D%7D

Convert euro sign from a PHP sent JSON string to a Java string (utf-8)

i am building an Android application that prints to a thermal printer and i have problems with printing € symbol.
Specifically, my Android application:
makes a GET request to a php file
the php file build a JSON object
one of my JSON values contains a "\u20AC", that is, euro sign in unicode.
php file stringifies the JSON + send it back to Android
which sends the data UTF-8 encoded (plain text)
My PHP sending-back code is, basically, something like that:
header("Content-Type: text/html; charset=UTF-8\n");
...
$currency_symbol = '\u20AC';
...
$blah = array("id"=>$order['id'], ... , "currency_symbol"=> $currency_symbol);
echo json_encode( $blah );
exit;
Before all that, i was able to print the € sign to a printer by:
Changing to correct code page of thermal printer
Calling the following code:
new String("\u20ac").getBytes( Charset.forName("Windows-1252") ) );
then sending euro bytes directly to printer.
With JSON solution, i can not render the euro sign anymore as every try, even the previous working one, it always renders this to printer (but not the sign):
\u20AC
PS. I have no problem with the other UTF-8 strings as i am able to print them by that:
String.format("- " + json_obj.getString("address") + "\n").getBytes( charset )
where json_obj is the encoded JSON that came from PHP and charset in what code page the printer is set to (as Charset).

I solved it by using the following code:
String currency_symbol_hex = order_obj.getString("currency_symbol");
String currency_symbol_str = Character.toString((char) Integer.parseInt(currency_symbol_hex,16));
BT_write( String.format("%s", currency_symbol_str).getBytes( Charset.forName("Windows-1252") ) );
where order_obj.getString("currency_symbol") is a JSON value sent by the PHP and contains only 20AC (rather \u20AC) and BT_write basically writes the bytes to a connected Bluetooth socket.
The currency_symbol is stored in JSON like that:
return array(
...
"euro"=>array("symbol"=>"€","symbol_unicode"=>"20AC")
-------^
...
);
and i just return it through the currency_symbol key in my JSON.
This is obviously not the best solution as different currencies may require different character set than Windows-1252 and also requires special case on PHP side (sending 2 unicode codes eg. \u0631.\u0639. (the Omani rial currency symbol) wont work unless you use an array and parse each one etc.) but at least its a start. Thanx!

Store base64 encoded string in HBase

I have a very specific requirement of storing PDF data in Hbase columns. The source of Data is Mongo DB, from where the base64 encoded data is read and I will need to bulk upload it to Hbase table.
I realized that in base64 encoded string there are a lot of "\n" character which splits the entire string into parts. Not sure if it is because of this, but when I store the string as it is, using a put :
put.add(Bytes.toBytes(ColFamilyName), Bytes.toBytes(columnName), Bytes.toBytes(data.replaceAll("\n","").toString()));
It is storing only the first line from the entire encoded string. Eg :
If the actual content was something like this :
"JVBERi0xLjQKJaqrrK0KNCAwIG9iago8PAovQ3JlYXRvciAoQXBhY2hlIEZPUCBWZXJzaW9uIDEu
" +
"MSkKL1Byb2R1Y2VyIChBcGFjaGUgRk9QIFZlcnNpb24gMS4xKQovQ3JlYXRpb25EYXRlIChEOjIw\n" +
"MTUwODIyMTIxMjM1KzAzJzAwJykKPj4KZW5kb2JqCjUgMCBvYmoKPDwKICAvTiAzCiAgL0xlbmd0\n" +
It is storing only the first line which is :
JVBERi0xLjQKJaqrrK0KNCAwIG9iago8PAovQ3JlYXRvciAoQXBhY2hlIEZPUCBWZXJzaW9uIDEu
in the column. Even after trying to remove the "\n" manually it is the same output.
Could someone please guide me in the right direction here ?

Currently, I am also working on Base64 encoding. As per my understanding, you should try using
org.apache.hadoop.hbase.util.Base64.encodeBytes(byte[] source, int option)
method where DONT_BREAK_LINES can be used as an option.
Please let me know if this works fine.

Managed to solve it. The issue was when reading the Base64 encoded data from MongoDB Source. Read the data from Mongo DB document DBObject as:
jsonObj.get("receiptContent").toString().replaceAll("\n","")
And stored it as such in Hbase. Even from the Hue HBase UI Browser I can see the PDF content now.

Get parameter + replace with space when processing in action class

I encrypt a text "good-bye, friend" using BasicTextEncryptor. So the encrypt value looks like below,
3qe80L1ap+cR2zRU9csFwOffw5NtWTueLRYgSXyjctI=
Then I email a URL to the user where the above parameter as a token.
Then the user copies the below URL and presses enter,
http://localhost:8080/token=3qe80L1ap+cR2zRU9csFwOffw5NtWTueLRYgSXyjctI=
But when I access the parameter in Struts 2 application through the action method it gives me the encrypt parameter as below,
3qe80L1ap cR2zRU9csFwOffw5NtWTueLRYgSXyjctI=
The + is replaced by " ". So when I decrypt it, it gives me EncryptionOperationNotPossibleException.
Does struts decode the + to " " assuming browser + is a encode character? In that case it ok before I proceed with decrypt, I replace the space with + ?

A better way would be to "URL encode" the string before appending it to actual URL.
URLEncoder.encode("3qe80L1ap+cR2zRU9csFwOffw5NtWTueLRYgSXyjctI=", "ISO-8859-1");
This would make sure the token is correctly decoded.
To, answer your question, struts does not have any role in decoding the URL parameter. Its the core functionality of the application server to decode the URL parameter. So every HTTP parameter is subjected to decoding before reaching the application code.
Whatever is decoded by the server is available by to the application (i.e. Struts in your case. )
Now to explain why the + is not reaching your struts.
java.net.URLDecoder.decode("3qe80L1ap+cR2zRU9csFwOffw5NtWTueLRYgSXyjctI="));
it returns 3qe80L1ap cR2zRU9csFwOffw5NtWTueLRYgSXyjctI=
which means that + is not getting URL Decoded.
So, reiterating, every HTTP parameter (querystring or form POST) is subjected to decoding before reaching the application code.
When you URL encode your string, + is encoded as %2B and your struts application will receive the correct decoded string.

You'll need to not put the base64 encoded string there, but encode it using the UrlEncoder, like the following:
URLEncoder.encode("3qe80L1ap+cR2zRU9csFwOffw5NtWTueLRYgSXyjctI=", "UTF-8")
That way you can put it in the link.

Consider using a so called URL safe variant of Base64. The most common variant, described in RFC 4648, uses - and _ instead of + and / respectively, and omits padding characters (=).
Most implementations of Base64 support this URL safe variant too, though if yours doesn't, it's easy enough to do manually.

URLs cannot contain spaces. URL encoding normally replaces a space with a + sign.
Thus the server decodes normally + sign to the space. See URLEncoder docs or read Java URL encoding of query string parameters.

Convert and Display the UTF8 Encoded String

I have a JSON response which i want to store in DB and display in text view or edit text. This json response is encoded by UTF-8 format.
Response is somthing like
"currencies": [[0,"RUR"," ",1,0],[1,"EUR","â¬",1.44,100],[2,"GBP","£",1.6,100],[3,"JPY","Â¥",0.0125,100],[4,"AUD","$",1.1,100]]}
where â¬,£,Â¥ are currency symbol. I have to decode this and then display. This symbols are symbol in Unicode (transferrred as UTF8). How can I convert this encoded symbol. Plz help.
I tried this but it didnt works:
byte[] b = stringSymbol.getBytes("UTF-8"); // â¬,£,Â¥
final String str = new String(b);

You're showing the text with non-currency symbols... it's as if you're taking the original text, then encoding that as UTF-8, then decoding it as ISO-8859-1.
It's just text - you shouldn't need to do anything to it afterwards, and you should never see it in this broken format. If you have to convert the text back to bytes and then to a string again, that means you've already lost, basically.
Check the headers on the HTTP response which returns the JSON - I suspect you'll find that it's claiming the data is ISO-8859-1 rather than UTF-8. The actual encoding has to match the encoding that's specified in the headers, otherwise you end up with this sort of effect.
Another possibility is that whatever's returning the JSON is accurately giving you the data that it knows about, and that the data is broken upstream. You should follow the data step by step (assuming you own all the links in the chain) until you can see where you're first encountering this brokenness.