Prevent Gson from escaping Unicode symbols - java

I have Map<String, String> which later I want to serialize to JSON using Gson. Some of keys of this map contain Unicode characters like \uf177 etc. The problem appears when I try to serialize such a map to JSON, let's say I have Map<String, String> containing:
"TEST_KEY" -> "\uf177"
then, when serialized using Gson, I have:
{
"TEST_KEY": "\\uf177"
}
Which is not what I want, I want these Unicode symbols to be as they are when serialized. Is there a way to achieve this? Would appreciate any help,
UPDATE
Code which produces the issue:
projectI18nFileContent = commentsRemover.transform(projectI18nFileContent);
//find json map which represents translations
Matcher fullTranslationsMapMatcher = translationsMapSerializedToJsonPattern.matcher(projectI18nFileContent);
if (!fullTranslationsMapMatcher.find()) {
throw new IllegalArgumentException(format("%s \n does not contain valid translations json map", projectI18nFileContent));
}
String translationsMapSerializedToJson = fullTranslationsMapMatcher.group();
String newTranslationsMapSerializedToJson = gson.toJson(newTranslations);
//replace old json translations map with a new
return projectI18nFileContent.replace(translationsMapSerializedToJson, newTranslationsMapSerializedToJson);
This piece of code is dedicated to change content of i18n file for javascript project, this is the reason why unicode should not be escaped (otherwise it's just resolved not correctly)
Thanks,
Cheers

According to Json spec the backslashes must to be escaped.
So you shouldn't try to prevent this. It's correct behavior.

Related

How to convert utf8 string to escape string in JSON Java?

I want to convert a UTF-8 string to escape \uXXX format in value of JSON Object.
I used both JSON Object and Gson, but did not work for me in this case:
JSONObject js = new JSONObject();
js.put("lastReason","nguyễn");
System.out.println(js.toString());
and
Gson gson = new Gson();
String new_js = gson.toJson(js.toString());
System.out.println(new_js);
Output: {"test":"nguyễn"}
But i am expect that my result is:
Expected Output: {"test":"nguy\u1EC5n"}
Any solutions for this case, please help me to resolve it.
You can use apache commons-text library to change a string to use Unicode escape sequences. Use org.apache.commons.text.StringEscapeUtils to translate the text before adding it to JSONObject.
StringEscapeUtils.escapeJava("nguyễn")
will produce
nguy\u1EC5n
One possible problem with using StringEscapeUtils might be that it will escape control characters as well. If there is a tab character at the end of the string it will be translated to \t. I.e.:
StringEscapeUtils.escapeJava("nguyễn\t")
will produce an incorrect string:
nguy\u1EC5n\t
You can use org.apache.commons.text.translate.UnicodeEscaper to get around this but it will translate every character in the string to a Unicode escape sequence. I.e.:
UnicodeEscaper ue = new UnicodeEscaper();
ue.translate(rawString);
will produce
\u006E\u0067\u0075\u0079\u1EC5\u006E
or
\u006E\u0067\u0075\u0079\u1EC5\u006E\u0009
Whether it is a problem or not is up to you to decide.

Symbols and special characters in maps as keys THROUGH YML FILE

Is it possible to store characters such as : ; / , . in the keys in map in Java that's mapped through application.yml?
I'm trying to store URIs and a value associated with them through a mapping in application.yml file however since it's a key, it doesnt save any of the special characters
http://localhost:8080 becomes httplocalhost8080
Is there any way to preserve those lost characters?
~~EDIT~~
I've tried escape characters, double, and single quotes around the keys and still once they are in the hashmap, they are ignored.
The property in the yml file has this structure:
uri-example-config:
keyToValueMap:
http://localhost:8080 : String1
http://exmaple.com: String2
http://moreURIs.com: String2
And then my config is:
#Configuration
#ConfigurationProperties(prefix = "uri-example-config")
#EnableConfigurationProperties
#Getter
#Setter
public class URIProperties {
private Map<String, String> keyToValueMap = new HashMap<>();
}
~~~~EDIT 2~~~~
Yes, i can easily map.put("http://localhost:8080"...) and the strings is preserved as expected. Doing it through yml file with config and properties causes this issue.
Java HashMap accept any Object (String is an Object) as key !
Also, HashMap will not write/change your String.
I think your problem comes from the loading of your .yml
Yaml accept special characters as key if its in double quote.
- "http://localhost:8080"
Check your serializer / deserializer
Edit
There is an explanation on the github of the project
You should surround your string with brackets
- '[http://localhost:8080]'
Yes it is
For your example in java you just have to escape it right.
Map testmap=new HashMap<String,Object>();
Object testObject="a value Object";
testmap.put("http://localhost:8080",testObject);
for (Object s:testmap.keySet()) {
System.out.println(s);
}
Will print out:
http://localhost:8080
Maybe you can create an URI like this:
URI uri = new URI("http", null, "localhost", "8080", "/my/api");
String uriAsString = uri.toASCIIString();
And then set it into your map as a string.
I suggest you to look at this post, you might find a solution there:
HTTP URL Address Encoding in Java
Yes, it is possible to use characters like : ; / , . in a String key Map, if the key is of type String so the Map can accept any string with any character.
The key should be of type a class that implements equals and hashCode methods. If these methods aren't available so they will be inherited from the Object class.

List of HashMap into JSONs string with new line in Java

I have to convert a List into jsons string with new line.
Right now the code which i am using converts the List of HashMap into single jsons string. like below:
List<HashMap> mapList= new ArrayList<>();
HashMap hashmap = new HashMap();
hashmap.add("name","SO");
hashmap.add("rollNo","1");
mapList.put(hashmap);
HashMap hashmap1 = new HashMap();
hashmap1.add("name","SO1");
hashmap1.add("rollNo","2");
mapList.put(hashmap1 );
Now I am converting it into jsons string using ObjectMapper and the output would be
ObjectMapper mapper = new ObjectMapper();
String output = mapper.writeValueAsString(mapList);
Output:
[{"name":"SO","rollNo":1},{"name":"SO1","rollNo":2}]
Its working fine but I need the output inthe format shown below, i.e for every HashMap there should be new line in the JSON string.
[{"name":"SO","rollNo":1},
{"name":"SO1","rollNo":2}]
If i clearly understand the question, you can use:
output.replaceAll(",",",\n");
or you can go through each HashMap.Entry and call
mapper.writeValueAsString(entry);
or use configuration
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(SerializationConfig.Feature.INDENT_OUTPUT, true);
I suggest a slightly different path, and that is use a custom serializer, as outlined here for example.
It boils down to have your own
public static class MgetSerializer extends JsonSerializer<Mget> {
Which works for List for example.
The point is: I would avoid to "mix" things, as: having a solution where your code writes part of the output, and jackson creates other parts of the output. Rather enable jackson to do exactly what you want it to do.
Beyond that, I find the whole approach a bit dubious in the first place. JSON strings do not care about newlines. So, if you care how things are formatted, rather look into the tools you are using to look at your JSON.
Meaning: why waste your time formatting a string that isn't meant for direct human consumption in the first place? Browser consoles will show you JSON strings in a "folded" way, and any decent editor has similar capabilities these days.
In other words: I think you are investing your energy in the wrong place. JSON is a transport format, and you should only worry about the content you want to transmit, not in (essentially meaningless formatting effects).
You can use String methods to change/replace the output's String. However, this is not correct for json Strings as they may contain commas or other characters that you should escape in the String replace methods.
Alternatively, you should parse the Json String and use JsonNode on the Json as below:
ObjectMapper mapper = new ObjectMapper();
String output = mapper.writeValueAsString(mapList);
JsonNode jsonNode = mapper.readTree(output);
Iterator<JsonNode> iter=jsonNode.iterator();
String result = "[";
while(iter.hasNext()){
result+=iter.next().toString() + ",\n";
}
result =result.substring(0,result.length()-2) + "]";
System.out.println(result);
Result:
[{"rollNo":"1","name":"SO"},
{"rollNo":"2","name":"SO1"}]
This approach will work for String containing characters like comma, for example consider the input hashmap.put("n,,,ame","SO");
The result is:
[{"n,,,ame":"SO","rollNo":"1"},
{"rollNo":"2","name":"SO1"}]
Update: Output updated to include [ and ] and commas between rows.
Update: Fixed the output accordingly

Interpolate JSON values into a string

I am writing an application/class that will take in a template text file and a JSON value and return interpolated text back to the caller.
The format of the input template text file needs to be determined. For example: my name is ${fullName}
Example of the JSON:
{"fullName": "Elon Musk"}
Expected output:
"my name is Elon Musk"
I am looking for a widely used library/formats that can accomplish this.
What format should the template text file be?
What library would support the template text file format defined above and accept JSON values?
Its easy to build my own parser but there are many edge cases that needs to be taken care of and I do not want to reinvent the wheel.
For example, if we have a slightly complex JSON object with lists, nested values etc. then I will have to think about those as well and implement it.
I have always used org.json library. Found at http://www.json.org/.
It makes it really easy to go through JSON Objects.
For example if you want to make a new object:
JSONObject person = new JSONObject();
person.put("fullName", "Elon Musk");
person.put("phoneNumber", 3811111111);
The JSON Object would look like:
{
"fullName": "Elon Musk",
"phoneNumber": 3811111111
}
It's similar to retrieving from the Object
String name = person.getString("fullName");
You can read out the file with BufferedReader and parse it as you wish.
Hopefully I helped out. :)
This is how we do it.
Map inputMap = ["fullName": "Elon Musk"]
String finalText = StrSubstitutor.replace("my name is \${fullName}", inputMap)
You can try this:
https://github.com/alibaba/fastjson
Fastjson is a Java library that can be used to convert Java Objects into their JSON representation. It can also be used to convert a JSON string to an equivalent Java object. Fastjson can work with arbitrary Java objects including pre-existing objects that you do not have source-code of.

ISO-8859-1 encoded strings out of /into JSON in Java

My application has a Java servlet that reads a JSONObject out of the request and constructs some Java objects that are used elsewhere. I'm running into a problem because there are strings in the JSON that are encoded in ISO-8859-1. When I extract them into Java strings, the encoding appears to get interpreted as UTF-16. I need to be able to get the correctly encoded string back at some point to put into another JSON object.
I've tried mucking around with ByteBuffers and CharBuffers, but then I don't get any characters at all. I can't change the encoding, as I have to play nicely with other applications that use ISO-8859-1.
Any tips would be greatly appreciated.
It's a legacy application using Struts 1.3.8. I'm using net.sf.json 2.2.4 for JSONObject and JSONArray.
A snippet of the parsing code is:
final JSONObject a = (JSONObject) i;
final JSONObject attr = a.getJSONObject("attribute");
final String category = attr.getString("category");
final String value = attr.getString("value");
I then create POJOs using that information, that are retrieved by another action class to create JSON to pass to the client for display, or to pass to other applications.
So to clarify, if the JSON contains the string "Juan Guzmán", the Java String contains something like Juan Guzm?_An (I don't have the exact one in front of me). I'm not sure how to get the correct diacritical back. I believe that if I can get a Java String that contains the correct representation, that Mezzie's solution, below, will allow me to create the string with the correct encoding to put back into the JSON to serve back.
I had the same issue and I am using the same technology as you are. In our case, it was UTF 8. so just change that to UTF-16
public static String UTF8toISO( String str )
{
try
{
return new String( str.getBytes( "ISO-8859-1" ), "UTF-8" );
}
catch ( UnsupportedEncodingException e )
{
e.printStackTrace();
}
return str;
}

Categories

Resources