When I try to serialize an object to JSON using Jackson object mapper, it works perfectly.
{"id":1291741231928705024,"uuid":null,"email":"kannanrbk.r#gmail.com"}
Whereas, when I try to access it using spring rest controller. The long value numbers are rounded off, the last 3 digits.
I read existing questions in the stackoverflow, most of them suggest changing the datatype to string. But we used the Long value reference in most of the places, changing datatype will need some refactoring.
I did my initial analysis:
We are using Jackson ObjectMapper
From Spring, it indirectly calls MappingJackson2HttpMessageConverter
This problem might be somewhere around the JSONParser, where it treats any number as a double (15 digits) and after that, it's rounded off
Is there any way to fix this issue?
Is there any way to fix this issue?
There is no problem with Jackson/Java/Spring Boot, but with JavaScript/Browser.
Trying to reproduce the issue I serialized the same object and got this using curl:
$ curl localhost:8080
{"id":1291741231928705024,"uuid":null,"email":"kannanrbk.r#gmail.com"}
Here the number is correctly serialized.
The same json viewed in Firefox does truncate:
However the "Raw Data" tab displays the number correctly:
.
In JavaScript 1291741231928705024 is not safe integer (see Number.isSafeInteger()):
Number.isSafeInteger(1291741231928705024);
false
The number is greater than 2^53 - 1 so it gets rounded. Even more confusing situations are possible in JavaScript:
> 1291741231928705024 === 1291741231928705022
true
Possible solution
First of all check your client against this kind of problems. If it can safely deserialize such numbers then you're safe.
Or you can serialize longs as Strings (as you mentioned in the question), this is what Twitter proposes in its Twitter IDs (snowflake) article:
To allow Javascript and JSON parsers to read the IDs, Twitter objects include a string version of any ID when responding with JSON. Status, User, Direct Message, Saved Search and other IDs in the Twitter API are therefore returned as both an integer and a string in JSON responses.
Try with bigInt for your primary key
Related
I am getting an InputCoercionException from Jackson when reading JSONs that contain too large numbers.
Caused by: com.fasterxml.jackson.core.exc.InputCoercionException: Numeric value (2328461700) out of range of int (-2147483648 - 2147483647)
Is it possible to configure Jackson to replace values that are out of range - for example, with a -1? If not, is there any solution to parse the value without changing the type to double?
Thank you.
As far as I can see you cannot replace values out of range using Jackson's Deserialization Features.
But you can easily create a custom deserializer and handle the values as you need.
There are tutorials and good examples on Baeldnung's or Jenkov's blogs.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 2 years ago.
Improve this question
I read and write a set of numerical values to a JSON file - they are all the 'double' datatype in Java. An example is the following:
double length = 22.4
double height = 13.1
double width = 17.9
double mass = 15.0
A JSONArray, 'objects', will then contain JSONObjects, each of which will contain the above measurements. Using the same values as above, it will look something like this (pay attention ot the datatypes):
"objects" :
[
{
"length" : 22.4,
"width" : 17.9,
"height" : 13.1,
"mass" : 15
}
]
The first issue posed by such a document is the fact that reading 'length', 'width' and 'height' will return a datatype 'BigDecimal' which seems unnormal for the org.json library which to my knowledge only returns double and integer. This is a problem because I need to convert the 'BigDecimal' values back to double although I wrote the JSON object using standard java double values. Additionally, the org.json library seems to convert any 'double' datatypes to integers when read (notice how mass is '15' and not '15.0'? This is a problem because as you can probably tell, the numerical value of "mass" may be different for another JSONObject (and thus indeed be a double). You can probably imagine the hell, that is iterating through this JSONArray when random values are an integer...
In summary, my problem is the fact that reading JSON files with the org.json library means that numerical values are only returned as doubles or BigDecimals. My question is if I can specify what datatype I would like org.json to write? All I want, is to read and write all my numerical values as a double - I don't want org.json to choose which datatype it will write to the JSON file.
My ideas were the following:
Writing everything as a string and converting back to double when reading
My attempts to solve this that haven't worked or are redundant to me:
Using the valueOf method to change between datatypes
Writing 9999 conditions to find the right instances to convert back to double (which sucks)
Using doubleValue() method
All I want, is to read and write all my numerical values as a double - I don't want org.json to choose which datatype it will write to the JSON file.
For what it is worth, the JSON library is not choosing a data type. It is choosing a character based representation to express a value. And there is some logical justification for it doing what it is doing. JSON is (or at least originated as) a subset of Javascript. In Javascript, integer and floating point are not distinct types. There is just one primitive data type for all numbers in Javascript.
I don't think that this can be solved using the org.json code. The stripping of trailing zeros after a decimal point happens in the static JSONObject::numberToString. Since it is a static method, you can't override it in a subclass.
(Take a look at the source code.)
My suggestions:
Work with the current behavior. For example, your code could be written know which fields are supposed to have floating point values, and use doubleValue() to retrieve their values.
Find an alternative JSON library that doesn't do this. (Based on my reading of the code, I think Jackson would be OK. I haven't checked any other libraries ...)
You could create your own private fork of the org.json library and modify it, but then you would be stuck with the overhead of maintaining that fork indefinitely.
I think that 3 is a bad idea. I have only included it for completeness.
I'm building Android application which interacts with REST API built on .NET.
If my table in SQL Server has 2 rows with the following datetime values:
2019-01-01 00:00:00.000
2019-01-01 00:00:00.113
Then the returned json will have the following values:
2019-01-01T00:00:00
2019-01-01T00:00:00.113
So I don't know how to provide the pattern for setDateFormat when creating an instance of Gson.
If I use GsonBuilder().setDateFormat("yyyy-MM-dd'T'HH:mm:ss"), then my gson can generalize on both cases but it loses millisecond part.
If I use GsonBuilder().setDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS"), then my gson won't lose millisecond part in the second case but it will throw an exception when dealing with the first case.
How can I successfully parse time in two cases but still achieve millisecond? Any provided solution would be appreciated. I don't mind if things I have to is server side or client side.
After a workaround, I found solution myself, thanks to peter.petrov
It is because I configure my API to return data in json format, rather than xml, in my WebApiConfig.cs. So I feel I can't control how it builds its time format. But I finally found it. This is my WebApiConfig.cs file:
var json = config.Formatters.JsonFormatter;
json.SerializerSettings.PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects;
json.SerializerSettings.DateFormatString = "yyyy-MM-dd'T'HH:mm:ss.fff";
config.Formatters.Remove(config.Formatters.XmlFormatter);
Now I server always includes millisecond, no matter what.
That's it.
My goal is to synchronize abitrary rows of data by using the JSON-Format.
As I do not know the exact scheme for the rows (it is a general sync method), my datamodel apparently has to rely on "Object". So in Java I will have an array of Map<String,Object> to be synchronized with the server.
Translating such a row into JSON would give something like
{{"string":"stringvalue"},{"double1":1234.567},{"double2":1234.0},{"long":1234}}
so far, so good - no problem with moshi - everything works as expected.
Now the Problem: When I try to deserialize that JSON with moshi, I get back a double-value for the "long" member. Moshi converts all numbers to Doubles. But unfortunately not all numbers can be safely converted to doubles. Very big integers (aka longs) have a problem with the limited precision of doubles. And rounding-effects also might exist.
I opened an issue with moshi, but unfortunately that was closed. Maybe I wasn't clear enough. (Issue 192)
JSON has no concept of integer - only numbers and Strings. But the subtle detail from "double2" from the example above might lead to a solution for my problem:
If a number does not contain a decimal-point, it is an integer and should be converted to a long.
As longs can not be losslessly converted to doubles, I need a method to intercept the parser before the value is converted to double. But how to do that?
Moshi has this handy concept of JsonAdapters - but unfortunately I currently do not see how I can use them in this case:
The input-type of such an JsonAdapter would have to be Object because I can not cast a generated double to long. So I have to intercept the parser before he converts any value.
But how to return more than one type from there? (I would have to return String, Double or Long from there - or if I can limit the inputs to only numbers I would at least have to return Longs or Doubles.)
(My backend is written in PHP and automatically produces the desired output: Integers are written without a decimal-point.)
I am afraid it's not possible without changing Moshi source code. The JSON string source passes through JsonReader which converts all numbers to double. I could not find a way how to alter this behavior since all subclasses are package-protected.
Can anyone recommend a JSON library for Java which allows me to give it chunks of data as they come in, in a non-blocking fashion? I have read through A better Java JSON library and similar questions, and haven't found precisely what I'd like.
Essentially, what I'd like is a library which allows me to do something like the following:
String jsonString1 = "{ \"A broken";
String jsonString2 = " json object\" : true }";
JSONParser p = new JSONParser(...);
p.parse(jsonString1);
p.isComplete(); // returns false
p.parse(jsonString2);
p.isComplete(); // returns true
Object o = p.getResult();
Notice the actual key name ("A broken json object") is split between pieces.
The closest I've found is this async-json-library which does almost exactly what I'd like, except it cannot recover objects where actual strings or other data values are split between pieces.
There are a few blocking streaming/incemental JSON parsers (as per Is there a streaming API for JSON?); but for async nothing yet that I am aware of.
The lib you refer to seems badly named; it does not seem to do real asynchronous processing, but merely allow one to parse sequence of JSON documents (which multiple other libs allow doing as well)
If there were people who really wanted this, writing one is not impossible -- for XML there is Aalto, and handling JSON is quite a bit simpler than XML.
For what it is worth, there is actually this feature request to add non-blocking parsing mode for Jackson; but very few users have expressed interest in getting that done (via voting for the feature request).
EDIT: (2016-01) while not async, Jackson ObjectMapper allows for convenient sub-tree by sub-tree binding of parts of the stream as well -- see ObjectReader.readValues() (ObjectReader created from ObjectMapper), or short-cut versions of ObjectMapper.readValues(...). Note the trailing s in there, which implies a stream of Objects, not just a single one.
Google Gson can incrementally parse Json from an InputStream
https://sites.google.com/site/gson/streaming
I wrote such a parser: JsonParser.java. See examples how to use it:JsonParserTest.java.