I am developing a REST service using Spring Boot and Jackson. For this service the caller can specify the list of fields they want in the result. Suppose we had this java class for which we return the fields to the user:
public class Customer {
public int id;
public String firstName;
public String lastName;
public String addressLine1;
public String addressLine2;
public String city;
public String state;
public String zip;
public String phone;
}
The client would call our service:
https://myhost.net/getCustomer?id=2112&lof=id,firstName,lastName
Where the lof parameter is the "list of fields" that the client wants.
This is obviously a simple example, we are implementing something much larger and many more fields are being returned. In addition we are returning many occurrences of objects in 1 call. The requirement is that the client can limit the fields to only the fields they are interested in.
Annotations are out of the question for obvious reasons.
How can I control how the objects are serialized on a per request basis?
Thanks much!
Paul
What you could do:
Register a serialize that will take this field list in consideration https://www.baeldung.com/jackson-custom-serialization
Create a wrapper (or proxy) that will return null for fields that should not be fetched and use #JsonInclude(Include.NON_NULL) annotation, but there you loose the distinction between not present and null field
You can implement it using dynamic #JsonView, but, as it was mentioned in one of the comments, graphql is the way to go.
I'm designing a REST service and am running into the issue that for a given object, I have multiple "states".
The object as it arrives on the initial POST operation.
The Object I store in our DB
The Object I return on a GET
The Object I expect on a PATCH
e.g.
class MyObject {
// Unwanted on POST
// Required on PATCH
// Included on GET
#JsonProperty("id")
private UUID id;
// Everywhere
#NonNull
#JsonProperty("name")
private String name;
// Field I need for internal processing but don't want included in REST.
private AuditTrail stuff;
#JsonCreator
#Builder
public MyObject(...) { ... }
}
...
#Get
public ResponseEntity myFunction(HttpServletRequest request,
#RequestBody #Valid MyObject requestBody) {
...
}
The issue I am running into is that on POST, when the id is omitted, the deserialization fails. I got around it using #JsonIgnoreProperties(), but now on PATCH, where I do want the id present, things work if it is omitted.
Another alternative we toyed with was to have two objects. The first one with the common fields for POST and the other extending from it with the rest, but it feel messy, especially as we deal with objects more complex than the simple example.
It's not actually a problem since I validate and sanitize inputs anyway, but I was wondering if there is a clean way in Jackson to solve this issue.
If you are planning a rest service then you don't need the id in the body anyway. The id will come from the url as a pathvariable:
POST myobjects
GET myobjects/{id}
PATCH myobjects/{id}
I'm using spring boot with mysql to create a Restful API. Here's an exemple of how i return a json response.
first i have a model:
#Entity
public class Movie extends DateAudit {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String name;
private Date releaseDate;
private Time runtime;
private Float rating;
private String storyline;
private String poster;
private String rated;
#OneToMany(mappedBy = "movie", cascade = CascadeType.ALL, orphanRemoval = true)
private List<MovieMedia> movieMedia = new ArrayList<>();
#OneToMany(mappedBy = "movie", cascade = CascadeType.ALL, orphanRemoval = true)
private List<MovieReview> movieReviews = new ArrayList<>();
#OneToMany(mappedBy = "movie", cascade = CascadeType.ALL, orphanRemoval = true)
private List<MovieCelebrity> movieCelebrities = new ArrayList<>();
// Setters & Getters
}
and correspond repository:
#Repository
public interface MovieRepository extends JpaRepository<Movie, Long> {
}
Also i have a payload class MovieResponse which represent a movie instead of Movie model, and that's for example if i need extra fields or i need to return specific fields.
public class MovieResponse {
private Long id;
private String name;
private Date releaseDate;
private Time runtime;
private Float rating;
private String storyline;
private String poster;
private String rated;
private List<MovieCelebrityResponse> cast = new ArrayList<>();
private List<MovieCelebrityResponse> writers = new ArrayList<>();
private List<MovieCelebrityResponse> directors = new ArrayList<>();
// Constructors, getters and setters
public void setCelebrityRoles(List<MovieCelebrityResponse> movieCelebrities) {
this.setCast(movieCelebrities.stream().filter(movieCelebrity -> movieCelebrity.getRole().equals(CelebrityRole.ACTOR)).collect(Collectors.toList()));
this.setDirectors(movieCelebrities.stream().filter(movieCelebrity -> movieCelebrity.getRole().equals(CelebrityRole.DIRECTOR)).collect(Collectors.toList()));
this.setWriters(movieCelebrities.stream().filter(movieCelebrity -> movieCelebrity.getRole().equals(CelebrityRole.WRITER)).collect(Collectors.toList()));
}
}
As you can see i divide the movieCelebrities list into 3 lists(cast, directos and writers)
And to map a Movie to MovieResponse I'm using ModelMapper class:
public class ModelMapper {
public static MovieResponse mapMovieToMovieResponse(Movie movie) {
// Create a new MovieResponse and Assign the Movie data to MovieResponse
MovieResponse movieResponse = new MovieResponse(movie.getId(), movie.getName(), movie.getReleaseDate(),
movie.getRuntime(),movie.getRating(), movie.getStoryline(), movie.getPoster(), movie.getRated());
// Get MovieCelebrities for current Movie
List<MovieCelebrityResponse> movieCelebrityResponses = movie.getMovieCelebrities().stream().map(movieCelebrity -> {
// Get Celebrity for current MovieCelebrities
CelebrityResponse celebrityResponse = new CelebrityResponse(movieCelebrity.getCelebrity().getId(),
movieCelebrity.getCelebrity().getName(), movieCelebrity.getCelebrity().getPicture(),
movieCelebrity.getCelebrity().getDateOfBirth(), movieCelebrity.getCelebrity().getBiography(), null);
return new MovieCelebrityResponse(movieCelebrity.getId(), movieCelebrity.getRole(),movieCelebrity.getCharacterName(), null, celebrityResponse);
}).collect(Collectors.toList());
// Assign movieCelebrityResponse to movieResponse
movieResponse.setCelebrityRoles(movieCelebrityResponses);
return movieResponse;
}
}
and finally here's my MovieService service which i call in the controller:
#Service
public class MovieServiceImpl implements MovieService {
private MovieRepository movieRepository;
#Autowired
public void setMovieRepository(MovieRepository movieRepository) {
this.movieRepository = movieRepository;
}
public PagedResponse<MovieResponse> getAllMovies(Pageable pageable) {
Page<Movie> movies = movieRepository.findAll(pageable);
if(movies.getNumberOfElements() == 0) {
return new PagedResponse<>(Collections.emptyList(), movies.getNumber(),
movies.getSize(), movies.getTotalElements(), movies.getTotalPages(), movies.isLast());
}
List<MovieResponse> movieResponses = movies.map(ModelMapper::mapMovieToMovieResponse).getContent();
return new PagedResponse<>(movieResponses, movies.getNumber(),
movies.getSize(), movies.getTotalElements(), movies.getTotalPages(), movies.isLast());
}
}
So the question here: is it fine to use for each model i have a payload class for the json serialize ? or it there a better way.
also guys id it's there anything wrong about my code feel free to comment.
I had this dilemma not so long back, this was my thought process. I have it here https://stackoverflow.com/questions/44572188/microservices-restful-api-dtos-or-not
The Pros of Just exposing Domain Objects
The less code you write, the less bugs you produce.
despite of having extensive (arguable) test cases in our code base, I have came across bugs due to missed/wrong copying of fields from domain to DTO or viceversa.
Maintainability - Less boiler plate code.
If I have to add a new attribute, I don't have to add in Domain, DTO, Mapper and the testcases, of course. Don't tell me that this can be achieved using a reflection beanCopy utils like dozer or mapStruct, it defeats the whole purpose.
Lombok, Groovy, Kotlin I know, but it will save me only getter setter headache.
DRY
Performance
I know this falls under the category of "premature performance optimization is the root of all evil". But still this will save some CPU cycles for not having to create (and later garbage collect) one more Object (at the very least) per request
Cons
DTOs will give you more flexibility in the long run
If only I ever need that flexibility. At least, whatever I came across so far are CRUD operations over http which I can manage using couple of #JsonIgnores. Or if there is one or two fields that needs a transformation which cannot be done using Jackson Annotation, As I said earlier, I can write custom logic to handle just that.
Domain Objects getting bloated with Annotations.
This is a valid concern. If I use JPA or MyBatis as my persistent framework, domain object might have those annotations, then there will be Jackson annotations too. If you are using Spring boot you can get away by using application-wide properties like mybatis.configuration.map-underscore-to-camel-case: true , spring.jackson.property-naming-strategy: SNAKE_CASE
Short story, at least in my case, cons didn't outweigh the pros, so it did not make any sense to repeat myself by having a new POJO as DTO. Less code, less chances of bugs. So, went ahead with exposing the Domain object and not having a separate "view" object.
Disclaimer: This may or may not be applicable in your use case. This observation is per my usecase (basically a CRUD api having 15ish endpoints)
We should each layer separate from other. As in your case, you have defined the entity and response classes. This is right way to separate things, we should never send the entity in the response. Even for request thing we should have a class.
What the issue if we are sending entity instead of response dto.
Not available to modify them because we already expose it with our client
Sometimes we don't want to serialize some fields and send as response.
Some overhead are there to translate request to domain, entity to domain etc. But its okay to keep more organized. ModelMapper is the best choice for translation purpose.
Try to use construct injection instead of setter for mandate dependency.
It is always recommended to separate DTO and Entity.
Entity should interact with DB/ORM and DTO should interact with client layer(Layer for request and response) even if the structure of Entity and DTO same.
Here Entity is Movie and
DTO is MovieResponse
Use your existing class MovieResponse for request & response.
Never use Movie class for request & response.
and the class MovieServiceImpl should contain business logic for converting Entity to DTO, Or you can use Dozer api to do auto conversion.
The reason for sepating:
In case you need to add/remove new elements in Request/response you dont have to change much code
if 2 entity have 2 way mapping(e.g. one-to-many/many-to-many relationship) then
JSON object cant be created if object have nested data, this will throw error while serializing
if Anything changed in DB or Entity, then this will not affect JSON Response(most of the time).
Code will be clear and easy to maintain.
On one side you should separate them because sometimes some of the JPA annotations which you use in your model don't work well with the json processor annotations. And yes, you should keep the things separated.
What if you later decide to change your data layer? Will you have to rewrite all your client side?
On the other side, there is this problem of mapping. For that, you can use a library with a small performance penalty.
DTO is a design pattern and solves the problem of fetching as maximum useful data from a service as possible.
In case of a simple application as yours, the DTOs tend to be similar to the Entity classes. However for certain complex applications, DTOs can be extended to combine data from various entities to avoid multiple requests to the server and thus save valuable resources and request-response time.
I would suggest not to duplicate the code in a simple case like this and use model classes in response to the APIs as well. Using separate response classes as DTOs will not solve any purpose and will only make maintaining the code difficult.
While most people have answered pros and cons of using DTO objects, I would like to give my 2 cents. In my case DTO was necessary because not all fields persisted in database were captured from user. There were a few fields which were computed based on user input(of other fields) and were not exposed to users. Also, it can also reduces the size of payload which could result in better performance in such cases.
I advocate for separating the "Payload" or "Data" object from the "Model" or "Display" object. Pretty much always. This just keeps things easier to manage.
Here's an example:
Let's say you need to hit an API that gives you data about cats for sale. Then you parse the data into a cat model object and populate a list of cats that is then displayed to the user. Cool.
But now you want to integrate another API and pull cats from 2 databases. But you run into a problem. One API returns furColor for the color and the new one returns catColor for the color.
If you were using the same object to also display the info, you have some options:
Add both furColor and catColor to the model object, make them both optional, and do some kind of computed property to check which one is set and use that one to display the color
In reality, this is rarely an option because the responses will usually be much more different than just one value like this so you would likelly need a whole new parser anyway
Add a new data object and then also a new adapter and then have to do some kind of check to know which adapter to use when
Something else that still isn't pretty or fun to work with
However, if you create a data object that catches the response, and then a display object that has only the info needed to populate the list, this becomes really easy:
You have a data object that captures the response from the first API
Now make a data object that captures the response from the second API
Now all you need is some kind of simple mapper to map the response to the Display Object
Now both will be converted to a common simple display object, and the same adapter can be used to display the new cats without additional work
This also will make storing the data locally much cleaner.
I used RestTemplate to call some 3rd-Party APIs, and RestTemplate will convert received JSON to java POJO automatically like this:
Result result = restTemplate.getForObject(url, Result.class);
But sometimes the JSON structure is quite simple, such as the two examples:
{"access_token":"abcdefg","expires_in":7200} //only need access_token
{"status":0,"result":{"x":25,"y":46}} //only need "x" and "y"
Should I create POJOs for every JSON response in different structure?
For example, the POJO for the first JSON structure:
public class TokenResult {
private String access_token;
private String expires_in;
/* Getters and Setters */
......
}
For the second JSON structure:
public class CoordResult {
private String status;
private Coordinate result;
/* Getters and Setters */
......
}
public class Coordinate {
private String x;
private String y;
/* Getters and Setters */
......
}
I don't think it is elegant to do so, because some JSON structures are very simple and some are used only once like the "access_token" response.
Any ideas on how to avoid too many simple POJOs?
The example you have there are perfectly fine. Even if it feels you have too many POJOs with property members that are not used this IMO is best and future proof approach. Imagine in near future you need to implement functionality that will require access to TokenResult.getExpiresIn() this would mean you would still end up refactoring the base POJO and adding that method and class member.
In similar example with the CoordResult I see the status as a quite important property that would need checking once the response is received so mapping that to a class member in a POJOs is very good idea.
This will make your code more resilient and also predictive with well defined structure and encapsulating all available data from the response.
Though if you have similar responses in terms of context and structure you can always extend a base POJO and add the relevant class members to a child class.
Another benefit is that this all makes the client code easy to read as we can tell what we expect from the response objects and that can be used in different implementations many of which may not applicable and visible now.
If you decide you don't need all properties of a JSON response you can still use #JsonIgnoreProperties annotation on the class and it'll exclude any unknown elements from the JSON mapping.
EX:
import org.codehaus.jackson.annotate.JsonIgnoreProperties;
#JsonIgnoreProperties(ignoreUnknown = true)
public class TokenResult {
private String access_token;
// you don't need this
// private String expires_in;
}
If you're looking for just some values from the response you could try the org.json package ( http://www.json.org/javadoc/ ). It has methods to access values in a JSON response.
String jsonStr = "{\"access_token\":\"abcdefg\",\"expires_in\":7200}";
String accessToken = new JSONObject(jsonStr).getString("access_token");
This approach gets a little more tricky trying to nested values.
I am using Spring and Java and implementing REST Based services. I have a set of developers who develop for mobile,iPad and Web too. Consider I have a bean
Class User{
private String Name;
private Integer id;
private String photoURL;
private ArrayList<String> ProjectName;
private ArrayList<String> TechnologyList;
private ArrayList<String> InterestList;
//Getters and setters
}
While the Web Developers need the entire fields and mobile developers just require two fields from it whereas the iPad requires something in between mobile and web.
Since I am using jackson as a parser, is there a way where while requesting to the controller I can specify which all data I require and avoid the others. For example consider I do a GET request like
GET>http://somedomain.com/users?filter=name,id,photoUrl
Which returns me a JSON structure something like
{
"name":"My Name",
"id":32434,
"photoUrl":"/sss/photo.jpg"
}
Sameway if someone asks for some more fields, they could be filtered. Please let me know how this can be done so that my API remains generic and useable for all.
You can achieve what you want but some extra work is necessary. I can offer you two solutions.
1. Return a Map
Simply put every property that is requested into the map.
2. Use Jacksons Object Mapper directly
Jackson lets you set filters that specify which properties are serialized or ignored.
FilterProvider filter = new SimpleFilterProvider().addFilter("myFilter",
SimpleBeanPropertyFilter.filterOutAllExcept(requestedProperties));
String json = objectMapper.writer(filter).writeValueAsString(value);
You can then return the JSON string directly instead of an object.
For both solutions you would ideally write a class that does the job. But if you do that you could as well write your own message converter. You could extend the MappingJackson2HttpMessageConverter, for instance, and overwrite the writeInternal method to suit your needs. That has the big advantage that you don't need to change your controllers.
The straightforward solution is to implement custom Jackson JSON serializer that will get field names that should be serialized from thread local storage and then serialize only fields which names are presented in that context. For other hand, in controller you can grab all allowed fields names from url and store them into thread local context. Hope this helps.