I have a large CSV file which needs to be parsed in order to make post requests to a REST API.The post request body is a json string. The first line in the file has keys and following lines are the values, e.g.
FirstName,LastName
John,Doe
Mark,Twain
So the post body will be something like {"FirstName":"John", "LastName":"Doe"}
This file will be used to create test data for the developers. I will provide a simple page where the developers can enter an account number and hit Submit. The goal is to parse this file and make POST requests to a REST API service.
I want to avoid reading the file each time a request comes in but rather cache these requests on startup to avoid the hassle of reading/parsing the file each time, so that each time a request comes in the request body simply needs to be retrieved from a cache . Is caching these POST request on startup the right way to go here ?
Also there might be a need to add more CSV files just to have variety of data.
What would be a way to make this scalable ?
The format(key names) of the CSV file will be same so each file can be parsed in the same way.
It can be implemented in N number of ways based on your requirement , here is a simple example. You can keep a class like
class Contact{
private String firstName;
private String lastName;
}
and in your other class which holds this value
class RestEnterpriseService{
List<Contact> contacts;
#PostConstruct
init(){
contacts = readFromJSON("JSON URL");
}
}
This can be done in several ways.
If you are using an Spring Framework or Java EE container, you can use the PostConstruct style solution as #AvinashSingh said above.
If you are in some Servlet container, you can do startup tasks using Servlet's Listener or other ways.
Also you can implement it your self, there are so many ways you can do that. Like using an guard variable to detect whether your contacts list is cached; or use skills which like Singleton Design Pattern.
Related
I have a Spring Boot REST API that I'm building. Im slightly stuck on the correct way to design my API in a way that protects each individual users' data. For example, consider the following database relations:
User -> (Has Many) Projects -> (Has Many) Tasks. (A User has-many Projects, and a Project has-many tasks).
For example, if I design my endpoints in the following way:
GET /api/v1/projects/{projectId}
POST /api/v1/projects/{projectId}/tasks
Just as a simple example for the above, how can I make sure, when creating new tasks for a certain project, that the project belongs to the logged in user?
Currently, I am using JWT tokens via Spring Security as my authentication strategy, and included in the payload of the token I have my Users' id. So with every request I can retrieve the user, but surely that's incredibly inefficient to be making so many requests to the database and check if the user actually has a given project.
Some solution I was thinking about is to simply have endpoints designed like this:
/api/v1/users/{userId}/projects/{projectId}/tasks
And then I can use the user id in the JWT payload and compare it to the user id in the request parameter. But then that would mean with every new relation in my database, the length of the url is going to be massive :) Also I guess it would mean all the business logic would be inside the User service for the whole application, right? Which seems a little odd to me... but maybe I'm wrong.
Im not sure if thats an issue or not, but just trying to design the API to be as elegant as possible.
Thanks again!
Checking if the user has permissions to a project on every request is the correct solution. Consider cases when many other applications / users are calling your API. You want to make sure that your API is as secure as possible and cannot be manipulated from the frontend.
To make it more efficient you should have a way/query to check associations in your database like a simple query that returns true/false which should be quicker than retrieving all the data and comparing in Java code.
And when possible combine multiple database queries into one, like for one of your examples:
GET /api/v1/projects/{projectId}
in this case, don't run a query to get a user's information and a query for the requested project. Instead you could do a single query with a join between the user's table and the project table which should only return a project if the user is associated with it. The best way really depends on how your database is structured.
Adding a user id into the API URL is just redundant information. Just because the user id in the token matches the user id in the URL doesn't mean the user has any kind of permissions to any project.
Another solution to be avoided is to include the user's project ids in the JWT token which you can then compare without making a database request. This is bad for several reasons:
The token should only have required information for the user to access the API, it shouldn't have business logic
Depending on how much business logic you store in the token the token can become large in size. See this post for a discussion on size limits: What is the maximum size of JWT token?
If there is a way for the someone other than the user (like admin) to add/remove a user's association to a project then that change will not be reflected in the token until the token's data is refreshed
EDIT:
On the spring side I have used the #PreAuthorize annotation before to handle these types of method checks. Below is pseudo code as an example.
#RestController
public class MyController {
#PostMapping
#PreAuthorize("#mySecurityService.isAllowed(principal, #in)")
public SomeResponseType api1(SomeRequestType requestData) {
/* this is not reached unless mySecurityService.isAllowed
returns true, instead a user gets a 401/403 HTTP response
code (i don't remember the exact one) */
}
}
#Service
public class MySecurityService {
/*
Object principal - this is spring's UserDetails object that is
returned from the AuthenticationProvider. So basically a Java
representation of the JWT token which should have the
user's username.
SomeRequestType requestData - this is the request data that was
sent to the API. I'm sure there is a way to get the project ID
from the URL here as well.
*/
public boolean isAllowed(Object principal, SomeRequestType requestData) {
/*
take the user's username from the principal, take the
project ID from the request data and query the database
to check authorization, return true if authorized
make this check efficient
*/
return false;
}
}
The annotation and the security service can then be applied to multiple methods. You can have many different security services depending on what your are checking.
There are other ways available too https://www.baeldung.com/spring-security-method-security and this has to be enabled in spring's configuration (also explained in the link).
Hi so if I understood it correctly you want to automatically assign the task that is going to be created with "POST /api/v1/projects/{projectId}/tasks" to the current logged in user.
You could try to add a Parameter 'Principal principal' to your rest controller. The Principal is the user that is sending the request.
After you have your Prinicipal, you could write a simple convert method(for example: convertPrincipalToUser(Principal principal) which returns you the user. Finally you can add your user to the corresponding task)
Here is some more information about it:
https://www.baeldung.com/get-user-in-spring-security
I am currently working on a monitoring application using Spring Cloud Sleuth. Currently I try to collect as much information about my requests as possible.
To keep the approach as scalable as possible I use GenericFilterBeans and HandlerInterceptorAdapter to access information from the requests sent to the REST-API.
I am struggling with getting parameters of a REST-call where the parameters are mapped from the URL like in the following code snippet:
#RequestMapping(
value = {"/{service}/{route_id}/book", "/accounting-core-service/{service}/{route_id}/book"},
method = RequestMethod.GET)
#ResponseBody
public ModelAndView book(#PathVariable(value="service") String serviceName,
#PathVariable(value = "route_id") int routeId,
HttpServletResponse response) {
/*Do something*/
}
The question is not, whether it is good practice or not to write it like so. The question is whether there is an approach similar to Filter or Interceptor (or the proper use of them) to access those parameters.
A requirement is, that it can be applied easily to an application by adding very few lines of code. Annotating every Method call manually or manually inserting the code to write the parameters into the trace from within the method is not feasible for me.
If you need more information feel free to ask. I will provide you with all information you need to help me with my problem.
Although not officially supported (as it's not written in the reference documentation), Spring MVC holds that information as request attributes.
You could create your own HandlerInterceptor, ordered right after the Sleuth one, and get that information from the request like this:
// "/{service}/{route_id}/book"
String matchingPattern = (String) request
.getAttribute(HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE);
// "service" => "fooService", "route_id" => "42"
Map<String, String> templateVariables = (Map<String, String>) request
.getAttribute(HandlerMapping.URI_TEMPLATE_VARIABLES_ATTRIBUTE);
Note, the template variables are already decoded at that point, which is probably what you want anyway.
You can create a Filter that intercept all the requests.
For each request you can retrieve also this informations:
query parameters
body of request
url
header parameters
You can save all this data as you need.
This is the best way you can do that.
If you need to group all urls like /{service}/{route_id}/book in a "family" of urls you can do that splitting the url and check if it is part of the family, but when a new family is added in your code you need to update the filter (or configure something in an external file or database).
When developing a Web Application in Java, launching in Tomcat, I need to be able to create (dynamically) a new static address (link,URL) in the server that will be used to view the information of a new item, let's call it new_item_001, which have been just created by one user.
Say I want to create a new address
www.domain.com/webapp/items/new_item_001
which can be used to render a view of the contents of new_item_001.
Which is the best approach to do this?
Should I dynamically create a new servlet class for this view?
Should I dynamically create the folder items and one html file new_item_001 for this item inside of it?
Should I edit the server address mapping rules to create this static address and map it to a central servlet which somehow knows which item to display?
I understand the question is ill posed, and that I am far from even understanding the issue, so I would like some guidelines on what to look for.
None of the above.
You should simply have a servlet mapped to /items/*. When a request come to this servlet, analyze the actual path of the request, extract the part after /items/ to know the actual value (new_item_001) in your example, get the data corresponding to this item from the database, and send it to the browser.
Using a true MVC framework like Spring MVC would make that much easier. You could simply map a method of a controller using
#RequestMapping("/items/{itemId}")
public Item getItem(#PathVariable("itemId") String itemId) {
...
}
and let the framework do all the URL parsing for you.
I would like to tackle this in a simple way. Creating a servlet for each created item would be overkill and become quite cumbersome to manage after a successful run of the application for some time.
Changing/editing server mapping URL looks very naive approach and is not scaling too. Let configuration be there and change them only when you actually need to change them.
My suggestion is to create one servlet that handles all these requests. For example, you may save item information on a datastore or on file system(i.e images uploaded by user etc..). Next time a GET request is received by the application to fetch saved information of an item, servlet should be able to reference the item on database associated with the item id on the URL. If you don't wish to expose item id/surrogate key in the database, you can also have a simple mapping between them by implementing your own logic. Frameworks like Spring MVC do a good job in mapping URLs to resources like this should you wish to use a framework.
Additionally to minimize the number of requests to the same item, you can also implement an HTTP caching strategy(i.e. ETAG, If-Modified-Since) by instructing your web server at the time of first GET request from a user.
I'm trying to understand how to create and modify links in Spring HATEOAS.
For example, say I have two collections, one at api/users and another at api/event. I would like to associate a user api/user/56 with an event api/event/21. For arguments sake this is a many-to-many - a user may attend many events, an event may have many users.
As I understand it, the restful way of doing this is to use the URIs as primary keys, so I might post the following to api/user/56/events;
{
attends: "http://localhost:9090/api/event/21"
}
The endpoint then needs to be able to parse that URL and extract the ID (in this case 21) and the controller (EventController.class) so that I can persist this.
Question 1: Is this the correct way of dealing with relationships in Spring Hateoas in terms of the REST API?
Question 2: How can I resolve this url in a controller to a usable handle on the data (for example a reference to the appropriate controller/method, a primary key, etc)
Research
RestTemplate can be used to request the data from the controller inside the request mapped method, like so;
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<EventResource> response = restTemplate.getForEntity(attendsUrl, EventResource.class);
EventResource eventResource = response.getBody();
However I don't believe that eventResource should return an Id field as part of the data - it's not very restful and this would be exposed on the API. One approach is to have a parameter "includePK=true" but again this doesn't feel right - it's just hiding the problem. Moreover the idea of the server making requests to it's own API in this manner seems circuitous.
Update
There is an open question for this here https://github.com/spring-projects/spring-hateoas/issues/292. Based loosely on some of the comments (by user kevinconaway) from that issue I have made a quick util class that offers an easy solution here: SpringHateoasUtils. The solution boils down to;
String mapping = DISCOVERER.getMapping(targetClass, targetMethod);
UriTemplate template = new UriTemplate(mapping);
//values is key/value map of parameters that the referenced method accepts
Map<String, String> values = uriTemplate.match(uri);
SpringHateoasUtils makes this slightly nicer but it still feels like it should be a feature. I'll seek to get something in the spring code for this - when it's clear what is happening with this I'll answer this question.
Look at the answer here:
POSTing a #OneToMany sub-resource association in Spring Data REST
Question 1) Yes this is how you post links/relations. With URIs.
Question 2) The URI of the resource actually IS its ID from the client's perspective. The server internally automatically resolves this URI into the actual model instance with
org.springframework.data.rest.core.UriToEntityConverter.convert(...)
I am having some trouble understanding this. Can someone help me better understand this?
MVC
Model --> Java- Spring Framework
View ---> templating language(JSP velocity) & Javascript
DB --> SQL
Q-1)
Now, When I open a particular page, I can't visualize the flow. I've read about DAO, controller , service etc and I understand them individually but I am really confused when I club all together what's the order of execution? Whats the flow exactly ? Is it that first the view is loaded then it sends JS request to fetch the necessary data from backend and then the controller and service are invoked and the DAO queries the db? Then how does the API come into picture? DAO deals with the API?
Q-2)
Why do we need xyz.properties? I have removed a module from my page. If I remove a particular js file(related to that module) from the scripts.properties, then ideally that js should not get executed at all right? Then still why would I see the api call to fetch the data related to that module? I don't see the module but I sure see the api call. Why is that?
DB doesn't enter in MVC model. And you're forgetting a principal element in your analysis: the Controller. The flow goes like this:
Client performs a request to an URL
The application server gets the URL and passes the handling to the web application.
The web application using Spring MVC will handle the URL processing to the Controller: DispatchServlet, which is a Servlet.
The DispatchServlet will try handle the URL. If there's an URL mapping, then it will pass it to the class (mapped in the spring.xml config or decorated with #Controller annotation).
This controller (which in fact is part of the model) will handle the request. It will call services, daos, etc (Model) and return the necessary data to complete the response to the DispatchServlet.
The DispatchServlet will finish the request handling and, in the end, will generate the results e.g. a text/json response, or it will forward to a JSP file (View).
For question two, I never have used such scripts.properties file, so I don't know what you're talking about. Usage of a properties file is to store application properties that should not change until an application redeploy. They have 3 main advantages:
They can be easily manipulated by human users. It's no rocket science to add, edit or remove values.
Since it is a plain text, it's easier to version using a version control system like SVN, Git or another of your preference.
It provides a faster access since it is usually in the same disk as the application, so there's no much time penalty when accessing to its contents compared to a database configuration. But since it is in disk, it still has a disadvantage against RAM access only.
In simple layman's term, MVC explained in pictorial form
(inputing data) (data related part) (display rendering)
-request mapping -classes -JSP (Script,CSS,HTML)
-request param -interface -velocity
Controller ------------->Model--------------->View
||
\/
(data processing logic) (access to Databse)
-optimization -JDBC
-business logic -SQL
Service--------------------->DAO