I have been given a API link of the form of a URL and query string. And following is my approach,
Query string format means that a GET request is to be fired.
I also assume that this can be done with the HttpURLConnection in Java
I have some data list that I'm retrieving from db
How would I fire for each data in list? Is a simple for loop not going to be enough for such a sophisticated task?
The API link is a trivial link with query string with data from db to be appended to one at a time.
Would like to hear how you would approach this task and see if my approach lacks somewhere.
You are right in doubting the simple for loop approach. It would be slow. The request is blocking, so you'll be waiting for the result of request 1 before firing request 2. Look into doing this asynchronously, firing multiple requests at once.
It's hard to say more without details on the API. Is it an online web service? Something internal created by another department? If it does not exist, consider asking for a version of that function that can receive multiple parameters at once, instead of having to do tons of tiny calls.
Related
I have a drop down which lets you select the name of the country and then in the other bottom the province/state is populated based on the country selected.
When the user selects the country, a query is made and then province is updated respectively
I am running into a scenario where there's a race condition between the query response arrival, which ends up displaying the incorrect data . How should I handle this ?
eg: User selects country A, query is fired, network is slow, meanwhile user changes the country to B and then another request is fired, The response of B comes back quickly, but response of A comes after, Now the screen is in a state where country says B but the province are of A
Notice: I don't want to block the country selector while the query response is being awaited
any suggestions on resolving this ?
You may want to add a simple filter on your query response, such
that the code to display runs if the response matches the current
query. Configure your callback code carefully, and you can have only the latest
response displayed, since you know which country is currently selected.
(Admittedly, this might not the best way to do this, but it should work.)
More generally, you would have to develop this with some combination of network request(s) and caching/local database.
If the size of your dataset is not too large, or if it is not going to be updated too often, you might simply want to put it into a local database, e.g. Room DB.
If your data doesn't match this specification and it is subject to future updates in the back-end, but the overall dataset is pretty small, try caching the dataset (countries and provinces) at app start. This way your network calls are light, you always have the latest data, your UI is no longer dependent on asynchronous operations, and your code is simpler. You could also use data binding here, if appropriate.
If your dataset is very large such that fetching it all at app start is an unacceptable overhead, only then should you work purely with network request(s) each time the user changes a filter option. One option is that your callback code is configured to ignore any response that doesn't relate to the currently selected option. Another option is that your HTTP-related schema, for request and/or response, can contain some information about which "country" generated the HTTP request so that you can check for it in the response object. (The latter option can be considered overkill in most cases, but sometimes in complex scenarios, you might have to do such stuff for one or more reasons, such as functional definition, simplicity, neatness, etc.)
P.S.: I assume you are already using appropriate libraries for HTTP calls.
Consider there are 2 Rest APIs implemented using Java and spring .
One to raise a request(https://example.com/some-service/Requests)(1st API)(our application consumes),It does some processing at the back end but it does not return me the actual result and instead returns me success response.
The needed response to my request takes some time for example 13 mins and this actual response is sent through another API which our application exposes, for example(<ourApplication.com/notifyRaisedRequests>)(2nd API).
I want to write the code in java such that response from 2nd API should not take more than 13 mins after I raise request using 1st API.If more than 13 mins execute failure part and if less than 13 mins execute success part of code
How can this be achieved using Spring boot,Java.There should be someway to keep checking in spring boot that whether 2nd API was called or not within the time interval(13mins). Any help is appreciated.
The code which raises the request:
Response r = post("/Requests");
db.insert(r.getUniqueId(), now);
The code which receives the notification:
void listener(NotificationRequest r) {
db.delete(r.getUniqueId());
// do the success action
}
The code which runs periodically checking the outstanding requests:
for (DbRecord r: db.selectAllRecords()) {
if (r.time - now > 15 minutes) {
db.delete(r.id);
// do the failure action
}
}
This is all pseudocode obviously.
This isn't really specific to Java, it sounds more like an architectural concern.
A user or a process requests resource X by calling the first api
An "OK" (200) response is returned
A user or a process requests resource Y (notify) by calling the second api
If less than Z amount of time has passed between the call to Y and the call to X then send response A; otherwise send response B
One way to tackle this, as suggested in the comments, is to use a database to keep a unique record with a timestamp for each request to the fist api (X). When the second api (Y) is called, it's straghtforward: get the difference in time and act accordingly.
As for generating a unique identifier for each request to the first api, depending on where the record is stored you could do different things but my suggestion would be to keep it really simple and use a random uuid: UUID.randomUUID().
This implies that the response the first api sends back to whatever requested it will have that same uuid in it somewhere since this is necessary for the second api.
For example:
request to api X is received by a spring rest controller (or similar). Somewhere in there (i.e. service layer called by the controller) a unique identifier will be created and a record will be saved in the database. This unique identifier must be returned in the response.
request to api Y is made with the uuid returned in the X's response. This can then be used to recover more information that was saved in the database for that request (such as the timestamp).
EDIT
As per more indications in the comments, three components are needed: a rest controller, a service for data storage / retrieval and a scheduler.
A call is made to first api via controller which uses the service to store information specific to that request. The controller's response returns the uuid to the user for reference.
A scheduler uses that same service to periodically check if the second call has been made or not and decides what to do based on the amount of time lapsed.
If a second call is made, it also uses the same service to save the relevant data and/or delete old records as necessary.
I would suggest you have to analyze your code and API calls first. Exactly where it is consuming more time like while fetching the data from DB(In this case you need to optimize your DB query)or processing the records from client/DB or etc..
You can use any monitoring tool present in the market, for the first analysis.You will get a idea of your API calls, method execution time, thread dump etc...
This link gives some tool names.
https://geekflare.com/api-monitoring-tools/
If you are using spring, then use the spring build library to get the metrics of the application.
Use the below link for more reference::
https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-features.html
I am designing a web service that wraps a very large data source and I would be very grateful for any suggestions whether my design is appropriate or I am missing something substantially better.
So here is the problem:
We have several data sources that all provide the same interface with the "most important" method being RowIterator select(Table table, String where). Now, functionally everything is going fine for all our implementations but the problem is that the web service that we need to wrap around one of the sources would (in a naive implementation) upon receiving a query
wait for the wrapped data source to return the whole result set
marshal the whole result set before sending it to the client
at the client side unmarshal the whole result set before returning it to the caller
Only after this sequence would the caller be able to see the first row. This is a quite disappointing behavior as the caller has to wait unnecessessarily for the whole result set twice. I want to have some pipelining, instead. The caller must be able to see the first results while the service is still sending rows. Now I am planning to overcome this by implementing some kind of paging that is encapsulated in my client-side row iterator. The service would maintain a session id (with a timeout) that is created upon receiving a query and can be used to fetch chunks of data. The session id could already be returned before sending the actual query to the wrapped data source. The client would then fetch chunks (pages) until a chunk is empty or smaller than the expected (= requested) chunk size.
So, in this design the caller would be able to see the first results while the service is still sending rows. However, I am wondering whether there is a way to efficiently pipeline results on a per-row basis using a SOAP web service?
Also, would it be possible to return the results to the caller without repeatedly asking for more results?
In the end I used MTOM to transmit the data in binary and used blocking queues at the client and the server to achieve the desired parallelism. I sketched this here: Streaming large SOAP attachments
I wanted to fire HQL queries from GWT client. But As it comes asynchronously, If I want to fire a sequence of queries, How do I do that?
Asynchronously all the data is transferred to client and used. But If I want to fire the queries in sequence how do I do that?
Mainly, the order of processing those HQL results should not be changed.
Let me know if you have any queries. Thanks in advance.
GWT Client will fire your queries in the order you have defined in client file i.e
yourClient.java
Query q1; //execute method first request
Query q2; // execute method second request
The Queries will run on sequence first q1 then q2 but q2 will not wait for the completion of q1. Once request send you can not assure about the order inwhich you will get response it might be possible that q2 will execute and return the response first while q1 is in progress.
Mostly operations for making sequence from client side is bit expensive. i.e you can use the queries on success method of AsyncCallback. When response of q1 will comes then q2 will be executed but if you have several queries then its an expensive solution.
So best thing is that make a Service method execute your all queries at server side and return map of result then populate it in your client side in any sequence you want to show the data.
Using Hibernate with GWT is not as simple as you may think. Google has a good documentation for the use of Hibernate. Did you already read that documentation?
I can recommend the use of Gilead. I'm not sure if this would solve your problem because I'm not exactly sure what your problem is.
GWT is asynchronous so you have to build everything based on request->callback
You can do two things in a case like this:
Have only a single RPC service that gathers the results for all queries and returns them. Remember that the services are plain java servlets
Have many RPC services that do one query each. Make the query to the first and in the callback you can display (or gather) whatever result and invoke the second query and in its callback do the same procedure... and so on.
Scenario Imagine a REST service that returns a list of things (e.g. notifications)
Usage A client will continually poll the REST service. The REST service retrieves records from the database. If records are available, they are converted into JSON and returned to the client. And at the same time, the retrieved records are purged from the DB.
Problem How do you handle the problem if the REST endpoints encounters a problem writing the results back to the client ? By that time, the records have been deleted.
Deleting the records will always be a dangerous proposition. What you could do instead is include a timestamp column on the data. Then have your REST url include a "new since" timestamp. You return all records from that timestamp on.
If the notifications grow to be too large you can always setup an automated task to purge records more than an hour old - or whatever interval works well for you.
It sounds like a strange idea to delete DB records after reading access. Possible problems immediately leap into mind: Network trouble prevent the client reading the data, multiple clients cause each other to see incomplete lists, et.al.
The RESTful apporach might be like this:
Give each notification a specific URI. Allow GET and DELETE on these URIs. The client may trigger the record deletion once it successfully received and processed the notification.
Provide an URI to the collection of current notifications. Serve a list of notification data (ID, URI, timestamp, (optional:) content) upon GET request. Take a look at the Atom protocol for ideas. Optional: Allow POST to add a new notification.
With this approach all reading requests stay simple GETs. You may instrument the usual HTTP caching mechanisms on proxies and clients for performance improvement.
Anyway: Deleting a DB entry is a state change on the server. You must not do this upon a GET request. So POST will be you primary choice. But this does not help you much, since the communication might still not be reliable. And polling qith POSTs smells a lot more like Web-Services than REST.
This could be a possible solution.
The REST service can query the database for a list of notifications.
It marks each item in the list ,by say setting a flag in the database.
It then delivers all this notification records to the client.
If the server has successfully sent the results to the client, all marked records are deleted.
In case of failure, the marked records are unmarked,so that they get delivered during the subsequent polling interval.
I hope you got my point.
We did this with special timestamp paramter.
Requests
Request with timestamp.min, this return all items, and server timestamp;
Request with timestamp from server, return items from hat time stamp, and delete prevoius, return server time stamp;
Please note that we did all this with post. means that virtually we sent command (not query get).