Develop Reference

How to constantly check if time is up for auction site

I am working on a project where we are creating an auction site that works on a weekly basis. The plan is to start all auctions on Monday and end all on Friday.
So far I have figured out that I need a database that holds the start and end date so I can check to see how much time left and so. But I need to be able to constantly check and see if the time is up or not and I do not know how to proceed. What is the proper way to do this?
We are using Java8 with Spring and react as frontend.

two solution:
Use websocket, server set a timer which due at Friday, and once timer expired, send the event to client.
client side do timer also.

You have 3 layers in play here:
Frontend (React)
Backend (Java8/Spring app)
Database
Now you need to figure out how to propagate data between those layers.
Propagating from backend to frontend can be done using either polling or websockets.
Propagating from database to backend can be done using either polling or database triggers.
I'd personally connect React with Spring App via websockets. Then I'd have some background task polling the database and pushing the discovered changes to connected websocket clients.
Here is Springs own tutorial on websockets https://spring.io/guides/gs/messaging-stomp-websocket/

I think you are looking for a pull model. Basically your Java application needs to pull the end date from database at certain intervals. You can write a cron job for that.Quartz cron is one of the popular Java based frameworks out there http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html . It handles distributed system also. So if your application is having multiple instances, Quartz can cover it for you.
Another variant in pull model, you can read the entry with end dates in JVM local cache or some other cache(Redis, Memcache) & run a cron on that. But you have to maintain cache consistency with database.
Which one you choose depends on your business use case(how frequently end date changes, how frequently you need to check for end dates etc.).
Other option was to go for push model. But push model won't work with traditional databases for this case.

Possible option - is to extend org.springframework.web.servlet.handler.HandlerInterceptorAdapter
And write logic of checking current time against your time range in this class - with throwing of an exception if check fails.
Potential optimization - cache values from DB (at least for some time, for example - 15 minutes - as it will help to decrease number of actual calls to the database.

Should I use AKKA for the periodical task

I have a terminal server monitor project. In the backend, I use the Spring MVC, MyBatis and PostgreSQL. Basically I query the session information from DB and send back to front-end and display it to users. But there is some large queries(like searching total users, total sessions, etc.), which slow down the system when user opens the website, So I want to do these queries as asynchronous tasks so the website could be opened fast rather than waiting for the query. Also, I would check terminal server state periodically from DB(every hour), and if terminal server fails or average load is too high, I would notifying admins. I do not know what should I use, maybe AKKA, or any other way to do these two jobs(1.do the large query asynchronously 2. do some periodical query)? Please help me, thanks!

You can achieve this using Spring and caching where necessary.
If the data you're displaying is not required to be "in real-time", but it can be "near real-time" you can read the data from the DB periodically and cache it. Your app then reads from the cache.
There's different approaches you can explore.
You can try to create a materialized view in PostgreSQL which will hold the statistic data you need. Depending on your requirements you have to see how to handle refresh intervals etc.
Another approach is to use application level cache - you can leverage Spring for that(Spring docs). You can populate the cache on start up and refresh it as necessary.
The task that runs every hour can be implemented again leveraging Spring (Spring docs) #Scheduled annotation.
To answer your question - don't use Akka - you have all the tools necessary to achieve the task in the Spring ecosystem.

Akka is not very relevant here, it is for event-driven programming model which deals with concurrency issues to build highly scalable multithreaded applications.
You can use Spring task scheduler for running heavy queries periodically. If you want to keep it simple, you can solve your problem by simply storing the data like total users, total sessions etc, in the global application context. And periodically update this data from database using spring scheduler. You can also store the same in a separate database table, so that this data can be easily loaded at the initialization time.
I really don't see why you need "memcached", "materialized views", "Websockets" and other heavy technologies and frameworks, for a caching a small set of data. All you need is maintain a set of global parameters in your application context, keep them updated using a scheduled task as frequently as desired.

Is Spring Integration suitable for web-farm processing of "reliable queue"?

Sorry if title is confusing, let me explain my question.
Our team need to develop web service which is suppose to run on several nodes (web farm - horizontal scaling). We know how to implement this "manually", but we're pretty excited about Spring Integration which is new to us - so we really trying to understand whether this is good fit for our scenario - and if so we'll try to make use of it.
Typical scenario:
Sevaral servers ("nodes") running same web application (lets call it "OurWebService")
We need to pull files from external systems ("InboundExtSystems")
Process this data with help of other external systems (involves local resource-consuming operations) ("UtilityExtServices")
Submit processing results to another set of external systems ("OutboundExtSystems")
Non-functional requirements:
Due to performance reasons we cannot query UtilityExtServices by demand -AND- local processing also CPU-intensive. So we need to have queue, in order to control pace at which we performing requests and process results
We expect several nodes will equally pull tasks from this queue and process them
We need to make sure that every queued task pulled from InboundExtSystems will be handled - we need to guarantee that none of them will disappear.
We need to make sure timeouts are handled as well. If task processing timed out - we need to "requeue" this task (and make sure previous handled will not submit results for this task)
We need to be able to perform rolling updates. Like let's say 5 nodes are processing queue. We want to be able to sequentially stop-upgrade-start each node without noticeably impacting system performance.
So question is: is spring integration perfect fit for such case?
If answer is "Yes", could you kindly name primary components we should use primarily?
p.s. Sure enough we would probably also need to pick something as a message bus and queue acessible by every node (maybe redis, hazelcast or maybe rabbitmq, not sure what is more appropriate)

Yes, it's a good fit. I would suggest rabbitmq for the transport/queuing and the Spring Integration AMQP enpoints.
Rolling updates shouldn't be an issue unless you change the format of the messages sent between nodes). But even then you could handle it relatively easily by moving to a new set of queues.

Combining java spring/thread and database access for time-critical web applications

I'm developing an MVC spring web app, and I would like to store the actions of my users (what they click on, etc.) in a database for offline analysis. Let's say an action is a tuple (long userId, long actionId, Date timestamp). I'm not specifically interested in the actions of my users, but I take this as an example.
I expect a lot of actions by a lot of (different) users par minutes (seconds). Hence the processing time is crucial.
In my current implementation, I've defined a datasource with a connection pool to store the actions in a database. I call a service from the request method of a controller, and this service calls a DAO which saves the action into the database.
This implementation is not efficient because it waits that the call from the controller and all the way down to the database is done to return the response to the user. Therefore I was thinking of wrapping this "action saving" into a thread, so that the response to the user is faster. The thread does not need to be finished to get the reponse.
I've no experience in these massive, concurrent and time-critical applications. So any feedback/comments would be very helpful.
Now my questions are:
How would you design such system?
would you implement a service and then wrap it into a thread called at every action?
What should I use?
I checked spring Batch, and this JobLauncher, but I'm not sure if it is the right thing for me.
What happen when there are concurrent accesses at the controller, the service, the DAO and the datasource level?
In more general terms, what are the best practices for designing such applications?
Thank you for your help!

Take a singleton object # apps level and update it with every user action.
This singleton object should have a Hashmap as generic, which should get refreshed periodically say after it reached a threshhold level of 10000 counts and save it to DB, as a spring batch.
Also, periodically, refresh it / clean it upto the last no.# of the records everytime it processed. We can also do a re-initialization of the singleton instance , weekly/ monthly. Remember, this might lead to an issue of updating the same in case, your apps is deployed into multiple JVM. So, you need to implement the clone not supported exception in singleton.

Here's what I did for that :
Used aspectJ to mark all the actions of the user I wanted to collect.
Then I sent this to log4j with an asynchronous dbAppender...
This lets you turn it on or off with log4j logging level.
works perfectly.

If you are interested in the actions your users take, you should be able to figure that out from the HTTP requests they send, so you might be better off logging the incoming requests in an Apache webserver that forwards to your application server. Putting a cluster of web servers in front of application servers is a typical practice (they're good for serving static content) and they are usually logging requests anyway. That way the logging will be fast, your application will not have to deal with it, and the biggest work will be writing a script to slurp the logs into a database where you can do analysis.

Typically it is considered bad form to spawn your own threads in a Java EE application.
A better approach would be to write to a local queue via JMS and then have a separate component, e.g., a message driven bean (pretty easy with EJB or Spring) which persists it to the database.
Another approach would be to just write to a log file and then have a process read the log file and write to the database once a day or whenever.
The things to consider are: -
How up-to-date do you need the information to be?
How critical is the information, can you lose some?
How reliable does the order need to be?
All of these will factor into how many threads you have processing your queue/log file, whether you need a persistent JMS queue and whether you should have the processing occur on a remote system to your main container.
Hope this answers your questions.

Can REST in practice really be stateless? [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
Consider the situation.
I am writing a statistical analysis app. The app has multiple tiers.
Frontend UI written for multiple device types, desktop, browser,
mobile.
Mid-tier servlet that offers a so-called REST service to
these frontend.
Backend that performs the extreme computation of the statistical
processing.
Which communicates with a further backend database
Due to the reason that statistical analysis requires huge amount of processing power, you would never dream of delegating such processing to the front-end.
The statistical analyses consists of procedures or a series of
work-flow steps.
Some steps may require so much processing power, you would not want
to repeat them.
If you have a work-flow of 20 steps, you cannot execute step 20
without first executing step 19, which cannot be executed without
first executing step 18, so on and so forth.
There are observation points, such that, for example, the
statistician must inspect results of steps 3, 7, 9, 14, 19 before
telling to client-side to proceed to the next step.
Each of these steps are a so-called request to the REST service, to
tell the backend supercomputer to progressively set up the
statistical model in memory.
There are many workflows. Some workflows may incidentally share step
results. e.g., Flow[dry]:Step[7] may share Flow[wet]:Step[10]. Due
to the amount of processing involved, we absolutely have prevent
repeating a step that might incidentally have already be
accomplished by another flow.
Therefore, you can see that in the so-called REST service being designed,
it is not possible that each request be independent of any previous request.
Therefore, how true can the following statement be?
All REST interactions are stateless. That is, each request contains
all of the information necessary for a connector to understand the
request, independent of any requests that may have preceded it.
Obviously, the application I described, requires that request be dependent on previous request. There are three possibilities that I can see concerning this app.
My app does not comply to REST, since it cannot comply to stateless requests. It may use the JAX-RS framework, but using JAX-RS and all the trappings of REST does not make it REST, simply because it fails the stateless criteria.
My app is badly designed - I should disregard trying to avoid the temporal and financial cost restacking up a statistical model even if it took 5 - 15 minutes for a workflow. Just make sure there is no dependence on previous requests. Repeat costly steps when necessary.
The stateless criteria is outdated. My understanding of REST is outdated/defective in that REST community have been constantly ignoring this criteria.
Is my app considered RESTful?
New Question: ISO 9000
Finally, in case my app is not completely considered RESTFul, would all references to "REST" need to be omitted to pass ISO 9000 certification?
new edit:
REST-in-piece
OK, my colleague and I have discussed this and decided to call such an architecture/pattern REST-in-piece = REST in piecemeal stages.

ISTM, you're reading too much into to statelessness. A REST API supports traditional CRUD operations. The API for CouchDB is good example of how DB state is updated by a series of stateless transactions.
Your task is to identify what the resources are and the "state transfers" between them. Each step in your workflow is a different state transfer, marked by a different URI. Each update/change to a resource has an accompanying POST/PATCH or an idempotent PUT or DELETE operation.
If you want to gain a better of understanding of what is means to be RESTful and the reasons behind each design choice, I recommend spending a hour reading Chapter 5 of Roy Fielding's Dissertation.
When making design choices, just think about what the principles of RESTful design are trying to accomplish. Setup your design so that queries are safe (don't change state) and that they are done in a ways that can be bookmarkable, cacheable, distributable, etc. Let each step in the workflow jump to a new state with a distinct URI so that a user can back-up, branch out different ways, etc. The whole idea is to create a scalable, flexible design.

You are updating an in memory model via a REST api. This means that you are maintaining state on the server between requests.
The REST-ful way of addressing this would be to make the client maintain the state by simply processing the request and returning all the information for constructing the next request in the response. The server then reconstructs the in memory model from the information in the request and then does its thing. That way, if you operate in a e.g. a clustered environment, any of the available servers would be able to handle the request.
Whether or not this is the most efficient way to do things depends on your application. There are loads of enterprise applications that use a server side session and elaborate load balancing to ensure that clients always use the same nodes in a cluster. So having server side state is an entirely valid design choice and there are plenty of ways of implementing this robustly. However, server side state generally complicates scaling out and REST in the purest sense is all about avoiding server side state and avoiding complexity.
A workaround/compromise is persisting the state in some kind of database or store. That way your nodes can fetch the state from disk before processing a request.
It all depends on what you need and what is acceptable for you. As the previous commenter mentioned, don't get too hung up on this whole stateful-ness thing. Clearly somebody will have to maintain state and the question is merely what the best place is to put that state for you and how you access it. Basically there are a couple of tradeoffs that basically have to do with various what-if type scenarios. For example, if the server crashes, do you want your client to re-run the entire set of requests to reconstruct the calculation or do you prefer to simply resend the last request? I can imagine that you don't really need high availability here and don't mind the low risk that something occasionally goes wrong for your clients. In that case, having the state on the server side in memory is an acceptable solution.
Assuming your server holds the computation state in some hash map, a REST-ful way of passing the state around then could be simply sending back the key for the model in the response. That's a perfectly REST-ful API and you can change the implementation to persist the state or do something else without changing the API when needed. And this is the main point of being REST-ful: decouple the implementation details from the API. Your client doesn't need to know where you put the state or how you store it. All it needs is a resource representation of that state that can be manipulated.
Of course the key should be represented as a URI. I recommend you read Jim Webber's "REST in practice". It's a great introduction to designing REST-ful APIs.