User-friendly migration of data between instances of an application - java

We have a web-based system that is essentially a survey application.
Up until now we've been loading new surveys in directly into the database (as flyway migrations). This is obviously not tenable long-term, and we need to put the power to create new surveys into the hands of the admin users. One aspect of this is adding the UI so that admin users can create new surveys.
However, we want the admin users to create and test any new surveys in a test environment first before letting it loose in production. As these surveys are complex, there is a real risk that a survey that is setup in a test environment is subtly different from how someone does the setup in the production environment. My aim is to remove this risk.
My question pertains to the technical mechanism(s) we could implement so that the admin users can, on a self-service basis, "migrate" surveys from their test environment (where they've done all of their testing and verification) to the production environment - such that we have a high confidence that the migrated survey(s) will be setup identically. Applications like Moodle already have this sort of functionality built-in.
I've considered:
Allowing users to create a checksum of a survey (perhaps based on the hashcode of all the elements) so they can know that the surveys across the two environments are identical. The issue then is that it would be annoying to try and find the exact difference between the surveys on the two instances.
Writing a UI that dumps raw SQL as the export and allowing these users to just splatting it into the production database, but it's incredibly bad from so many perspectives
Constructing our own domain-specific language that describes these surveys - but writing an input/output parser just seems so complex
Dumping the data to JSON (given that it's a RESTful web app, most of this is already done) and sucking it back in
At this stage, I'm leaning towards the JSON method, but would be keen to hear any other ideas or libraries that we could easily integrate into a Spring Boot app that would assist with this process.

To be honest, solving that kind of problem by introducing independent instances sounds like a terrible idea: Not only the migration part is tricky. You will have to ensure compatibility at all times. Imagine an admin uses an outdated version or does not configure his instance correctly (every admin's dream). Even if the migration itself produces no errors the results may be different in the end (and that's what you actually want to avoid).
As I understand it, you currently look at a survey as a bulk of raw information that somehow appears in your database and then gets displayed by the application. From that point of view you obviously can't tell the difference between a test survey and a production survey.
That's why you really should extend your model!
There is much more than just a survey. There is a survey draft and/or a survey preview area, a survey creation process that users go through, a process that spins of a production survey from a draft, etc.
If you add those concepts to your application a use case like the one you described should be easy to implement (you already mentioned a planned UI, right?).
Hope I could help you. Sorry, I know it doesn't exactly answer your question.

Related

JAVA Spring to MS Azure Logic App and Functions

I am exploring options how to convert my existing java application which has Sprint framework to Azure Cloud server less "Logic App" and "Functions" concept
without re-writing.
As far as i can see, there is no accurate information in microsoft websites.
Can any one please suggest me on the road map, that, how i must lift and shift my java spring frame work to Azure Function?
I am aware of the fact that there will be little modifications that i might need to do, that is ok with me. But not a complete re-write into some other language.
Thanks !
Hard to say exactly but with the new Java support as long any package you are using can be resolved with Maven it should work in Functions. The potential bigger question is what can remain as-is. Likely each of your controller methods would become Azure Functions - the method signature would change but the code inside should remain largely the same (HttpTrigger with a request message to a route). The models should be able to stay untouched. If you had any orchestration or workflows that would become orchestrated by Logic Apps - but since Logic Apps has no-code, it would mean re-creating that workflow/orchestration in Logic Apps.
Java is still in preview so you may hit some snags here and there, but let us know if you have any other questions along the way.

Designing a java GUI program with user access control

I want to make an application in java that has a GUI and a database. I will be using MYSQL as the database and my recent experience with JavaFX has been quite pleasant so that's what I'll be using to code.
My issue is I want user access control and want to show GUI components based on who or what type of user logs in.
So let us say it's a school management system and if a teacher logs in the teacher can access his or her course and homework assignments ( via GUI of course) but not any other teacher's. If an admin user logs in he/she can make changes to student profile, etc. but nothing else.
Could someone point me in the right direction with regards to concepts, tutorials, books, sample code, etc. so I can accomplish something like this
if(user==teacher){
stage.setscene(teacherOnlyScene);
stage.show();
}
Thank you so much! Apologies if its too subjective. This is the one site that I use like 90% of the time for answers to all my programming related questions.
The scope of your question is probably too large to be successfully answered here as you are asking about a fairly sizeable and potentially complex portion of the architecture of your final system. User access systems are generally quite complex especially if they are highly configurable e.g. you can grant users privileges outside their basic group privileges etc.
Before starting to write any code have a think about what you want to achieve. You've already come up with the idea of using user groups which I think is very sensible but there are some questions you need to ask. Can users belong to multiple groups? Can permissions be assigned to users or can they only be assigned to groups? Is the administrator a special case or can it be handled as just another group?
Have a look at the way Linux handles users and groups to assign permissions. It's a very well thought out system should offer some inspiration. Additionally the JavaEE 6 documentation orangegoat linked to provides a good overview of security in Java EE (but be warned it's fairly complex).
Having said that I feel I must encourage you to consider making this a web based application rather than using JavaFX. It doesn't sound like you have any particularly complex graphical requirements and it also sounds like you want multiple concurrent access to the system / database (e.g. more than one teacher can be checking their assignments at a time).
Perhaps I'm a little biased but I think a JavaEE based web application is a better match for these requirements. The business logic and most of the complexity of the multi-user side of things would be handled well by a bunch of EJB's with all the persistence handled by JPA. The GUI would be written in JSF and PrimeFaces and the whole lot hosted on GlassFish. The real benefit of this set up is that you have a zero installation footprint which will save you a ton of time when rolling out new versions. Going the JavaEE route will probably make it more difficult for you but I feel it's the correct solution given the requirements.

Implementing a Dynamic Award System

I have been developing a Online Poker Game. But I keep hitting a wall. I want to implement Awards into the system, but I want them to be dynamic. Meaning I don't want to recompile for every award I would like to add.
I have thought about using Python code for each award. Then when the server checks to see if the user qualifies for the award it runs the python script with Jython (server is in Java and Netty NIO) and if the function returns a certain value I award the award to the user. Which could work but is there maybe a more efficient technique out there that will not force me to run hundreds of python scripts each time I need to check if a user got a award.
And when are the best times to do these checks? I have tought about a hook system where I will specify the hooks like ( [onconnect][ondisconnect][chatmessage.received] ). Which also could work but feels a bit crude and I will still have to run all the scripts from the database.
If I were you, I'd have a totally separate process that grants awards. It runs perhaps once a day on the underlying database that contains all your player/game data.
Your core customer-facing app knows about awards, but all it knows about them is data it loads from the DB -- something like a title, image, description, maybe how many people have the award, etc., and (based on DB tables) who has won the award.
Your "award granter" process simply runs in batch mode, once per day / hour etc, and grants new awards to eligible players. Then the core customer-facing app notifies them but doesn't actually have to know the smarts of how to grant them. This gives you the freedom to recompile and re-run your award granter any time you want with no core app impact.
Another approach, depending on how constrained your awards are, would be to write a simple rules interface that allows you to define rules in data. That would be ideal to achieve what you describe, but it's quite a bit of work for not much reward, in my opinion.
PS -- in running something like an online poker server, you're going to run into versions of this problem all the time. You are absolutely going to need to develop a way to deploy new code without killing your service or having a downtime window. Working around a java-centric code solution for awards is not going to solve that problem for you in the long run. You should look into the literature on running true 24/7 services, there are quite a few ways to address the issue and it's actually not that difficult these days.
There are a number of options I can think of:
OSGi as described above - it comes at a cost, but is probably the most generic and dynamic solution out there
If you're open to restart (just not recompile), a collection of jars in a well known folder and spring give you a cheaper but equally generic solution. Just have your award beans implement a standard interface, be beans, and let spring figure #Autowire all the available awards into your checker.
If you award execution is fairly standard, and the only variation between awards are the rules themselves, you can have some kind of scripted configuration. Many options there, from the python you described (except I'd go for a few big script managing all awards), to basic regular expressions, with LUA and Drools in the middle. In all cases you're looking at some kind of rules engine architecture, which is flexible in term of what the award can trigger on but doesn't offer much flexibility in term of what an award can lead to (i.e. perfect for achievements).
Some comments to the answer with batch ideas:
Implementing a Dynamic Award System
That batch processes can be on separate server/machine, so you can recompile the app or restart the server at any time. Having that new awards can be handled using for example the mentioned approach with adding jars and restarting the server, also new batch jobs can be introduced at any time and so on. So your core application is running 99% of the time, batch server can be restarted frequently. So separate batch machines is good to have.
When you need to deploy new version of core app I think you can just stop, deploy and start it with maintenance notice to users. That approach is used even by top poker rooms with great software (e.g. FullTiltPoker did so, right now it is down due to the license loss, but their site says 'System Update' :) ).
So one approach for versions update is to redeploy/restart in off-hours.
Another approach is real time updates. As a rule it is done by migrating users bunch by bunch to new version. So at the same time some users are using old version, some - new. Not pretty cool for poker software were users with different versions can interact. But if you are sure in the versions 'compatibility' you can go with that approach, checking user's client version for example on login.
In my answer I tried to say that you need not introduce 24/7 support logic to your code. Leave that system availability problems for hardware (failovers, loadbalancers, etc.). You can follow any good techniques you used to write code, only you need to remember that your crucial core logic is deployed not frequently (for example once a week), and batch part can be updated/restarted at any time if needed.
As far as I understand you, you probably do not have to run external processes from your application nor use OSGI.
Just create a simple Java interface and implement each plugin ('award') as a class implementing the interface. You could then just simply compile any new plugin and load it in as a class file from your application at run-time using Class.forName(String className).
Any logic you need from such a plugin would be contained in methods on the interface.
http://download.oracle.com/javase/1,5.0/docs/api/java/lang/Class.html#forName(java.lang.String)

Considering Portlet Architecture for Project - Is this reasonable?

I have a new application to write, and the timeline is tight - I'd like to overview the requirements and why I'm considering Portlets over Servlets; however, I have no experience with Portlets and therefore would like someone who is familiar to let me know if I'm missing anything, or if I'm doomed to spend too much time learning the new technology.
Project Requirements Overview :
I need to build a system that allows clients to build simple websites through our own UI (the building tools must be completely customizable as I'm working with a talented design team) - think of a "easy as 1,2,3" page builder where you specify templates, content, etc. Of course we also need to be prepared to handle special requests where clients or internal people can tweak the pages. Also, we will offer special components and pages that interact with our back-end system.
Each client website must be linked to a separate domain
I should be able to create multiple interfaces for editing pages and content - so that more advanced users are able to make more advanced changes
A web designer without programming skills can create templates for the look and feel of the sites - while developers can then add the dynamic functionality
Why I'm Considering Portlets :
As I drew up extravagant designs, the one thing I keep coming to is realizing, this is going to be difficult to deliver quickly, and the quicker I deliver, the more likely I am to have coded myself into a corner. So I wanted to be able to manage aspects of the application separately, but just don't have the time to do so (since it would be entirely on me to create that system). Also, in order to keep on time, I will need to delegate parts of the project carefully, if its totally home built I need to be REALLY careful what I delegate. Then I found out about "liferay" which lead me to investigate portlets.
So basically the Portlet architecture doesn't seem to prohibit anything I need, plus we can add all our special needs as hot deployed portlets - pages can be created as HTML and enhanced with Velocity - and if some requirements change, there is a standard I've adhered to and therefore can make changes to configuration. I may even want to use liferay as it does nearly everything I want (although I would need to provide a much simpler page editing interface).
Conclusion :
So that's the project requirements and why I'm thinking about going for portlets, but I'm not sure if it's really a fit, or if it just SEEMS like a fit, does it seem reasonable to those of your familiar with Portlets?
There's nothing you mention that speaks against adopting portlets - quite contrary: Sounds like a good fit.
The only tricky part might be that you don't have any portlet experience, so estimation will likely be a bit off while you learn this environment, but this would apply to every environment that you don't know yet. Compared with a servlet approach you'll start with quite a lot of functionality out of the box, so it sounds completely reasonable. And you don't even need to maintain this functionality yourself...
If you can you might want to consider getting trained in the environment of your choice to get to speed quickly. (You tagged the question with "liferay" and Liferay offers training all over the world - Disclaimer: I work for Liferay)

What is the best way to migrate an existing messy webapp to elegant MVC? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I joined a new company about a month ago. The company is rather small in size and has pretty strong "start-up" feel to it. I'm working as a Java developer on a team of 3 others. The company primarily sells a service to for businesses/business-type people to use in communicating with each other.
One of the main things I have been, and will be working on, is the main website for the company - from which the service is sold, existing users login to check their service and pay their bills, new users can sign up for a trial, etc. Currently this is a JSP application deployed on Tomcat, with access to a database done thru a persistence layer written by the company itself.
A repeated and growing frustration I am having here (and I'm pretty happy with the job overall, so this isn't an "oh no I don't like my job"-type post) is the lack of any larger design or architecture for this web application. The app is made up of several dozen JSP pages, with almost no logic existing in Servlets or Beans or any other sort of framework. Many of the JSP pages are thousands of lines of code, they jsp:include other JSP pages, business logic is mixed in with the HTML, frequently used snippets of code (such as obtaining a web service connection) is cut and paste rather than reused, etc. In other words, the application is a mess.
There have been some rumblings within the company of trying to re-architect this site so that it fits MVC better; I think that the developers and higher-ups are beginning to realize that this current pattern of spaghetti code isn't sustainable or very easily scalable to add more features for the users. The higher-ups and developers are wary of completely re-writing the thing (with good reason, since this would mean several weeks or months of work re-writing existing functionality), but we've had some discussions of (slowly) re-writing certain areas of the site into a new framework.
What are some of the best strategies to enable moving the application and codebase into this direction? How can I as a developer really help move this along, and quickly, without seeming like the jerk-y new guy who comes into a job and tells everyone that what they've written is crap? Are there any proven strategies or experiences that you've used in your own job experience when you've encountered this sort of thing?
Your best bet is probably to refactor it slowly as you go along. Few us of have the resources that would be required to completely start from scratch with something that has so many business rules buried in it. Management really hates it when you spend months on developing an app that has more bugs than the one you replaced.
If you have the opportunity to build any separate apps from scratch, use all of the best practices there and use it to demonstrate how effective they are. When you can, incorporate those ideas gradually into the old application.
First pick up a copy of Michael Feather's Working Effectively with Legacy Code. Then identify how best to test the existing code. The worst case is that you are stuck with just some high level regression tests (or nothing at all) and If you are lucky there will be unit tests. Then it is a case of slow steady refactoring hopefully while adding new business functionality at the same time.
In my experience, the "elegance" of an app typically has more to do with the database design than anything. If you have a great database design, including a well-defined stored procedure interface, good application code tends to follow no matter what platform you use. If you have poor database design, no matter what platform you use you'll have a very difficult time building elegant application code, as you'll be constantly compensating for the database.
Of course, there's plenty of room between great and poor, but my point is that if you want good application code, start by making sure your database is up to snuff.
This is harder to do in applications that are only in maintenance mode because it is hard to convince management that rewriting something that is already 'working' is worth doing. I would start by applying the principles of MVC to any new code that you are able to work on (ie. move business logic to something akin to a model, put all your layout/view code in one place)
As you gain experience with new code in MVC you can start seeing opportunities to change the existing code subtly so that it also falls in line. It can be a very slow process, but if you can show the benefits of this way of doing it then you will be able to convince others and get the entire team on board.
I would agree with the slow refactoring approach; for example, take that copy-and-pasted code and extract it into whatever the appropriate Java paradigm is (a class, perhaps? or better yet, use an existing library?). When your code is really clean and terse, yet still lacking in overall architectural strategy, then you'll be able to make things fit into an overall architecture much more easily.
Iteratively refactor. Also look for new features that can be done completely in the new framework as a way to show the value of the new framework.
My suggestion would be to find the rare pages that do not require an import of other JSPs, or have very few imports. Treat each JSP imported as a black box, and refactor these pages around them (iteratively, testing each change and making sure it works before continuing). Once these are cleaned up, you can continue finding pages with more and more imports, until finally you have refactored the imports.
When refactoring, note the parts that try to access resources not given to the page and try to take this out to a controller. For instance, anything that accesses the database should be inside a controller, let the JSP handle the display of the information the controller gives to it via a forward. In this way you will develop several servlets, or things like servlets, for each page. I would suggest using a front-controller based framework for this refactoring (from personal experience I recommend Spring and its Controller interface), so that each controller isn't a separate Servlet but is rather delegated to from a single servlet that is mapped appropriately.
For the controller, it is better to do database hits all at once rather than attempt them piecemeal. Users can and generally do tolerate a page load, but the page output will be much faster if all the database data is given to the rendering code rather than the rendering code hanging and not giving data to the client while it is trying to read yet another piece of data from the database.
I feel your pain and wish you luck in this endeavor. Now, when you have to maintain an application that abuses Spring Webflow, that's another story :)
The best way is to print out the code, crumple it up, and throw it out. Don't even recycle the paper.
You've got an application that is written in 1,000+ line long JSPs. It probably has a god-awful Domain Model (if it even has one at all) and doesn't just MIX presentation with business logic, it BLENDS it and sits there and keeps stirring for hours. There is no way to take out the code that is crappy and move into an MVC Controller class and still be doing the right thing, you'll just end up with a MVC app with an anemic domain model or one that has stuff like Database calls in Controller code, you're still failing.
You could try a new app that does the right thing and then have the two apps talk to each other, but that's new complexity in itself. Also you're likely going to be doing the same amount of work that you were going to do if you just started from scratch, but you might have an easier time trying to convince your bosses that this is a better approach.

Categories

Resources