Related
I have just been asked by my company to rewrite a largish (50,000 single lines of code) Java application (a web app using JSP and servlets) in Clojure. Has anyone else got tips as to what I should watch out for?
Please bear in mind that I know both Java AND Clojure quite well.
Update
I did the rewrite and it went into production. It's quite strange as the rewrite ended up going so fast that it was done in about 6 weeks. Because a lot of functionality wasn't needed still it ended up more like 3000 lines of Clojure.
I hear they are happy with the system and its doing exactly what they wanted. The only downside is that the guy maintaining the system had to learn Clojure from scratch, and he was dragged into it kicking and screaming. I did get a call from him the other day saying he loved Lisp now though.. funny :)
Also, I should give a good mention to Vaadin. Using Vaadin probably accounted for as much of the time saved and shortness of the code as Clojure did.. Vaadin is still the top web framework I have ever used, although now I'm learning ClojureScript in anger! (Note that both Vaadin and ClojureScript use Google's GUI frameworks underneath the hood.)
The biggest "translational issue" will probably be going from a Java / OOP methodology to a Clojure / functional programming paradigm.
In particular, instead of having mutable state within objects, the "Clojure way" is to clearly separate out mutable state and develop pure (side-effect free) functions. You probably know all this already :-)
Anyway, this philosophy tends to lead towards something of a "bottom up" development style where you focus the initial efforts on building the right set of tools to solve your problem, then finally plug them together at the end. This might look something like this
Identify key data structures and transform them to immutable Clojure map or record definitions. Don't be afraid to nest lots of immutable maps - they are very efficient thanks to Clojure's persistent data structures. Worth watching this video to learn more.
Develop small libraries of pure, business logic oriented functions that operate on these immutable structures (e.g. "add an item to shopping cart"). You don't need to do all of these at once since it is easy to add more later, but it helps to do a few early on to facilitate testing and prove that your data structures are working..... either way at this point you can actually start writing useful stuff interactively at the REPL
Separately develop data access routines that can persist these structures to/from the database or network or legacy Java code as needed. The reason to keep this very separate is that you don't want persistence logic tied up with your "business logic" functions. You might want to look at ClojureQL for this, though it's also pretty easy to wrap any Java persistence code that you like.
Write unit tests (e.g. with clojure.test) that cover all the above. This is especially important in a dynamic language like Clojure since a) you don't have as much of a safety net from static type checking and b) it helps to be sure that your lower level constructs are working well before you build too much on top of them
Decide how you want to use Clojure's reference types (vars, refs, agents and atoms) to manage each part mutable application-level state. They all work in a similar way but have different transactional/concurrency semantics depending on what you are trying to do. Refs are probably going to be your default choice - they allow you to implement "normal" STM transactional behaviour by wrapping any code in a (dosync ...) block.
Select the right overall web framework - Clojure has quite a few already but I'd strongly recommend Ring - see this excellent video "One Ring To Bind Them" plus either Fleet or Enlive or Hiccup depending on your templating philosophy. Then use this to write your presentation layer (with functions like "translate this shopping cart into an appropriate HTML fragment")
Finally, write your application using the above tools. If you've done the above steps properly, then this will actually be the easy bit because you will be able to build the entire application by appropriate composition of the various components with very little boilerplate.
This is roughly the sequence that I would attack the problem since it broadly represents the order of dependencies in your code, and hence is suitable for a "bottom up" development effort. Though of course in good agile / iterative style you'd probably find yourself pushing forward early to a demonstrable end product and then jumping back to earlier steps quite frequently to extend functionality or refactor as needed.
p.s. If you do follow the above approach, I'd be fascinated to hear how many lines of Clojure it takes to match the functionality of 50,000 lines of Java
Update: Since this post was originally written a couple of extra tools/libraries have emerged that are in the "must check out" category:
Noir - web framework that builds on top of Ring.
Korma - a very nice DSL for accessing SQL databases.
What aspects of Java does your current project include? Logging, Database transactions, Declarative transactions/EJB, web layer (you mentioned JSP, servlets) etc. I have noticed the Clojure eco-system has various micro-frameworks and libraries with a goal to do one task, and do it well. I'd suggest evaluate libraries based on your need (and whether it would scale in large projects) and make an informed decision. (Disclaimer: I am the author of bitumenframework) Another thing to note is the build process - if you need a complex setup (dev, testing, staging, prod) you may have to split the project into modules and have the build process scripted for ease.
I found the most difficult part was thinking about the database. Do some tests to find the right tools you want to use there.
We have an agile enterprise application built on JSP and Servlet without any design strategy.
This application was built in early 2002 considering 1000 users. After 2002, we received lots of requests from the marketing partners.
Currently, the application has lots of spaghetti code with lots of Ifs and elses. One class has more than 20,000 lines of code with a huge body of functions without abstraction.
Now, we need to support billions of records,
what we need to do immediately and gradually?
We have to refactor the application?
Which framework, we need to use?
How the usage of the framework will be helpful to the end users?
How to convince the leaders to do the refactoring?
How to gain the faster response time as compare to the current system?
Here is how I would approach this if I had appropriate company resources at my disposal (yeah right):
Get a good QA process going, with automated regression testing set up before making significant changes. I don't care how good you are, you can't put a system like that under unit test and reasonably control for regressions.
Map out interdependencies, see how much an individual class can be tested as a unit.
How do you eat an elephant? One bite at a time. Take a given piece of required functionality (preferably something around the increase load requirements) and refactor the parts of the class or classes that can be worked on in isolation.
Learn how do 3 above by reading Working Effectively with Legacy Code.
Convenient way to refactor the application.
There are no "convenient" or "easy" ways to refactor an existing codebase, especially if the codebase looks like spaghetti.
... what we need to do immediately and gradually?
That's impossible to answer without understanding your system's current architecture.
We have to refactor the application?
On the one hand, the fact that you have a lot of poorly designed / maintained code would suggest that it needs some refactoring work.
However, it is not clear that it will be sufficient. It could be that a complete rewrite would be a better idea ... especially if you need to scale up by many orders of magnitude.
Which framework, we need to use?
Impossible to answer without details for your application.
How the usage of the framework will be helpful to the end users?
It might reduce response times. It might improve reliability. It might allow more online users simultaneously. It might do none of the above.
Using a framework won't magically fix a problem of bad design.
How to convince the leaders to do the refactoring?
You need to convince them that the project is going to give a good return on investment (ROV). You / they also need to consider the alternatives:
what happens if you / they do nothing, or
is a complete rewrite likely to give a better outcome.
How to gain the faster response time as compare to the current system?
Impossible to answer without understanding why the current system is slow.
The bottom line is that you probably need someone from outside your immediate group (e.g. an external consultant) to do a detailed review your current system and report on your options for fixing it. It sounds like your management don't trust your recommendations.
These are big, big questions. Too broad for one answer really.
My best advice is this: start small, if you can. Refactor piece by piece. And most importantly, before touching the code, write automated tests against the current codebase, so you can be relatively sure you haven't broken anything when you do refactor it.
It may not be possible to write these tests, as the code may not be testable in it's current format. But you should still make this one of your main goals.
By definition refactoring shouldn't show any difference to the users. It's only to help developers work on the code. It sounds like you want to do a more extensive rewrite to modernize the application. Moving to something like JSF will make life a lot easier for developers and will give you access to web component libraries to improve the user experience.
It is a question which needs a lengthy answer. To start with I would suggest that the application is tested well and is working as per the specification. This means there are enough unit, integration and functional tests. The functional tests also have to be automated. Once these are in place, a step by step refactoring can take place. Do you have enough tests to start?
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I find the nature of this question to be quite suited for the practical-minded people on Stack Overflow.
I'm setting out on making a rather large-scale project in Java. I'm not going to go into specifics, but it is going to involve data management, parsing of heterogeneous formats, and need to have an appealing interface with editor semantics. I'm a undergraduate student and think it would evolve into a good project to show my skill for employment -- heck, ideally it would even be the grounds for a startup.
I write to ask you what shortcuts I might not be thinking about that will help with a complicated project in Java. Of course, I am planning to go at it in Eclipse, and will probably use SWT for the GUI. However, I know that Java has the unfortunately quality of overcomplicating everything, and I don't want to get stuck.
Before you tell me that I want to do it in Python, or the like, I just want to reiterate why I would choose Java:
Lots more experience with algorithms in Java, and there will be quite a bit of those.
Want a huge library of APIs to expand functionality. ANTLR, databases, libraries for dealing with certain formats
Want to run it anywhere with decent performance
I am open-minded to all technologies (most familiar with Java, perl, sql, a little functional).
EDIT: For the moment, I am giving it to djna (although the low votes). I think all of your answers are definitely helpful in some respect.
I think djna hit better the stuff I need to watch out for as novice programmer, recognizing that I'm not taking shortcuts but rather trying not to mess up. As for the suggestions of large frameworks, esp. J2EE, that is way too much in this situation. I am trying to offer the simplest solution and one in which my API can be extended by someone who is not a J2EE/JDBC expert.
Thanks for bringing up Apache Commons, although I was already aware. Still confused over SWT vs. Swing, but every Swing program I've used has been butt ugly. As I alluded to in the post, I am going to want to focus most on the file interchange and limited DB features that I have to implement myself (but will be cautious -- I am aware of the concurrency and ACID problems).
Still a community wiki to improve.
Learn/use Spring, and create your project so it will run without a Spring config (those things tend to run out of control), while retaining the possibility to configure the parameters of your environment in a later stage. You also get a uniform way to integrate other frameworks like Hibernate, Quartz, ...
And, in my experience, stay away from the GUI builders. It may seem like a good deal, but I like to retain control of my code all the time.
Google-collections:
http://code.google.com/p/google-collections/
Joda Time for all date and time manipulations.
One of Hibernate or iBATIS to manipulate data in a database
Don't forget the IDE: Eclipse, Netbeans or IDEA if you have some cash and like it
Apache Commons has a lot of time saving code that will you most likely need and can reuse.
http://commons.apache.org/
A sensible build and configuration platform will help you along the way, Ant:
http://ant.apache.org/
or Maven (preferably):
http://maven.apache.org/
Especially as the size of the project, and the number of modules in the project increase.
There's getting the "project" completed efficiently and there are "short cuts". I suspect the following may fall into the "avoiding wasted effort" category rather be truly short cuts but if any of them get you to then end more quickly then I perhaps they help.
1). Decomposition and separation of concerns. You've already identified high-level chunks (UI, persistence layer, parser etc.). Define the interfaces for the provider classes as soon as possible and have all dependent classes work against those interfaces. Pay a lot of attention to the usability of those interfaces, make them easy to understand - names matter. Even something as simple as the difference between setShutdownFlag(true) and requestShutdown(), the second has explicit meaning and hence I prefer it.
Benefits: Easier maintenance. Even during initial development "maintenance" happens. You will want to change code you've already written. Make it easy to get that right by strong encapsulation.
2). Expect iterative development, to be refining and redesigning. Don't expect to "finish" any one component in isolation before you use it. In other words don't take a purely bottom up approach to developing your componenets. As you use them you find out more information, as you implement them you find out more about what's possible.
So enable development of higher level components especially the UI by mocking and stubbing low level components. Something such as JMock is a short-cut.
3). Test early, test often. Use JUnit (or equivalent). You've got mocks for your low level components so you can test them.
Subjectively, I feel that I write better code when I've got a "test hat" on.
4). Define your error handling strategy up front. Produce good diagnostics when errors are detected.
Benefits: Much easier to get the bugs out.
5). Following on from error handling - use diagostic debugging statements. Sprinkle them liberally throughout your code. Don't use System.out.println(), instead use the debugging facilities of your logging library - use java.util.logging. Interactive debuggers are useful but not the only way of analysing problems.
Don't discount Swing for the GUI. There are a lot of good 3rd party Swing libraries available (open source and commercial) depending on your needs e.g.
JGoodies Form Layout
SwingX
JFreeChart
Use logging framework (i.e. Log4J) and plan for it. Reliable logging system saves a lot of time during bug fixing.
For me big time-savers are:
use version-control (Subversion/Git)
use automatic builds (Ant/Make/Maven)
use Continuous-integration (Hudson/CruiseControl)
the apache-commons-libraries are very useful
Test Driven Development
Figure out the functionality you want and write tests to demonstrate the functionality. Then write just enough code to pass the tests. This will have some great side effects:
By writing just enough code to satisfy your requirements you will not be tempted to overbuild, which should reduce complexity and make your code cleaner.
Having tests will allow you to "refactor with confidence" and make changes to the code knowing you're not breaking another part of the system.
You're building quality in. Not only will you have assurance that you code "works, but if you really want to use this code as a sort of "resume" for potential employers, this will show them that you place a lot of value on code quality. This alone will set you apart from the majority of the developers out there.
Aside from that, I would agree that other big time savers are Spring, having an automated build (Maven), and using some form of source control.
For data persistency, if you're not certain that you must use SQL, you should take a look at alternate data-storage libraries:
Prevayler: an in memory database systems, using developing ideas like "crash only components", that make it easy to use and very performatic. May be used even for simple applications where you just need to save some state. It has a BSD License.
BerkleyDB Java Edition: a storage system with different layers of abstraction, so one can use it as a basic key-value storage (almost an in-disk hashtable) to a fully transactional ACID system. Take a look at it's licensing information because even some commercial software may use it for free.
Be aware that they have trade-offs when compared with each other or with a standard SQL database! And, of course, there may be other similar options around, but these are the ones I know and like. :)
PS: I was not allowed to post the respective links because I'm a new user. Just google for them; first hits are the right ones.
You can save time between restarts by using JavaRebel (commercial). Briefly, this tool allows you to write your code in Eclipse and have the code changes picked up instantly.
Spring Roo can not only get your project quickly kickstarted (using a sound architecture) but also reduce your ongoing maintenance. If you're thinking of using Spring, servlets, and/or JPA, you should definitely consider it. It's also easy to layer on things like security.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I'm so tired of having to learn yet another Java web framework every other day.
JSP, Struts, Wicket, JSF, JBoss Seam, Spring MVC to name just a few - all this countless frameworks out there try to address the same issues. However, none of them really solves the fundamental problems - that's why there are still coming up more and more new ones all the time.
Most do look very bright and shiny on the first impression because they simplify doing simple things.
But as soon as it comes to the implementation of a real world use case one is running into problems.
Often the frameworks don't provide any help but are hindering one and limiting the options by forcing things to be implemented according to the frameworks own logic and environment.
In short, I see the following disadvantages when using a framework:
There mostly is a steep learning curve and you first need to understand sometimes quite academic concepts and know meaning and location of a bunch of configuration files before you can get started.
The documentation usually is more or less awful, either missing a public accessible online reference, is helpless outdated, confuses different incompatible versions or all of this together and often doesn't provide any helpful examples.
The framework consist of zillions of classes which makes it practically impossible to understand the intended use only by browsing the sources.
Therefore you need to buy some "XYZ in action for dummies in 21 days" kind of books which have a bad user interface because they are missing a full text search and are heavy to carry around.
To really use one of this frameworks you need to learn by heart how things can be done the way the framework requires it by remembering the adequate classes and method names until your head is full of stupid and useless information you can't use for anything else.
There is a big overhead, slowing down your applications performance and making your brain feeling numb when try to understand what really is going on.
In the real world there is usually no time to get well familiar with something new because of the pressure of being productive. As a consequence of this learning by doing approach one always looks only for the fastest way to get the next task done rather than really understanding the new tool and it's possibilities.
The argument that following a standard would allow people who are new to a project to quickly get started is not valid in my view because every project uses a different framework even within the same company (at least in my case).
It seems to me that the following quote from Albert Einstein fits here very well:
“We can't solve problems by using the same kind of thinking we used when we created them.”
Back in my good old PHP coding days when coding still was fun and productive, I used to write my own frameworks for most things and just copy-pasted and adopted them from one project to the next.
This approach paid out very well, resulting in fast development, no overhead at all and a framework which actually was mightier than most Java frameworks out there but with only a few hundred lines of code in a single file plus some simple mod_rewrite rules.
This certainly wasn't solving all problems of web development, but it was simple, fast and straight to the point.
While perfectly adjusted to the requirements of the current project, it also was easy expandable and had a very high performance due to zero overhead.
So why all that hassle with using this frameworks, why not throwing them all away and going back to the roots?
What should I say to my boss when we're starting tomorrow the next project with a new framework again?
Or are there maybe frameworks which really make a difference?
Or some hidden advantages I have ignored?
Back in my good old PHP coding days
when coding still was fun and
productive, I used to write my own
frameworks for most things and just
copy-pasted and adopted them from one
project to the next. This approach
paid out very well, resulting in fast
development, no overhead at all and a
framework which actually was mightier
than most Java frameworks out there
Forgive me for believing that not one second.
but with only a few hundred lines of
code in a single file plus some simple
mod_rewrite rules. This certainly
wasn't solving all problems of web
development, but it was simple, fast
and straight to the point.
So basically you developed your own framework over the course of months or years, tailored to your own needs, and could work very fast with it because you knew it intimately.
And yet you can't understand why others do the same and then try to turn the result into something useable by everyone?
Where's this great framework you developed? If it's so powerful and easy to use, where are the dedicated communities, thousands of users and hundreds of sites developed with it?
every project uses a different
framework even within the same company
(at least in my case)
Well, that's your problem right there. Why would you throw away the expertise gained with each framework after each project?
The idea is to choose one framework and stick with it over multiple projects so that you get proficient in it. You have to invest some time to learn the framework, and then it saves you time by allowing you to work on a higher level.
The problem with coming up with your own framework is that you will make all of the same mistakes that all of the established frameworks have already stumbled on and addressed. This is true particularly when it comes to security.
Just ask Jeff and the guys about what they had to consider when implementing the WMD in stack overflow. I'd rather use what they have produced in a project rather than implement it from scratch. That is just one example.
Here is a quote from Kev from the thread What’s your most controversial programming opinion? which fit's in here really well:
I think that the whole "Enterprise" frameworks thing is smoke and mirrors. J2EE, .NET, the majority of the Apache frameworks and most abstractions to manage such things create far more complexity than they solve.
Take any regular Java or .NET OMR, or any supposedly modern MVC framework for either which does "magic" to solve tedious, simple tasks. You end up writing huge amounts of ugly XML boilerplate that is difficult to validate and write quickly. You have massive APIs where half of those are just to integrate the work of the other APIs, interfaces that are impossible to recycle, and abstract classes that are needed only to overcome the inflexibility of Java and C#. We simply don't need most of that.
How about all the different application servers with their own darned descriptor syntax, the overly complex database and groupware products?
The point of this is not that complexity==bad, it's that unnecessary complexity==bad. I've worked in massive enterprise installations where some of it was necessary, but even in most cases a few home-grown scripts and a simple web frontend is all that's needed to solve most use cases.
I'd try to replace all of these enterprisey apps with simple web frameworks, open source DBs, and trivial programming constructs.
The problem is of course not just with Java frameworks. I've lost count of the number of C++ MFC projects I've seen floundering around trying to shoe-horn their requirements into the Document/View model (which really only workks for text and graphic editors - database applications are particularly difficult to shoehorn).
The secret of succesful framework use is to change your application to match the framework, not the other way around. If you can't do that, don't even think of using the framework - it will end up being more work than if you had written the app from scratch with the help of some good, reliable and well-documented utility libraries.
So are you saying we should deal in sockets and HTTP every time we want to build a web application!
The servlet container itself can be considered a framework, since it handles all these messy details, and leaves you to write much simpler Servlets/Filters/Listeners (ie: 'extensions' of the framework specific to your application).
All any framework tries to do is separate mundane, repeatable, error-prone, legwork code from the fun application-specific code.
However, for a small application, you can get away with simply having a Model 2 MVC approach which uses only JSPs and Servlets.
Example:
class MyController extends HttpServlet {
public void doGet(HttpServletRequest request, HttpServletResponse response) throws ... {
MyBean model = // do something
request.setAttribute("model", model);
request.getRequestDispatcher("/view.jsp").forward(request, response);
}
}
Then as your app becomes more sophisticated, you could look at using Spring MVC to provide looser coupling (and hence more flexible) of controllers, view resolvers etc..
I share your pain when confronted with yet another framework that doesn't do the trick.
Having survived ten years of jsp, struts, EJB, EJB2, struts2, jsf and now more recently all the new web services framworks, the xslt horrors and wsdl-first nightmares, I am definitely fed up.
There is a number of problems with frameworks. They leak so you have to learn more -not less, inhouse frameworks have huge costs, using external frameworks costs too (but much less), as they seldom deliver, and then you end up writing enourmus chunks of xml-configuration and spend days correcting case and spelling errors that you had seen immediately in your favorite content helping code editor.
Maybe the answer is to find less pompous toolkits that tries to solve a problem but not redefine the world, but that is hard too as the fundamental application model (html over http) is awkward - at best.
Add the fact that there seems to be a lot of complicators around, people that seems to be obsessed of trading boring simple problems to complex (but hard) interresting problems ( maybe a variant of Eric Sink’s Axiom of Software Development mentioned above.)
Add the hubris of developers that knows it all and do not hesitate to write a new framework to solve all the hard problems for you, Only that they can't, leaving 10% left, only much harder to fix now.
I have no .NET experience, but the .NET world seems to be less crowded with theorists and complicators, and maybe the lingering stink of VB is scaring them off, but everytime I hear someone telling me that they have spent 1500 hours on their maven config (hello?), I am seriously considering deleting "java" from my resume.
...what was the question again? Are there any frameworks that make a difference?
EDIT - added Stripes and QueryDSL.
I would try Stripes or GWT with QueryDSL + Hibernate or OpenJPA (with annotations) just for the fact that you actually develop in Java, and try to limit the use of wsdl-first web-services, xml-centric frameworks, EJB and ESBs (not the beer) as much as possible.
I once had to work on a project trying to implement it in JSF. It was a nightmare.
Most of the working time was spent to just making things compile. The fact that no less than half of what was compiled didn't work was another story. Almost no tutorials. Documentation is basically an automated source code export with no human comments. How can one be expected to work like this?
Of several frameworks we'd seen only Sun's was able to create a new project that is compilable at all! The other could only produce bunch of stuff it took many days to ring to a compilable state.
The web was almost silent. To any search there we no more than 20 pages of search results, with useful first 1-3. In that relevant what was found, a half of people was crying for help, the other half declared they had cried for help, noone came, they lost time and interest and dropped that technology.
So we spent times and only made something simple that could have been done in a few weeks with ASP.NET for example.
Then we looked at alternative JSF frameworks. To our surprise we found them all quite incompatible.
With not surprise at all, we joined the ranks of those who dropped JSF as well.
Consider the counter point. I am working at a shop right now that doesn't use any frameworks beyond the JSP standard. Everyone has a different way of doing things and we are very lax about concepts like de-coupling and security concerns like validation.
While I don't think use of frameworks automatically makes you a better coder, I do think by using a standard design pattern implemented by most frameworks and by having easy access to utility functions like validation, I think the chances are you are going to be forced to code up to a certain standard.
In web application design you aren't inventing the wheel every time, so you either end up rolling your own solution to common tasks, or using a framework. I make the assumption that by using a commonly used framework rather then roll your own, you are going to get underlying code that is well tested and flexible.
There is nothing wrong with rolling your own solution as an academic pursuit but I accept that there are people out there putting a lot more time into a robust solution then I may be able to spend. Take log4j for example, pretty easy to roll your logger, but log4j is well tested and maintained and they have taken the time to improve on flexibility and performance to a degree that most roll your own loggers can't touch. The end result is a framework that is robust but also simple enough to use in even the most basic applications.
What worked for me is: you shouldn't just learn any web framework you hear about, take a look at it, see if it makes you code comfortably, ask around stackoverflow or forums to see its advantages and disadvantages, then learn it and learn it good and just stick with it until you feel its broken or plain outdated. Any of the webframeworks you wrote about is good by itself and a fun to work with if you "REALLY" know what it does. if you don't you are just wandering in a desert with no compass! I've also found the 21 days book is a sure way for you NOT to master a framework or a technology. Docs is surely something to consider while adopting a f/w it also helps if you look around the code yourself (actually this is what helps me the best when I faced with some behavior that I find wierd.
1-So why all that hassle with using this frameworks, why not throwing them all away and going back to the roots?
if you go back to the roots you rewrite code that does the same thing again and again + most of these f/ws being open source means they are probably better off with maintenance than you would do alone to your own f/w.
2-What should I say to my boss when we're starting tomorrow the next project with a new framework again?
this is my first time working with this f/w I don't see why should we use this f/w I already know X and I am really good at it. bare in mind the cost of me learning this f/w, the cost of rework that has to be done due to my ignorance of such a f/w. I think we are better off using X, if this is a specific requirement we should fight for it and only do it if we really have to stating the previous notes.
Or are there maybe frameworks which really make a difference?
only those who address the way you think not the way you write code (think struts at its golden age enforcing the MVC pattern).
Or some hidden advantages I have ignored?
can't think of any tbh.
You have the same problem in PHP: more frameworks than you have fingers to count them on, each being the best and greatest (although you have some hints: pure PHP5 design vs. PHP4 compatibility, Rails philosophy (inflexible folder hierarchy, auto-generated code) vs. library approach...) and you spend more time searching and exploring the possibilities than writing your code!
But in PHP it allows to pre-solve common problems, like I18N support, integration of plugins, management of sessions and authentication, database abstraction, templates, Ajax support, etc. Avoiding to re-invent the wheel on each project, and to fall in common traps for newbies.
Of course, there are some hints for Java frameworks too: big or small? well documented or not? widely used or confidential? for XML fans or not? Etc.
I suppose most frameworks aim at large projects, where learning time isn't a big problem, scalability and ease of deployment are important, etc. They are probably overkill for small projects.
There is also a trend in such frameworks to aim at doing a consistent set of loosely coupled libraries rather a monolithic framework. That's the case, in the PHP world, for the Zend framework (some even deny the usage of word 'framework'...).
So it solves the issue of "resolving common problems" without getting in the way.
So you think it's better if we all invent the wheel in every project?
You might see an excess of frameworks as a problem, and it does make it harder to choose your own set. But on the other hand, you don't have to try each one; and even if you do, you'll end up preferring some of them. You will have a favorite framework for ORM, another for web development, IoC, etc.
It does help to read up on some forums to learn which are the most popular ones; they must be popular for a reason, and even if it's not the right reason (like being technically superior, maybe it's popular just among managers because of the buzzword overload or whatever), knowledge of said framework will be helpful because you will be able to participate in several projects that use it.
Plus, using a framework instead of writing your own will save you a lot of problems. Bugs are not always found and solved by the authors; that's often done by users of the framework. You said you ended up with your own private framework in PHP; I bet it wasn't bug-free, but maybe you didn't know it since you were the only user and the only coder.
However I disagree on some points that you have mentioned but I agree with you regarding the boring work.
Yes all web applications are about pages displaying forms, collecting data, making validation, sending the data for storage in Database, and filtering the stored data by search forms and displaying the result in tables and selecting one or more records for manipulation (CRUD, or business actions that all about changing status in database).
however I'm working for just 4 years plus of course my 4 years academic study.
I feel this type of development is boring as you are not inventing algorithms, off course you got happy when you discover new framework and will be happier if you integrated one of the AI engines into your application, but at the end I feel that this work is dummy works, or lets say machine work, so why we don't automate all of this stuff.
yes another framework ;)
MDA Model Driven Architecture, in brief is about transforming from PIM (Platform Independent Model) to PSM (Platform Specific Model), i.e for example from UML to Code.
And this may solves your problem of learning curve and technology changes as you will only need to be good at modeling, as there are some frameworks that implements the MDA specs such as AndroMDA as it have cartridges that take the Class Diagrams, Use cases, Sequence Diagrams, and Activity Diagrams and generate Database creation script, POJOs, hibernate mapping, Spring/EJB, JSF/Struts, .NET code.
Off course such frameworks will not generate 100% of the code but will generate a big percent, and off course you will ask whither this framework will solve complex and tricky scenarios of requirements? today I will say no, tomorrow yes.
so why you and I don't invest in the development of this great framework.
I want to move a legacy Java web application (J2EE) to a scripting language - any scripting language - in order to improve programming efficiency.
What is the easiest way to do this? Are there any automated tools that can convert the bulk of the business logic?
Here's what you have to do.
First, be sure you can walk before you run. Build something simple, possibly tangentially related to your main project.
DO NOT build a piece of the final project and hope it will "evolve" into the final project. This never works out well. Why? You'll make dumb mistakes. But you can't delete or rework them because you're supposed to evolve that mistake into the final project.
Next, pick a a framework. What? Second? Yes. Second. Until you actually do something with some scripting languages and frameworks, you have no real useful concept of what you're doing. Once you've built something, you now have an informed opinion.
"Wait," you say. "To do step 1 I had to pick a framework." True. Step 1, however, contains decisions you're allowed to revoke. Pick the wrong framework for step 1 has no long-term bad effects. It was just learning.
Third, with your strategic framework, and some experience, break down your existing site into pieces you can build with your new framework. Prioritize those pieces from most important to least important.
DO NOT plan the entire conversion as one massive project. It never works. It makes a big job more complex than necessary.
We'll use Django as the example framework. You'll have templates, view functions, model definitions, URL mapping and other details.
For each build, do the following:
Convert your existing model to a Django model. This won't ever fit your legacy SQL. You'll have to rethink your model, fix old mistakes, correct old bugs that you've always wanted to correct.
Write unit tests.
Build a conversion utility to export old data and import into the new model.
Build Django admin pages to touch and feel the new data.
Pick representative pages and rework them into the appropriate templates. You might make use of some legacy JSP pages. However, don't waste too much time with this. Use the HTML to create Django templates.
Plan your URL's and view functions. Sometimes, these view functions will leverage legacy action classes. Don't "convert". Rewrite from scratch. Use your new language and framework.
The only thing that's worth preserving is the data and the operational concept. Don't try to preserve or convert the code. It's misleading. You might convert unittests from JUnit to Python unittest.
I gave this advice a few months ago. I had to do some coaching and review during the processing. The revised site is up and running. No conversion from the old technology; they did the suggested rewrite from scratch. Developer happy. Site works well.
If you already have a large amount of business logic implemented in Java, then I see two possibilities for you.
The first is to use a high level language that runs within the JVM and has a web framework, such as Groovy/Grails or JRuby and Rails. This allows you to directly leverage all of the business logic you've implemented in Java without having to re-architect the entire site. You should be able to take advantage of the framework's improved productivity with respect to the web development and still leverage your existing business logic.
An alternative approach is to turn your business logic layer into a set of services available over a standard RPC mechanisim - REST, SOAP, XML-RPC or some other simple XML (YAML or JSON) over HTTP protocol (see also DWR) so that the front end can make these RPC calls to your business logic.
The first approach, using a high level language on the JVM is probably less re-architecture than the second.
If your goal is a complete migration off of Java, then either of these approaches allow you to do so in smaller steps - you may find that this kind of hybrid is better than whole sale deprecation - the JVM has a lot of libraries and integrates well into a lot of other systems.
Using an automated tool to "port" the web application will almost certainly guarantee that future programming efficiency will be minimised -- not improved.
A good scripting language can help programming efficiency when used by good programmers who understand good coding practices in that language. Automated tools are usually not designed to output code that is elegent or well-written, only code that works.
You'll only get an improvement in programming efficiency after you've put in the effort to re-implement the web app -- which, due to the time required for the reimplementation, may or may not result in an improvement overall.
A lot of the recommendations being given here are assuming you -- and just you -- are doing a full rewrite of the application. This is probably not the case, and it changes the answer quite a bit
If you've already got J2EE kicking around, the correct answer is Grails. It simply is: you probably already have Hibernate and Spring kicking around, and you're going to want the ability to flip back and forth between your old code and your new with a minimum amount of pain. That's exactly Groovy's forte, and it is even smoother than JRuby in this regards.
Also, if you've already got a J2EE app kicking around, you've already got Java developers kicking around. In that case, learning Groovy is like falling off a ladder -- literally. With the exception of anonymous inner classes, Groovy is a pure superset of Java, which means that you can write Java code, call it Groovy, and be done with it. As you become increasingly comfortable with the nicities of Groovy, you can integrate them into your Java-ish Groovy code. Before too long, you'll be writing very Groovy code, and not even really have realized the transition.