Develop Reference

(.Net + SQL Server +Azure ) vs (Java + Oracle +Google App Engine) which is better for this project specifically?

I have to develop an ERP System for a 2,000+ end users organisation.
Could you please suggest me with comparable points that among (Java or .Net)
in which technology I should invest money and time? Although I have done
some average projects in both, but this project is going to be very big in near
future in terms of scalability.
I want to know your experiences and some tips from you people, so that I can develop
and deploy this project efficiently.
I rate .Net > Java for this project only due to less development time available.
We have to use some Rapid App Development technology.
I have to deploy this on Cloud (Azure or Google App engine).
It will be better if I got answers from those people who works in both (.Net and Java).
I will appreciate answers from your experiences.

I would suggest creating a very small proof-of-concept project in both technologies, which do something real - like allow people to log in, see messages, and allow them to type in new messages, and log out again.
Even if the project is laughably small, if you do it well, you will have a finished product on each platform which have shown you by experience how things works and if you like the way you had to do them. You will be able to see if you can debug in the cloud, if you can profile when load testing, if you can do fast work inhouse which then works well when deployed to the cloud.
And you will need to figure out things. Are the online resources good? How responsive is the StackOverflow community for each platform when you ask questions?
Personally, I consider the ".NET is Windows-only" to be important. Except for that I do not believe there is any technical showstopper for either platform.

I think both approaches can be used to deliver this successfully. I would expect you to have the same amount of success/pain with either choice. When it comes to making a decision you should base it on the amount of expertise that you have to hand. That is, your own and that of your existing colleagues and the resources that you can acquire (new recruits, contractors, consultants etc.).
That said a couple of technical notes:
The Java approach tends to have more freedom, i.e. more frameworks and choice of technologies for various solutions (although GAE will bring in some restrictions).
There is less choice in the .NET space, but that is not always a bad thing. E.g. you tend not end up in tireless debates about the logging frameworks.
Java is starting to age as a language and C# is a bit nicer, however there a number of newer languages that run on the Java VM (Scala, Groovy, Ruby, Clojure).

Are there any good Java thread migration packages?

There has been a tremendous amount of research on migrating Java threads between different JVMs on different machines, but is there some publicly available, maintained, de facto implementation that everyone uses? If there is not one good implementation, but multiple ones, which one do you guys recommend and why?
(Note: I'm not talking about database migrations, I'm not asking about Hadoop, RMI, actors, or looking for any other way of doing distributed computing, I need information specifically on thread migration for Java, not Scala [yes I'm aware of the continuations in bleeding edge] or anything else which can sit on the JVM.)

Not sure this is what you are after but...
I know of a couple projects using javaflow to migrate between JVMs. The project needs a few more users to finally see a release though. And contributors that are up to the task are hard to come by. But maybe give it a try. It has a few restrictions/overhead compared to jvm-native migrations.
You might also might want to have a look into this paper about mobile agent migrations.

Distributed ETL question

Looking for any recommendations for an ETL system for 200+ distributed systems (Windows, AS400, Linux etc).
We collect data each month from all of our customers (regardless of system type), bring it back, process it all together and send the aggregate solutions back to them. I'm tasked with automating this system - any suggestions on how to do this robustly, I really don't want to re-invent the wheel. I don't own any of the systems I'm pulling data from, which has made this task more difficult but can install a client.
I've created a prototype client/server architecture in Java with FTP for transport but it feels brittle to me. I should note that all of the extract/transformation code for the different systems already exists in Java (albeit legacy).
I should mention we pull data once per month currently, but working towards weekly.
Any insight is appreciated.

I think it would depend on how the project will become. If this porject will be adding more requirement and there is some money involved, the ETL tool might be good idea.
However, if you have fixed output(the report) now and it is not intended to go anywhere, the custom ETL might be worth it. The reason is the most ETL tools have various output format(Chart, text file etc) and convinience to use the tool but the bottom line is Data moving part is almost universal for all the tools. Even with any other ETL tool, you need to implement same query you are doing now, plus you need to learn the tool. Who knows? Some tool might involved in 200+ site installation.
Recently, our company spent a lot of money to buy report tools & servers & human resource to build good ETL since our in-house ETL has been critisized for the slowness and not professional looking(You know it is not using popular ETL tools. It is bunch of script command). With all the money spending, the project faced on almost dead end.
One more thing. I don't understand how Java & FTP is involved in this process.
Can you directly connect the DB in your customer system using SQL?
If you could, using SQL & stored procedure is always better idea than using JAVA & FTP.
Hope it would help.

Preparing for a Java professional work environment before interviews

I recently graduated in Computer Engineering. I don't have a permanent job yet. The recession is making a good difference in my aim and reality. I don't want to just sit down. I want to sharpen my capabilities. I want to learn and practice in a professional work environment.
Now my question is: What are the tools and practices followed in a professional working environment? I mean IDEs, team working tools,debugging tools, unit testing tools etc?
What are the frameworks in Java EE which are must-know, or which will bring me advantages in my job interviews?
Thanks in advance.

Some initial suggestions:
IDE: Eclipse or IntelliJ
Source code management tools like git, Subversion, Mercurial, CVS
JUnit for testing
Read "Code Complete", then sleep with it under your pillow. Maybe keep a copy in your bathroom as well.

IntelliJ is by far the best Java IDE, but it is commercial. Between Eclipse and NetBeans I would recommend NetBeans, it is closer to what an IDE should look and/or do(nothing against Eclipse, but I couldn't get used with it...maybe is just something wrong with me :) ).

You might find this poll (long but) interesting... http://www.java-forums.org/new-java/7315-what-you-using-write-your-code.html
We've been running a co-op (i.e. cadette) program for the last five (or so) years. Here's a list of stuff (in no particular order) that co-ops are clueless about that we really really wish they knew "the basics of" when they arrived:
Testing - Everyone tests, but (IMHO) few do so really effectively. Myself included.
Team development - How to work together on a codebase which is bigger than anyone.
Understanding legacy systems - Hey, this code is twice is my age? WTF?
Project management - That dark arts of delivering quality on time and on budget.
Business analysis - BA's are full of it! You at-least need to know enough to smell the effluent.
Known when to say NO - Practice saying this in front of a mirror: "No sir, I'm sorry, you really can't have this Ferrarri for the price of a second-hand mini."
The technology really is the easy bit. Having said that, your first position is likely to be as a code-monkey... so the more tecken ze spreken, the more likely you are to get a foot in the door.
So I suggest
Tackle the J2EE 1.4 Tutorial
... also look at a web-app MVC framework like Struts 1 or (better) JSF; and
... also look at Hibernate - the psuedo standard persistence layer.
... also look at "early" DHTML - manipulating a html-DOM with javascript.
... also look at Swing, Applets, but try not to drown in it.
then (and only then) tackle the J2EE 1.5 Tutorial (EJB3).
I'm still in the process myself. I've been at it for two months. Anymore than two hours at-a-time makes my brain hurt, a lot... you can teach an old dog new tricks, you just have to do it slowly.
Like already stated: Every position will have it's own technology set... two people sitting next to each other may use vastly different product groups. What I advise you to learn instead of "all the techs" is the process of learning technologies, and the considerations in selecting appropriate technologies to suit a particular problem, and organisation.
The best thing you can do to improve your chances of actually getting that job is to join a "Young IT Professionals" group (like this one in Australia)... they'll probably do a "mock interview" day... constructive criticism (as apposed to spoonfeeding) will improve your job applications, your CV, and your interviewability. Hanging-out with a bunch of people who shre your interests is informative, and fun. You'll miss that "brain stimulation" in the break between uni and IT work.
On the recession thing... I graduated B.Bus(Computing) a month after the dot-com bubble burst. I worked in a supermarket, drove a cab, delivered pizza, did some builders labouring, flogged PC's, vended bad financial advise to people who can't add-up for ${nameless_charlitans}, flogged anti-virus software, tutored at the local TAFE college, got a few short-term contracts writing software for a big accounting firm... then I started voluntarily fixing up the PC's at my local employment exchange (I was there waiting for appointments for hours-on-end anyway, and it was better than doing nothing)... The IT-dude there got me an interview with "a mate" (i.e. my job was never advertised), and the rest is history. Funny how stuff works out. I feel for ya', just please don't take the knock-backs personally... you're likely to get a lot of them... it's just how the game is played.
Cheers. Keith.

For the long term, work on an open source project. You'll learn a lot, and probably more quickly than you'd learn from a job.

My Java colleagues use this for most things.
Eclipse
Oracle or MySQL
Struts
Subversion
Bugzilla
JUnit
They also use Sun's Java Composite Application Platform Suite (JCAPS) for some things.

I think you need to take a close look at the companies you are applying to. The amount of "toolage" in the java world is way too much for you to become familiar with on your own in order to become attractive to any random employer.
I personally vote that the majority of it is overcomplicated crap, despite all the marketing hype that it gets. Find a company that looks like it works with stuff that you're interested in, find out what sort of stuff they use and then familiarize yourself with that. You probably won't be able to demonstrate job-level experience, but you will show yourself to be enthusiastic.

I'm going to go the route of suggesting what I feel are the most popular choices out there, which hopefully will give you more exposure to land that first job. Once you get your feet wet, I would recommend investing some of your free time to evaluate the other options out there. There is so much in the Java ecosystem to explore.
IDE: From people that I talk to, Eclipse seems to be the most popular. Netbeans is solid too but its future is in doubt since Oracle is working on acquiring Sun. A lot of folks swear by IntelliJ but I think it's much smaller community (and costs money).
SCM: Subversion and CVS are probably the most popular and Eclipse has built-in support for both.
Unit Testing: JUnit, it's the de-facto standard.
Builds: Ant, again the de-facto standard for build automation.
Frameworks: I would recommend either (1) Spring and Hibernate, the wildly-popular open source stack or (2) EJB and JPA, the Java standards. My personal preference is towards #1. As far as UI goes, there are a lot of options available, so it's harder to recommend. Struts and Spring MVC are more "old school" classic MVC frameworks whereas GWT and JSF are more "component-oriented" frameworks. You also have less popular ones like Wicket and Tapestry.

Assuming you can get your CV/resume read by someone and get an interview:
Get a copy of Effective Java by Josh Bloch, read it, memorize it and understand it. A lot of interviewers (and I have done more than my fair share) use it as a good source of techniques that people should know and understand.
In terms of tools - you can't go wrong if you know:
Eclipse
Spring
Hibernate
Ant/Maven/Hudson
JUnit
Log4J
These are all Open Source (and hence will fit anyones budget). Most Java shops will use at least one of these!

With below tools you can make sample/demo applications
IDE : JDevloper/Eclipse/Netbeans
Server: Tomcat/Jboss/Glassfish
Technology: Hibernate,Spring,EJB,Struts,Log4J,SLF4J,Hudson,Ant,Maven

Weblogic,Jetty,JBoss or Tomcat with
-Maven build tool (indirectly background Ant and library versioning)
-Prettify ("readify")
-Minify (spaceless)
-Refactoring (batch rename units)
-XML
-Stress test tool
Old tools e.g. rsync, emacs, awk, xargs, dd handle the largest files. less is good for streaming files (shift+F)

This really depends on the jobs in your area. I suggest you take an analytical approach.
Use a job search engine for you area and do keyword searches. This will give you hard numbers of job skills people are looking for in your area.
You may also find these pages useful 10 Hot Skills for 2009 The 2009 IT Salary Guide

Write an own homepage! It shows that you have fun when you write software (so you do it in your free time).

Build something you're interested in building and choose technologies that seem correct to do the job. At the same time, take the extra time -- as you should when working as well -- to use tools that might be a hassle but you think will bring you an advantage. Knowing frameworks like Spring or Maven or whatever may or may not be relevant. A good employer doesn't care what you know, but how you know what you know. Can you learn?
Use some note repository (like Evernote or my own, TheKbase :) and start making notes about your world. Whatever knowledge you have digested should be instantly retrievable and not based on your limited memory.
I was always interested in Swing and teaching, and strangely I got a job teaching Swing (which I knew a bit), but also teaching all kinds of stuff that I didn't know like Struts and XML security (you read the books and put in the time). Plus I taught IDEs, which I always liked even, though everyone thought I was an immature programmer because I wanted autocomplete :) Again, following preferences...
My point is that unless something about learning a framework or tool particularly appeals to you, don't bother learning it. On the other hand, if frameworks are particularly annoying for you (for me they have been), take the most annoying and daunting and build something with it. It's a rite of passage that most people do because an employer asks for it, which is too late, IMO.

You already have a good list of tools from the answers above. Here are a few things you could do to get more attractive to potential employers.
Participate in an Open-Source Project
To become a better programmer, learn
a language completely different from
Java. A good starting point could be
other languages on the JVM - Clojure
/ Scala.
Gain expertise in specific areas in J2EE that will make you stand out - security, performance analysis, etc.
Create your own website using the tools and the frameworks you have learnt

Setting up a new Java development shop

I'm setting up a Java development shop, currently just for myself as the only developer, but with hopes of needing to hire others as the business grows. Obviously I'm hoping to set it up right so that as more people come in, they can be productive right away. Please help suggest things I want to do, and tools to do them.
Here's what I think I need:
Distributed source code/revision control (Subversion?)
Bug tracking (does Trac do this?)
documentation (both internal and customer facing)
team communication
frequent automated building
maybe something to make sure automatic tests pass as part of the check-in process?

I like Hudson for Continuous Integration builds, and I like JIRA for issue tracking. Eclipse has plugins for both.
Hudson can watch software repositories and rebuild those projects that use the changed resources.
If you need more documentation than javadoc can cover (which is quite a lot) then consider a Wiki. Easy to use, and with a bit of structure you can massage it into a PDF.
Source control is a bugger. Too many to choose from. For a small development team start with either subversion or CVS (which is old but has supreme IDE support) and when you outgrow that and know your needs, then migrate to a better one. Most have migration tools from svn or cvs. It is harder to move from e.g. git to Mercurial, and you defintively want one with more than one implementation. Remember to have good backups of the source control repository - it IS your business. Frequent rsyncs, often tapes.
EDIT: You also want decent hardware. For the Continuous Integration server, the fastest build machine you can afford. For yourself the largest monitor you can afford (not in size, in resolution) for your primary monitor and as many extra monitors as you can afford to have (including adapters to your computer). I have found that Mac's use the pixels better than Windows, so that might also be a point.
My primary monitor is pivoted 90 degrees. This allows me to see many lines at once instead of a few long lines. (For some reason tradition says that editing areas should be wide and short, which may work in word but not in code where lines should not be wider than 72 characters)
Note on Eclipse: Use the source repository to have a single workspace per project! Use the Java Editor Save Function to reformat your code everytime you save - this makes it more readable up front, and goes better with the source repository as changes are marked in the correct version.
Edit: The reason for the CI server needing to be better than your development machine is because it will run all your tests every time you check stuff into your source repository. After a while, that WILL take time.
Personally I have found tests working well for library routines. They specify what works and what doesn't. It is harder to write good tests for whole applications, but you may want to look into that from the beginning, as it allows you to ensure that everything works for every check in. Write a comment if you are not familiar with the concept.
Whatever you choose for the individual parts, you will be glad if they can work together. Hudson knows how to talk to JIRA for instance. JIRA knows how to look in CVS.

To see what a big amount of people thinks to make a good software development environment read this article on the 12 points of joels test
This list does not include the things more important for you. Getting clients getting them to pay and manage legal stuff and taxes.

Most important, the right staff:
get great people who find work and handle customers (aka sales)
get software engineers who are smart and get things done (http://www.joelonsoftware.com/items/2007/06/05.html)
get someone who knows about accounting and the local legal and tax regulations, so you don't get any surprises
Tools / Processes:
use a distributed version control system like git or mercurial
jira or the like for bug/issue tracking
continuous integration with hudson or cruisecontrol
wiki system to share the teams tangible knowledge
unit tests, clover, checkstyle, findbugs, ...
From a managerial point of view i would try
daily standup meetings (checkout scrum) to keep the team updated members to commit by saying what the did, do, and will do
timeboxed meetings, everything else sucks.
plan iterations/sprints
let team do task time estimations
pair programming (gets you better code)
code reviews (builds trust)
weekly in house "techtalks" to build a strong sense for the team
twitter like communication tool to keep all insync and informed with minimal distraction
develop team towards dynamic languages (groovy, scala, ...)
yourself, listen to what the guys at http://manager-tools.com/ have to say...
good luck!

We have been using:
version control:
subversion - Its not distributed but it is accessible over a few different protocols if firewalls are an issue. I'm not sure if distributed version control is necessary for us and reading Eric Sink's take is entertaining at least
issue tracking :
Fogbugz - You get some team discussion and communication for free with it because of the built-in wiki and discussion boards.
continuous integration:
CruiseControl - we had been talking about switching to Hudson, but Cruise is working really well right now - it runs our unit tests.
dev environment:
Netbeans and Eclipse - There is really no reason to pay for a Java IDE. An import point for getting going fast is that Netbeans and Eclipse both store all of their project data as text files which version control nicely. See this question. We had giant headaches when using an IDE which used binary project files.
profiler:
JDK VisualVM - Its free and it works. I used to really like YourKit, but VisualVM does so much now.
documention:
Combination of javadoc and fogbugz wiki pages plus the cruise dashboard for internal. For external we are using RoboHelp and we dislike it.
other tools:
Findbugs - huge help in catching things that are sometimes really stupid and sometimes amazing quirks that you'd have never realized. PMD is good for some of this as well.
We find chat tools to be really helpful for communication. We used to have access to Sametime and it had a giant conferencing feature that was really great. That was taken away for an unknown reason by the overlords though.

Here is what development stack our team of five developers is using over a year now:
Eclipse IDE (worked better for us than NetBeans)
Maven as a project comprehension and build tool -- simply a must tool!
Nexus: The Maven Repository Manager -- serves as a local Maven repo for proxying, and for managing internal and 3rd party libs; simple in use and is really necessary if you're going to use Maven
Subversion for source versioning -- was chosen mainly due to very good IDE support (Subclipse for Eclipse IDE)
Trac as a bug tracking and requirement management tool -- it nicely integrates with Subversion, has very useful plugins including blog and discussion plugins; also it can be integrated with Eclipse Mylyn.
Hudson as continuous integration, which nicely integrates with Subversion, Maven and Trac -- very valuable even for a small team.
Sonar code quality management platform -- a tool which integrates a large number of code quality matrix with intuitive web interface supporting code review and drill-down facility for analysis of the problems; integrates with Maven.
In our case this development stack is running under Ubuntu (workstation components: Eclipse IDE, Maven) and CentOS (server components: Maven, Nexus, Subversion, Trac, Hudson, Sonar).
As for the documentation, LaTeX (TexLive and Kile under Ubuntu) works just great supporting high quality PDF generation. The documentation source can be managed by Subversion the same way as the application source. Allows making of simple several page document and large multi-chapter books.
Hope this helps.

Distributed source code/revision control (Subversion?)
Subversion is good enough unless you want to use this as an opportunity to learn Git or Mercurial.
Bug tracking (does Trac do this?)
Trac works fine if you don't mind CamelCaseTurningIntoWikiWords. JIRA is fancier but (as noted) not free, and I find its data-ink ratio annoyingly low. Good once you learn which parts of the UI to ignore, though.
documentation (both internal and customer facing)
Internally, I would prefer communication over documentation unless you're really willing to commit to spending time keeping internal documentation updated. Javadoc your integration points (internal/external APIs), but try to keep your code and build scripts as self-documenting as you can.
That said, requirements docs are useful. I'd suggest using one tracking system for bugs and requirements. JIRA lets you create hierarchical issues, but I wouldn't bother with that till you need it.
Trac includes a Wiki, which can be handy although too much content will get stale.
Customer-facing docs are all over the map -- depends a lot on what you're doing. If you need big fat manuals (either printed or HTML or on-line help) FrameMaker + the DITA Open Toolkit is a nice combo, but a FrameMaker license is going to set you back $1000.
At a certain point this shades into corporate communications (marcom and PR), which are kind of a different animal.
team communication
Google Talk (or the instant messaging framework of your choice), email, and the occasional Skype teleconference are probably all you need.
An internal wiki and/or internal blogs might be nice but you probably don't need them right away, even once you've added a few more developers. Trac includes a wiki.
frequent automated building
Switching from CruiseControl to Hudson has made everyone here much happier -- two teams made the same decision independently and both are pleased with it. Hudson's flexible, easy to configure, and shiny.
I have yet to be convinced by Maven. I suspect its value depends on how much you want automated downloads of the latest versions of all your open-source libraries.
Edited (Feb. 2010) to add: Several months of Maven 2 experience now and I'm still not convinced. I think (like many frameworks, e.g. Rails) it may work well if you're starting from scratch and structuring your project according to its conventions, but if you already have your own structure in place it's a lot less advantageous.
maybe something to make sure automatic tests pass as part of the check-in process?
I'd advise against it. Most of the time the tests will be passing (or should be, anyway) As long as continuous integration notifies you quickly of failures, and you also have stable automated builds (a couple of days' worth of nightly builds, tagged iteration builds, etc.), you want more and faster checkins, not fewer and slower.

Subversion is not a distributed approach -- go for Mercurial, Bazaar or git instead!
Yes, Trac does do bug tracking (among other things -- check it out!).
Documentation is indeed a must but I'm not sure what you're asking -- tools for it? Why not just javadoc?
For communication you can have many tools, such as skype, email, IM of many kinds, and so forth -- you need to express your specific issues better to get specific advice, I think. Google Wave once it matures may be just great, but it's not production-ready yet.
For continuous build check out CruiseControl -- of course it also run tests &c.
You can write "triggers" for any of the build systems I've mentioned (and even good old svn;-) to run some test suite and reject the commit if it fails.

A few more items:
IDE
Billing Software - will you be charging by the
hour? If so you might want to track
what your time is going towards.
offsite backup of some kind.
Each one of your bullet points is probably worthy of a community wiki by themselves. Though in the end you might not care so much about best of breed in each area, but care more about how well they all integrate with your IDE or with each other.
Also, if you really want to get new teammates up and running quickly, consider putting as much of your dev environment into source control as you can, so you can just checkout your "dev-env" project onto a new computer and be up and running instantly!

One of your specs (in your question) says:
maybe something to make sure automatic
tests pass as part of the check-in
process?
I would suggest this is essential. Check out this matrix of continuous integration servers to see which one fits your requirements.

If you're starting off with just yourself as the only developer but scaling up at some point, depending on what it is you're developing, it might be worth to check out a Platform as a Service (PaaS) offering, like https://www.openshift.com or https://www.heroku.com/, both which offer developer accounts, but you can scale up to paid accounts as and when needed.
OpenShift have CI cartridges for Jenkins so you can spin up DVCS (git), CI using Jenkins, and an app server like JBoss AS or WildFly all in one go in a matter of minutes... depending on your needs and how much time you want to dedicate to setting this up yourself locally, I'd be more inclined to look at using a PaaS.

Rather than use Trac, consider Redmine. It allows you to have multiple projects stored in the one issue tracking system. You can then use the same setup for each project, rather than having to have N instances of Trac.

For starters, SVN will be enough for you. Definitely get a bugtracker. JIRA is good but isn't free. Enforce a rule "No commits without a bugtrecker ticket". This way you will be able to track the development. Do a cruise control and run a build + unit tests after every checkin into the main branch. Bigger changes should be made on a separate branch and then merged into the main branch.
Invest in a good IDE (I recommend intelliJ IDEA) and a good profiler (I recommend JProfiler). They're not free, but they are definitely worth their price.