Develop Reference

How to share entity between REST service between two microservices?

I have created two micro-services using java. I need to make a REST api call from service A to service B. The data sent will be in JSON format. Using jax-rs I need to create entity class in both the service.
Since both the entity class be same in both the projects. Do i
Create an common jar and use is for all my entity/domain objects? Does this make my microservice more tightly coupled?
Do i create the same class in both the microservice projects? This will just mean repeating the work in both the projects?
Is there a better way to communicate between the sevices?

In terms of having your two micro services independent and having them also independent in the future I would also duplicate the code. We had the exact same situation before. Several microservices seem to use some "common" classes that can be put to a seperate jar.
In the end we had following situation:
several (5+) services using the same JAR
turned out that classes that we thought are the same, seemed to have slightly different semantics in different services
a change on one of the classes more or less forced us to have a release on every microservice, when it came to releasing (no independency here anymore)
developers tend to see "common" behavior everywhere, so you most likely end up with some "Helper/Utility" classes there as well which is in the meanwhile considered a code smell in OOP
Long story short, in the meanwhile we switched to having the code duplicated, which gives us the freedom to handle our mircoservices really independently, as we only need to stick to the service contract. What happens internally is fully up to the service and we don't have to release all services in the end of an iteration. I'm not saying that the other option is wrong, but it turned out that it was not suitable for us. If you really see common classes between two services and you are sure you don't mess your common library up with other crap, your save to go.
EDIT
Maybe as follow up, we had the same discussion in regards of tests (unit and integration) having share test code in some common classes. In the end this was hell, as every slight change in code or acceptance criteria made 50% of tests fail. Meanwhile our strategy is to not share anything on test level and have everything right at the tests place. By that you are super fast in eliminating or changing tests. In the end the lesson for us was to keep business code as clean and elegante as suitable and the test code in a way to give us the least headache possible.
Edit2
Meanwhile, we define all our REST interface with open api specifications and create the actual DTO objects that are exchanged via the maven plugin openapi-generator. The spec resides in the project that implements the interface and it is published to artifactory. The project implementing the client pulls it and creates DTOs based on that. By that, you have a single point of truth and no need to write DTO boilerplate code.

I'd say it depends on the situation. If you use a shared package, this will introduce a coupling between the two projects. This makes sense, if both of the project build up on the same data classes and therefore will have the same dto objects to work with. Ideally you would have your own nexus which simplifies the usage of the shared artefact.
Otherwise, if only a few classes are redundant I probably would implement it in each sevice separately, which decouples them too.
I am afraid that you need to decide which one the right solution is for your project.

This is common situation where we as developer gets confused. I would suggest to have a common jar(shared) which can be used in both micro services (A and B). It is nothing but sharing a third resource as we use third-party libraries.
In my current project we were in the same situation and we found the best approach to have separate shared libraries(api-shared as name) and consuming it as jar in different micro-services.
In your second approach you ended up with redundant code and also difficult to maintain. Lets say if you have any changes in entity then you have to change in both the entities which is not quite a good way to synchronize the thing.
All in all I would suggest you to use shared jar for both micro services.
Regards
Techno

Given a choice of adding unit tests or integration tests to an existing system, which is better to start with and why?

I'm currently consulting on an existing system, and I suspect the right next step is to add unit tests because of the types of exceptions that are occurring (null pointers, null lists, invalid return data). However, an employee who has a "personal investment" in the application insists on integration tests, even though the problems being reported are not related to specific use cases failing. In this case is it better to start with unit or integration tests?

Typically, it is very difficult to retrofit an untested codebase to have unit tests. There will be a high degree of coupling and getting unit tests to run will be a bigger time sink than the returns you'll get. I recommend the following:
Get at least one copy of Working Effectively With Legacy Code by Michael Feathers and go through it together with people on the team. It deals with this exact issue.
Enforce a rigorous unit testing (preferably TDD) policy on all new code that gets written. This will ensure new code doesn't become legacy code and getting new code to be tested will drive refactoring of the old code for testability.
If you have the time (which you probably won't), write a few key focused integration tests over critical paths of your system. This is a good sanity check that the refactoring you're doing in step #2 isn't breaking core functionality.

Integration tests have an important role to play, but central to the testing of your code is unit-tests.
In the beginning, you will probably be forced to do integration tests only. The reason is that your code base is very heavily coupled (just a wild guess since there are no unit tests). Tight coupling means that you cannot create an instance of an object for test without creating a lot of related objects first. This makes any tests integration-tests per definition. It is crucial that you write these integration tests, as should be used as base lines for your bug-finding/refactoring efforts.
Write tests that document the bug.
Fix the bug so all created unit-tests are green.
It is time to be a good boyscout (leave the campsite/code in better order that it was when you entered) : Write tests that documents the functionality of the class that contained the bug.
As a part of your boyscout efforts, you start to decouple the class from others. Dependency Injection is THE tool here. Think that no other classes should be constructed inside other classses -- they should be injected as interfaces instead.
Finally, when you have decoupled the class, you can decouple the tests as well. Now, when you are injecting interfacing instead of creating concrete instances inside the tested class, you can make stubs/mocks instead. Suddenly your tests have become unit-tests!!
You can create integration tests as well, where you inject concrete classes instead of stubs and mocks. Just remember to keep them far away from the unit-tests; preferably in another assembly. Unit-tests should be able to run all the time, and run very fast don't let them be slowed down by slow integration tests.

The answer to the question depends on the context in which it is being asked. If you are looking to bring an existing codebase, and you are considering rewriting or replacing large portions of the code then it will be more valuable to design a comprehensive set of integration tests around the components you wish to rewrite or replace. On the other hand, if you are taking responsibility for an existing system that needs to be support and maintained, you might want to first start with unit tests to make sure that your more focused changes do not introduce errors.
I'll put it another way. If someone sends you an old car, take a look at it. If you are going to replace all of the components right away, then don't bother testing the minute performance characteristics of the fuel injector. If, on the other hand, you are going to be maintaining the car, as is, go ahead and write targeted unit tests around the components you are going to be fixing.
General rule, code without unit tests are brittle, systems without integrations are brittle. If you are going to be focused on low-level code changes, write Unit Tests first. If you are going to be focused on system-level changes, write integration tests.
And, also, make sure to ignore everything you read on sites like this. No one here knows the specifics of your project.

Choosing between integration tests and unit tests is highly subjective. It depends on various metrics of the codebase, most notably cohesion and coupling of the classes.
The generic advice that I would provide is that if classes that are loosely coupled, then test setup is going to consume lesser time, and hence, it would be much easier to start writing unit tests (especially against the more critical classes in the codebase).
On the other hand, in the event of high coupling, you might be better off writing integration tests against the more critical code paths, starting especially with a class that is loosely coupled (and resident much higher up in the execution stack). At the same time, attempts must be made to refactor the classes involved to reduce coupling (while using the integration tests as a safety net).

Java Dependency injection: XML or annotations

Annotations becoming popular. Spring-3 supports them. CDI depends on them heavily (I can not use CDI with out of annotations, right?)
My question is why?
I heard several issues:
"It helps get rid of XML". But what is bad about xml? Dependencies are declarative by nature, and XML is very good for declarations (and very bad for imperative programming).
With good IDE (like idea) it is very easy to edit and validate xml, is not it?
"In many cases there is only one implementation for each interface". That is not true!
Almost all interfaces in my system has mock implementation for tests.
Any other issues?
And now my pluses for XML:
You can inject anything anywhere (not only code that has annotations)
What should I do if I have several implementations of one interface? Use qualifiers? But it forces my class to know what kind of injection it needs.
It is not good for design.
XML based DI makes my code clear: each class has no idea about injection, so I can configure it and unit-test it in any way.
What do you think?

I can only speak from experience with Guice, but here's my take. The short of it is that annotation-based configuration greatly reduces the amount you have to write to wire an application together and makes it easier to change what depends on what... often without even having to touch the configuration files themselves. It does this by making the most common cases absolutely trivial at the expense of making certain relatively rare cases slightly more difficult to handle.
I think it's a problem to be too dogmatic about having classes have "no idea about injection". There should be no reference to the injection container in the code of a class. I absolutely agree with that. However, we must be clear on one point: annotations are not code. By themselves, they change nothing about how a class behaves... you can still create an instance of a class with annotations as if they were not there at all. So you can stop using a DI container completely and leave the annotations there and there will be no problem whatsoever.
When you choose not to provide metadata hints about injection within a class (i.e. annotations), you are throwing away a valuable source of information on what dependencies that class requires. You are forced to either repeat that information elsewhere (in XML, say) or to rely on unreliable magic like autowiring which can lead to unexpected issues.
To address some of your specific questions:
It helps get rid of XML
Many things are bad about XML configuration.
It's terribly verbose.
It isn't type-safe without special tools.
It mandates the use of string identifiers. Again, not safe without special tool support.
Doesn't take any advantage of the features of the language, requiring all kinds of ugly constructs to do what could be done with a simple method in code.
That said, I know a lot of people have been using XML for long enough that they are convinced that it is just fine and I don't really expect to change their minds.
In many cases there is only one implementation for each interface
There is often only one implementation of each interface for a single configuration of an application (e.g. production). The point is that when starting up your application, you typically only need to bind an interface to a single implementation. It may then be used in many other components. With XML configuration, you have to tell every component that uses that interface to use this one particular binding of that interface (or "bean" if you like). With annotation-based configuration, you just declare the binding once and everything else is taken care of automatically. This is very significant, and dramatically reduces the amount of configuration you have to write. It also means that when you add a new dependency to a component, you often don't have to change anything about your configuration at all!
That you have mock implementations of some interface is irrelevant. In unit tests you typically just create the mock and pass it in yourself... it's unrelated to configuration. If you set up a full system for integration tests with certain interfaces using mocks instead... that doesn't change anything. For the integration test run of the system, you're still only using 1 implementation and you only have to configure that once.
XML: You can inject anything anywhere
You can do this easily in Guice and I imagine you can in CDI too. So it's not like you're absolutely prevented from doing this by using an annotation-based configuration system. That said, I'd venture to say that the majority of injected classes in the majority of applications are classes that you can add an #Inject to yourself if it isn't already there. The existence of a lightweight standard Java library for annotations (JSR-330) makes it even easier for more libraries and frameworks to provide components with an #Inject annotated constructor in the future, too.
More than one implementation of an interface
Qualifiers are one solution to this, and in most cases should be just fine. However, in some cases you do want to do something where using a qualifier on a parameter in a particular injected class would not work... often because you want to have multiple instances of that class, each using a different interface implementation or instance. Guice solves this with something called PrivateModules. I don't know what CDI offers in this regard. But again, this is a case that is in the minority and it's not worth making the rest of your configuration suffer for it as long as you can handle it.

I have the following principle: configuration-related beans are defined with XML. Everything else - with annotations.
Why? Because you don't want to change configuration in classes. On the other hand, it's much simpler to write #Service and #Inject, in the class that you want to enable.
This does not interfere with testing in any way - annotations are only metadata that is parsed by the container. If you like, you can set different dependencies.
As for CDI - it has an extension for XML configuration, but you are right it uses mainly annotations. That's something I don't particularly like in it though.

In my opinion, this is more a matter of taste.
1) In our project (using Spring 3), we want the XML-configuration files to be just that: configuration. If it doesn't need to be configured (from end-user perspective) or some other issue doesn't force it to be done in xml, don't put the bean-definitions/wirings into the XML-configurations, use #Autowired and such.
2) With Spring, you can use #Qualifier to match a certain implementation of the interface, if multiple exist. Yes, this means you have to name the actual implementations, but I don't mind.
In our case, using XML for handling all the DI would bloat the XML-configuration files a lot, although it could be done in a separate xml-file (or files), so it's not that valid point ;). As I said, it's a matter of taste and I just think it's easier and more clean to handle the injections via annotations (you can see what services/repositories/whatever something uses just by looking at the class instead of going through the XML-file looking for the bean-declaration).
Edit: Here's an opinion about #Autowired vs. XML that I completely agree with: Spring #Autowired usage

I like to keep my code clear, as you pointed. XML feets better, at least for me, in the IOC principle.
The fundamental principle of Dependency Injection for configuration is that application objects should not be responsible for looking up the resources or collaborators they depend on. Instead, an IoC container should configure the objects, externalizing resource lookup from application code into the container. (J2EE Development without EJB - Rod Johnson - page 131)
Again, it just my point of view, no fundamentalism in there :)
EDIT: Some useful discussions out there:
http://forum.springsource.org/showthread.php?t=95126
http://www.theserverside.com/discussions/thread.tss?thread_id=61217

"But what is bad about xml?" It's yet another file to manage and yet another place to have to go look for a bug. If your annotations are right next to your code it's much easier to mange and debug.

Like all things, dependency injection should be used in moderation. Moreover, all trappings of the injections should be segregated from the application code and relegated to the code associated with main.
In general applications should have a boundary that separates the abstract application code from the concrete implementation details. All the source code dependencies that cross that boundary should point towards the application. I call the concrete side of that boundary, the main partition, because that's where 'main' (or it's equivalent) should live.
The main partition consists of factory implementations, strategy implementations, etc. And it is on this side of the boundary that the dependency injection framework should do it's work. Then those injected dependencies can be passed across the boundary into the application by normal means. (e.g. as arguments).
The number of injected dependencies should be relatively small. A dozen or less. In which case, the decision between XML or annotations is moot.

Also don't forget Spring JavaConfig.

In my case the developers writing the application are different that the ones configuring it (different departments, different technologies/languages) and the last group doesn't even has access to the source code (which is the case in many enterprise setups). That makes Guice unusable since I would have to expose source code rather than consuming the xmls configured by the developers implementing the app.
Overall I think it is important to recognize that providing the components and assembling/configuring an application are two different exercises and provide if needed this separation of concerns.

I just have a couple of things to add to what's already here.
To me, DI configuration is code. I would like to treat it as such, but the very nature of XML prevents this without extra tooling.
Spring JavaConfig is a major step forward in this regard, but it still has complications. Component scanning, auto-magic selection of interface implementations, and semantics around CGLIB interception of #Configuration annotated classes make it more complex than it needs to be. But it's still a step forward from XML.
The benefit of separating IoC metadata from application objects is overstated, especially with Spring. Perhaps if you confined yourself to the Spring IoC container only, this would be true. But Spring offers a wide application stack built on the IoC container (Security, Web MVC, etc). As soon as you leverage any of that, you're tied to the container anyway.

XML has the only benefit of a declarative style that is defined clearly separated from the application code itself. That stays independent from DI concerns. The downsides are verbosity, poor re-factoring robustness and a general runtime failure behaviour. There is just a general (XML) tool support with little benefit compared to IDE support for e.g. Java. Besides this XML comes with a performance overhead so it usually is slower than code solutions.
Annoations often said to be more intuitive and robust when re-factoring application code. Also they benefit from a better IDE guidance like guice provides. But they mix application code with DI concerns. An application gets dependent on a framework. Clear separation is almost impossible. Annotations are also limited when describing different injection behaviour at the same place (constructor, field) dependent on other circumstances (e.g. robot legs problem). Moreover they don't allow to treat external classes (library code) like your own source. Therefore they are considered to run faster than XML.
Both techniques have serious downsides. Therefore I recommend to use Silk DI. It is declarative defined in code (great IDE support) but 100% separated from your application code (no framework dependency). It allows to treat all code the same no matter if it is from your source or a external library. Problems like the robot legs problem are easy to solve with usual bindings. Furthermore it has good support to adapt it to your needs.

Dependency Injection: What's wrong with good old-fashioned refactoring?

DI creates an extra layer of abstraction so that if your implementation class ever changes you can simply plug in a different class with the same interface.
But why not simply refactor when you want to use a different implementation class? Other languages like Python and Ruby work fine this way. Why not Java?

That is an incorrect characterization of dependency injection. It is not that you have one implementation of a particular interface that changes over time; rather, it is possible that there will be many different implementations of an interface all at once, and which implementation will be used can vary over multiple different runs of the program. For example, in your actual program, you might want to use one implementation, while during unit testing, you might want to "mock out" that implementation with an alternative version that is easier to test. In this case, refactoring is not a solution, because you need to be able to test all the time without interrupting the rest of the development process.
It should also be noted that dependency injection is usually used as a solution to the Singleton anti-pattern; it allows one to have a singleton object that can be easily mocked out during testing. And, if later it turns out that the singleton assumption really is incorrect, that singleton can be replaced with various implementations.
Some resources which you may find helpful in better understanding the topic:
Java on Guice: Dependency Injection the Java Way
Big Modular Java with Google Guice
Singletons are Pathological Liars
Why Singletons are Evil
Root Cause of Singletons
Dependency Injection Myth: Reference Passing

So you are saying Python and Ruby can't have dependency injection? Or Java can't work fine without DI?
Besides you've missed the one of the most characteristic of DI, that you can have Dynamic DI, not just at compile time, but at run time. In Software Engineering there is always a question of is there too much Abstraction and too little and it really comes down to how you design a solution to your problem

Not quite. The issue here is that when you write a code snippet like:
Runnable r = new MyFooRunnable();
you essentially decide that the Runnable you will need is a MyFooRunnable (and not a MyBarRunnable or a third one). Occasionally you will want to postpone that decision from compile time to deployment time, so that the deployer can decide how the individual modules your application consists of are to be glued together.
Traditionally this has been done with factories, but this just moves the actual decision around in code and you still have to know all the possibilities when coding the factory or let it read instructions from a configuration file (which tends to be fragile to refactoring).
Dependency Injection is a formalization of configured factories in a way so the code does not need to know hardly anything about how things work. This is also why annotations have been found so useful for pointing out where the Dependency Injection should happen. If running the code in a non-DI setting (like a junit test) then there does not happen anything (which would have been hard to do with Factories littered all over).
So, Dependency Injection used liberally allows you to write modules that "snap" well together without knowing of each other at compile time. This is very similar to the jar-file concept, but it has taken longer to mature.

How to best test Java code?

I have been working on a comparatively large system on my own, and it's my first time working on a large system(dealing with 200+ channels of information simultaneously). I know how to use Junit to test every method, and how to test boundary conditions. But still, for system test, I need to test all the interfacing and probably so some stress test as well (maybe there are other things to do, but I don't know what they are). I am totally new to the world of testing, and please give me some suggestions or point me to some info on how a good code tester would do system testing.
PS: 2 specific questions I have are:
how to test private functions?
how to testing interfaces and avoid side effects?

Here are two web sites that might help:
The first is a list of open source Java tools. Many of the tools are addons to JUnit that allow either easier testing or testing at a higher integration level.
Depending on your system, sometimes JUnit will work for system tests, but the structure of the test can be different.
As for private methods, check this question (and the question it references).
You cannot test interfaces (as there is no behavior), but you can create an abstract base test classes for testing that implementations of an interface follow its contract.
EDIT: Also, if you don't already have unit tests, check out Working Effectivly with Legacy Code; it is a must for testing code that is not set up well for testing.

Mocking is a good way to be able to simulate system tests in unit testing; by replacing (mocking) the resources upon which the other component depends, you can perform unit testing in a "system-like" environment without needing to have the entire system constructed to do it.
As to your specific questions: generally, you shouldn't be using unit testing to test private functions; if they're private, they're private to the class. If you need to test something, test a public method which uses that private method to do something. Avoiding side effects that can be potentially problematic is best done using either a complete test environment (which can easily be wiped back to a "virgin" state) or using mocking, as described above. And testing interfaces is done by, well, testing the interface methods.

Firstly, if you already have a large system that doesn't have any unit tests, and you're planning on adding some, then allow me to offer some general advice.
From maintaining the system and working with it, you'll probably already know the areas of the system which tend to be buggiest, which tend to change often and which tend not to change very much. If you don't, you can always look through the source control logs (you are using source control, right?) to find out where most of the bug fixes and changes are concentrated. Focus your testing efforts on these classes and methods. There's a general rule called the 80/20 rule which is applicable to a whole range of things, this being one of them.
It says that, roughly on average, you should be able to cover 80 percent of the offending cases by doing just 20% of the work. That is, by writing tests for just 20% of the code, you can probably catch 80% of the bugs and regressions. That's because most of the fragile code, commonly changed code and worst offending code makes up just 20% of the codebase. In fact, it may be even less.
You should use junit to do this and you should use something like JMock or some other mocking library to ensure you're testing in isolation. For system testing/integration testing, that is, testing things while they're working together, I can recommend FitNesse. I've had good experience with it in the past. It allows you to write your test in a web browser using simple table-like layouts, where you can easily define your inputs and expected outputs. All you have to do is write a small backing class called a Fixture, which handles the creation of the components.

Private functions will be tested when the public functions that call them. Your testing of the public function only cares that the result returned is correct.
When dealing with API (to other packages or URLS or even to file/network/database) you should mock them. A good unit test should run in a few milliseconds not in seconds. Mocking is the only way to do that. It means that bugs between packages can be dealt with a lot easier than logical bugs at the functional level. For Java easymock is a very good mocking framework.

You may have a look on this list : Tools for regression testing / test automation of database centric java application? for a list of interesting tools.
As you seem to already use Junit extensively it means that you're already "test infected", that is a good point...
In my personal experience, the most difficult thing to manage is data. I mean, controlling very acutely the data agaisnt which the tests are runned.

The lists of tools given before are useful. From personal experience these are the tools I find useful:
Mocking - Mockito is an excellent implementation and has clever techniques to ensure you only have to mock the methods you really care about.
Database testing - DBunit is indespensible for setting up test data and verifying database interactions.
Stress testing - Jmeter - once you see passed the slightly clunky gui this is a very robust tool for setting up scenarios and running stress tests.
As for general approach start by trying to get tests running for the usual "happy paths" through your application these can form a basis for regression testing and performance testing. Once this is complete you can start looking at edge cases and error scenarios.
Although this level of testing should be secondary to good unit testing.
Good luck!