I've recently tried to include elasticsearch into one of my projects. Therefore I looked up the different possibilities to do so. It basically came down to use either a) the transport client (internal protocol cluster nodes too use to communicate = my understanding) or b) implement it all as REST HTTP calls on my own.
As b) doesn't look like a good option, and I'm used to spring data anyways I tried using spring-data-elasticsearch which was updated recently to version 3.0.0 to support es 5.5.0 which is great as the last version was very old.
Well, I tried everything but didn't manage to get it to work with my project, as this really seems to be some kind of dependency hell you get into. My project is too big, I just can't upgrade it to spring boot 2.0.0M4 which I believe would be required to support the new spring-data-elasticsearch version - right?
So, could somebody please advice on the current and preferred way for now and the future is on how to deal with elasticsearch in spring projects? Do we really have to implement the whole REST API ourselves? Am I missing something?
Here is the REST client which should be used, if there is one used, right?
This is the compatibility chart I know which is old.
Thanks!
Unfortunately there are quite a few options out there:
There's the transport client you have mentioned. While not (yet) deprecated, it's not the way forward and I would not start a new project with it (if possible).
There is the low level REST client (mentioned in the blog post you've linked), which was added in 5.0, but is compatible with older versions as well. While it works, you don't really want to use it, since it's pretty low level (as the name suggests).
The high level REST client has just been released with 5.6 — this is what you want to use going forward (and it will decouple you from a specific Spring Boot / Data version). Right now it only supports the index, delete, aggregate, search, and bulk APIs, but for regular operations this should be enough. More APIs will be added in the future and you can always fall back to the low level client if you need to do other tasks (like setting up a specific mapping for an integration test).
With the release of Spring Data Kay, it also supports Elasticsearch 5.x. If you need to use Spring Data, this is what you'll want to use, but keep in mind that it's a community project and development can be slower than everybody wants (though they do accept pull requests ;-) ). As you've discovered, you'll need to use Spring Boot 2 to have Spring Data Kay built in. This might be a good reason to upgrade or it might be a blocker for you.
How about writing a small service which would be a standalone application in spring boot 2, with the newest ES dependency, and giving it a REST API compatible with your requirements for the rest of the project? You could communicate with it via RestTemplate for starters.
This way you can start breaking up your large project into smaller subprojects and eventually maybe even microservices. Also spring boot 2 has some really nice reactive options, which really come in handy for processing data.
Related
Spring seems to be deprecating its RMI:
As of Spring Framework 5.3, support for several remoting technologies is now deprecated for security reasons and broader industry support. Supporting infrastructure will be removed from Spring Framework for its next major release.
But I can't find any easy equivalent alternative to it. REST would sound like the best choice for simple stuff, but it does not cover solutions where the task is long running and/or a continuous flow of results is needed. So the implementation would be very absurdly painful with REST. Some consider submitting the tasks via REST to queues and then possibly querying for the results separately - but this sounds like an overkill and blows the amount of work needed to sky-high for something really simple that used to be available.
Is there a good alternative or some framework that uses the non-deprecated technologies and is officially available that deals with the above mentioned problems? When something is deprecated it usually indicates that something better is available that would be the better solution, so can somebody help me on educating me what it is?
Based on the spring documentation:
https://docs.spring.io/spring-framework/docs/current/reference/html/integration.html#remoting-web-services
I can say, that probably these alternatives remain in future versions, but I did not try them:
Jax-WS
Spring Web Services
AMQP
I feel, that Spring Web Services is Spring's favorite.
We will see when the 6.0 documentation is finished.
I have come across the same problem using Hessian. I think you still can make your own implementation. Spring only deprecates their own integrations to RMI technologies.
I think there are plenty of other technologies, but the message is clear. Spring will not make any own integrations.
I can think of these alternatives:
Hessian: still usable, but implement your own integration
gRPC
messaging (rabbit, pulsar, kafka, ...)
SOAP
I've been using Spring for about a year, and I'm comfortable enough using it, but I've avoided jumping under the hood for the most part.
I'm tasked with upgrading a large, mission critical enterprise application, from Spring 3.0.x to Spring 4.1.x.
What are the best practices for making a large, inevitably finicky and complex change like this? (Anything above and beyond 'throw in the jar files and see what happens' and 'read the documentation here: http://spring.io/' would be very helpful)
The system:
Java 6 - jax-b/-p/-ws/, Apache Commons,
Spring 3.0.5 - the usuals (core, context, beans etc), MVC, AOP, ORM, JDBC, Acegi
Hibernate 3.5
Tomcat 6
0 unit tests or automated testing of any kind.
Maven dependency management and build automation.
Half controllers using annotations for request response mapping, half using simpleFormController pattern, half autowired, half hooked up with xml.
Hundreds of views, scores of controllers.
Steps I've taken so far:
Prepared a (mostly automated) regression testing script (so that I can ensure I haven't broken anything)
I've started reading through the 'upgrade guides' one at a time, "upgrading to 3.1", "upgrading to 3.2" and making notes on things that sound familiar, but I think I'd need to have a much deeper grasp of our system, and spring in general, before I could be confident of this as an exhaustive approach. This just generally feels like a haphazard approach, which is not what I want for such a complex change.
My questions:
What steps/procedures are considered 'best practice' in these for a job like this?
Does anything jump out at you as a 'gotcha' for a job like this?
Obviously, there won't be "standard" set of recommended practices because every migration/upgrade is different. Here're my thoughts:
Requirements, requirements, requirements
Regression testing script is great start. If there is a complete documentation of the features/functionality, then your "success criteria" for migration is straightforward.
If the documentation is incomplete/non-existent, then double and triple check to make sure that all 'requirements' are captured with your tests. Might be a good idea to create documentation too. And have the product manager/supervisor sign off on it. You'll be surprised at how many 'hidden' requirements exist even in simple systems. There is a big risk of underestimating the effort needed for migration without comprehensive requirement.
It is extremely critical to set the right expectations in terms of timelines. Perhaps an agile approach with biweekly demos of how much progress you've made will help keep everyone on the same page.
Spring projects have evolved a lot. Budget for learning time.
This could be a big gotcha. Spring projects and Java development have evolved a lot since Spring 3.x. Big changes include:
Java 8 features
JavaConfig (as opposed to xml configuration)
Acegi is now Spring Security
Spring projects typically use Spring Boot
Switch from Maven to Gradle for building projects
Full CI using Jenkins (or other CI tools)
Unit and integration testing have moved on to using annotations (and mock frameworks)
Well, it is not easy to answer you question since there are many things to be taken into account.
First of all I can suggest you to use the Migrating from earlier versions of the Spring Framework guide that's coming directly from the 'source'.
I would especially draw you attention to the 'Enforced minimum dependency versions' section that recommends you the minimum version level of some wide used libraries.
Obviously in the moment you insert these new versions they're bringing with them some transitive dependencies that might generate conflicts.
Take also a look to the dependency updates section.
Also remember to correctly define the scope of the dependencies in your pom files, since many of them could be provided by the infrastructure you're using (i.e. Tomcat).
I think you will be required to move to Java 7 or 8 and also Tomcat should be updated to version 7 or better 8.
Moreover try to automate as much as you can your building and testing environment with maven along with adopting a CI environment like Jenkins (or Hudson if you prefer the product).
It is also very important to perform unit testing of every single little method/piece of code, since it will make integration tests easier.
You should also become familiar with Spring 4.x new features and try to exploit them especially those regarding testing improvements.
A little resume of new features is the following:
Removed Deprecated Packages and Methods
Java 8 Support
Java EE 6 and 7 become the baseline
Groovy Bean Definition DSL
Core Container Improvements
General Web Improvements
WebSocket, SockJS, and STOMP Messaging
Testing Improvements with extreme use of annotations
Take also a look to Spring MVC Test Tutorial by Petri Kainulainen that can give you a lot of informations about testing.
You have to have answer to the following before you proceed.
Is the need to upgrade is only the libraries and runtime for some sort of dependencies ?
OR
You really want to get the most out of Spring 4.x ?
Once you decide this you can take proper course. Those regression scripts you have created will help in both the scenarios. If you can think of some crude throwaway utility that will hit every public api with some valid input and capture the output and be able t compare this in the both worlds that may help but it may not be applicable in your situation.
So if you want to get the benefit of the Spring 4.x I would suggest you focus on productivity aspects and create an inventory of these things.
You may redesign the whole app in Spring 4 as if it is a new application.
Once you can envision the future state. The next problem reduces to going from Point A to Point B i.e. a matter of best migration path.
From Migrating from Spring 3 to Spring 4, you would probably get some help from the Spring project's
Spring Integration 3.0 to 4.0 Migration Guide on Github.
Hope it helped!
We are looking at which REST framework to use and where to run it.
Given an existing WebSphere (6.1.0.17) environment would you use Jersey of Wink?
Or would you recommend a different platform?
As a third option why not use Spring's own RESTful features in Spring MCV. This is easy to code, test and maintain and (obviously) works for Spring right out of the box. The ease of managing code based on Spring REST is due to an architecture based on MVC, IoC and annotations. And you can choose best in class solutions to enhance your solutions such as XStream for serialization, Jackson JSON Support, and Springs own REST Template for a REST Client or unit-tests.
At its heart Spring MVC's REST features deal with REST as an Architecture not a Protocol and blends it well with other proven approaches such as IoC and MVC. When considering a framework the following REST litmus test is useful: see innoq.com/blog/st/2010/07/rest_litmus_test_for_web_frame.html.Springs approach checks more of the boxes, thanks to it's effective content delivery.
One thing all these frameworks fall down on is HATEOAS support, a way of in which the REST response contain links that correspond to all the actions that the client can currently perform from this resource. Most frameworks promises to introduce this soon, but a critical eye will be needed to understand what the best approach to this will be.
When delivering JAX-RS solutions such as Wink and Jersey, I've found more emphasis on conforming to a Java standard and less flexibility in the architecture (such as using other 3rd party packages). If conforming to JAX-RS is a requirement consider Restlets, Wink or Jersey. Otherwise, maybe it's worth considering Spring MVC's REST support.
I've deployed this kind of solution to WAS 7 without any issues or dependency conflicts.
I have no experience in using Wink, but I can share some of my experience in using Jersey running on Websphere.
The pros... very easy to use, it took me 5 minutes to get familiarized and I'm ready writing my first hello world. Jersey does automatically generate WADL for you based on your Resources package, albeit pretty basic... but you can customize the WADL documentation yourself.
The cons... sigh, I like Jersey, but with Websphere, it almost made me cry. First (this is not related to Websphere), I'm using Spring in my project and the Jersey release I use is 1.2. The problem is that release uses Spring 2.5 whereas I'm using Spring 3.x. So, I pulled out Jersey's Spring 2.5 to use Spring 3.x. Everything works fine. However, the Jersey test framework started to fail. I'm not able to boot up the built-in Grizzly server to test my web services anymore. Second, I'm currently tied to Websphere 6.1 which uses JDK 5. I believe all Jersey releases after 1.2 are compiled with JDK 6... that means, I'm out of luck until my company upgrade to WAS 7.
It is indeed a very stable Rest framework, very easy to use... but in my case, I'm just bummed out because all existing unit tests for the web services are currently commented out because I upgraded my Spring release to 3.x. I'm sure the latest Jersey release should be using Spring 3.x by now, yet I can't use them because I'm still using JDK 5 in WAS 6.1.
So, it's up to you to decide. By the way, I'm still using Jersey 1.2 in my project running in WAS 6.1.
On websphere 6.x the prefered method is jersey because it is easier to implement and it supports JAX-RS 1 and 2. (Watch out for the JVM version issues, Websphere 6 might not support the latest version)
Now starting in Websphere 8.x IBM introduced Apache Wink that will actually bring dependency and classloader issues if used together with jersery (because of the same interface implementations for JAX-RS 1 and 2).
There are a lot of examples on how to overcome this issues and make it work but I don't believe is worthy and perhaps Wink will have a better performance within websphere.
IBM recommends to use Apache wink. In fact, IBM has his own implementation of Apache wink.
I vote for Wink.
The reasons:
It's developed by HP and IBM guys. So I believe that IBM guys tested it with Websphere.
Actually AFAIK Wink is built-in in the Websphere 7.*
Also see this video
As a seasoned Spring user I was assuming that Spring Integration would make the most sense in a recent project requiring some (JMS) messaging capabilities (more details). After some days working with Spring Integration it still feels like a lot of configuration overhead given the amount of channels you have to configure to bring some request-response (listening on different JMS queues) communications in place.
Therefore I was looking for some background information how Camel is different from Spring Integration, but it seems like information out there are pretty spare, I found:
http://java.dzone.com/articles/spring-integration-and-apache (Very neutral comparison between implementing a real-world integration scenario in Spring Integration vs. Camel, from December 2009)
http://hillert.blogspot.com/2009/10/apache-camel-alternatives.html (Comparing Camel with other solutions, October 2009)
http://raibledesigns.com/rd/entry/taking_apache_camel_for_a (Matt Raible, October 2008)
Question is: what experiences did you make on using the one stack over the other? In which scenarios would you recommend Camel were Spring Integration lacks support? Where do you see pros and cons of each? Any advise from real-world projects are highly appreciated.
We choose Camel over Spring-Integration because the fluent API is really nice. We actually use it in Spring projects and use Spring to configure part of it. The programming API's are clear and there is a large set of sensible components.
We did a small scale shootout and basically at that time for our requirement Camel won. We use it mainly to transfer internal datafiles to/from external parties which usually requires format conversions sending it using ftp/sftp/... or attaching it to an email and sending it out.
We found the edit-compile-debug cycle reduced. Using groovy to experiment setting up routes are added bonuses.
Spring-Integration is a great product too, and I am quite sure it would satisfy our needs too.
I only recommend Spring Integration if you already have got a Spring project and you have just to add some "basic" integration using File, FTP, JMS, JDBC, and so on.
Apache Camel has two main advantages:
Many, many more technologies are supported.
Besides, a (good) XML DSL, there are fluent APIs for Java, Groovy and Scala.
Because Apache Camel has very good integration with Spring, I would even use it instead of Spring Integration in most Spring projects.
If you need more details, you can read my experiences in my blog post: Spoilt for Choice: Which Integration Framework to use – Spring Integration, Mule ESB or Apache Camel?
I have recently conducted a Camel vs Spring Integration shoot-out with the aim to integrate Apache Kafka. Despite being an avid Spring developer, I sadly found my suspicion with Spring's ever-growing Project stack confirmed: Spring is awesome as IOC-Container to serve as glue for other framework, but it fails at providing viable alternatives to those frameworks. There might be exceptions to this, namely everything to do with MVC, where Spring came from and where it does a great job, but other attempts to provide new functionality on top of container features fall short for three reasons and the SI Kafka use case confirms all of them:
Introduction of a long-winded difficult to use DSL for XML-configuration.
Pages of xml-configuration code to get all framework components wired-up.
Missing resources to provide functionality on par with dedicated frameworks.
Now, back to the results of my shoot-out: most importantly I am impressed by Camels overall concept of routes between endpoints. Kafka seamlessly integrates with this concept and three lines of configuration are enough to get everything up-and-running. Problems encountered during the process are neatly addressed by ample documentation from the project team as well as a lot of questions on Stackoverflow. Last but not least, there is a comprehensive integration into Spring that leaves no wishes unfulfilled.
With SI on the contrary, the documentation for the Kafka integration is quite intense and still fails to explain clearly how to integrate Kafka. The integration of Kafka is pressed into the SI-way of doing things, which adds extra complexity. Other documentation, e.g. on Stackoverflow is also less plentiful and less helpful than for Camel.
My conclusion: cobbler stick to your trade - use Spring as a container and Camel as system integration framework.
It really depends on what you want to do. If you need to extend something to build your own messaging solution Spring Integration has the better programming model. If you need something that supports many protocols without custom code, Camel is ahead of Spring Integration.
Having a small scale shootout is a very good idea, just make sure you're trying to do the type of things that you'd typically be doing in the project.
--disclaimer: I'm a Spring Integration committer
Most comparisons of Camel and SI that I've seen don't take the following into account:
1.) The effect that Spring Boot has had on developer productivity for Spring Integration
2.) The effect of Spring XD has had on making Spring Integration applications available with no code compilation - also Spring XD sources and sinks are simply Spring Integration channel adapters, when you're looking to extend Spring XD.
3.) The effect of Spring XD has had on making unifying Spring Integration, Spring Batch, Spring Data (+Hadoop!) in one stack, effectively bringing batch and stream processing, HDFS/Apache Hadoop support, and much more to Spring Integration.
4.) The effect of the soon-to-be-released Spring Integration 4.0 Java DSL https://github.com/spring-projects/spring-integration-extensions/wiki/Spring-Integration-Java-DSL-Reference
For your consideration,
/Pieter (disclaimer I work at Pivotal)
We are using Spring Integration for our application and now considering to move to Apache Camel as we encountered lots of issues with Spring Integration framework. Here are couple of issues.
The CachingConnectionFactory which Spring provides opens 1000's of idle connections in IBM MQ and there is no guarantee that these connections are reused. And still these connections will stay open forever which creates troubles on the MQ side. Had to restart the application every week in lower environments just to refresh the connections. Apache Camel also provides Caching and the connections seems to go up/down based on the load.
Spring doesn't provide mappers for QoS parameters. Even if you enable QoS, the delivery mode and expiration/timetolive properties will get lost (I am going to raise a JIRA issue for this). Apache Camel handles this and QoS parameters are sent to upstream applications and not dropping it.
I am right now working on issues with handling the exceptions and transactions with Apache Camel which Spring seemed to handle better with AOP.
Apache Camel is a very good framework and very complete too. But if your application uses spring, my personal advice is to use Spring Integration.
Spring Integration is the integration EIP complaint framework of Spring-Source ecosystem. It has excellent integration with the ecosystem: Spring boot, Batch, XD; even the core uses same abstraction starting from Spring Framework 4. Some of the messaging abstraction were moved in the framework, as proof that the basic messaging abstraction of Spring Integration is very strong. Now Spring framework for instance use the messaging abstraction for Spring Web, web socket support.
Another good thing in a Spring application with Spring integration respect to use Apache Camel is that with Spring integration, you can use only one Application Context. Remember that the Camel Context is a Spring context. if you have the chance of use a new Spring version, I suggest to use Spring Integration Java DSL for configuration. I use it on my new projects, and it feels more readable and clear. I hope that this reflection can help you for the your evaluations.
Actually, I would say FTP has graduated its incubation period. You can do a simple search on SI forums/JIRA to see what new features were implemented and bugs that were fixed. From various chatter it seems like there is already some production usage out of it, so I would suggest to give it a second look and of course communicate your concerns to us via
http://forum.springsource.org/forumdisplay.php?42-Integration
https://jira.springsource.org/browse/INT
Cheers
Oleg
Disclaimer: I am Spring Integration committer
One reason to use Camel over Spring Integration is when you need a more featureful EIP set. Spring Integration doesn't provide abstractions over things such as ThreadPool.
Camel does provide additional constructs for this simplifying some of the aspects of working with concurrent code:
http://camel.apache.org/camel-23-threadpool-configuration.html
If you have no need for this sort of thing and just want to connect file, JMS, FTP endpoints etc... then just use Spring Integration.
Camel act as middleware for application where one can perform data modeling, transformation of message values and choreography of messages.
If your current application is in Spring and require features which are supported by Spring Integration of EIP then Spring Integration is the best option else require more third party supports/protocols/file formats etc
I'm looking for a light version of REST for a Java web application I'm developing.
I've looked at RESTlet (www.restlet.org) and the REST plugin for Struts 2, but I haven't made up my mind. I'm leaning towards RESTlet, as it seems to be lighter.
Has anyone implemented a RESTful layer without any of the the frameworks or with the frameworks?
Any performance issues that you've seen because of the new web layer?
Did the introduction of REST added unmanageable or unreasonable complexity to your project? (Some complexity is understandable, but what I mean is just plain overkilling your design just to add REST)
I'm a huge fan of JAX-RS - I think they've done a great job with that specification. I use it on a number of projects and its been a joy to work with.
JAX-RS lets you create REST resources using POJOs with simple annotations dealing with the URI mappings, HTTP methods and content negotiation all integrated nicely with dependency injection. There's no complex APIs to learn; just the core REST concepts (URIs, headers/response codes and content negotiation) are required. FWIW JAX-RS is quite Rails-ish from the controller point of view
There are a number of JAX-RS implementations out there - see this thread for a discussion.
My personal recommendation is to use Jersey as its got the biggest, most active community behind it, has the best features at the time of writing (WADL support, implicit views, spring integration, nice REST client API); though if you are using JBoss/SEAM you might find RESTeasy integrates a little better.
I'm a big fan of Restlet, but I usually use it to implement apps whose primary role is to be a RESTful web service. It sounds like you're looking to add a RESTful API to an existing application. If that's the case, JAX-RS's (or Enunciate's) annotation-based approach might be a better fit for your project.
As for Restlet, I can tell you that I've been very impressed with the developers and the community; they're very active, engaged, responsive, and committed to a stable, efficient, reliable, and effective framework. My single favorite aspect of the framework is that it is a ground-up implementation of the REST paradigm; therefore there is no impedance-mismatch between a Restlet app's external API and internal implementation. I also really like how flexible it is - it can run inside a Java application container/server such as JBoss, Tomcat, Jetty, etc, or standalone, with an embedded HTTP server library.
Well, I've used Enunciate quite a bit. It uses simple annotations to provide either REST and/or SOAP endpoints.
http://enunciate.codehaus.org
Plus, Ryan Heaton has always provided top-notch support for things, too.
You know there is a new JCP API for Accessing RESTful Services, also:
JAX-RS JCP311
https://jsr311.dev.java.net/
The open source version is called Project Jersey
I am working on a REST API for gliffy.com and we ended up rolling our own. We didn't want to have to bring in Struts 2, Spring, or any other framework. I looked at RESTLet and found it incredibly confusing and over complicated.
Apache has an implementation of the JAX-RS spec, but it is not finalized and also has some oddities to it. We're tentatively planning to open source our solution, but that's not for a few months.
Rolling your own is easy, though. The Servlet Specification gives you everything you need, and you can easily connect to a database via Hibernate (see http://www.naildrivin5.com/daveblog5000/?p=39 for how to set up JPA without using EJB3).
I found restlet to be a really elegant architecture. I'm working in the .net world so it was not an option for me, but I was able to build my own framework following the same basic principles of restlet.
I have found the conversion of our WCF contract-based SOA application to REST based one has significantly simplified the application,