We're running a Spring Batch Web-Application for Importing CSV Files into a Database. This Web-Application is currently evolving and is constantly extended by new jobs.
the current update procedure looks like this:
1. Write new Code
2. Build a war file
3. Deploy the newly build war file and replace the whole Web Application on the Tomcat Webserver
This might bring us into trouble, when the running system is currently importing / writing Files to the Database.
I wanted to know if there is a smart way to maybe upgrade the spring batch jobs seperately ?
I already thought about splitting the Project into many different Web-Applications but this might be a lot of overhead with all the libraries bundled into the war file(s).
Are there any best practices for building that sort of Application ?
Thanks for your Help !
This packaging model is known to cause a lot of issues like the one you are facing. I recommend to package your jobs as separate jars and make your application launch those jobs in separate processes. With this model, you can deploy/upgrade jobs without impacting the web application used to launch them.
For the record, Spring Batch Admin suffered from this packaging model (as described here) and the recommended replacement is Spring Cloud Data Flow (which uses the model I described previously)
Related
Hi folks need some opinions here.
I already have a spring boot application holding all my rest APIs running on tomcat that ships in with spring-boot-starter-web.
I would like to set up jobs using spring batch that will be schedule via kubernetes. The idea is to share the same business logic instead of creating a standalone batch project which i need to maintain double business logic.
Question, scheduling via kubernetes meaning i will be firing java -jar someJar --spring.batch.jobNames=xxx in container, doing that it will also start up all my RestApis right? which in turn unnecessary and waste of resources. Anyway to mitigate this or my understanding is wrong?
The way I would implement this is by extracting the common business logic in a separate module, and make the batch app and the webapp depend on that module.
I have some questions about Spring Boot and Hibernate.
I discussed with one developer and he said me he compile his Rest Api, developed with Spring Boot, in Jar and used the tomcat server provided by Spring Boot and deploy the jar on the server. But in my case, i use the war packaging and i deploy on my tomcat server but he said it’s less performing.
I don’t know why and i asking me if you developed in microservices your Resp Api with Spring boot and you use his solution, do you create multiple tomcat instance ?
About hibernate, i used HQL for some queries and he said me it’s bad way because it’s dangerous with Sql injection, it’s true ?
I need some answers of people who can advice me.
Sorry if my english, it’s not great and thank you very much, in advance, for your answer.
In the Java Cloud space there are 2 major directions:
Java EE:
Create a Docker base image which includes a Java EE application/ web server such (e.g. Tomcat, WildFly, GlassFish,...)
Create a (thin) WAR file
Create a Docker image based on your base image which deploys the WAR file to the application server
Spring Boot:
Create a single JAR file using the Spring Boot Maven Plugin
Create a Docker image which executes the JAR file
I guess you could also mix the two approaches (create a Spring Boot WAR file), but the single JAR file approach is much more common with Spring Boot (I'm currently using this approach).
Some Java EE experts such as Adam Bien promote the first approach (less dependencies, smaller WAR files, smaller Docker images, more standard APIs (though what "standard" means is currently changing with the transition from Java EE to Jakarta EE)).
I can't tell you which one has a higher performance. Either way there will be one server started per Docker image.
Regarding Hibernate and SQL injection:
I'm not using HQL but JPQL, but in the end it's more or less the same as with JDBC: Don't string concatenate queries with input from the user. Always use some kind of prepared statements with variables to format the user input.
If you are using spring boot you implement more or less a standalone application (even if it´s a webapplication), therefore yes there will be an own embedded tomcat, jetty, whatever instance for each spring boot application.
SQL injection has nothing to do with the way how you deploy an application, it´s only a question how you create your HQL queries.
However, that´s an interesting discussion. In our company we are quite old fashioned and create an WAR file of our webapplication, and deploy it on our Glassfish server where serveral other web applications are running under the same instance.
Florian
I'm currently developing a JavaFX application which uses embedded Derby database. The application needs fixed data of around 13000 products. I can deploy app without any problem, and after deployment, if I copy database files to deployed app's directory, the app can find and use that database. My question is what is the best practice to deploy this kind of application with the initial data? Thanks!
(Note: I'm using Intellij IDE and Maven if it helps deployment for my purpose.)
I would use flywaydb to run database initialization scripts upon startup. I have done this in the past and it works well.
It has the additional benefit in that if you release a new version, you can just include scripts to evolve the schema for that new version if necessary.
I have 50,000,000 files that need to be processed using 3-5 different filters configured in workflows
I plan to use microservice architecture
My Questions
i want to use spring integration and batch, to run the workflows. and design the workflows, do you agree or is there another java based system you recommend?
can spring batch can handle "long running i.e. days" workflows.
can spring batch/integration load xml files on the fly
I think Spring Batch is pretty good for this job, below my answers.
I recommend you Spring Batch for this job. It's easy to use and in combination with Spring Workflow are good for the workflow desing.
Yes, it's really good. You need to configure it well.
I'm not sure what are you saying with on the fly. (batch files or configuration files). For batch files yes. For configuration files, it depends on how you load the configuration and how you will use the context.
IMHO Spring Batch can process files based on multiple filters. It can also be easily customized to fit most of your needs and has really fast processing speeds. However, I haven't tried it with anything close to 50,000,000 files, so can't vouch for that.
To run a Spring Batch application as a microservice, take a look at Spring Boot and Spring Cloud Task. Also, look into Spring Cloud Dataflow for orchestration.
I'm looking for a Java Web Container (like jetty and tomcat) or a tool in which I can create/remove server instances through a management console.
The problem is that my organization needs to create different instances of a test server for quality control testing (against different database configurations). Currently, I'm having to manually copy a Tomcat "catalina_base" template directory and make any changes needed for the test being run. It would be nice to have a unified interface where I could click a button to create a new instance (and click another to remove it).
Edit 1
Must be able to run on Windows Server 2003.
Edit 2
I'm getting a lot of answers that have to do with build, so I'm going to add some extra information about the application. The application is a standard Java EE web application that is built using an ANT script. We use a container managed JNDI DataSource for connecting to the database. Our procedures, which are hold overs from 20+ years ago, have dictated that every new database change needs to be in a new schema.
So say a customer reports that our application is displaying a calculation wrong - the first thing we do is create a new database schema, then we run the create script for all of the database objects, and lastly copy the data from production for testing to that new schema. When we've fixed the bug (either application side or database side), our quality control person, needs the fixed application and the schema within the DataSource changed to that of the new "test environment". When they've completed their testing, we stage the code to be included in the next scheduled release.
The catch is, this process is multiplied by a number of developers and a number of concurrent bugs fixed and features added. Currently, there are over 20+ Tomcat instances managing different versions of the application and database objects. I'm constantly having to create new Tomcat instances and remove old ones, as features are added and quality control has completed.
it sounds like what you really need is a build a deployment tool like Continuum
You can do so with jetty.You can create your own Java Class with the specified configuration as an embedded server and run it from the prompt or through some web interface.
you can check this out
http://docs.codehaus.org/display/JETTY/Embedding+Jetty
Have you thought about a configuration management tool like Chef?
It sounds like you should just update your application's build script to be able to accept different parameters for things like "use test1.properties for database settings" or "use prod2.properties", and then deploy that rebuilt application.