I am new to Spring batch framework. Can we create a batch job and run that without using maven?
I have seen spring batch samples connecting spring batch with maven..
Can anyone give me a sample Spring batch project that can run without using maven dependencies? Or at least give an idea about this.
Yes, run them in a container like tomcat see here for more information; Spring-Batch-standalone-Server-architecture.
Related
Background
I have built a console java application using kotlin and gradle.
The gradle file creates a fat jar which I can run from the command line using
java -jar <project>.jar
The jar contains the application.properties file from which properties are read.
Problem
I would like to specify on the command line that the application.properties file should be read from some external path.
When using spring boot, I have used
java -jar -Dspring.config.location=somepath/application.properties <project>.jar
and this works.
But it does not seem to be working in the non-spring boot application
Question
Is it possible to specify external configuration on the command line for non spring boot applications?
Spring boot has a whole chapter in the documentation which deals with various ways of configuration.
Obviously if you don't have spring boot you should implement something similar to it by yourself.
First thing you should decide - at which level you need the configuration to be integrated into your application:
Do you only want to read the key/values from command line or maybe rely on environment variables or system properties?
In general, what is the source of your configuration: Yaml? Properties file? maybe consul or etc.d?
Do you want to create a java object that reflects the configurations that you've read (like classes annotated with #ConfigurationProperties in spring boot do?
Do you want to support only one source of configuration or you want the various sources of configurations to be supported?
If you ware using Spring, do you want configuration properties to be automatically injected into beans?
If you're planning to use properties/yaml (like application.properties in spring boot) - where do you want to place them? Non spring boot application won't read them "auto-magically", you'll have to implement this logic.
Are you planning to deal with profiles (non-spring-boot application still supports flavors of loading different beans depending on specified profile).
Spring boot has answered all these questions and more.
Here are some options that you might want to give a try to if you're running outside the spring boot context but still have spring application:
Since spring 3.1, I guess, there is a#PropertySource annotation that you can use to make spring load properties from the file in the classpath or some "place" in the filesystem. This article summarizes the usage of this method as well as compares what spring boot has up on its sleeves as opposed to regular spring application. This is also a nice tutorial that covers regular spring features.
Something out of spring eco-system but still can be useful: apache common configuration project. There are some workarounds to integrate it with spring application, see here
Considering all the answers here I concluded that though it is possible to enable external configuration in applications that are not spring boot, it does require some effort.
Therefore I decided to use Spring Boot in the container.
We're running a Spring Batch Web-Application for Importing CSV Files into a Database. This Web-Application is currently evolving and is constantly extended by new jobs.
the current update procedure looks like this:
1. Write new Code
2. Build a war file
3. Deploy the newly build war file and replace the whole Web Application on the Tomcat Webserver
This might bring us into trouble, when the running system is currently importing / writing Files to the Database.
I wanted to know if there is a smart way to maybe upgrade the spring batch jobs seperately ?
I already thought about splitting the Project into many different Web-Applications but this might be a lot of overhead with all the libraries bundled into the war file(s).
Are there any best practices for building that sort of Application ?
Thanks for your Help !
This packaging model is known to cause a lot of issues like the one you are facing. I recommend to package your jobs as separate jars and make your application launch those jobs in separate processes. With this model, you can deploy/upgrade jobs without impacting the web application used to launch them.
For the record, Spring Batch Admin suffered from this packaging model (as described here) and the recommended replacement is Spring Cloud Data Flow (which uses the model I described previously)
Flyway has several integration options.
I'm trying to determine what the pros/cons are of using the Gradle integration vs the Spring Boot integration given that your project is already using both Spring Boot and Gradle.
The only thing I can think of is that if you want to be able to do migrations without starting the application or want to save time by not migrating every time you start the app then the Gradle choice could be better.
Think of it as build time vs run time.
In general you will build an artifact once and deploy it to many environments, so run time is a much better fit.
However sometimes build time makes sense. This is primarily for situations where you need a fully migrated database as part of the build, in order to for example generate code based on the structure of that database using frameworks like jOOQ or QueryDSL.
I have 50,000,000 files that need to be processed using 3-5 different filters configured in workflows
I plan to use microservice architecture
My Questions
i want to use spring integration and batch, to run the workflows. and design the workflows, do you agree or is there another java based system you recommend?
can spring batch can handle "long running i.e. days" workflows.
can spring batch/integration load xml files on the fly
I think Spring Batch is pretty good for this job, below my answers.
I recommend you Spring Batch for this job. It's easy to use and in combination with Spring Workflow are good for the workflow desing.
Yes, it's really good. You need to configure it well.
I'm not sure what are you saying with on the fly. (batch files or configuration files). For batch files yes. For configuration files, it depends on how you load the configuration and how you will use the context.
IMHO Spring Batch can process files based on multiple filters. It can also be easily customized to fit most of your needs and has really fast processing speeds. However, I haven't tried it with anything close to 50,000,000 files, so can't vouch for that.
To run a Spring Batch application as a microservice, take a look at Spring Boot and Spring Cloud Task. Also, look into Spring Cloud Dataflow for orchestration.
I am a newbie to Spring and Groovy integration.
I have some strange requirement. Please help in achieving it.
I have Groovy scripts in a database and in a flow I want to get a script from the database and execute the script.
But the problem here is I want the Spring container inside that script and I want to call any beans of Spring container from Groovy script.
Now the problem is how to specify the Spring container to scan the Groovy scripts in database. Also Groovy scripts get added and removed from the database. I also don't want to restart the application when a new script is added.