I am implementing a batch application using springboot 2.4.3 + jsr352. There is a simple batchlet class(sleepybatchlet) defined. I am trying to reference it in the JSL. but It fails saying classnotfound exception when the job is started using joboperator.
sleepy-batchlet.xml:
<job xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/jobXML_1_0.xsd" restartable="true" version="1.0" id="sleepy-batchlet">
<step id="step1">
<batchlet ref="sleepyBatchlet">
<properties>
<property name="sleep.time.seconds" value="#{jobParameters['sleep.time.seconds']}" />
</properties>
</batchlet>
</step>
</job>
Below is my batchlet class which is annotated with #Named
#Named
public class SleepyBatchlet extends AbstractBatchlet{
private final static Logger logger = Logger.getLogger(SleepyBatchlet.class.getName());
private Map<ReportMetaData,byte[]> pdfMetadataMap;
/**
* Logging helper.
*/
protected static void log(String method, Object msg) {
System.out.println("SleepyBatchlet: " + method + ": " + msg);
// logger.info("SleepyBatchlet: " + method + ": " + String.valueOf(msg));
}
/**
* This flag gets set if the batchlet is stopped. This will break the batchlet
* out of its sleepy loop.
*/
private volatile boolean stopRequested = false;
/**
* The total sleep time, in seconds.
*/
#Inject
#BatchProperty(name = "sleep.time.seconds")
String sleepTimeSecondsProperty;
private int sleepTime_s = 3;
#Inject
private JschFileUtil jschFileUtil;
#Override
public String process() throws Exception {
log("process", "entry");
System.out.println("Test");
return "exitStatus";
}
/**
* Called if the batchlet is stopped by the container.
*/
#Override
public void stop() throws Exception {
log("stop:", "");
stopRequested = true;
}
}
Defined the bean in java configuration class as well.
#Autowired
private SleepyBatchlet sleepyBatchlet;
#Bean
public Batchlet fooBatchlet() {
return sleepyBatchlet;
}
But for some reason, Its not getting referenced in the JSL. Can someone please suggest what needs to be done to use the bean created already ?
I am trying to reference it in the JSL. but It fails saying classnotfound exception when the job is started using joboperator.
This is because you are referring to the class by its name and not its fully qualified name.
I added sample springboot+jsr352 application here . github.com/MekalaJ/demo
In your example, you need to update your step definition as follows:
<batchlet ref="com.example.demo.batch.SleepyBatchlet">
<properties>
<property name="sleep.time.seconds" value="#{jobParameters['sleep.time.seconds']}" />
</properties>
</batchlet>
Related
Is it possible to get a list of defined jobs in Spring Batch at runtime without using db? Maybe it's possible to get this metadata from jobRepository bean or some similar object?
It is possible to retrieve the list of all job names using JobExplorer.getJobNames().
You first have to define the jobExplorer bean using JobExplorerFactoryBean:
<bean id="jobExplorer" class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
<property name="dataSource" ref="dataSource"/>
</bean>
and then you can inject this bean when you need it.
To list jobs defined as beans, you can just let the spring context inject them for you all the bean types of type Job into a list as below:
#Autowired
private List<? extends Job> jobs;
..
//You can then launch you job given a name.
Alternative strategy to get list of job names that are configured as beans one can use the ListableJobLocator.
#Autowired
ListableJobLocator jobLocator;
....
jobLocator.getJobNames();
This does not require a job repository.
I use these code to list and execute jobs
private String jobName = "";
private JobLauncher jobLauncher = null;
private String selectedJob;
private String statusJob = "Exit Status : ";
private Job job;
ApplicationContext context;
private String[] lstJobs;
/**
* Execute
*/
public ExecuteJobBean() {
this.context = ApplicationContextProvider.getApplicationContext();
this.lstJobs = context.getBeanNamesForType(Job.class);
if (jobLauncher == null)
jobLauncher = (JobLauncher) context.getBean("jobLauncher");
}
/**
* Execute
*/
public void executeJob() {
setJob((Job) context.getBean(this.selectedJob));
try {
statusJob = "Exit Status : ";
JobParameters jobParameters = new JobParametersBuilder().addLong("time", System.currentTimeMillis()).toJobParameters();
JobExecution execution = jobLauncher.run(getJob(), jobParameters);
this.statusJob = execution.getStatus() + ", ";
} catch (Exception e) {
e.printStackTrace();
this.statusJob = "Error, " + e.getMessage();
}
this.statusJob += " Done!!";
}
Our application built and runs on :-
spring.version = 3.2.8.RELEASE
spring.amqp.version = 1.2.0.RELEASE
rabbitmq.version = 3.1.3 (client)
RabbitMQ Server version is 3.1.5
We wanted to upgrade the rabbitmq server from 3.1.5 to 3.3.5 and we did that successfully.
Now we wanted to upgrade the application to use latest version of spring-amqp, RabbitMQ java client, so we have upgrade the following components :-
spring.version = 3.2.8.RELEASE
spring.amqp.version = 1.3.0.RELEASE
rabbitmq.version = 3.2.4 (client)
RabbitMQ Server version is 3.3.5
However after upgrading to spring-amqp to 1.3.0, our application started to hung. Basically we start many listener containers during the application startup, and starting each listener container now takes exactly 60 secs to allow to the next step
after digging deep I found that, the program gets hanging in run() method in SimpleMessageListenerContainer class :-
org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer
public void run() {
boolean aborted = false;
int consecutiveIdles = 0;
int consecutiveMessages = 0;
try {
try {
SimpleMessageListenerContainer.this.redeclareElementsIfNecessary(); // Here is where the programing thread hangs.
this.consumer.start();
this.start.countDown();
}
As mentioned above code, the thread gets hung at redeclareElementsIfNecessary() method, and this method is introduced in this version of spring-rabbit only. I have no clue why its getting hanging there, whatever suggested parameter's that I pass to this SimpleMessageListenerContainer it doesn't seems to be working.
If I revert back to spring-amqp 1.2.0 release with the new RabbitMQ server 3.3.5 all seems to be working fine, but things are not working with new spring-amqp client.
I am kind of stuck here for couple of days now. Spring / Rabbitmq master's out there, can you please help me solving this problem?
Thanks for your quick response, however it seems the code is not reaching to that point and it just get hung just above the snippet that you have provide,
I have comment below on where the code gets hung exactly below
Set<String> queueNames = this.getQueueNamesAsSet();
Map<String, Queue> queueBeans = ((ListableBeanFactory) applicationContext).getBeansOfType(Queue.class); // The code started to hung here
for (Entry<String, Queue> entry : queueBeans.entrySet()) {
Queue queue = entry.getValue();
if (queueNames.contains(queue.getName()) && queue.isAutoDelete()
&& this.rabbitAdmin.getQueueProperties(queue.getName()) == null) {
if (logger.isDebugEnabled()) {
logger.debug("At least one auto-delete queue is missing: " + queue.getName()
+ "; redeclaring context exchanges, queues, bindings.");
}
this.rabbitAdmin.initialize();
break;
}
}
Actually we have upgraded to the latest version of spring-amqp only, that is
spring.version = 3.2.8.RELEASE
spring.amqp.version = 1.3.6.RELEASE
rabbitmq.version = 3.3.4 (client)
RabbitMQ Server version is 3.3.5
however we faced exactly the same issue, so just to find out from which version the issue started I ran down to the lower versions upto 1.3.0, seems the issue is starting in 1.3.0 version of spring-amqp itself. that's the reason.
I have attached the requested information including thread dumps which are based on spring-amqp 1.3.6 only.
Here is the configuration our listener container where the programs hangs, as you can see we have our own SimpleMessageLinstenerContainer which acts as a wrapper for the acutal spring's SimpleMessageListenerContainer,
I have also attached this custom wrapper file for your reference.
<bean id="tlogOutOfCycleMessageListenerPrototype" class="com.myorg.ips.cnccommon.support.amqp.SimpleMessageListenerContainer" scope="prototype">
<property name="channelTransacted" value="true" />
<property name="transactionManager" ref="transactionManager" />
<property name="concurrentConsumers" value="1" />
<property name="taskExecutor" ref="tlogOutOfCycleMessageListenerPool" />
<property name="messageListener" ref="tlogMLAOutOfCycle" />
<property name="errorHandler" ref="tlogOutOfCycleMessageHandler" />
<property name="autoStartup" value="false" />
<property name="instanceNameForLogging" value="site1TlogOutOfCycleMessageListener"/>
<!-- A dummy connection factory which will never be used -->
<property name="connectionFactory" ref="switchCompositeConnectionFactoryPrototype"/>
</bean>
Our wrapper class SimpleMessageListenerContainer.java
package com.myorg.ips.cnccommon.support.amqp;
import org.apache.commons.lang.builder.ToStringBuilder;
import org.slf4j.cal10n.LocLogger;
import org.springframework.util.ErrorHandler;
import com.myorg.ips.amqp.SwitchSiteSupport;
import com.myorg.ips.logging.LoggerFactory;
import com.myorg.ips.system.config.InitialisableSiteAware;
import static com.myorg.ips.logging.SystemWideLogMessages.ERROR_AMQP_FAILED_TO_START_LISTENER;
import static com.myorg.ips.logging.SystemWideLogMessages.INFO_AMQP_STOPPING_LISTENER;
/**
*
* Wrapper for the Spring SimpleMessageListenerContainer which simply allows us to delay (or prevent startup). Can also restart on command.
*
*/
public class SimpleMessageListenerContainer extends org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer implements InitialisableSiteAware {
private static final LocLogger logger = LoggerFactory.getLogger(SimpleMessageListenerContainer.class);
private boolean autoStart = true;
private ErrorHandler exposedErrorHandler;
private boolean springBeanInitialisationAttempted = false;
private boolean springBeanInitialised = false;
private String instanceNameForLogging;
#Override
public void initialize() {
// Do nothing -- we will instead perform the Spring bean initialisation later on via the factory bean, after the connection factory has been set
springBeanInitialisationAttempted = true;
springBeanInitialised = false;
}
#Override
public void initialise() {
SwitchSiteSupport.initialiseIfSiteAware(getMessageListener());
SwitchSiteSupport.initialiseIfSiteAware(getErrorHandler());
// If this object is a Spring bean, we should now complete the initialisation that the Spring framework attempted earlier
if (springBeanInitialisationAttempted && !springBeanInitialised) {
springBeanInitialised = true;
super.initialize();
if (isAutoStartup()) {
start();
}
}
}
#Override
public void configureForSite(final MultiHostConnectionFactory configuredConnectionFactory) {
setConnectionFactory(configuredConnectionFactory);
SwitchSiteSupport.configureIfSiteAware(getMessageListener(), configuredConnectionFactory);
SwitchSiteSupport.configureIfSiteAware(getErrorHandler(), configuredConnectionFactory);
setInstanceNameForLogging(SwitchSiteSupport.replaceWithSiteAlias(instanceNameForLogging, configuredConnectionFactory));
}
#Override
//CHECKSTYLE:OFF Unfortunately the parent springframework class throws and exception, so so do we
protected void doStart() throws Exception {
//CHECKSTYLE:ON
if (autoStart) {
logger.debug("Starting message listener " + instanceNameForLogging);
super.doStart();
logger.debug("Started message listener " + instanceNameForLogging);
}
}
/**
* Start this listener
*/
public void start() {
autoStart = true;
try {
doStart();
//CHECKSTYLE:OFF Unfortunately the parent springframework class throws and exception, so that is what we catch
} catch (Exception e) {
//CHECKSTYLE:ON
logger.error(ERROR_AMQP_FAILED_TO_START_LISTENER, e);
}
}
/**
* Stop listener
*/
public void stop() {
logger.info(INFO_AMQP_STOPPING_LISTENER, getBeanName());
autoStart = false;
doStop();
}
/**
* Stop and start this listener
*/
public void restart() {
stop();
start();
}
/**
* Store the errorHandler in a subclass-specific property so that we can retrieve it later
* #param errorHandler errorHandler
*/
#Override
public void setErrorHandler(final ErrorHandler errorHandler) {
this.exposedErrorHandler = errorHandler;
super.setErrorHandler(errorHandler);
}
/**
* Return the exposed errorHandler
* #return errorHandler
*/
public ErrorHandler getErrorHandler() {
return exposedErrorHandler;
}
public void setInstanceNameForLogging(final String instanceNameForLogging) {
this.instanceNameForLogging = instanceNameForLogging;
}
#Override
public String toString(){
return ToStringBuilder.reflectionToString(this);
}
}
Good catch anyway!
I'm just right now working with that code for Spring AMQP 1.4.
Would you mind sharing:
The configuration for the ListenerContainer on which you hang
A debug analiz for the code from that redeclareElementsIfNecessary()
Actually now that code looks like:
if (queueNames.contains(queue.getName()) && queue.isAutoDelete()
&& this.rabbitAdmin.getQueueProperties(queue.getName()) == null) {
if (logger.isDebugEnabled()) {
logger.debug("At least one auto-delete queue is missing: " + queue.getName()
+ "; redeclaring context exchanges, queues, bindings.");
}
this.rabbitAdmin.initialize();
break;
}
So, it may heppen only on the auto-delete Queue.
Or do you have another picture?..
UPDATE
According to your ThreadDump. This is illegal:
at com.vocalink.ips.amqp.AmqpMessageListenerManager.initialise(AmqpMessageListenerManager.java:106)
at com.vocalink.ips.amqp.SwitchSiteSupport.initialiseIfSiteAware(SwitchSiteSupport.java:29)
at com.vocalink.ips.system.config.AbstractSiteAwareComponentCachingFactory.createAndConfigureSiteAwareComponent(AbstractSiteAwareComponentCachingFactory.java:51)
You can't start component within initialization phase. Or leave it to the container, or just do start manually somewhere at runtime, when all of your beans are already created.
For example you can do that using ApplicationListener<ContextRefreshedEvent>.
I have some service bean which is accessible by identifier someSpecificService which I need to modify.
Beans are defined in different xml files and are collected together in runtime. So one big xml file is created where all these xmls are imported:
context.xml
....
<import path="spring1.xml" />
<import path="spring2.xml" />
...
So there is following configuration:
<-- definitions from spring1.xml -->
<alias name="defaultSomeSpecificService" alias="someSpecificService" />
<bean id="defaultSomeSpecificService" class="..."/>
....
<!-- definitions from spring2.xml -->
<alias name="myOwnSomeSpecificService" alias="someSpecificService" />
<bean id="myOwnSomeSpecificService" class="..." /> <!-- how to inject previously defined someSpecificService into this new bean? -->
I would like to override someSpecificService from spring1.xml in spring2.xml, however I do need to inject previously defined bean defaultSomeSpecificService and all I know is its alias name someSpecificService which I need to redefine to new bean myOwnSomeSpecificService.
Is it possible to implement?
One solution would be to avoid trying to override the definition, by creating a proxy for the service implementation to intercept all calls towards it.
1) For the sake of the example, suppose the service would be something like:
public interface Service {
public String run();
}
public class ExistingServiceImpl implements Service {
#Override
public String run() {
throw new IllegalStateException("Muahahahaha!");
}
}
2) Implement an interceptor instead of myOwnSomeSpecificService:
import org.aopalliance.intercept.MethodInterceptor;
import org.aopalliance.intercept.MethodInvocation;
public class SomeSpecificServiceInterceptor implements MethodInterceptor {
#Override
public Object invoke(MethodInvocation invocation) throws Throwable {
String status;
try {
// allow the original invocation to actually execute
status = String.valueOf(invocation.proceed());
} catch (IllegalStateException e) {
System.out.println("Existing service threw the following exception [" + e.getMessage() + "]");
status = "FAIL";
}
return status;
}
}
3) In spring2.xml define the proxy creator and the interceptor:
<bean id="serviceInterceptor" class="com.nsn.SomeSpecificServiceInterceptor" />
<bean id="proxyCreator" class="org.springframework.aop.framework.autoproxy.BeanNameAutoProxyCreator">
<property name="beanNames" value="someSpecificService"/>
<property name="interceptorNames">
<list>
<value>serviceInterceptor</value>
</list>
</property>
</bean>
4) Running a small example such as:
public class Main {
public static void main(String[] args) {
Service service = new ClassPathXmlApplicationContext("context.xml").getBean("someSpecificService", Service.class);
System.out.println("Service execution status [" + service.run() + "]");
}
}
... instead of the IllegalStateException stacktrace you'd normally expect, it will print:
Existing service threw the following exception [Muahahahaha!]
Service execution status [FAIL]
Please note that in this example the service instance is not injected in the interceptor as you asked because I had no user for it. However should you really need it, you can easily inject it via constructor/property/etc because the interceptor is a spring bean itself.
I want to create a session-scoped bean to monitor activations and passivations of HTTP sessions. The bean is very simple:
package my.log;
import javax.servlet.http.HttpSessionActivationListener;
import javax.servlet.http.HttpSessionEvent;
import org.apache.log4j.Logger;
public class SessionLoggingListenerBean implements HttpSessionActivationListener {
private final Logger LOG = Logger.getLogger(this.getClass());
public SessionLoggingListenerBean() {
LOG.info("SessionLoggingListenerBean starting");
}
public void init() {
LOG.info("SessionLoggingListenerBean init");
}
public void sessionDidActivate(HttpSessionEvent event) {
LOG.info("Session " + event.getSession().getId() + " activated");
}
public void sessionWillPassivate(HttpSessionEvent event) {
LOG.info("Session " + event.getSession().getId() + " will passivate");
}
}
Bean definition in application context:
<bean id="sessionLoggingListenerBean" class="my.log.SessionLoggingListenerBean" scope="session" init-method="init" lazy-init="false"/>
With this configuration there is no logs from this class, even from the constructor or init() method. Apparently, Spring does not create this bean.
By trial and error I checked that Spring instantiates such a bean when it is needed by another bean, e.g. used by UI. Is there any other (better) way? Is it a bug in Spring?
Spring version used: 2.0.8.
HttpSessionActivationListener is part of the javax.servlet.http package. That should give you a hint that it should be managed by the Servlet container. In your case, you aren't registering the Listener with your ServletContext, neither through the web.xml or a SerlvetContainerInitializer.
Through web.xml you wouldn't be able to make it both a Spring and Servlet container managed object so instead these workarounds exist, first, second.
If you are using a WebApplicationInitializer, you can instantiate your AnnotationConfigWebApplicationContext, have the SessionLoggingListenerBean bean created, retrieve it and use it with
SessionLoggingListenerBean yourBean = context.getBean(SessionLoggingListenerBean.class);
servletContext.addListener(yourBean);
After some experimenting I think it is better not to use Spring for this purpose. I've modified the class to implement also HttpSessionListener and Serializable:
public class SessionLoggingListener implements HttpSessionListener,
HttpSessionActivationListener, Serializable {
private static final long serialVersionUID = -763785365219658010L;
private static final Logger LOG = Logger.getLogger(SessionLoggingListener.class);
public SessionLoggingListener() {
LOG.info("SessionLoggingListener created");
}
public void sessionDidActivate(HttpSessionEvent event) {
LOG.info("Session " + event.getSession().getId() + " activated");
}
public void sessionWillPassivate(HttpSessionEvent event) {
LOG.info("Session " + event.getSession().getId() + " will passivate");
}
public void sessionCreated(HttpSessionEvent event) {
final HttpSession session = event.getSession();
LOG.info("Session " + session.getId() + " created. MaxInactiveInterval: " + session.getMaxInactiveInterval() + " s");
session.setAttribute(this.getClass().getName(), this);
}
public void sessionDestroyed(HttpSessionEvent event) {
LOG.info("Session " + event.getSession().getId() + " destroyed");
event.getSession().removeAttribute(this.getClass().getName());
}
}
and added it to web.xml:
<listener>
<listener-class>
evo.log.SessionLoggingListener
</listener-class>
</listener>
From now on, whenever a new session is created, the listener binds to it (session.setAttribute(...)). This is necessary to make the container notice the listener about session activation or passivation.
In case of Spring and sessions - according to this forum thread Spring does not load session beans until they are requested:
Session bean is treated as a special form of "prototype". That means it will follow prototype symantics in creating an instance.
For me this is an unintuitive and not well documented behavior.
I'm trying to create a Spring Batch job using a ListItemReader<String>, ItemProcessor<String, String> and ItemWriter<String>.
The XML looks like the following,
<job id="sourceJob" xmlns="http://www.springframework.org/schema/batch">
<step id="step1" next="step2">
<tasklet>
<chunk reader="svnSourceItemReader"
processor="metadataItemProcessor"
writer="metadataItemWriter"
commit-interval="1" />
</tasklet>
</step>
<step id="step2">
<tasklet ref="lastRevisionLoggerTasklet"></tasklet>
</step>
</job>
<bean id="svnSourceItemReader"
class="com.example.repository.batch.SvnSourceItemReader"
scope="prototype">
<constructor-arg index="0">
<list>
<value>doc1.xkbml</value>
<value>doc2.xkbml</value>
<value>doc3.xkbml</value>
</list>
</constructor-arg>
</bean>
<bean id="metadataItemProcessor"
class="com.example.repository.batch.MetadataItemProcessor"
scope="prototype" />
<bean id="metadataItemWriter"
class="com.example.repository.batch.MetadataItemWriter"
scope="prototype" />
The reader, processor and writer are vanilla,
public class SvnSourceItemReader extends ListItemReader<String> {
public SvnSourceItemReader(List<String> list) {
super(list);
System.out.println("Reading data list " + list);
}
#Override
public String read() {
String out = (String) super.read();
System.out.println("Reading data " + out);
return out;
}
}
public class MetadataItemProcessor implements ItemProcessor<String, String> {
#Override
public String process(String i) throws Exception {
System.out.println("Processing " + i + " : documentId " + documentId);
return i;
}
}
public class MetadataItemWriter implements ItemWriter<String> {
#Override
public void write(List<? extends String> list) throws Exception {
System.out.println("Writing " + list);
}
}
The job is started like this, but on a schedule of every 10 seconds.
long nanoBits = System.nanoTime() % 1000000L;
if (nanoBits < 0) {
nanoBits *= -1;
}
String dateParam = new Date().toString() + System.currentTimeMillis()
+ "." + nanoBits;
param = new JobParametersBuilder().addString("date", dateParam)
.toJobParameters();
JobExecution execution = jobLauncher.run(job, param);
When the application starts, I see it read, process and write each of the three items in the list passed to the reader.
Reading data doc1.xkbml
Processing doc1.xkbml : documentId doc1
Writing [doc1.xkbml]
Reading data doc2.xkbml
Processing doc2.xkbml : documentId doc2
Writing [doc2.xkbml]
Reading data doc3.xkbml
Processing doc3.xkbml : documentId doc3
Writing [doc3.xkbml]
Because this sourceJob is on a scheduled timer, every 10 seconds I expected to see that list processed, but instead I see on all subsequent runs.
Reading data null
Does anyone know why this is happening? I'm new to Spring Batch and just can't get my hands around the issue.
Thanks /w
The problem is that you marked your reader as scope="prototype". It should be scope="step".
In Spring-batch there are only two scopes: singleton (the default) and step.
From the javadoc:
StepScope: Scope for step context. Objects in this scope use the
Spring container as an object factory, so there is only one instance
of such a bean per executing step. All objects in this scope are
(no need to decorate the bean definitions).
and
Using a scope of Step is required in order to use late binding since
the bean cannot actually be instantiated until the Step starts, which
allows the attributes to be found.
During the Spring context startup look at your log and you will see this line:
INFO: Done executing SQL script from class path resource
[org/springframework/batch/core/schema-hsqldb.sql] in 9 ms.
Reading data list [doc1.xkbml, doc2.xkbml, doc3.xkbml]
as you can see your reader has already been created and managed as a singleton; dynamic beans in spring-batch context should be managed with the special step scope so that Spring will create a fresh copy of the bean every time a step is executed.
In your reader, ListItemReader.read() is written as:
public T read() {
if (!list.isEmpty()) {
return list.remove(0);
}
return null;
}
In each read items are removed from original list! The reader is constructed once and, on second job execution, the list is empty!
Just an additional information: you can also use JavaConfig instead of the xml config file, and annotate the reader bean declaration with #StepConfig.
ex:
#Configuration
#EnableBatchProcessing
public class MyConfig {
...
#Bean
#StepScope
public ItemReader<HeadingBreakevenAssociation> readerHeadingBreakevenAssociationList(){
ItemReader<Person> itemReader = new ListItemReader<Person>(myList);
return itemReader;
}
}