I have some points to resolve it in spring integration, I'm kind of newbie in spring integration.
The scenario is to process three dbf files and extract the data and send it to HTTP rest service as JSON if there is no response from that service during 3 mins should wait 10 mins and attempt again. The rest service will reply with json.
now I have three to four things I can't solve it:
I need to read the reply from rest service and depend on that reply decide either to retry or to finish the process. (beside if I send message to rest and no response should also retry)
I need to get multiple files from dynamic folder names for example ({wherever}/20140101/HERE WILL FIND THE THREE DBF FILES). there is any possibility to make folder name as pattern or as date with certain format? and the file adapter can process many files or files by file.
there is any possibility to process dbf files through spring integration and transform it to json
below my spring integration configuration:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/file http://www.springframework.org/schema/integration/file/spring-integration-file.xsd
http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task.xsd
http://www.springframework.org/schema/integration/http http://www.springframework.org/schema/integration/http/spring-integration-http.xsd"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-file="http://www.springframework.org/schema/integration/file"
xmlns:int-http="http://www.springframework.org/schema/integration/http"
xmlns:task="http://www.springframework.org/schema/task">
<int:service-activator input-channel="filesInChannelDBF"
output-channel="requestChannel" >
<bean class="com.mm.integration.serviceactivator.FileProcessor" />
</int:service-activator>
<int:channel id="requestChannel" />
<int-http:outbound-gateway request-channel="requestChannel"
url="http://localhost:8090/receiveGateway" http-method="POST" />
<int-file:inbound-channel-adapter id="filesInChannelDBF"
directory="file:input" filename-pattern="FILENAME*">
<int:poller id="poller" fixed-rate="100" task-executor="executor" />
</int-file:inbound-channel-adapter>
<task:executor id="executor" pool-size="10" />
</beans>
and here my pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven- 4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mm.integration</groupId>
<artifactId>XXXX</artifactId>
<version>0.0.1-SNAPSHOT</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.1.9.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-integration</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
and here my spring entry point.
#RestController
#EnableAutoConfiguration
#EnableIntegration
#ImportResource("classpath:spring/config/concurrentFileProcessing-config.xml")
public class MainApp {
#RequestMapping(value="/receiveGateway" , method=RequestMethod.POST)
public String testGateway(String jSon){
System.out.println("Starting process the message [reciveing]");
return "{HelloMessage: \"Hello\"}";
}
public static void main(String[] args) throws Exception {
SpringApplication.run(MainApp.class, args);
}
}
1- I need to read the reply from rest service and depend on that reply decide either to retry or to finish the process.
Any processor (e.g. <int-http:outbound-gateway>) endpoint can be configured with <request-handler-advice-chain>. And there is out-of-the-box advices for you: RequestHandlerRetryAdvice and ExpressionEvaluatingRequestHandlerAdvice. You can specify the retry advice as first one and expression as the second. With the last you can make some decision to throw or not an exception to initiate that wrapping retry. Because this one can retry only on exception. It would be better than get deal with RetryContext to have an ability to specify the custom RetryPolicy independently of exceptions.
In addition consider to use request-factory with timeout options to get deal with no response case.
2- I need to get multiple files from dynamic folder names for example ({wherever}/20140101/HERE WILL FIND THE THREE DBF FILES).
Consider to use RecursiveLeafOnlyDirectoryScanner to scan those timestamp directories from the specified root dir. From other side you even can provide any DirectoryScanner implementation to achieve the requiremnts.
3- there is any possibility to process dbf files through spring integration and transform it to json
No, Spring Integration doesn't support that feature. You can consider to use some existing tool to read DBF (https://github.com/iryndin/jdbf) and then build JSON manually.
From other side feel free to contribute it back to the framework as an extension with appropriate adapters.
Related
I am using spring boot application with appliaction.properties.
In the project there is XML file written by another company to configure Stomp, and there is hardcoded IP of the server, but I want to change those IP to take value from application.properties.
is there any way to do it?
application.properties
property.env=SLA
property.endpoint=http://172.1.1.139/router
tcmanager.xml
<tcmanager>
<channelbuilders>
<channelbuilder>
<class>com.company.tc.stomp.TcStompChannelBuilder</class>
<config>
<properties>
<property>brokerURL=[property.endpoint]</property> //this is the place
<property>login=login</property>
<property>passcode=pass</property>
</properties>
<in> [...] </in>
<out> [...]</out>
</config>
</channelbuilder>
</channelbuilders>
<processingbuilders>
<processingbuilder>
<class>com.company.tc.template.camel.MainProcessingBuilder</class>
<config>
<properties>
<property>p1=1</property>
</properties>
</config>
</processingbuilder>
</processingbuilders>
</tcmanager>
I'm trying to intercept rest service calls with an aspect in the following manner
package mypackage.services.Service;
#Component
public class Service {
#Override
public Response helloService() {
return handleResult("Hello test " + new Date());
}
}
#Component
#Aspect
public class AuditLog {
#Before("execution(* mypackage.services.Service.*(..))")
public void beforeServcie(JoinPoint jp){
log.info("Before ",jp.getSignature().getName());
}
}
I'm using the following maven dependencies
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aop</artifactId>
<version>4.3.6</version>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjrt</artifactId>
<version>1.8.10</version>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>1.8.10</version>
</dependency>
This maven plugin
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>aspectj-maven-plugin</artifactId>
<version>1.0</version>
</plugin>
And my configuration xml contains
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:aop="http://www.springframework.org/schema/aop"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop.xsd">
<context:component-scan base-package="mypackage"/>
<aop:aspectj-autoproxy proxy-target-class="true" />
also in the Application class I've added the following annotation
#Configuration
#EnableAspectJAutoProxy(proxyTargetClass=true)
public class Configuration{
...
}
On startup, by logging beans in the ApplicationContext, I can see that the aspect class "AuditLog" is being created.
I've set 2 breakpoints, but the debugger does not stop at the "beforeServcie" method but it does stop at the "helloService".
What am I missing?
Try this
execution(* mypackage.services.Service.*.*(..))
instead of
execution(* mypackage.services.Service.*(..))
If you are using spring-boot then instead of automatically adding dependency jars you can do
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
If you're using XML config <aop:aspectj-autoproxy ... /> then no need to have #EnableAspectJAutoProxy. It probably doesn't matter since AFAIK XML config wins over annotation config but better to avoid duplication
I am not quite sure why do you need aspectj-maven-plugin since Spring implements AOP by proxy and AFAIK this plugin is only needed for compile-time, post compile-time or load time weaving which are different concepts, see Spring AOP vs AspectJ
Now all the above mentioned points may not resolve your issue but the following might
execution(* mypackage.services.Service.Service.*(..))
And, don't set proxyTargetClass=true, let it be default false.
Explanation
The format is execution(<return type> <package name>.<class name>.<method name>(..)
The package name here is mypackage.services.Service and the class name is Service.
I'm currently working on a Maven based project in Eclipse which I want to connect to a database using Hibernate + JPA. I have created a persistence.xml file and placed it in the directory <project>/src/main/resources/META-INF. When I run the application it will throw the very known exception No Persistence provider for EntityManager named
The contents of my persistence.xml:
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.1"
xmlns="http://xmlns.jcp.org/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence
http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
<persistence-unit name="NetworkMonDB">
<provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>
<exclude-unlisted-classes>false</exclude-unlisted-classes>
<properties>
<property name="javax.persistence.jdbc.driver" value="com.mysql.cj.jdbc.Driver" />
<property name="hibernate.hbm2ddl.auto" value="validate" />
<property name="hibernate.dialect" value="org.hibernate.dialect.MySQLDialect" />
</properties>
</persistence-unit>
How I'm initializing this in my application:
public void initializeDatabase(){
final HashMap<String, String> dbConfig = new HashMap<String, String>();
dbConfig.put("javax.persistence.jdbc.url", this.properties.getProperty("database.uri"));
dbConfig.put("javax.persistence.jdbc.user", this.properties.getProperty("database.usr"));
dbConfig.put("javax.persistence.jdbc.password", this.properties.getProperty("database.psw"));
final EntityManagerFactory factory = Persistence.createEntityManagerFactory("NetworkMonDB", dbConfig); //exception thrown at this point.
this.entityManager = factory.createEntityManager();
}
And here are my Maven Dependencies:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.2.4.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>5.2.4.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
<version>5.3.2.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate.javax.persistence</groupId>
<artifactId>hibernate-jpa-2.1-api</artifactId>
<version>1.0.0.Final</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>6.0.5</version>
</dependency>
I conducted some further tests (see edits for previous comments on identifying the problem) and it seems the problem comes from my currently used security policy. While I have a security policy file to use when the application is built and packaged in a jar file, I do not have one that will work inside the Eclipse IDE. As soon as I figure out what the best way to configure the security policy I will post it as an answer, unless someone else has an answer.
The problem as described above was caused by a handling of the security policy and security manager. Since this application will eventually read from the hard drive, and work with RMI I need to use a security manager and security policy. However, when working inside the IDE you don't actually need to set a codebase value in the security policy file.
My current solution is to have this as my development area security policy:
grant {
permission java.security.AllPermission;
}
And then the security policy in use when I deploy my application would look like this:
grant codeBase "file:./MyApplication.jar" {
permission java.security.AllPermission;
}
I know that eventually I will need to provide a better more specific security policy for deployment that grants specific permissions to specific file locations and network values, but this should get me going until then.
How can I setup my camel projects to completely rely on the Spring Dependency injection system while being xml free using the maven camel run plugin. I have tried a ton of configurations, but I still seem to be stuck with a "shell context" file that just imports my java configuration file. Is there anyway to get rid of this? Note: also on latest camel version of 2.17.1
Camel Route
#Component
public class TestRoute extends SpringRouteBuilder {
#Override
public void configure() throws Exception {
from("timer://foo?repeatCount=1")
.log("Hello World");
}
}
Java Config
#Configuration
#ComponentScan("com.mcf.xml.free.route")
public class RouteJavaConfig extends CamelConfiguration {
}
Maven Camel Plugin
<plugin>
<groupId>org.apache.camel</groupId>
<artifactId>camel-maven-plugin</artifactId>
<version>${camel.version}</version>
<configuration>
<configClasses>com.mcf.xml.free.config.RouteJavaConfig</configClasses>
</configuration>
</plugin>
Plugin Context that lets it all work that I want to get rid of.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<context:annotation-config/>
<bean class="com.mcf.xml.free.config.RouteJavaConfig"/>
</beans>
Is there some way to have the maven camel run plugin accept the Java Config instead of a Spring Context file? Also I haven't tested this yet, but If removing this file is going to cause an issue deploying to Apache Karaf is there a way to have it configured to use my Java Config as well?
You can start your Spring Java Config application using a main class you create yourself, and then use the java-maven-exec plugin to run it. This example has no XML file at all.
There is an example as part of Apache Camel at: https://github.com/apache/camel/tree/master/examples/camel-example-spring-javaconfig
I think your scenario is a great fit for using spring boot with camel. Spring quotes:
Absolutely no code generation and no requirement for XML configuration
on their page. Spring-Boot
Here is one Spring boot-camel example you can have a look at to check: Camel-Spring Boot example
I need to wait some time before shut down my OSGI context.( Give some time to finish the tasks that are currently running). I came across with shutdown.wait.time property in the extenderProperties bean.
Can anyone please let me know how can I use this OSGi fragments to achieve my goal? I think I can attach a fragment bundle to my existing OSGI bundle.
Thanks in advance. appreciate your help.
You'll need to create a bundle with two files in it: META-INF/MANIFEST.MF and META-INF/spring/extender/extender.xml (the xml file could be named anything with an xml extension, but must be in the META-INF/spring/extender folder). Your MANIFEST.MF file will need to contain the OSGi manifest header Fragment-Host of org.springframework.osgi.extender. If you're using the maven-bundle-plugin, your plugin configuration would look something like this:
...
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.3.5</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Fragment-Host>org.springframework.osgi.extender</Fragment-Host>
</instructions>
</configuration>
</plugin>
...
Your extender.xml file will need to define a java.util.Properties bean with a name of extenderProperties. It should contain a property named shutdown.wait.time with a value in milliseconds (eg. 30000 for 30 seconds). The file may look like this:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd">
<util:properties id="extenderProperties">
<prop key="shutdown.wait.time">30000</prop>
</util:properties>
</beans>
Then deploy the bundle into your environment. You may need to restart the Spring OSGi bundles (or your server) depending on the order this fragment bundle is installed in relation to the Spring DM bundles.