I researched on ElasticSearch and knew how to connect it but I need to create a model using anotation #Document() like this.
#Document(indexName = "resource", type = "person", shards = 1, replicas = 0)
public class Person {
#Id
private String id;
private String firstName;
private String lastName;
}
I create a Repository accordingly:
public interface PersonRepository extends ElasticsearchRepository<Person ,Long> { }
My purpose is that I don't want create model anymore. I want to use Json file instead so my repository should be like this:
public interface PersonRepository extends ElasticsearchRepository<String,Long> { }
My problem is that I don't know the configuration of ElasticSearch in order to create indexName and type without using #Document(). I wonder if my idea is possible. I hope everyone can give me some hints. I really appreciate your supports. Thank you very much in advance.
This is my current configuration:
#Configuration
#PropertySource(value = "classpath:elasticsearch.properties")
#EnableElasticsearchRepositories(basePackages = "spring.demo")
public class ElasticsearchConfiguration {
#Resource
private Environment environment;
#Bean
public Client client() throws UnknownHostException {
String host = environment.getProperty("elasticsearch.host");
int port = Integer.parseInt(environment.getProperty("elasticsearch.port"));
String cluster = environment.getProperty("cluster.name");
Settings settings = Settings.settingsBuilder()
.put("cluster.name", cluster)
.build();
Client client = TransportClient.builder().settings(settings).build()
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(host), port));
return client;
}
#Bean
public ElasticsearchOperations elasticsearchTemplate() throws UnknownHostException {
return new ElasticsearchTemplate(client());
}
}
Related
I'm unable to make works queue listener with Spring Boot and SQS
(the message is sent and appear in SQS ui)
The #MessageMapping or #SqsListener not works
Java: 11
Spring Boot: 2.1.7
Dependencie: spring-cloud-aws-messaging
This is my config
#Configuration
#EnableSqs
public class SqsConfig {
#Value("#{'${env.name:DEV}'}")
private String envName;
#Value("${cloud.aws.region.static}")
private String region;
#Value("${cloud.aws.credentials.access-key}")
private String awsAccessKey;
#Value("${cloud.aws.credentials.secret-key}")
private String awsSecretKey;
#Bean
public Headers headers() {
return new Headers();
}
#Bean
public MessageQueue queueMessagingSqs(Headers headers,
QueueMessagingTemplate queueMessagingTemplate) {
Sqs queue = new Sqs();
queue.setQueueMessagingTemplate(queueMessagingTemplate);
queue.setHeaders(headers);
return queue;
}
private ResourceIdResolver getResourceIdResolver() {
return queueName -> envName + "-" + queueName;
}
#Bean
public DestinationResolver destinationResolver(AmazonSQSAsync amazonSQSAsync) {
DynamicQueueUrlDestinationResolver destinationResolver = new DynamicQueueUrlDestinationResolver(
amazonSQSAsync,
getResourceIdResolver());
destinationResolver.setAutoCreate(true);
return destinationResolver;
}
#Bean
public QueueMessagingTemplate queueMessagingTemplate(AmazonSQSAsync amazonSQSAsync,
DestinationResolver destinationResolver) {
return new QueueMessagingTemplate(amazonSQSAsync, destinationResolver, null);
}
#Bean
public QueueMessageHandlerFactory queueMessageHandlerFactory() {
QueueMessageHandlerFactory factory = new QueueMessageHandlerFactory();
MappingJackson2MessageConverter messageConverter = new MappingJackson2MessageConverter();
messageConverter.setStrictContentTypeMatch(false);
factory.setArgumentResolvers(Collections.singletonList(new PayloadArgumentResolver(messageConverter)));
return factory;
}
#Bean
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSqs) {
SimpleMessageListenerContainerFactory factory = new SimpleMessageListenerContainerFactory();
factory.setAmazonSqs(amazonSqs);
factory.setMaxNumberOfMessages(10);
factory.setWaitTimeOut(2);
return factory;
}
}
I notice also that org.springframework.cloud.aws.messaging.config.SimpleMessageListenerContainerFactory and org.springframework.cloud.aws.messaging.config.annotation.SqsConfiguration run on startup
And my test
#RunWith(SpringJUnit4ClassRunner.class)
public class ListenTest {
#Autowired
private MessageQueue queue;
private final String queueName = "test-queue-receive";
private String result = null;
#Test
public void test_listen() {
// given
String data = "abc";
// when
queue.send(queueName, data).join();
// then
Awaitility.await()
.atMost(10, TimeUnit.SECONDS)
.until(() -> Objects.nonNull(result));
Assertions.assertThat(result).equals(data);
}
#MessageMapping(value = queueName)
public void receive(String data) {
this.result = data;
}
}
Do you think something is wrong ?
I create a repo for exemple : (https://github.com/mmaryo/java-sqs-test)
In test folder, change aws credentials in 'application.yml'
Then run tests
I had the same issue when using the spring-cloud-aws-messaging package, but then I used the queue URL in the #SqsListener annotation instead of the queue name and it worked.
#SqsListener(value = { "https://full-queue-URL" }, deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void receive(String message) {
// do something
}
It seems you can use the queue name when using the spring-cloud-starter-aws-messaging package. I believe there is some configuration that allows usage of the queue name instead of URL if you don't want to use the starter package.
EDIT: I noticed the region was being defaulted to us-west-2 despite me listing us-east-1 in my properties file. Then I created a RegionProvider bean and set the region to us-east-1 in there and now when I use the queue name in the #SqsMessaging it is found and correctly resolved to the URL in the framework code.
you'll need to leverage the #Primary annotation, this is what worked for me:
#Autowired(required = false)
private AWSCredentialsProvider awsCredentialsProvider;
#Autowired
private AppConfig appConfig;
#Bean
public QueueMessagingTemplate getQueueMessagingTemplate() {
return new QueueMessagingTemplate(sqsClient());
}
#Primary
#Bean
public AmazonSQSAsync sqsClient() {
AmazonSQSAsyncClientBuilder builder = AmazonSQSAsyncClientBuilder.standard();
if (this.awsCredentialsProvider != null) {
builder.withCredentials(this.awsCredentialsProvider);
}
if (appConfig.getSqsRegion() != null) {
builder.withRegion(appConfig.getSqsRegion());
} else {
builder.withRegion(Regions.DEFAULT_REGION);
}
return builder.build();
}
build.gradle needs these deps:
implementation("org.springframework.cloud:spring-cloud-starter-aws:2.2.0.RELEASE")
implementation("org.springframework.cloud:spring-cloud-aws-messaging:2.2.0.RELEASE")
I am trying to use both Cassandra and MySQL in my project. Some data will be saved into Cassandra and some to Mysql. I had been using mySql for last 1 yr in the same project and now since I'm Expanding it, I want to add Cassandra DB also.
My Cassandra Configuration file is as follows.
#Configuration
#PropertySource(value = {"classpath:META-INF/application.properties"})
#EnableCassandraRepositories(basePackages = {"com.example.repository"})
public class CassandraConfig {
#Autowired
private Environment environment;
private static final Logger LOGGER = LoggerFactory.getLogger(CassandraConfig.class);
#Bean
public CassandraClusterFactoryBean cluster() {
CassandraClusterFactoryBean cluster = new CassandraClusterFactoryBean();
cluster.setContactPoints(environment.getProperty("spring.cassandra.contactpoints"));
cluster.setPort(Integer.parseInt(environment.getProperty("spring.cassandra.port")));
return cluster;
}
#Bean
public CassandraMappingContext mappingContext() {
return new BasicCassandraMappingContext();
}
#Bean
public CassandraConverter converter() {
return new MappingCassandraConverter(mappingContext());
}
#Bean
public CassandraSessionFactoryBean session() throws Exception {
CassandraSessionFactoryBean session = new CassandraSessionFactoryBean();
session.setCluster(cluster().getObject());
session.setKeyspaceName(environment.getProperty("spring.cassandra.keyspace"));
session.setConverter(converter());
session.setSchemaAction(SchemaAction.NONE);
return session;
}
#Bean
public CassandraOperations cassandraTemplate() throws Exception {
return new CassandraTemplate(session().getObject());
}
}
My Repository is
public interface NewRepository extends CassandraRepository<ID>{
}
Now I'm trying to save an entity to it using the reposiroty
repo.save(entity);
where repo is the object for NewRepository.
But it shows InvalidDataAccessApiUsageException: unknown Type.
Where am i wrong.
Thank You in advance.
I use javaee 8 for my project and have this classes :
public class PersonRepository {
#Inject
#Mongo(collection = "users")
private MongoCollection collection;
// define some methods ....
}
#ApplicationScoped
public class MongoProducer {
#Inject
private MongoClient mongoClient;
#Produces
#Mongo
protected MongoCollection produceCollection(InjectionPoint ip) {
Mongo mongo = getMongoAnnotation(ip);
return mongoClient.getDatabase("sample").getCollection(mongo.collection());
}
private Mongo getMongoAnnotation(InjectionPoint ip) {
return ip.getAnnotated().getAnnotation(Mongo.class);
}
}
#MongoClientDefinition(
name = "mongoClient",
dbName = "sample",
username = "admin",
password = "adminpass"
)
public class MongoConnectionConfig {
}
Unfortunately mongoClient has not any method for get database name currently connected to that. (maybe i cant find that !)
How can scan MongoClientDefinition annotation from produceCollection method ?
I want set database name from that annotation . (replace "sample") .
Note : I created CDI extension for MongoClientDefinition .
simple idea is BeanManager :
I modified my extension and add getter method for dbName .
full project on github
#ApplicationScoped
public class MongoProducer {
#Inject
private MongoClient mongoClient;
#Inject
private BeanManager bm;
#Produces
#Mongo
protected MongoCollection produceCollection(InjectionPoint ip) {
String dbName = bm.getExtension(MongoClientExtension.class).getDatabaseName();
Mongo mongo = getMongoAnnotation(ip);
return mongoClient.getDatabase(dbName).getCollection(collectionName,mongo.collection());
}
private Mongo getMongoAnnotation(InjectionPoint ip) {
return ip.getAnnotated().getAnnotation(Mongo.class);
}
}
I have a properties file for defining dynamic endpoints (name, host and port).
So I created a bean named EndPoint with name, host and port as member variables.
How to inject server's host and port dynamically by using Spring #Value and SpEL?
endpoint.properties
names=server1,server2,server3
endpoint.server1.host=192.168.1.101
endpoint.server1.port=10101
endpoint.server2.host=192.168.1.102
endpoint.server2.port=10102
endpoint.server3.host=192.168.1.103
endpoint.server3.port=10103
EndPoint.java
public class EndPoint {
/** name */
private String name;
// hardcode server1 here
// how to inject server's host dynamically by name?
// #Value("${endpoint.#{this.name}.host}")
#Value("${endpoint.server1.host}")
private String host;
#Value("${endpoint.server1.port}")
private int port;
public EndPoint(String name) {
this.name = name;
}
}
EndPointBeanConfig.java
#Configuration
#PropertySource( //
value = { "classpath:conf/endpoint.properties" }, ignoreResourceNotFound = true//
)
public class EndPointBeanConfig {
#Autowired
private Environment env;
#Bean(name = "endPoints")
public List<EndPoint> endPoints() {
final List<EndPoint> endPoints = new ArrayList<EndPoint>();
final String[] names = env.getProperty("names").split(",");
for (final String name : names) {
final EndPoint endPoint = endPoint(name);
endPoints.add(endPoint);
}
return endPoints;
}
#Bean(name = "endPoint")
#Scope("prototype")
public EndPoint endPoint(String name) {
return new EndPoint(name);
}
}
I am just giving suggestion, Go with YAML/Properties for this type of implementation in your project.
This link helpful for your requirement
My Problem
I am using an LdapRepository to get user information from an Ldap server. This Ldap server is also queried by Spring Security to authenticate user.
My issue is, that Spring Security is able to find and identify users, while I'm unable to find users through my LdapRepository, using the same LdapContextSource. Querying the LdapRepository in any way does not return results (null or empty Lists).
What I have tried
Using the ldapsearch tool directly - Works
Using LdapQuery instead of the findByUsername method - Does not work
Testing methods like findAll() (CrudRepository) - Returns an empty List
Trying to get logs from spring-ldap - Seems to be impossible?
Used ldapsearch command: ldapsearch -x -H ldaps://<domain> -b o=<org> uid=<uid>
Viewing the traffic in Wireshark (using ldap instead of ldaps) looks like no query is executed by the LdapRepository at all, the connection just opens and closes with 0 results.
Relevant code
Configuration of LdapContextSource
#Bean
public LdapContextSource contextSource() {
LdapContextSource contextSource = new LdapContextSource();
contextSource.setUrl("ldaps://<domain>");
contextSource.setBase("o=<org>");
return contextSource;
}
SecurityConfiguration
#Configuration
#EnableWebSecurity
#EnableGlobalMethodSecurity(prePostEnabled = true, securedEnabled = true)
public class SecurityConfiguration extends WebSecurityConfigurerAdapter {
private final AuthenticationManagerBuilder authenticationManagerBuilder;
private final LdapContextSource ldapContext;
public SecurityConfiguration(AuthenticationManagerBuilder authenticationManagerBuilder, LdapContextSource ldapContext) {
this.authenticationManagerBuilder = authenticationManagerBuilder;
this.ldapContext = ldapContext;
}
#PostConstruct
public void init() {
try {
authenticationManagerBuilder
.ldapAuthentication()
.contextSource(ldapContext)
.userSearchFilter("(uid={0})");
} catch (Exception e) {
throw new BeanInitializationException("Security configuration failed", e);
}
}
}
UserRepository
#Repository
public interface UserRepository extends LdapRepository<User> {
User findByUsername(String username);
List<User> findByUsernameLikeIgnoreCase(String username);
}
User
#Entry(
objectClasses = {})
public class User {
#Id
private Name id;
#Attribute(name = "uid", readonly = true)
private String username;
public Name getId() {
return id;
}
public String getUsername() {
return username;
}
}
I found the solution.
Spring seems to assume the objectClass of User is User, if it is not set explicitly. Setting the correct objectClasses fixes this issue.