SQSListener not consuming messages from queue - java

I cannot see the messages in the SQS queue being consumed by the #SqsListener
import org.springframework.cloud.aws.messaging.listener.annotation.SqsListener; //others
#Component
public class Consumer{
private static final Logger logger = LoggerFactory.getLogger(Consumer.class);
#SqsListener(value = "TEST-MY-QUEUE")
public void receiveMessage(String stringJson) {
System.out.println("***Consuming message: " + stringJson);
logger.info("Consuming message: " + stringJson);
}
}
My configuration (Here I print the client queues, and I can deffo spot the queue I want to consume - TEST-MY-QUEUE . It prints the URL correctly in the region. I am also able to see the region loaded correctly (same as queue) in regionProvider
#Configuration
public class AwsConfiguration {
#Bean
#Primary
AmazonSQSAsync sqsClient() {
AmazonSQSAsync amazonSQSAsync = AmazonSQSAsyncClientBuilder.defaultClient();
System.out.println("Client queues = " + amazonSQSAsync.listQueues()); //The queue I want to consume is here
return amazonSQSAsync;
}
#Bean
AwsRegionProvider regionProvider() {
DefaultAwsRegionProviderChain defaultAwsRegionProviderChain = new DefaultAwsRegionProviderChain();
System.out.println("Region = " + defaultAwsRegionProviderChain.getRegion());
return defaultAwsRegionProviderChain;
}
#Bean
public SimpleMessageListenerContainer simpleMessageListenerContainer(AmazonSQSAsync amazonSQSAsync, QueueMessageHandler queueMessageHandler) {
SimpleMessageListenerContainer simpleMessageListenerContainer = new SimpleMessageListenerContainer();
simpleMessageListenerContainer.setAmazonSqs(amazonSQSAsync);
simpleMessageListenerContainer.setMessageHandler(queueMessageHandler);
simpleMessageListenerContainer.setMaxNumberOfMessages(10);
simpleMessageListenerContainer.setTaskExecutor(threadPoolTaskExecutor());
return simpleMessageListenerContainer;
}
#Bean
public QueueMessageHandler queueMessageHandler(AmazonSQSAsync amazonSQSAsync) {
QueueMessageHandlerFactory queueMessageHandlerFactory = new QueueMessageHandlerFactory();
queueMessageHandlerFactory.setAmazonSqs(amazonSQSAsync);
QueueMessageHandler queueMessageHandler = queueMessageHandlerFactory.createQueueMessageHandler();
return queueMessageHandler;
}
public ThreadPoolTaskExecutor threadPoolTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10);
executor.setMaxPoolSize(10);
executor.initialize();
return executor;
}
And pom.xml (Java 11, spring boot, spring cloud aws)
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>2.5.6</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws-core</artifactId>
<version>2.2.6.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws-autoconfigure</artifactId>
<version>2.2.6.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bootstrap</artifactId>
<version>3.0.3</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws</artifactId>
<version>2.2.6.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws-messaging</artifactId>
<version>2.2.6.RELEASE</version>
</dependency>
I noticed very similar issues in the questions here and I changed my dependencies in pom.xml to be spring-cloud-starter-aws-messaging but didnt fix for me. I double checked the names (queue, annotation) all seems fine
When I run my app, starts fine but I dont see any logs or exception. Not one message consumed.
What am I missing?
Thank you

You are using a third party API. To use invoke Amazon Simple Queue Service (SQS) from a Java project, use the Official AWS SDK for Java V2. If you are not aware how to use this SDK, see this DEV Guide:
Developer guide - AWS SDK for Java 2.x
For AWS SQS specific information, see:
Working with Amazon Simple Queue Service
This has links to AWS Github where you will find POM dependencies, code, etc.

At the end it was an issue with the config (using the credentials)
In application.yml
credentials:
useDefaultAwsCredentialsChain: true #Will use credentials in /.aws
And then in the AWSConfig class where you create the AmazonSQSAsync, just make it use that config
public AmazonSQSAsync amazonSQSAsync() {
DefaultAWSCredentialsProviderChain defaultAWSCredentialsProviderChain = new DefaultAWSCredentialsProviderChain();
return AmazonSQSAsyncClientBuilder.standard().withRegion(region)
.withCredentials(defaultAWSCredentialsProviderChain)
.build();

Related

Thymeleaf Spring 5 cannot find html template inside web-inf

I want to send email using thymeleaf for the email content, but it seems it cannot find the html template inside WEB-INF/templates/notifications.
I am using thymeleaf spring 5 on spring boot. Spring boot (2.0.2.RELEASE) and Thymeleaf-spring5 (3.0.11.RELEASE)
Below are my configuration/changes:
POM
...
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.2.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
...
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>
<dependency>
<groupId>org.thymeleaf.extras</groupId>
<artifactId>thymeleaf-extras-springsecurity4</artifactId>
</dependency>
<dependency>
<groupId>org.thymeleaf</groupId>
<artifactId>thymeleaf-spring5</artifactId>
<version>3.0.11.RELEASE</version>
</dependency>
...
NotificationTemplateConfiguration
#Configuration
public class NotificationTemplateConfiguration {
#Bean
#Qualifier(value = "myTemplateEngine")
public SpringTemplateEngine springTemplateEngine() {
SpringTemplateEngine templateEngine = new SpringTemplateEngine();
templateEngine.addTemplateResolver(myTemplateResolver());
return templateEngine;
}
#Bean
public SpringResourceTemplateResolver myTemplateResolver(){
SpringResourceTemplateResolver myTemplateResolver = new SpringResourceTemplateResolver();
myTemplateResolver.setPrefix("WEB-INF/templates/notifications/");
myTemplateResolver.setSuffix(".html");
myTemplateResolver.setTemplateMode(TemplateMode.HTML);
myTemplateResolver.setCharacterEncoding(StandardCharsets.UTF_8.name());
// I tried adding these lines but still it does not work
// myTemplateResolver.setOrder(0);
// myTemplateResolver.setCheckExistence(true);
return myTemplateResolver;
}
}
EmailService
#Component
public class EmailService {
...
#Autowired
#Qualifier(value = "myTemplateEngine")
private SpringTemplateEngine m_myTemplateEngine;
...
private String buildContent() {
final Context context = new Context();
context.setVariable("recvName", getRecvName());
context.setVariable("inquiry", getInquiry());
context.setVariable("logoImageUrl", getLogoImageUrl());
// This is causing the error as it cannot find `WEB-INF/templates/notifications/inquiry-notification.html`, but this file really exists
return m_templateEngine.process("inquiry-notification", context);
}
public void sendEmail() throws MessagingException {
MimeMessage message = m_javaMailSender.createMimeMessage();
MimeMessageHelper helper = new MimeMessageHelper(message, true, ConstantUtil.CHARACTER_ENCODING);
String content = buildContent();
helper.setFrom(getFromEmail());
helper.setReplyTo(getNoReplyEmail());
helper.setText(content, true);
helper.setSubject(getSubject());
helper.setTo(getRecipientEmail());
m_javaMailSender.send(message);
}
}
Now, I am getting an error on buildContent():
Caused by: java.io.FileNotFoundException: class path resource [WEB-INF/templates/notifications/inquiry-notification.html] cannot be opened because it does not exist
at org.springframework.core.io.ClassPathResource.getInputStream(ClassPathResource.java:180)
at org.thymeleaf.spring5.templateresource.SpringResourceTemplateResource.reader(SpringResourceTemplateResource.java:103)
at org.thymeleaf.templateparser.markup.AbstractMarkupTemplateParser.parse(AbstractMarkupTemplateParser.java:223)
... 16 common frames omitted
Problem
The file WEB-INF/templates/notifications/inquiry-notification.html is existing and it is inside WEB-INF/templates/notifications even if I check the war file. The problem is on m_templateEngine.process("inquiry-notification", context) as it cannot find inquiry-notification even if it exists. If I comment out this and just return a hard coded string (for testing only)... it will send the email without any error.
This WEB-INF/templates/notifications/inquiry-notification.html really exists, but I am out of idea on why it cannot find it.
Any idea on why it cannot find the file inside WEB-INF and how to fix it?
Update:
If I change the prefix into:
myTemplateResolver.setPrefix("classpath:/templates/notifications/");
and move the folder from WEB-INF/templates/notifications/ into resources/templates/notifications.
Everything works, but I want to use WEB-INF/templates/notifications/ and not resources/templates/notifications.
As what #Ralph commented, it can be seen in the exception that it is reading from class path resource and not context resource.
My problem now is how can I make it read from context resource (WEB-INF/templates/notifications) and not from class path resource (resources/templates/notifications).

NPE Testing Kafka Producer Using Embedded Kafka

I've written a basic spring boot service that consumes some data via rest API and publishes it to rabbitmq and kafka.
To test the service class handling kafka producing, I followed this guide: https://www.baeldung.com/spring-boot-kafka-testing
In isolation, the test (KafkaMessagingServiceImplTest) works perfectly both in intellij idea and via mvn on the command line. Running all project tests in idea works fine. However, when I run all project tests via maven on the command line, this test fails with an NPE when trying to make the assertion on the payload String.
I've narrowed down the location of the root problem to another test class (AppPropertiesTest) which is solely testing my AppProperties component (which is a component I use to pull config from application.properties in a tidy way). When, and only when, the tests within that test class are run alongside the failing test using 'mvn clean install' in project root, does the NPE show up. Commenting out the tests in this class or annotating it with #DirtiesContext fixes the problem. Apparently something loaded into the spring context by this test class causes an issue with the timing/order of events/countdownlatch in the other test. Of course, I don't want to use #DirtiesContext as it can lead to a much slower build as the project increases in complexity. It also does not explain the problem.. and I can't handle that :)
AppPropertiesTest uses constructor injection to inject the AppProperties component. It also extends a abstract class 'GenericServiceTest' which is annotated by:
#SpringBootTest
#TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
and contains nothing else. As you probably know, the SpringBootTest annotation builds a test spring context and wires in boilerplate to allow effective testing of a spring app's dependency injection etc. and the TestConstructor annotation allows constructor injection in some of my tests. FWIW, I have tried removing the TestConstructor annotation and using plain old Autowiring in the AppProperties class to see if it makes a difference but it does not.
The failing test class also extends GenericServiceTest, as it requires the spring context to inject some of the dependencies such as the consumer and the messaging service being tested and AppProperties instance within etc.
So I know where the problem lies but I don't know what the problem is. Even when the test fails with the NPE, I can see in the logs that the consumer has successfully consumed the message before the failure, as per the Baeldung guide :
TestKafkaConsumer : received payload='ConsumerRecord(topic = test-kafka-topic, partition = 0, leaderEpoch = 0, offset = 0, CreateTime = 1618997289238, serialized key size = -1, serialized value size = 43, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = This is a test message to be sent to Kafka.)'
However, the payLoad is null when we get back to the assertion. I've tried all kinds of things like Thread.sleep() in the failing test to give it more time and I've increased the await() timeout but no joy.
I find it bizarre that the tests are fine in IDEA and in isolation. Now it's starting to drive me a little crazy and I can't debug it because the problem doesn't occur in my IDE.
If anyone has any ideas, it would be greatly appreciated!
Thanks.
EDIT: Someone very reasonably suggested that I add some code so here goes :)
The Failing Test (fails at assertTrue(payload.contains(testMessage)) because payLoad is null). The autowired kafkaMessagingService simply has the dependencies of AppProperties and KakfaTemplate injected and calls kafkaTemplate.send():
#EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:9092", "port=9092" })
class KafkaMessagingServiceImplTest extends GenericServiceTest {
#Autowired
#Qualifier("kafkaMessagingServiceImpl")
private IMessagingService messagingService;
#Autowired
private TestKafkaConsumer kafkaConsumer;
#Value("${app.topicName}")
private String testTopic;
#Test
public void testSendAndConsumeKafkaMessage() throws InterruptedException {
String testMessage = "This is a test message to be sent to Kafka.";
messagingService.sendMessage(testMessage);
kafkaConsumer.getLatch().await(2000, TimeUnit.MILLISECONDS);
String payload = kafkaConsumer.getPayload();
assertTrue(payload.contains(testMessage));
}
The TestConsumer (used to consume in the test above)
#Component
public class TestKafkaConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(TestKafkaConsumer.class);
private CountDownLatch latch = new CountDownLatch(1);
private String payload = null;
#KafkaListener(topics = "${app.topicName}")
public void receive(ConsumerRecord<?, ?> consumerRecord) {
LOGGER.info("received payload='{}'", consumerRecord.toString());
setPayload(consumerRecord.toString());
latch.countDown();
}
public CountDownLatch getLatch() {
return latch;
}
public String getPayload() {
return payload;
}
public void setPayload(String payload) {
this.payload = payload;
}
Project dependencies:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
<version>2.5.8.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
<!-- https://mvnrepository.com/artifact/org.mockito/mockito-all -->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.10.19</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.springframework.kafka/spring-kafka-test -->
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<version>2.5.6.RELEASE</version>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
AppPropertiesTest class (the context of which seems to cause the problem)
class AppPropertiesTest extends GenericServiceTest {
private final AppProperties appProperties;
public AppPropertiesTest(AppProperties appProperties) {
this.appProperties = appProperties;
}
#Test
public void testAppPropertiesGetQueueName() {
String expected = "test-queue";
String result = appProperties.getRabbitMQQueueName();
assertEquals(expected, result);
}
#Test
public void testAppPropertiesGetDurableQueue() {
boolean isDurableQueue = appProperties.isDurableQueue();
assertTrue(isDurableQueue);
}
}
The AppProperties class that the AppPropertiesTest class is testing:
#Component
#ConfigurationProperties("app")
public class AppProperties {
// a whole bunch of properties by name that are prefixed by app. in the application.properties file. Nothing else
}
The Generic service test class which both tests extend.
#SpringBootTest
#TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
public abstract class GenericServiceTest {
}
The failure (you can see on the line above the payload has been received and printed out).
2021-04-21 14:15:07.113 INFO 493384 --- [ntainer#0-0-C-1] service.TestKafkaConsumer : received payload='ConsumerRecord(topic = test-kafka-topic, partition = 0, leaderEpoch = 0, offset = 0, CreateTime = 1619010907076, serialized key size = -1, serialized value size = 43, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = This is a test message to be sent to Kafka.)'
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.791 s <<< FAILURE! - in
service.KafkaMessagingServiceImplTest
[ERROR] testSendAndConsumeKafkaMessage Time elapsed: 2.044 s <<< ERROR!
java.lang.NullPointerException
at service.KafkaMessagingServiceImplTest.testSendAndConsumeKafkaMessage(KafkaMessagingServiceImplTest.java:42)
The problem is that TestListener is a #Component so it is being added twice - the record is going to the other instance.
I added more debugging to verify the getter is called on a different instance.
#Component
public class TestKafkaConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(TestKafkaConsumer.class);
private final CountDownLatch latch = new CountDownLatch(1);
private String payload = null;
#KafkaListener(id = "myListener", topics = "${app.kafkaTopicName}")
public void receive(ConsumerRecord<?, ?> consumerRecord) {
LOGGER.info("received payload='{}'", consumerRecord.toString());
setPayload(consumerRecord.toString());
if (payload != null) {
LOGGER.info(this + ": payload is not null still");
}
latch.countDown();
if (payload != null) {
LOGGER.info(this + ": payload is not null after latch countdown");
}
}
public CountDownLatch getLatch() {
return latch;
}
public String getPayload() {
LOGGER.info(this + ": getting Payload");
return payload;
}
public void setPayload(String payload) {
this.payload = payload;
}
}
If you don't want to use #DirtiesContext, you can at least stop the listener containers after the tests complete:
#SpringBootTest
#TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
public abstract class GenericDataServiceTest {
#AfterAll
static void stopContainers(#Autowired KafkaListenerEndpointRegistry registry) {
registry.stop();
}
}
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Springcloud bus custom messages cannot be sent through rabbitmq

When using the springcloud bus, a custom message is created and sent through rabbitmq, but after the message is sent, it does not go to rabbitmq. When you try to call /actuator/bus-refresh, you can see the bus messages emitted from the console page of rabbitmq.
I tried to start a micro service to register a custom event listener but failed to receive it. However, if the sender registers a listener himself, he can receive it but does not send it from rabbitmq.
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.3.RELEASE</version>
<relativePath/>
</parent>
<dependencies>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bus-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Greenwich.SR1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
About #RemoteApplicationEventScan annotations, I code all under the same package, so I should be able to scan to TestEvent. I also tried to specify basepackage.
#SpringBootApplication
#RemoteApplicationEventScan
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
#RestController
#RequestMapping("test")
public class TestController {
#Autowired
private ApplicationContext context;
#RequestMapping("test")
public String test() {
final TestEvent testEvent = new TestEvent(this, context.getId(), null,"test");
context.publishEvent(testEvent);
return "success";
}
}
#Data
public class TestEvent extends RemoteApplicationEvent {
private String action;
public TestEvent(Object source, String originService, String destinationService, String action) {
super(source, originService, destinationService);
this.action = action;
}
}
When I called http://localhost:8080/actuator/bus-refresh I can see the information in the rabbitmq.
{" type ":" AckRemoteApplicationEvent ", "timestamp" : 1554350325406, "originService" : "application: 0: b3461fbec3536203a7020ff9d24bb11b", "destinationService" : "* *", "id" : "e6b875bd - 2402-494 - f - a870 - 4917324 d2c5c ackId ", "" :" af93075e - 55 d2-41 f8 - ba27 - e3c80cf19eea ", "ackDestinationService" : "* *", "the event" is: "org. Springframework. Cloud. Bus. Event. RefreshRemoteApplicationEvent"}.
But when I call to http://localhost:8080/test/test, I don't.
I came into the same issue a few days ago and it turned out that it was because the originService was incorrect. passing in context.getId() as originService doesn't work.
Short answer: use org.springframework.cloud.bus.BusProperties#id. You can inject BusProperties to your component. Or you can configure your own spring cloud bus id as stated in the document.
I am not 100% sure this is the proper way. Maybe I missed something in the document. It is just based on what I read from the source code of org.springframework.cloud.bus.BusAutoConfiguration, method acceptLocal.
Hope it works for you.

Exception in PortForwarder when adding swagger support to org.glassfish.jersey.server

I am trying to add swagger support to javax rest apis and ended up having the following exception. Any insights will be helpful.
Exception in thread "PortForwarder" java.lang.NoSuchMethodError: org.apache.commons.lang3.Validate.inclusiveBetween(JJJ)V
at com.offbynull.portmapper.mappers.upnpigd.externalmessages.ServiceDiscoveryUpnpIgdRequest$ProbeDeviceType.<init>(ServiceDiscoveryUpnpIgdRequest.java:128)
at com.offbynull.portmapper.mappers.upnpigd.externalmessages.ServiceDiscoveryUpnpIgdRequest$ProbeDeviceType.<clinit>(ServiceDiscoveryUpnpIgdRequest.java:104)
at com.offbynull.portmapper.mappers.upnpigd.UpnpIgdPortMapper.identify(UpnpIgdPortMapper.java:202)
at com.offbynull.portmapper.PortMapperFactory.discover(PortMapperFactory.java:58)
at com.swirlds.p2p.portforwarding.portmapper.PortMapperPortForwarder.execute(PortMapperPortForwarder.java:60)
at com.swirlds.p2p.portforwarding.portmapper.PortMapperPortForwarder.run(PortMapperPortForwarder.java:172)
at java.lang.Thread.run(Thread.java:748)
Code to initialise the server
public static void initREST(int port, String[] packages) {
URI baseUri = UriBuilder.fromUri("http://0.0.0.0").port(port).build();
ResourceConfig config = new ResourceConfig()
.register(new CORSFilter())
.register(JacksonFeature.class);
configureSwagger(config);
for (String pkg : packages) {
config.packages(pkg);
}
System.out.println("Attempting to start Grizzly on " + baseUri);
GrizzlyHttpServerFactory.createHttpServer(baseUri, config);
} catch (IOException e1) {
e1.printStackTrace();
}
}
Code to configure swagger:
private static void configureSwagger(ResourceConfig resourceConfig) {
resourceConfig.register(ApiListingResource.class);
resourceConfig.register(SwaggerSerializers.class);
BeanConfig config = new BeanConfig();
config.setVersion("v1");
config.setBasePath("/");
config.setResourcePackage("ipos.hashgraph.rest");
config.setPrettyPrint(true);
config.setScan(true);
}
Maven Dependencies:
<dependency>
<groupId>io.swagger</groupId>
<artifactId>swagger-jaxrs</artifactId>
<version>1.5.0</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-grizzly2-servlet</artifactId>
</dependency>

How to use java to access dynamodb without access credential on an ec2 instance with IAM role

I want to use Java to access Dynamodb on an Ec2 instance.
This Ec2 instance has been granted a IAM role, with which I can directly access the Dynamodb by using aws CLI: aws dynamodb list-table.
Now I try to access the Dynamodb via Java. The Java code should be able to assume role, but it didn't work.
public static void main(String[] args) throws Exception {
String ROLE_ARN = "arn:aws:iam::....";
AWSSecurityTokenServiceClient stsClient = new AWSSecurityTokenServiceClient();
AssumeRoleRequest assumeRequest = new AssumeRoleRequest()
.withRoleArn(ROLE_ARN)
.withDurationSeconds(3600)
.withRoleSessionName("demo");
AssumeRoleResult assumeResult = stsClient.assumeRole(assumeRequest);
BasicSessionCredentials temporaryCredentials = new BasicSessionCredentials(
assumeResult.getCredentials().getAccessKeyId(),
assumeResult.getCredentials().getSecretAccessKey(),
assumeResult.getCredentials().getSessionToken());
AmazonDynamoDBClient client = new AmazonDynamoDBClient(temporaryCredentials)
DynamoDB dynamoDB = new DynamoDB(client);
TableCollection<ListTablesResult> tables = dynamoDB.listTables();
Iterator<Table> iterator_t = tables.iterator();
System.out.println("Listing table names");
while (iterator_t.hasNext()) {
Table table = iterator_t.next();
System.out.println(table.getTableName());
}
}
When I ran the code on the ec2 instance, I got
Exception in thread "main" com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: Not authorized to perform sts:AssumeRole (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: 60313562-d462-11e6-a116-5bf8bb6a59ce)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1586)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1254)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1035)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:747)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:721)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:704)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:672)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:654)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:518)
at com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1188)
at com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1164)
at com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:419)
at com.spokeo.dynamo_elas.AccessAwsD.main(AccessAwsD.java:stsClient.assumeRole(assumeRequest))
Anybody knows how to solve this problem?
Thanks.
After a long time exploring, finally figured out the following solution.
AWSCredentialsProvider provider = new InstanceProfileCredentialsProvider();
AWSCredentials credential = provider.getCredentials();
AmazonDynamoDBClient client = new AmazonDynamoDBClient(credential);
client.setRegion(Region.getRegion(Regions.US_WEST_2));
DynamoDB dynamoDB = new DynamoDB(client);
TableCollection<ListTablesResult> tables = dynamoDB.listTables();
Also the dependencies in pom.xml needs to be configured correctly to avoid conflicts, say,
com.amazonaws
aws-java-sdk
1.11.72
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.8.5</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.dataformat/jackson-dataformat-cbor -->
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-cbor</artifactId>
<version>2.8.5</version>
</dependency>
When I've done this I've never had to do anything specifically with the role - indeed, I have no idea what role I'm using. I use something like:
AWSCredentialsProviderChain credentialsProvider;
try {
credentialsProvider = new DefaultAWSCredentialsProviderChain();
}
catch (Exception e) {
throw new RuntimeException("Error loading credentials", e);
}
AmazonDynamoDBClient client = new AmazonDynamoDBClient(credentialsProvider);
The advantage of using the default provider is that if I'm developing locally with a ~/.aws/credentials it is used. If I'm on the EC2 with IAM credentials then it is used.

Categories

Resources