In my application i need to send messages to the given topic name. The topic is already created by other person and in a config file they give only the topic Name. My work is to push the messages in the given topic Name. Is there any way to get topic ARN by topic NAME in java?
As stated in this answer using createTopic(topicName) is more direct approach. In case topic have been created before, it will just return you topic ARN.
I've done this one of two ways. The ARN is always the same pattern. So, you could just subscribe to "arn:aws:sns:<region>:<account>:<name>" where:
region is from Regions.getCurrentRegion(). Be careful with this as it is a bit expensive of a call and you'll need to handle not being on an EC2/Elastic Beanstalk instance.
account is from AmazonIdentityManagementClient.getUser().getUser().getArn(). You'll have to parse the account number from that. Same warning about not being in an EC2 environment.
name is what you have.
A simpler way is loop through the the topics and look for the name you want in the ARN. You will use the AmazonSNSClient listTopics method to do this. Remember that that method only returns the first 100 topics - you'll want to properly loop over the entire topic list.
This might be what you need. Supply the topic and get it from available topics.
import json
import boto3
def lambda_handler(event, context):
try:
topic = event['topic']
subject = event['subject']
body = event['body']
subscriber = event['subscriber']
sns_client = boto3.client('sns')
sns_topic_arn = [tp['TopicArn'] for tp in sns_client.list_topics()['Topics'] if topic in tp['TopicArn']]
sns_client.publish(TopicArn = sns_topic_arn[0], Message=body,
Subject=subject)
except Exception as e:
print(e)
What you can do, is to create a table that contains topic and its topicArn.
The topic arn can be retrieve by the console or using the api when you create a topic.
This way no need to loop or try to match a pattern.
Related
From the examples I have seen the below code snippet and it works fine. But the problem is that : I don't always have requirements of processing the input-stream and produce it to a sink.
What if I have an application where based on some events I have to only publish to a kafka topic so that down-stream applications can make certain decisions. That means, I don't really have an input-stream but I just know when something happens in my application, I need to publish a message to a particular topic of kafka. That is, I only need a sink.
I was going through examples but didn't find anything matching to my requirements. Is there a way to only configure a KafkaSink that exposes a method() to be called for publishing messages to a topic.
Many thanks in advance!!
String inputTopic = "flink_input";
String outputTopic = "flink_output";
String consumerGroup = "baeldung";
String address = "localhost:9092";
StreamExecutionEnvironment environment = StreamExecutionEnvironment
.getExecutionEnvironment();
FlinkKafkaConsumer011<String> flinkKafkaConsumer = createStringConsumerForTopic(
inputTopic, address, consumerGroup);
DataStream<String> stringInputStream = environment
.addSource(flinkKafkaConsumer);
FlinkKafkaProducer011<String> flinkKafkaProducer = createStringProducer(
outputTopic, address);
stringInputStream
.map(new WordsCapitalizer())
.addSink(flinkKafkaProducer);
You must have a source. You might want to implement a custom source, or you could use something like a NumberSequenceSource followed by an operator like a process function that emits whatever you know you want to write to the sink, followed by the sink.
That process function could, for example, transform the incoming events into whatever you want to write to Kafka, or it could ignore its inputs and use a timer to generate the events to be sent to Kafka.
Or you might find that async i/o is a better building block than a process function, depending on your requirements.
I have currently a huge problem for which I need help for it.
Currently I´m not loading all emails at once.
I found here the following function for it:
Message[] messages = emailFolder.getMessages(start, end);
I know I can use SortTerm for sorting the emails:
SortTerm sortTerm[] = new SortTerm[] { SortTerm.REVERSE, SortTerm.DATE };
Message messages = ((IMAPFolder) emailFolder).getSortedMessages(sortTerm);
But than I will load again all emails.
How can I use together:
- search
- sort
- and use getMessages(start, end)
A sample code would be very helpful.
Many thanks
To be clear, when using IMAP no messages are "loaded" when you call getMessages. All that happens is that the JavaMail client creates a Message object that refers to the message on the server, and sets it up so that the Message object will fetch the data for the message on the server when you ask for it.
You could create a SearchTerm that uses a pair of MessageNumberTerms to constrain the message to be within a certain range just as you were doing with "start, end". But you should ask yourself whether you really want to sort all the messages in the mailbox first by message number (effectively a forward sort by received date) and then reverse sort them by sent date. What exactly are you trying to accomplish?
Using spring-camel, I have built a route that consumes from a JMS topic (with JMSReplyTo expected to be set for each input message), splits the message into smaller chunks, sends them to a REST processsor, then aggregates the answers and should produce an output message to the destination pointed by JMSReplyTo. Unfortunately, camel implicitly utilises the JMSReplyTo destination in one of the intermediate steps (producing an unmarshalled POJO).
We have a functional requirement to adapt JMSReplyTo in order to provide a request-reply messaging service.
I am able to read the JMSReplyTo header before ending the route and I am explicitly converting it to CamelJmsDestinationName, which successfully overrides the destination for JMS component and produces the message on the output topic. I am not sure if this is the best approach and the problem is that camel still utilises the JMSReplyTo on its own.
My RouteBuilder configuration is as follows:
from("jms:topic:T.INPUT")
.process(requestProcessor)
.unmarshal().json(JsonLibrary.Jackson, MyRequest.class)
.split(messageSplitter)
.process(restProcessor)
.aggregate(messagesAggregator)
.unmarshal().json(JsonLibrary.Jackson, BulkResponses.class)
.process(responseProcessor)
.to("jms:topic:recipientTopic");
T.INPUT is the name of the input topic, while recipientTopic is just a placeholder that will be replaced by CamelJmsDestinationName.
I'm not keen on using CamelJmsDestinationName and a sort of a mocked up topic name in route configuration so I'm open to find a better solution. It would be great if camel utilised the JMSReplyTo automatically to produce the output message to the output topic.
Currently, the problem is that camel produces an intermediate output on the JMSReplyTo topic BUT the output is an unmarshalled MyRequest object, which results in an exception saying "ClassNotFoundException: (package name).MyRequest", which is obvious since this is only a class used in my internal processing - I don't want to produce this to the output topic. It seems like Camel does implicitly use the JMSReplyTo destination between requestProcessor and messageSplitter processing... Why? What am I doing wrong? What are the best practices?
Use "disableReplyTo=true" in Endpoint. Camel will not try to use any reply option.
Refer: https://camel.apache.org/jms.html for more details
I have found the answer... this is absurdly easy but I haven't seen it anywhere in the documentation.
You just need to call .stop() to mark the route as completed, and Camel will reply the body you configured in the last step to the destination configured in ${header.JMSReplyTo}. It's that simple.
So you can do:
from("jms:my-queue")
.unmarshall().json(JsonLibrary.Jsonb, InboundMessage.class)
.bean(SomeProcessingBean.class)
....
.log(LoggingLevel.INFO, "Sending reply to: " + simple("${header.JMSReplyTo}").getExpression().toString())
.marshall().json(JsonLibrary.Jsonb, ReplyMessage.class)
.stop();
And you will receive reply.
I wonder why no one has found this before... Nothing when I search the doc or here.... I must be dumb, or the doc is incomplete...but I am not dumb, so.
I'm currently working with Akka Streams (in Java) for a personal project and I'm having a hard time understanding how to send element to a Source.
The idea is to use a WebSocket to push content into the user's web browser. I've managed to use Akka Streams to create a request-response system, following the Akka HTTP documentation, but this is not what I want to do.
Looking into the Akka Streams documentation, I saw that there is Source.queue and Source.actorRef. But I don't understand how to put an element into the Source. Source.queue and Source.actorRef return a Source, which doesn't have the method offer (for Source.queue) or tell (for Source.actorRef).
My question is: how do I get the ActorRef for the Source created by Source.actorRef or the SourceQueueWithComplete for a Source created with Source.queue, to be able to send elements to my Source?
I searched the various Akka documentation but found no method to do that. And the majority of the code I found on the Internet is written in Scala, which doesn't seem to have the same problem.
The actor and queue from Source.actorRef and Source.queue, respectively, are the materialized values of those sources, meaning that they can be obtained only if the stream is running. For example:
final ActorRef actor =
Source.actorRef(Integer.MAX_VALUE, OverflowStrategy.fail())
.to(Sink.foreach(m -> System.out.println(m)))
.run(materializer);
actor.tell("do something", ActorRef.noSender());
It's no different in Scala:
implicit val materializer = ActorMaterializer()
val actor =
Source.actorRef(Int.MaxValue, OverflowStrategy.fail)
.to(Sink.foreach(println))
.run()
actor ! "do something"
I am intercepting messages that are sent through JBossESB. I am using pipeline interceptors to do so.
The problem is, that altough the sender is a service (for example PortReference < logical:BlueServiceESB#BlueListener >), the name of the receiver is a queue (not a service). That is logical because in some case, multiple services can receive messages from a given queue, but usually, each queue is mapped to only one service.
I would like to know which queue is mapped to which service, so I can display/save this information and have it displayed like message: service ---> service (not service ---> queue).
I know that I can get the name of the queue mapped to a service using the registry like this:
System.setProperty("javax.xml.registry.ConnectionFactoryClass", "org.apache.ws.scout.registry.ConnectionFactoryImpl");
// Retrieving information from the ESB Registry
Registry reg = RegistryFactory.getRegistry();
System.out.println(reg.findAllServices());
List<EPR> eprs = reg.findEPRs("FirstServiceESB", "SimpleListener");
System.out.println(eprs);
I would like to reverse this approach - queue is the input and service (EPR = end point reference = service) is the output. Is there any way how to do this or am I just trying to do the impossible here. I have found no tutorials or questions on this topic whatsoever.
Thanks for any tips!
As this question has 25 up-votes, this seems to be an useful feature. JBossESB is open source software. Thus, implement the feature yourself and commit it to the community! Or just create a change request hopping that somebody else will do it...
Try querying for all of the queues and building a reverse-lookup map. But I don't think there is any function that allows searching for services using a queue.