As already said in the title I'm looking for a way to receive mails with Integration Mail on Startup. Due to restrictions in the system I'm working in, the only way to get the mails is over the inbox and i should preferable only do it with integration mail. I know its possible with Javax.Mail, but its a lot of lines for just reading out the old mails on program start. I tired to do do it with a SearchTermStrategy with looking for already seen mails, but that ends up in a loop of reading old mails. Anybody knows a way to do it with integration mail?
I got a integration flow that constantly listens for mails, and i tried to do
do do one that reads out all the read mails, so i can keep track of them, the only problem with that is, that it checks for old mails via the SEEN Flag, and it cant differentiate between mails it already read out once that way, so it keeps reading out the same mails.
I also tried it with ImapMailReceiver(url).setSearchTermStrategy(searchTermStrategy())
with the strategy to get unseen mails, from trying with the debugger i saw there might be messages in the ImapMailReceiver, but I got no clue how to access them or if i can access those.
That's what I tried so far.
See MailReceivingMessageSource and something like OnlyOnceTrigger to let Inbound Channel Adapter to work only once on startup. After only one execution it is not going to look into the source any more:
public class OnlyOnceTrigger implements Trigger {
private final AtomicBoolean hasRun = new AtomicBoolean();
#Override
public Date nextExecutionTime(TriggerContext triggerContext) {
return this.hasRun.getAndSet(true) ? null : new Date();
}
}
I don't think the ImapIdleChannelAdapter is a good choice for this kind of tasks.
Related
Here I wanted to register to 2 endpoints and send requests to them. You can see this in the code below. I name one env1 and the other env2.
val client = Http.client
.configured(Transport.Options(noDelay = false, reuseAddr = false))
.newService("gexampleapi-env1.localhost.net:8081,gexampleapi-env2.localhost.net:8081")
So far everything is normal. But env1 instance had to be down for some reason(for a few hours' maintenance etc. not sure why.). Under normal circumstances, our expectation is that it continues to send requests through the env2 instance. But this didn't happen. Could not send requests to both servers. Normally it was working correctly, but it didn't work that day for a reason we don't know.
Since the event took place months ago, I only have the following log.
2022-02-15 12:09:40,181 [finagle/netty4-1-3] INFO com.twitter.finagle
FailureAccrualFactory marking connection to "gExampleAPI" as dead.
Remote Address:
Inet(gexampleapi-env1.localhost.net/10.0.0.1:8081,Map())
To solve the problem, we removed gexampleapi-env1.localhost.net:8081 host from the config file. and after restarting it continued to process requests. If you have any ideas about why we may have experienced this problem and how to avoid this next time, I would appreciate it if you could share them.
I have a spring boot web application with the functionality to update an entity called StudioLinking. This entity describes a temporary, mutable, descriptive logical link between two IoT devices for which my web app is their cloud service. The Links between these devices are ephemeral in nature, but the StudioLinking Entity persists on the database for reporting purposes. StudioLinking is stored to the SQL based datastore in the conventional way using Spring Data/ Hibernate. From time to time this StudioLinking entity will be updated with new information from a Rest API. When that link is updated the devices need to respond (change colors, volume, etc). Right now this is handled with polling every 5 seconds but this creates lag from when a human user enters an update into the system and when the IoT devices actually update. It could be as little as a millisecond or up to 5 seconds! Clearly increasing the frequency of the polling is unsustainable and the vast majority of the time there are no updates at all!
So, I am trying to develop another Rest API on this same application with HTTP Long Polling which will return when a given StudioLinking entity is updated or after a timeout. The listeners do not support WebSocket or similar leaving me with Long Polling. Long polling can leave a race condition where you have to account for the possibility that with consecutive messages one message may be "lost" as it comes in between HTTP requests (while the connection is closing and opening, a new "update" might come in and not be "noticed" if I used a Pub/Sub).
It is important to note that this "subscribe to updates" API should only ever return the LATEST and CURRENT version of the StudioLinking, but should only do so when there is an actual update or if an update happened since the last checkin. The "subscribe to updates" client will initially POST an API request to setup a new listening session and pass that along so the server knows who they are. Because it is possible that multiple devices will need to monitor updates to the same StudioLinking entity. I believe I can acomplish this by using separately named consumers in the redis XREAD. (keep this in mind for later in the question)
After hours of research I believe the way to acomplish this is using redis streams.
I have found these two links regarding Redis Streams in Spring Data Redis:
https://www.vinsguru.com/redis-reactive-stream-real-time-producing-consuming-streams-with-spring-boot/
https://medium.com/#amitptl.in/redis-stream-in-action-using-java-and-spring-data-redis-a73257f9a281
I also have read this link about long polling, both of these links just have a sleep timer during the long polling which is for demonstration purposes but obviously I want to do something useful.
https://www.baeldung.com/spring-deferred-result
And both these links were very helpful. Right now I have no problem figuring out how to publish the updates to the Redis Stream - (this is untested "pseudo-code" but I don't anticipate having any issues implementing this)
// In my StudioLinking Entity
#PostUpdate
public void postToRedis() {
StudioLinking link = this;
ObjectRecord<String, StudioLinking> record = StreamRecords.newRecord()
.ofObject(link)
.withStreamKey(streamKey); //I am creating a stream for each individual linking probably?
this.redisTemplate
.opsForStream()
.add(record)
.subscribe(System.out::println);
atomicInteger.incrementAndGet();
}
But I fall flat when it comes to subscribing to said stream: So basically what I want to do here - please excuse the butchered pseudocode, it is for idea purposes only. I am well aware that the code is in no way indicative of how the language and framework actually behaves :)
// Parameter studioLinkingID refers to the StudioLinking that the requester wants to monitor
// updateList is a unique token to track individual consumers in Redis
#GetMapping("/subscribe-to-updates/{linkId}/{updatesId}")
public DeferredResult<ResponseEntity<?>> subscribeToUpdates(#PathVariable("linkId") Integer linkId, #PathVariable("updatesId") Integer updatesId) {
LOG.info("Received async-deferredresult request");
DeferredResult<ResponseEntity<?>> output = new DeferredResult<>(5000l);
deferredResult.onTimeout(() ->
deferredResult.setErrorResult(
ResponseEntity.status(HttpStatus.REQUEST_TIMEOUT)
.body("IT WAS NOT UPDATED!")));
ForkJoinPool.commonPool().submit(() -> {
//----------------------------------------------
// Made up stuff... here is where I want to subscribe to a stream and block!
//----------------------------------------------
LOG.info("Processing in separate thread");
try {
// Subscribe to Redis Stream, get any updates that happened between long-polls
// then block until/if a new message comes over the stream
var subscription = listenerContainer.receiveAutoAck(
Consumer.from(studioLinkingID, updateList),
StreamOffset.create(studioLinkingID, ReadOffset.lastConsumed()),
streamListener);
listenerContainer.start();
} catch (InterruptedException e) {
}
output.setResult("IT WAS UPDATED!");
});
LOG.info("servlet thread freed");
return output;
}
So is there a good explanation to how I would go about this? I think the answer lies within https://docs.spring.io/spring-data/redis/docs/current/api/org/springframework/data/redis/core/ReactiveRedisTemplate.html but I am not a big enough Spring power user to really understand the terminology within Java Docs (the Spring documentation is really good, but the JavaDocs is written in the dense technical language which I appreciate but don't quite understand yet).
There are two more hurdles to my implementation:
My exact understanding of spring is not at 100% yet. I haven't yet reached that a-ha moment where I really fully understand why all these beans are floating around. I think this is the key to why I am not getting things here... The configuration for the Redis is floating around in the Spring ether and I am not grasping how to just call it. I really need to keep investigating this (it is a huge hurdle to spring for me).
These StudioLinking are short lived, so I need to do some cleanup too. I will implement this later once I get the whole thing up off the ground, I do know it will be needed.
Why don't you use a blocking polling mechanism? No need to use fancy stuff of spring-data-redis. Just use simple blocking read of 5 seconds, so this call might take around 6 seconds or so. You can decrease or increase the blocking timeout.
class LinkStatus {
private final boolean updated;
LinkStatus(boolean updated) {
this.updated = updated;
}
}
// Parameter studioLinkingID refers to the StudioLinking that the requester wants to monitor
// updateList is a unique token to track individual consumers in Redis
#GetMapping("/subscribe-to-updates/{linkId}/{updatesId}")
public LinkStatus subscribeToUpdates(
#PathVariable("linkId") Integer linkId, #PathVariable("updatesId") Integer updatesId) {
StreamOperations<String, String, String> op = redisTemplate.opsForStream();
Consumer consumer = Consumer.from("test-group", "test-consumer");
// auto ack block stream read with size 1 with timeout of 5 seconds
StreamReadOptions readOptions = StreamReadOptions.empty().block(Duration.ofSeconds(5)).count(1);
List<MapRecord<String, String, String>> records =
op.read(consumer, readOptions, StreamOffset.latest("test-stream"));
return new LinkStatus(!CollectionUtils.isEmpty(records));
}
I have a problem while trying my hands on the Hello World example explained here.
Kindly note that I have just modified the HelloEntity.java file to be able to return something other than "Hello, World!". Most certain my changes are taking time and hence I am getting the below Timeout error.
I am currently trying (doing a PoC) on a single node to understand the Lagom framework and do not have liberty to deploy multiple nodes.
I have also tried modifying the default lagom.circuit-breaker in application.conf "call-timeout = 100s" however, this does not seem to have helped.
Following is the exact error message for your reference:
{"name":"akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://hello-impl-application/system/sharding/HelloEntity#1074448247]] after [5000 ms]. Sender[null] sent message of type \"com.lightbend.lagom.javadsl.persistence.CommandEnvelope\".","detail":"akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://hello-impl-application/system/sharding/HelloEntity#1074448247]] after [5000 ms]. Sender[null] sent message of type \"com.lightbend.lagom.javadsl.persistence.CommandEnvelope\".\n\tat akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:595)\n\tat akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:605)\n\tat akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)\n\tat scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:866)\n\tat scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)\n\tat scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)\n\tat scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:864)\n\tat akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)\n\tat akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)\n\tat akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)\n\tat akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)\n\tat java.lang.Thread.run(Thread.java:748)\n"}
Question: Is there a way to increase the akka Timeout by modifying the application.conf or any of the java source files in the Hello World project? Can you please help me with the exact details.
Thanks in advance for you time and help.
The call timeout is the timeout for circuit breakers, which is configured using lagom.circuit-breaker.default.call-timeout. But that's not what is timing out above, the thing that is timing out above is the request to your HelloEntity, that timeout is configured using lagom.persistence.ask-timeout. The reason why there's a timeout on requests to entities is because in a multi-node environment, your entities are sharded across nodes, so an ask on them may go to another node, which is why a timeout is needed in case that node is not responding.
All that said, I don't think changing the ask-timeout will solve your problem. If you have a single node, then your entities should respond instantly if everything is working ok.
Is that the only error you're seeing in the logs?
Are you seeing this in devmode (ie, using the runAll command), or are you running the Lagom service some other way?
Is your database responding?
Thanks James for the help/pointer.
Adding following lines to resources/application.conf did the trick for me:
lagom.persistence.ask-timeout=30s
hello {
..
..
call-timeout = 30s
call-timeout = ${?CIRCUIT_BREAKER_CALL_TIMEOUT}
..
}
A Call is a Service-to-Service communication. That’s a SeviceClient communicating to a remote server. It uses a circuit breaker. It is a extra-service call.
An ask (in the context of lagom.persistence) is sending a command to a persistent entity. That happens across the nodes insied your Lagom service. It is not using circuit breaking. It is an intra-service call.
I'm building a client for trading with a remote server using FIX protocol and QuickFix/J API.
I can send order, receive price updates, cancel orders etc...
I'm asked now to "query API for current position of an instrument".
So let's say I can submit an order for buying an instrument, and it doesn't get executed, I would like to receive from the server some information like "you are LONG on intrument X with quantity Y etc".
Is it possible using QuickFix/J API?
I have written a method like this
static void positionReport() throws SessionNotFound{
quickfix.fix50.PositionReport order = new quickfix.fix50.PositionReport();
SessionID sessionId = (SessionID) initiator.getSessions().get(0);
order.set(new Account("1005390"));
order.set(new SecurityID("4663789"));
order.set(new SecurityExchange("XETR"));
order.set(new Symbol("SAP"));
Session.sendToTarget(order, sessionId);
}
which sends FIX messages like this
8=FIX.4.29=9835=AP34=4949=HIQ6_ORDER52=20140324-
15:54:10.14256=HIQFIX1=100539048=466378955=SAP207=XETR10=199
and receives messages like this:
8=FIX.4.29=9935=334=6949=HIQFIX52=20140324-15:54:10.89156=HIQ6_ORDER45=4958=Invalid
MsgType372=AP373=1110=242
As you can see I get "Invalid message" error
Check your counterparty's documentation.
FIX is a fairly "dumb" protocol. It just provides a communication infrastructure. The default message definitions are best thought of as a list of suggested messages that you can use. Even if one message type is supported by two counterparties, it's possible that each of the two counterparties could use it in totally different ways.
Most connection providers only use a subset of these messages. You should check their documentation to see if they support the PositionRequest message, and to see how they want you to set the fields in it.
No you cannot do that using Quickfix, unless and until the counterparty is modelled to give you FIX acknowledgements to your specific liking. That is why you can add your customized FIX fields to the FIX XML config file.
373 tag says 11 -> 11 = Invalid MsgType
58 confirms it for you again.
Check your FIX XML config and check if your message is complete and if your counterparty allows the messages of type AP.
I have developed a utility in Java using the Financial Information eXchange(FIX) protocol to pull data from an input stream. However, I currently have not found any test servers online to which I can connect to that implement the FIX protocol. Could someone please let me know how I can achieve this?
http://www.quickfixj.org/ have an open source fix engine. It comes with an example that has a simple server. You might want to use their library for the client too rather than rebuilding a FIX engine from scratch...
You can use CoralFIX to quickly fire up a test server. It takes care of all the session level FIX messages, such as Logon, ResendRequest, SequenceReset, Heartbeat so you are ready to start exchanging messages with your client. Below a simple example:
import com.coralblocks.coralfix.FixMessage;
import com.coralblocks.coralreactor.client.Client;
import com.coralblocks.coralreactor.nio.NioReactor;
import com.coralblocks.coralreactor.util.Configuration;
import com.coralblocks.coralreactor.util.MapConfiguration;
public class SimpleFixApplicationServer extends FixApplicationServer {
public SimpleFixApplicationServer(NioReactor nio, int port, Configuration config) {
super(nio, port, config);
}
#Override
protected void handleFixApplicationMessage(Client client, FixMessage fixMsg, boolean possDupe) {
// do whatever you want to do with the application message received from this client...
}
public static void main(String[] args) {
NioReactor nio = NioReactor.create();
MapConfiguration config = new MapConfiguration();
// print all messages received and sent to STDOUT for debugging purposes
// (default is false)
config.add("debugMessages", "true");
// accept as the client inbound sequence whatever
// sequence I receive in the first message coming from the client
// (default is false)
config.add("acceptInboundSeqFromClient", "false");
Server server = new SimpleFixApplicationServer(nio, 45451, config);
server.open();
nio.start();
}
}
A full explanation of the code above can be found here.
Disclaimer: I am one of the developers of CoralFIX.
I assume you haven't built it in a vacuum, but rather to talk to some broker's FIX server? If that's the case, speak to your broker as they are likely to have a test server you could connect to.
It depends if you want a hosted or a DIY solution. If you want to quickly bring up a FIX sessions without having to compile/build/setup then I would go for a hosted simulator.
We have an account with these guys http://www.fixsim.com/ and would recommend them (simple to setup and add more connections).
(I don't work for the company and get nothing to say this)
you can try to search more information here: http://fixprotocol.org/discuss/
I work for a broker, and you're more than welcome to connect to our Staging FIX servers. I do not know of any anonymous servers around, perhaps this is something the FIX community could set up..
Cheers,
Chris