I have four db servers with the same db structure but different data.
Currently when new data are inserted to database my application get this data, create template and send email.
I would like to separate sending email from my applications.
For example some thread which will start once per 10 minutes. It selects data from my four db servers, connect to mail server and send email to users.
It's possible with using JMS or something similar ?
Thanks for replys !
I did the same by creating a mail table (likely one per DB) and save the Template and data (or subject/body) in it. A separate process could be Quartz or your own pooling thread read that table and connect to mail server and send email and update email status.
In this way you can check status of any email at any given time and even you can resend any email. The table needs to purge/archive after some time may be after 1 day or 1 week depends on table size.
Related
I have an application that can be reduced/simplify to this flow:
user sends request to app A
app A inserts info about the request and user into DB ( marked as "B in progress" and "C in progress")
app A pushes the data into queue and returns to user
app B retrieves data from queue and process it
app B finishes processing the data and marks record in DB as "B done"
app C retrieves data from queue and process it
app C finishes processing the data and marks record in DB as "C done"
In other words, user sends request to app, app saves the record to the database and send it to queue, app B and C takes request from queue and process it ( each app does different thing but requires data from request ) and when they are done i want to mark the request in db as done for both APP.
This can be achieved, if all apps share DB. However sharing the DB like this between microservices is considered anti-pattern.
What are some design patterns to solve this? Am i really left with only option - make app A expose rest API and call the endpoint from app B and C to update the row in DB?
Thanks for help!
This more sounds like choreography , event driven process. Instead of DB, did u consider using Kafka where status gets enriched at each publish.
Евгений Кравцов:
I develop some tiny service with http backend and android app. And recently i felt the lack of knowledge in such a systems.
Case:
- Client makes order in app and send request to server
- Server successfuly recieved data and make a database row for order
- After database work completes, backend tries to respond to App with 200 success code.
- App user faced some internet connection problems and can not receive server response. App gets timeout exception and notify user, that order was not successful
- After some time, internet connection on user device restored and he send another request with same oreder.
- Backend recieves this again and create duplicate for previous order
So i got 2 orders in database, but user wanted to create only one.
Question: How can i prevent such a behavior?
You should add a key to your table in the db, for example defining the key as compound key of user_id, type - so specific user cannot make 2 orders with the same order type.
This will cause the second db insert to fail.
Last two days i've been searching suitable solution for the problem described below.
In my standalone notification-service module I have an abstract Message entity. Message has 'to', 'from', 'sentAt', 'receivedAt' and other attributes. The responsibility of the notification-service is to:
send new messages using different registered message providers (SMS, EMAIL, Skype , etc).
receive new messages from registered message providers
update status for already sent messages.
Notification-service module is developed as standalone module that is available by SOAP protocol. A lot of clients can use this module to send or searching through already received messages.
Clients want to attach some properties (~ smth like tags) while sending messages for further searching messages by these properties. These properties make a sense only in client's environment.
For example, Client A might want to send message and save following custom properties :
1. Internal system id of user whom system sends message
2. Distinguish flag (whether id related to users / admins or clients)
3. Notification flag (notification/alert/ ...)
Client B might want to send message and save another set of custom properties :
1. Internal system operator id (who sends sms)
2. Template id that was used to send message
Custom properties can be used by the clients to search already sent messages.
For example:
Client A could find SMS messages sent to administrator users in period between [Date 1; Date 2] that have 'alert' status.
Client B could find all notification sent by specified template.
Of course, data should be fetched page by page.
At first I created the following database model:
Database scheme
To find all messages with specified properties I tried to use query:
SELECT * FROM (SELECT message_id FROM custom_message_properties
WHERE CONCAT(CONCAT(key, ':'), value) IN ('property1:value1', 'property2:value2')
GROUP BY message_id having(count(*)) = 2)
as cmp JOIN message m ON cmp.message_id = m.id ORDER BY ID LIMIT 100 OFFSET 0
Query worked fine (although it seems me not very good) in database with small data. I decided to check results for ~ real awaited data .
So i generated 10 000 000 messages that have 40 000 000 custom properties and checked result. Execution time was ~ 2 minutes. The most time consumed operation was following sub-select:
SELECT message_id FROM custom_message_properties
WHERE CONCAT(CONCAT(key, ':'), value) IN ('property1:value1', 'property2:value2')
I understand that string comparison is very slow cause database index feature is not used. I decided to change database structure to merge 'key' and 'value' columns into single one. So i updated by database scheme :
Updated database scheme
I checked result again. Now execution time was ~20 seconds. It's much better but still is not suitable for production use.
So now I have no idea how to improve performance without significant changes in application architecture design.
The only one thought i have is to create separate table for each client with required client properties.
client(i)_custom_properties {
mid bigint, // foreign key references message (id)
p1 type1,
p2 type2,
......
pn type(n)
}
I have spent a lot of time while trying to find any useful information. I have also analyzed 'stackoverflow' database cause it seemed me that it should be quite the same. But in 'stackoverflow' there are ~ 50 000 different tags. Not so much that my database could have.
Any help is appreciated. Thanks, in advance!
Project environment that i use :
Postgres database (9.6)
Java 1.8
Spring modules (spring-boot, spring-data-jpa + hibernate, spring-ws, etc).
I have not found any suitable solution except creating additional table with client's properties for each client.
I know, that solution is not so flexible,
but now search query time is less than 1 second.
In future, I will try to solve the same problem using noSQL data storage.
I was checking for some alternatives for Quartz-scheduler.
Though this is not a complete replacement, I was trying out RabbitMQ Delayed Messages Plugin (suits for my use-case).
I was able to get the scheduling work but I was not to view the messages which are delayed(which are stored in Mnesia).
Is there a way to check the messages and/or number of messages in Mnesia?
Edit : I inferred that the messages are stored in Mnesia from the comment from here.
There is no way to check the messages that RabbitMQ is persisting in it's mnesia database.
RabbitMQ is not a generalized datastore. It is a purpose-built message broker and queueing system. The datastore it has in it is there to facilitate the persistence of messages, not to be queried and used as if it were a database on it's own.
To view the data inside MNESIA you could :
Write a simple Erlang program as this, as result you have:
(rabbit#gabrieles-MBP)5>
load:traverse_table_and_show('rabbit_delayed_messagerabbit#gabrieles-MBP').
{delay_entry,
{delay_key,1442258857832,
{exchange,
{resource,<<"/">>,exchange,<<"my-exchange">>},
'x-delayed-message',true,false,false,
[{<<"x-delayed-type">>,longstr,<<"direct">>}],
undefined,undefined, {[],[]}}},
{delivery,false,false,<0.2008.0>,
{basic_message,
{resource,<<"/">>,exchange,<<"my-exchange">>},
[<<>>],
{content,60,
{'P_basic',undefined,undefined,
[{<<"x-delay">>,signedint,100000}],
undefined,undefined,undefined,undefined,undefined,
undefined,undefined,undefined,undefined,undefined,
undefined},
..
OR in this way:
execute an Erlang shell session using:
erl -set-cookie ABCDEFGHI -sname monitorNode#gabrielesMBP
you have to use the same cookie that rabbitmq are using.
Typically $(HOME).erlang.cookie
execute this command:observer:start().
and you should have this:
Once you are connected to rabbitmq node open Table Viewer and from the menu Mnesia table as:
Here you can see your data:
How I can send file from one client (A) to another one (B) via socket? and vice versa, send file from B to A. I mean that make the client sender and receiver at the same time.
In other word, when muticlient connect to server, how I distinguish between clients ?
You need to implement you own communication message format in short a simple protocol .
You keep a list of all active sockets in a shared list/map , and based on the request from the message you pick up the apt client and push the desired message to that.
You can implement the actual message format as you want, but this can be the blueprint.
In this case lets say your client A sends message : 1. Client Id 2. File Start 3 X . File Content 4. File End
as soon as you get a connection you get the target client id , the file start message lets you understand the next message just needs to be diverted to target and file End message defines the transfer complete.
Also, you may would like to send Acknowledgement message from server to client, in order to eradicate transfer issues.
It is good way to manage client using their id(i.e. a unique long or string or any other for each user). At the time of connection to socket client send their id , store that is in collection. And when a user(Client) want to send file send with own id and Id of that user(Client) want to send.