I am currently working on a project where our main database is mongodb and for searching we use elasticsearch. We have inserted data in to mongodb by a java application. And we used river plugin to sync data. Up to now we have done syncing data between mongodb and elasticsearch manually by executing shellscript files mentioned below. (setup.sh && bash.sh)
//setup.sh
curl -XPOST http://localhost:9200/classdata -d #setup.json
//setup.json
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"classdata": {
"properties": {
"className": {
"type": "string"
},
"jarID": {
"index": "not_analyzed",
"type": "string"
},
"jarFileName": {
"index": "not_analyzed",
"type": "string"
},
"dependencies": {
"properties": {
"methodSignature": {
"type": "string"
},
"dependedntClass": {
"type": "string"
}
}
}
}
}
}
}
//bash.sh
curl -XPUT "localhost:9200/_river/classdata/_meta" -d '
{
"type": "mongodb",
"mongodb": {
"servers": [
{ "host": "127.0.0.1", "port": 27017 }
],
"options": { "secondary_read_preference": true },
"db": "E",
"collection": "ClassData"
},
"index": {
"name": "classdata",
"type": "classdata"
}
}'
But now our requirement has changed. Now we need to automate the process, like after inserting data in to mongodb we have to automatically sync data between elasticsearch and mongodb.
I have no idea how to do that. If some one know how to automate this process please help me.
I strongly recommend you monstache. It runs in the background and automately sync data from Mongodb to Elasticsearch. And you can configure to specify which db and what kind of operation(insert, update, delete...) you want to sync, the configuration options listed in here
MongoConnector Plugin supports data sync between MongoDB and Elastic Search.
1) Install Mongo Connector in your server.
`pip install mongo-connector`
2) Install Doc Manager based on target system. There are various implementations for Doc Manager based on the Target system. Install the one that supports Elastic Search and in particular the version that you have. Eg)
pip install 'mongo-connector[elastic5]'
3) Start Mongo Connector with configurations of the source(mongodb) and target systems. Eg)
mongo-connector -m <mongodb server hostname>:<replica set port> -t <replication endpoint URL, e.g. http://localhost:8983/solr> -d <name of doc manager>
Now data will be automatically synced up between the two systems.
For more information, use the following links,
https://www.mongodb.com/blog/post/introducing-mongo-connector
https://github.com/mongodb-labs/mongo-connector
https://github.com/mongodb-labs/mongo-connector/wiki/Usage%20with%20ElasticSearch
Related
Iam trying to run simulateprincipalpolicy through java SDK and getting incorrect results.
I have policy something like this and attached this policy to role 'Myrole':
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "id",
"Effect": "Allow",
"Action": [
"ec2:DescribeTags"
],
"Resource": "*"
}
]
}
Java Code :
SimulatePrincipalPolicyRequest simulatePrincipalPolicyRequest = new SimulatePrincipalPolicyRequest()
simulatePrincipalPolicyRequest.setPolicySourceArn("arn:aws:iam::123456789012:role/Myrole");
simulatePrincipalPolicyRequest.withActionNames("ec2:DescribeTags");
Result:
{
EvalActionName: ec2:DescribeTags
EvalResourceName: *
EvalDecision: implicitDeny
MatchedStatements: []
MissingContextValues: []
OrganizationsDecisionDetail: {AllowedByOrganizations: false}
EvalDecisionDetails: {}
ResourceSpecificResults: []
}
The response is incorrect because when i try to perform that action Im able to do so.
I've run into a similar situation by calling simulate-principal-policy directly:
AllowedByOrganizations: false indicates there is an Organization SCP, an organisation-wide service control policy applied that somehow the simulator interprets as its denying access.
The issue in my case is that there are some of Org SCP that deny all kind of access to some regions, but not to others.
The simulator seems to be unable to discern a given region from others, or at least I did not find a way to overcome this problem.
I send a Firebase notification from my java server.
I need to specify the sound to IOS to be "default", cause right now I get no notification sound.
I saw similar questions but none is in Java server code.
I understand that I should deliver something like this:
apns: {
payload: {
aps: {
sound: "default",
},
},
},
And my IOS developer says that he gets something like this:
FCM Message Data: {"data": {}, "messageId": "1646649783631300",
"mutableContent": true, "notification": {"body": "test", "ios": {},
"title": "test"}, "sentTime": "1646649783"}
What should I provide in notification data for the sound to be heard in IOS?
Folks,
I am using the v3.2.4 of the applicationinsights.jar on a Wildfly application server and am able to see all information go into Azure (Application Insights) portal.
https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-in-process-agent
However, I'm needing to do this for many application instances and am thinking it could be wise to suppress certain kinds of telemetry types (e.g. dependencies as one example) as it creates a lot of noise and data.
Is it possible to do this via the applicationinsights.json file?
Any guidance into this appreciated!
Update (5th Jan 2022): I am using a codeless solution whereby all configuration and thus suppression is done in the .json file.
Solutions involving v2 approaches via C#/java are out of scope (although this is what I have used in the past).
https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-telemetry-processors shows some ideas but it is not explicit with respect to supression of certain types as the default approach seems to push too much data to Azure.
You could try using sampling and sampling overrides (preview) to achieve the desired results. Though I am not certain if you can easily match only dependency calls by certain attributes. Sampling overrides are the reccomended way to filter out telemetry for cost reasons.
Example: Suppress collecting telemetry for a noisy dependency call
This will suppress collecting telemetry for all GET my-noisy-key redis calls.
{
"connectionString": "InstrumentationKey=00000000-0000-0000-0000-000000000000",
"preview": {
"sampling": {
"overrides": [
{
"attributes": [
{
"key": "db.system",
"value": "redis",
"matchType": "strict"
},
{
"key": "db.statement",
"value": "GET my-noisy-key",
"matchType": "strict"
}
],
"percentage": 0
}
]
}
}
}
You can also disable certain telemetry sources, though I admit that this is not exactly the same as what you are asking for:
{
"instrumentation": {
"azureSdk": {
"enabled": false
},
"cassandra": {
"enabled": false
},
"jdbc": {
"enabled": false
},
"jms": {
"enabled": false
},
"kafka": {
"enabled": false
},
"micrometer": {
"enabled": false
},
"mongo": {
"enabled": false
},
"rabbitmq": {
"enabled": false
},
"redis": {
"enabled": false
},
"springScheduling": {
"enabled": false
}
}
}
Below are few steps to suppress Application insights telemetry:
Create the telemetry process
Registering a Telemetry Process.
Adding the filters.
Enabling based on LogLevel.
This is one of the possible way.
You can get the full details from this blog
I am trying to send the foreground and background notification to selected Android and ios device through the Java code. I was using the following Json for both platform
{
"data": {
"appointmentId": 156913762771107,
"click_action": "com.invite",
"body": "Yay! Message"
},
"to": "eY0euHbZ0-s:APA91bFYUT-jxfuOLiqfJ5BsjGU9a2KHXE39g_gUkO4_hpmPIPeIqTauhNBYj50Jniyzx0_9aRTHoezugYPcshA9dDs60YoEWAno3gJEUYxfMzgtcjy4mVC5RuSqT_RRJN6RVkgWVyMf",
"notification": {
"click_action": "com.invite",
"body": "Yay! Message",
"sound": "default",
"badge": 1,
"content_available": true
}
}
Everything was working fine a few months back but now it stopped working. I tried to search for the common Json which can be used for both platforms but couldn't find the solution.
I have created Azure SQL Data warehouse and done pause and resume actions using Java SDK
Now I want to disable Geo backup policy when creation of Azure SQL Data warehouse.
How can I do it using Java SDK?
Refer below image for Geo backup policy in Azure SQL Data warehouse.
You could use Azure Rest API to do this.
PUT https://management.azure.com/subscriptions/***********/resourceGroups/shui156/providers/Microsoft.Sql/servers/shui156/databases/shui156/geoBackupPolicies/Default?api-version=2014-04-01
Body:
{
"id": "/subscriptions/***********/resourceGroups/shui156/providers/Microsoft.Sql/servers/shui156/databases/shui156/geoBackupPolicies/Default",
"name": "Default",
"type": "Microsoft.Sql/servers/databases/geoBackupPolicies",
"location": "East US",
"kind": null,
"properties": {
"state": "Disabled",
"storageType": null
}
}