I'm using Java High level rest client in my project and i want to limit my indices based on document count. I'm using rollover api but its not creating indices automatically.
The code will be given below.
I'm creating index pattern so that my custom analyzer will be applied to all other indices that follow the respective pattern.
PutIndexTemplateRequest request = new PutIndexTemplateRequest("testingtemplate");
request.source("{\n" +
" \"index_patterns\":[\n" +
" \"test_log-*\"\n" +
" ],\n" +
" \"settings\": {\n" +
" \"analysis\": {\n" +
" \"analyzer\": { \n" +
" \"my_analyzer\": {\n" +
" \"type\": \"custom\",\n" +
" \"tokenizer\": \"whitespace\",\n" +
" \"filter\": []\n" +
" }\n" +
" }\n" +
" }\n" +
" },\n" +
" \"mappings\": {\n" +
" \"properties\": {\n" +
" \"fullLog\": {\n" +
" \"type\": \"text\",\n" +
" \"analyzer\": \"my_analyzer\"\n" +
" }\n" +
" }\n" +
" }\n" +
" }",XContentType.JSON);
return client.indices().putTemplate(request,RequestOptions.DEFAULT).isAcknowledged();
My Rollover code. Here i want to rollover one when index gets one or document.
final RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
boolean isIndexTemplateCreated = createIndexTemplate(client);
System.out.println(isIndexTemplateCreated);
CreateIndexRequest request = new CreateIndexRequest("test_log-1");
request.alias(new Alias("temp_alias_new"));
CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);
RolloverRequest roll_req = new RolloverRequest("temp_alias_new",null);
roll_req.addMaxIndexAgeCondition(new TimeValue(7, TimeUnit.DAYS));
roll_req.addMaxIndexDocsCondition(1);
roll_req.addMaxIndexSizeCondition(new ByteSizeValue(5, ByteSizeUnit.GB));
RolloverResponse rolloverResponse = client.indices().rollover(roll_req, RequestOptions.DEFAULT);
Map<String,Object> map = new HashMap<>();
map.put("fullLog","Hi");
client.index(new IndexRequest("temp_alias_new").source(map), RequestOptions.DEFAULT);
map.put("fullLog","hello");
client.index(new IndexRequest("temp_alias_new").source(map), RequestOptions.DEFAULT);
But the code is not working and the rollover api is not creating indices automatically. All the 2 documents are stored only in test_log-1 index.
Is there any mistake in my code?
Thanks
Note: Rollover will not happen automatically.
Elasticsearch tries to rollover the index on getting a rollover request.
For example consider the following sequences:
You have a new index test_log-1 which is empty and is being pointed by the alias temp_alias_new.
If you try to rollover now, none of the conditions mentioned along with rollover request holds good as the index is new and empty. So this time rollover fails.
Add some documents to the index.
Now try the rollover with the condition maxIndexDocsCondition(1), it will rollover. Because the condition holds good.
Update
With the latest release of elastic search, you can use ILM (Index Life cycle Management) to automate the rollover.
Here is doc link for more info: https://www.elastic.co/guide/en/elasticsearch/reference/7.x/getting-started-index-lifecycle-management.html
Related
So in Java, I have this as my json String:
public static void main(String[] args) {
String stringJson = "{\n" +
"nodes: {\n" +
"Random-key-Here: {\n" +
"name: \"PRO-cnt-elastic2-4-i-0a414518a5b67\",\n" +
"transport_address: \"172.18.7.104:9300\",\n" +
"host: \"189.88.7.884\",\n" +
"ip: \"188.88.8.884:8880\",\n" +
"roles: [\n" +
"\"ingest\"\n" +
"],\n" +
"attributes: {\n" +
"zone: \"us-east-1a\"\n" +
"},\n" +
"tasks: {\n" +
"Random-key-Here: {\n" +
"node: \"-2688888mRPYHywWA\",\n" +
"id: 37202060,\n" +
"type: \"transport\",\n" +
"action: \"indices:data/write/reindex\",\n" +
"status: {\n" +
"total: 4869544,\n" +
"updated: 13920,\n" +
"created: 3654080,\n" +
"deleted: 0\n" +
"},\n" +
"description: \"blaaa\",\n" +
"start_time_in_millis: 1596456902705,\n" +
"running_time_in_nanos: 647855785005,\n" +
"cancellable: true,\n" +
"headers: { }\n" +
"}\n" +
"}\n" +
"}\n" +
"}\n" +
"}";
JsonObject jo = new JsonParser().parse(stringJson).getAsJsonObject();
Set s = jo.get("nodes").getAsJsonObject().entrySet().stream().map(e ->
e.getValue().getAsJsonObject().get("tasks").collect(toSet());
}
Important: notice inside the Json some of the Keys are actually a random string that I couldn't know what it is in advance... - so I Wrote them as "Random-key-Here"
In short: My goal is to retrieve the number of this field, "total: 4869544".
Above that's what I've tried. I've managed to have a list that contains one object member of a set, and it's the "Random-key-Here" as the key, and the value is the rest of my nested json.
I thought if I would repeat the same logic as I did to overcome the first "Random-key-Here" thing, like I did with
.entrySet().stream().map(e ->
e.getValue().getAsJsonObject().get("tasks")
then I would get to another level below, so instead of "tasks" I wrote "status"
and I tought then I would be able to get the rest of the nested json which is easier from then on... but instead, nothing is returned when doing that:
jo.get("nodes").getAsJsonObject().entrySet()
.stream()
.map(e -> e.getValue().getAsJsonObject().get("tasks").getAsJsonObject().entrySet()
.stream()
.map(ent-> ent.getValue().getAsJsonObject().get("status").getAsJsonObject().get("total").getAsJsonObject()))
would appreciate any help to get to the nested part I need which is: "total: 4869544".
and keep in mind, I can't simply just use this:
jo.get("nodes").getAsJsonObject().get("Random-key-Here").getAsJsonObject().get("tasks")
.getAsJsonObject().get("Random-key-Here").getAsJsonObject()
.get("status").getAsJsonObject().get("total").getAsInt()
because of "Random-key-Here".
Tnx for the helpers!
Answer to myself: So, on every unknown key that you have, you can use entrySet()
and then loop over it until you get to your desired field member.
so this usage solves the problem and can get you the nested Json even when your key is unknown in advance:
jo.get("nodes").getAsJsonObject().entrySet()
.forEach(node -> node.getValue().getAsJsonObject().get("tasks").getAsJsonObject().entrySet()
.forEach(task -> { int i = task.getValue().getAsJsonObject().get("status").getAsJsonObject().get("total").getAsInt();
System.out.println(i); }))
pls share if someone has a more efficient way to do so
In spring cloud contract (groovy) file I have issues with extracting a segment of request URL to use it as a parameter of in response body, e.g.
package contracts
import org.springframework.cloud.contract.spec.Contract
Contract.make {
description "Should return OK "
request {
url "/discovery.svc/something(\'${value(consumer(regex(".*")),producer('defaultSomething'))}\')"
method GET()
}
response {
status 200
body :
value(
consumer(file("response/defaultSomething.xml").file.write(
String.format("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
"<entry xmlns:metadata=\"http://docs.oasis-open.org/odata/ns/metadata\" xmlns:data=\"http://docs.oasis-open.org/odata/ns/data\" xmlns=\"http://www.w3.org/2005/Atom\" metadata:context=\"http://localhost:8082/discovery.svc/\$metadata#Environment/ContentServiceCapability\" xml:base=\"http://localhost:8082/discovery.svc\">\n" +
" <id>http://localhost:8082/discovery.svc/something('%1\$s')</id>\n" +
" <title></title>\n" +
" <summary></summary>\n" +
" <updated>2020-03-23T15:19:53.272668900Z</updated>\n" +
" <author>\n" +
" <name>SDL OData v4 framework</name>\n" +
" </author>\n" +
" <link rel=\"edit\" title=\"ContentServiceCapability\" href=\"something('%1\$s')\"></link>\n" +
" <link rel=\"http://docs.oasis-open.org/odata/ns/related/Environment\" type=\"application/atom+xml;type=entry\" title=\"Environment\" href=\"something('%1\$s')/Environment\"></link>\n" +
" <link rel=\"http://docs.oasis-open.org/odata/ns/relatedlinks/Environment\" type=\"application/xml\" title=\"Environment\" href=\"something('%1\$s')/Environment/\$ref\"></link>\n" +
" <category scheme=\"http://docs.oasis-open.org/odata/ns/scheme\" term=\"#Tridion.WebDelivery.Platform.ContentServiceCapability\"></category>\n" +
" <content type=\"application/xml\">\n" +
" <metadata:properties>\n" +
" <data:id>%1\$s</data:id>\n" +
" <data:LastUpdateTime metadata:type=\"Int64\">1580489088713</data:LastUpdateTime>\n" +
" <data:URI>http://localhost:8081/content.svc</data:URI>\n" +
" <data:ExtensionProperties metadata:type=\"#Collection(Tridion.WebDelivery.Platform.ContentKeyValuePair)\"></data:ExtensionProperties>\n" +
" </metadata:properties>\n" +
" </content>\n" +
"</entry>", fromRequest().url().split('/')[-1].split('\'')[-2]))),
producer(file("response/defaultSomething.xml")))
)
}
}
the problen is that fromRequest().url() doesn't seem to be working and when I try to insert
print fromRequest().url()
in response section (in order to debug) I get as a result:
DslProperty{
clientValue={{{request.url}}},
serverValue={{{request.url}}}}
instead of plain string URL. .toString() doesn't help neither. Do you have any ideas how can I get request.url as a plain string?
I was trying to insert mapping in the RestHighLevelClient of elasticsearch 6.2.1
From the following link I had found the following code for the insertion of mapping
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/master/java-rest-high-put-mapping.html
RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost(ipaddress, port, "http")));
client.indices().putMapping(putMappingRequest);
But I was unable to find putMapping(putMappingRequest) in the client.indices()
this is the maven dependency that I had added in the project
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.2.1</version>
Can anyone help me to find out correct jar file that suits my requirement or any other way to insert mapping using RestHighLevelClient
Any help is really appreciated.
Your link points to the documentation of an unreleased version. For 6.2.1, you need to use the CreateIndexRequest, like this:
CreateIndexRequest request = new CreateIndexRequest("twitter");
request.mapping("tweet",
" {\n" +
" \"tweet\": {\n" +
" \"properties\": {\n" +
" \"message\": {\n" +
" \"type\": \"text\"\n" +
" }\n" +
" }\n" +
" }\n" +
" }",
XContentType.JSON);
CreateIndexResponse createIndexResponse = client.indices().create(request);
I would want to send statistical information to my clients showing the number of transactions processed on every terminal or branch. I am using Apache Commons Email to send HTML emails.
I would like to send a pie-chart data like this one on the site.
My java code is basic extracted from.
It goes like:
public void testHtmlEmailPiechart()
throws UnsupportedEncodingException, EmailException, MalformedURLException {
HtmlEmail email = new HtmlEmail();
email.setHostName(emailServer);
email.setSmtpPort(587);
email.setSSLOnConnect(true);
email.setAuthentication(userName, password);
email.setCharset(emailEncoding);
email.addTo(receiver, "Mwesigye John Bosco");
email.setFrom(userName, "Enovate system emailing alert");
email.setSubject("Conkev aml Engine Statistics");
URL url = new URL("https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcROXe8tn1ljtctM53TkLJhLs6gEX56CvL0shvyq1V6wg7tXUDH8KRyVP30");
// URL url = new URL("http://www.apache.org/images/asf_logo_wide.gif");
String cid2 = email.embed(url, "logo.gif");
email.setHtmlMsg("<html>\n" +
" <head>\n" +
" <script type=\"text/javascript\" src=\"https://www.gstatic.com/charts/loader.js\"></script>\n" +
" <script type=\"text/javascript\">\n" +
" google.charts.load(\"current\", {packages:[\"corechart\"]});\n" +
" google.charts.setOnLoadCallback(drawChart);\n" +
" function drawChart() {\n" +
" var data = google.visualization.arrayToDataTable([\n" +
" ['Task', 'Hours per Day'],\n" +
" ['Work', 11],\n" +
" ['Eat', 2],\n" +
" ['Commute', 2],\n" +
" ['Watch TV', 2],\n" +
" ['Sleep', 7]\n" +
" ]);\n" +
"\n" +
" var options = {\n" +
" title: 'My Daily Activities',\n" +
" is3D: true,\n" +
" };\n" +
"\n" +
" var chart = new google.visualization.PieChart(document.getElementById('piechart_3d'));\n" +
" chart.draw(data, options);\n" +
" }\n" +
" </script>\n" +
" </head>\n" +
" <body>\n" +
" <div id=\"piechart_3d\" style=\"width: 900px; height: 500px;\">Piechart Data</div>\n" +
" </body>\n" +
"</html>");
email.setTextMsg("Your email client does not support HTML messages");
email.send();
}
My guess is that the JavaScript is not recognized because the code works like sending images,styling fonts and I have sent to my email address some sample mail. I would like your help or recommendation of any material I can read to achieve this as long as am using Java.The processes is automated running in the background so no user interface is involved.
Thanks.
We call a webservice from our C# app which takes about 300ms using WCF (BasicHttpBinding). We noticed that the same SOAP call does only take about 30ms when sending it from SOAP UI.
Now we also implemented a test accessing the webservice via a basic WebClient in order to make sure that the DeSer-part of the WCf is not the reason for this additional delay. When using the WebClient class the call takes about 300ms as well.
Any ideas on why Java compared to C# is about 10x faster in this regard? Is there some kind of tweaking possible on the .NET side of things?
private void Button_Click(object sender, RoutedEventArgs e)
{
executeTest(() =>
{
var resultObj = client.getNextSeqNr(new WcfClient()
{
domain = "?",
hostname = "?",
ipaddress = "?",
loginVersion = "?",
processId = "?",
program = "?",
userId = "?",
userIdPw = "?",
userName = "?"
}, "?", "?");
});
}
private void Button_Click_1(object sender, RoutedEventArgs e)
{
WebClient webClient = new WebClient();
executeTest(()=>
{
webClient.Proxy = null;
webClient.CachePolicy = new System.Net.Cache.RequestCachePolicy(System.Net.Cache.RequestCacheLevel.NoCacheNoStore);
webClient.Headers.Add("Content-Type", "application/xml");
webClient.Encoding = Encoding.UTF8;
var data = "<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:ser=\"SomeNamespace\">" +
" <soapenv:Header/>" +
" <soapenv:Body>" +
" <ser:getNextSeqNr>" +
" <!--Optional:-->" +
" <clientInfo>" +
" <!--Optional:-->" +
" <domain>?</domain>" +
" <!--Optional:-->" +
" <hostname>?</hostname>" +
" <!--Optional:-->" +
" <ipaddress>?</ipaddress>" +
" <!--Optional:-->" +
" <loginVersion>?</loginVersion>" +
" <!--Optional:-->" +
" <processId>?</processId>" +
" <!--Optional:-->" +
" <program>?</program>" +
" <!--Optional:-->" +
" <userId>*</userId>" +
" <!--Optional:-->" +
" <userIdPw>?</userIdPw>" +
" <!--Optional:-->" +
" <userName>?</userName>" +
" </clientInfo>" +
" <!--Optional:-->" +
" <name>?</name>" +
" <!--Optional:-->" +
" <schema>?</schema>" +
" </ser:getNextSeqNr>" +
" </soapenv:Body>" +
"</soapenv:Envelope>";
string result = webClient.UploadString("http://server:8080/service", "POST", data);
});
}
Am I missing something here? Any idea would be helpful... ;-)
Kind regards,
Sebastian
I just found the reason for this.
It's the 100-Expect Continue HTTP Header and the corresponding implementation in .NET. The .NET client wait 350ms as default on the server. This causes the delays. Java seems to have other default values here...
Just add the following line of code very early in your code:
System.Net.ServicePointManager.Expect100Continue = false;
Cheers!