Develop Reference

Any reasonable way to sort DynamoDB tables?

I am using spring-data-dynamodb 5.1.0 in my Spring Boot 2.5.4 setup. I am trying to get this sample code to work for sorting on SongTitle column on which I have defined a global secondary index.
Here's my entity object.`public class ImprovedMusic {
#Id
private ImprovedMusicId improvedMusicId;
public ImprovedMusic() {}
public ImprovedMusic(ImprovedMusicId improvedMusicId) {
this.improvedMusicId = improvedMusicId;
}
#DynamoDBHashKey(attributeName = "Artist")
public String getArtist() {
return improvedMusicId != null ? improvedMusicId.getArtist() : null;
}
public void setArtist(String artist) {
if (improvedMusicId == null) {
improvedMusicId = new ImprovedMusicId();
}
improvedMusicId.setArtist(artist);
}
#DynamoDBIndexRangeKey(attributeName = "SongTitle", globalSecondaryIndexNames={"GSISongTitle"})
#DynamoDBIndexHashKey(attributeName="SongTitle", globalSecondaryIndexNames = "GSISongTitle")
public String getSongTitle() {
return improvedMusicId != null ? improvedMusicId.getSongTitle() : null;
}
public void setSongTitle(String songTitle) {
if (improvedMusicId == null) {
improvedMusicId = new ImprovedMusicId();
}
improvedMusicId.setSongTitle(songTitle);
}
}
`
I invoke this code in my test
`Sort sortBy = Sort.by("SongTitle").descending;
Pageable page = PageRequest.of(1,5,sortBy);
Page aPage = myRepositor.findAll(page);`
Please note, I am not trying to search by SongTitle, I want this list to be sorted by the songtitle. The above test throws the following exception. Anything I am doing wrong here?
The number of conditions on the keys is invalid (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException;
Request ID: c0634958-d465-4560-8dce-3e51519b4ed6)
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: The
number of conditions on the keys is invalid (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException;
Request ID: c0634958-d465-4560-8dce-3e51519b4ed6) at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1799)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1

Grouping with Sentry

I want to group exception with Sentry, the exception comes from different servers, but I want all exception by type together, for example, all NPE be grouped. I know you can extend EventBuilderHelper and this is how sentry group things, but sentry java doesn't provide features to send an event with fingerprints of the method, error type, etc, as others SDK like this example in docs.sentry.io
function makeRequest(method, path, options) {
return fetch(method, path, options).catch(err => {
Sentry.withScope(scope => {
// group errors together based on their request and response
scope.setFingerprint([method, path, err.statusCode]);
Sentry.captureException(err);
});
});
}
this is what I try to do, but in this scope, don't have knowledge about method, error, etc.
package com.test;
import io.sentry.SentryClient;
import io.sentry.event.EventBuilder;
import io.sentry.event.helper.ContextBuilderHelper;
public class FingerprintEventBuilderHelper extends ContextBuilderHelper {
private static final String EXCEPTION_TYPE = "exception_type";
public FingerprintEventBuilderHelper(SentryClient sentryClient) {
super(sentryClient);
}
#Override
public void helpBuildingEvent(EventBuilder eventBuilder) {
super.helpBuildingEvent(eventBuilder);
//Get the exception type
String exceptionType =
if (exceptionType != null) {
eventBuilder.withTag(EXCEPTION_TYPE, exceptionType);
}
//Get method information and params
if (paramX != null) {
eventBuilder.withTag("PARAM", paramX);
}
}
}
the json send to the server has some information about the exception, but I dont know hoy to get it
...
"release": null,
"dist": null,
"platform": "java",
"culprit": "com.sun.ejb.containers.BaseContainer in checkExceptionClientTx",
"message": "Task execution failed",
"datetime": "2019-06-26T14:13:29.000000Z",
"time_spent": null,
"tags": [
["logger", "com.test.TestService"],
["server_name", "localhost"],
["level", "error"]
],
"errors": [],
"extra": {
"Sentry-Threadname": "MainThread",
"rid": "5ff37e943-f4b4-4hc9-870b-4f8c4d18cf84"
},
"fingerprint": ["{{ default }}"],
"key_id": 3,
"metadata": {
"type": "NullPointerException",
"value": ""
},
...

You can get the type of exception that was raised, but I have my doubts about getting the parameters related to functions in the trace
EventBuilderHelper myEventBuilderHelper = new EventBuilderHelper() {
public void helpBuildingEvent(EventBuilder eventBuilder) {
eventBuilder.withMessage("Overwritten by myEventBuilderHelper!");
Map<String, SentryInterface> ifs = eventBuilder.getEvent().getSentryInterfaces();
if (ifs.containsKey("sentry.interfaces.Exception"))
{
ExceptionInterface exI = (ExceptionInterface) ifs.get("sentry.interfaces.Exception");
for (SentryException ex: exI.getExceptions()){
String exceptionType = ex.getExceptionClassName();
}
}
}
};
If you look at the sendException method of the client, it initiates the ExceptionInterface with the actual Exception
public void sendException(Throwable throwable) {
EventBuilder eventBuilder = (new EventBuilder()).withMessage(throwable.getMessage()).withLevel(Level.ERROR).withSentryInterface(new ExceptionInterface(throwable));
this.sendEvent(eventBuilder);
}
And the constructor for the same is like
public ExceptionInterface(Throwable throwable) {
this(SentryException.extractExceptionQueue(throwable));
}
public ExceptionInterface(Deque<SentryException> exceptions) {
this.exceptions = exceptions;
}
So each exception get converted to a SentryException, but the original exception is not stored. So if you need params also, you will need to throw a custom exception with those parameter and also override the sendException method, not a straightforward way

Reindex part of Elasticsearch index onto new index via Jest

I have a test ElasticSearch 6.0 index populated with millions of records, likely to be in the billions in production. I need to search for a subset of these records, then save this subset of the original set into a secondary index for later searching. I have proven this out via querying ES on Kibana, the challenge is to find appropriate APIs in Java 8 using my Jest client (searchbox.io, version 5.3.3) to do the same. The ElasticSearch cluster is on AWS, so using a transport client is out.
POST _reindex?slices=10&wait_for_completion=false
{ "conflicts": "proceed",
"source":{
"index": "my_source_idx",
"size": 5000,
"query": { "bool": {
"filter": { "bool" : { "must" : [
{ "nested": { "path": "test", "query": { "bool": { "must":[
{ "terms" : { "test.RowKey": ["abc"]} },
{ "range" : { "test.dates" : { "lte": "2018-01-01", "gte": "2010-08-01"} } },
{ "range" : { "test.DatesCount" : { "gte": 2} } },
{ "script" : { "script" : { "id": "my_painless_script",
"params" : {"min_occurs" : 1, "dateField": "test.dates", "RowKey": ["abc"], "fromDate": "2010-08-01", "toDate": "2018-01-01"}}}}
]}}}}
]}}
}}
},
"dest": {
"index": "my_dest_idx"
},
"script": {
"source": <My painless script>
} }
I am aware I can perform a search on the source index, then create and bulk load the response records onto the new index, but I want to be able to do this all in one shot, as I do have a painless script to glean off some information that is pertinent to the queries that will search the secondary index. Performance is a concern, as the application will be chaining subsequent queries together using the destination index to query against. Does anyone know how to do accomplish this using Jest?

It appears as if this particular functionality is not yet supported in Jest. The Jest API It has a way to pass in a script (not a query) as a parameter, but I even was having problems with that.
EDIT:
After some hacking with a coworker, we found a way around this...
Step 1) Extend the GenericResultAbstractionAction class with edits to the script:
public class GenericResultReindexActionHack extends GenericResultAbstractAction {
GenericResultReindexActionHack(GenericResultReindexActionHack.Builder builder) {
super(builder);
Map<String, Object> payload = new HashMap<>();
payload.put("source", builder.source);
payload.put("dest", builder.dest);
if (builder.conflicts != null) {
payload.put("conflicts", builder.conflicts);
}
if (builder.size != null) {
payload.put("size", builder.size);
}
if (builder.script != null) {
Script script = (Script) builder.script;
// Note the script parameter needs to be formatted differently to conform to the ES _reindex API:
payload.put("script", new Gson().toJson(ImmutableMap.of("id", script.getIdOrCode(), "params", script.getParams())));
}
this.payload = ImmutableMap.copyOf(payload);
setURI(buildURI());
}
#Override
protected String buildURI() {
return super.buildURI() + "/_reindex";
}
#Override
public String getRestMethodName() {
return "POST";
}
#Override
public String getData(Gson gson) {
if (payload == null) {
return null;
} else if (payload instanceof String) {
return (String) payload;
} else {
// We need to remove the incorrect formatting for the query, dest, and script fields:
// TODO: Need to consider spaces in the JSON
return gson.toJson(payload).replaceAll("\\\\n", "")
.replace("\\", "")
.replace("query\":\"", "query\":")
.replace("\"},\"dest\"", "},\"dest\"")
.replaceAll("\"script\":\"","\"script\":")
.replaceAll("\"}","}")
.replaceAll("},\"script\"","\"},\"script\"");
}
}
public static class Builder extends GenericResultAbstractAction.Builder<GenericResultReindexActionHack , GenericResultReindexActionHack.Builder> {
private Object source;
private Object dest;
private String conflicts;
private Long size;
private Object script;
public Builder(Object source, Object dest) {
this.source = source;
this.dest = dest;
}
public GenericResultReindexActionHack.Builder conflicts(String conflicts) {
this.conflicts = conflicts;
return this;
}
public GenericResultReindexActionHack.Builder size(Long size) {
this.size = size;
return this;
}
public GenericResultReindexActionHack.Builder script(Object script) {
this.script = script;
return this;
}
public GenericResultReindexActionHack.Builder waitForCompletion(boolean waitForCompletion) {
return setParameter("wait_for_completion", waitForCompletion);
}
public GenericResultReindexActionHack.Builder waitForActiveShards(int waitForActiveShards) {
return setParameter("wait_for_active_shards", waitForActiveShards);
}
public GenericResultReindexActionHack.Builder timeout(long timeout) {
return setParameter("timeout", timeout);
}
public GenericResultReindexActionHack.Builder requestsPerSecond(double requestsPerSecond) {
return setParameter("requests_per_second", requestsPerSecond);
}
public GenericResultReindexActionHack build() {
return new GenericResultReindexActionHack(this);
}
}
}
Step 2) Use of this class with a query then requires you to pass in the query as part of the source, then remove the '\n' characters:
ImmutableMap<String, Object> sourceMap = ImmutableMap.of("index", sourceIndex, "query", qb.toString().replaceAll("\\\\n", ""));
ImmutableMap<String, Object> destMap = ImmutableMap.of("index", destIndex);
GenericResultReindexActionHack reindex = new GenericResultReindexActionHack.Builder(sourceMap, destMap)
.waitForCompletion(false)
.conflicts("proceed")
.size(5000L)
.script(reindexScript)
.setParameter("slices", 10)
.build();
JestResult result = handleResult(reindex);
String task = result.getJsonString();
return (task);
Note the reindexScript parameter is of type org.elasticsearch.script.
This is a messy, hack-y way of getting around the limitations of Jest, but it seems to work. I understand that by doing it this way there may be some limitations to what may be acceptable in the input formatting...

Unable to get data from java web service

Someone please help me i keep trying but not able to find out why i am unable to get the results.
I have created this java springboot web service where when I run the java application, a web browser page will open and when I type in the URL e.g localhost:8080/runbatchfileparam/test.bat the program will check if the test.bat file exist first. If it does, the web page will show a JSON result {“Result”: true} and the command in the batch file will be executed. If it does not exist, the web page will show {“Result”: false}.
I want to create an ASP.NET Web Service that will use the function created in the java web service. When I run the ASP.NET Web Application, a web browser page will open. User will type in URL something like this: localhost:12345/api/callbatchfile/test.bat. The java web service should be running and I should get either {“Result”: true} or {“Result”: false} when I run the C# ASP.NET Web Application too.
However I only get an empty {} without anything inside the brackets. Why is that so?
Here are my code in ASP.NET
TestController.cs
private TestClient testClient = new TestClient();
public async Task<IHttpActionResult> GET(string fileName)
{
try
{
var result = await testClient.runbatchfile(fileName);
var resultDTO = JsonConvert.DeserializeObject<TestVariable>(result);
return Json(resultDTO);
}
catch (Exception e)
{
var result = "Server is not running";
return Ok(new { ErrorMessage = result });
}
}
TestVariable.cs
public class TestVariable
{
public static int fileName { get; set; }
}
TestClient.cs
private static HttpClient client;
private static string BASE_URL = "http://localhost:8080/";
static TestClient()
{
client = new HttpClient();
client.BaseAddress = new Uri(BASE_URL);
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
}
public async Task<string> runbatchfile(string fileName)
{
var endpoint = string.Format("runbatchfile/{0}", fileName);
var response = await client.GetAsync(endpoint);
return await response.Content.ReadAsStringAsync();
}
WebApiConfig.cs
config.Routes.MapHttpRoute(
name: "TestBatchClient",
routeTemplate: "api/runbatchfile/{fileName}",
defaults: new { action = "GET", controller = "Test" }
);
Someone please do help me. Thank you so much.
EDIT
Java web service
Application.java
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
BatchFileController.java
private static final String template = "Sum, %s!";
#RequestMapping("/runbatchfile/{param:.+}")
public ResultFormat runbatchFile(#PathVariable("param") String fileName) {
RunBatchFile rbf = new RunBatchFile();
return rbf.runBatch(fileName);
}
ResultFormat
private boolean result;
public ResultFormat(boolean result) {
this.result = result;
}
public boolean getResult() {
return result;
}
RunBatchFile.java
public ResultFormat runBatch(String fileName) {
String var = fileName;
String filePath = ("C:/Users/attsuap1/Desktop/" + var);
try {
Process p = Runtime.getRuntime().exec(filePath);
int exitVal = p.waitFor();
return new ResultFormat(exitVal == 0);
} catch (Exception e) {
e.printStackTrace();
return new ResultFormat(false);
}
}

I am not sure if this helps.. but I suspect that the AsyncTask is not really executing...
var result = await testClient.testCallBatchProject(fileName);
I would try something like below:
await testClient.testCallBatchProject(fileName).Delay(1000);
Can you try and check if the same happens for a synchronous call? .. if it does, we can zero down on the above.

How to do "Object.defineProperty" in pure Rhino object model?

With Rhino 17R4, we can create properties in javascript using Object.defineProperty() method.
public class MyGlobalObject : org.mozilla.javascript.ScriptableObject
{
public static org.mozilla.javascript.Script ___compiledScript = null;
public MyGlobalObject()
{
org.mozilla.javascript.Context con = org.mozilla.javascript.Context.enter();
try
{
con.initStandardObjects(this);
string strScript = "Object.defineProperty(this,\r\n 'onload', \r\n{ set : function(val){this.set_onload(val);},\r\n get : function(){return this.get_onload();}, enumerable: true, configurable: true});";
this.defineFunctionProperties(new string[] { "set_onload", "get_onload" }, typeof(MyGlobalObject), org.mozilla.javascript.ScriptableObject.DONTENUM);
org.mozilla.javascript.Script sc = con.compileString(strScript, "", 1, null);
object result_onload = con.evaluateString(this, "this.onload == undefined;", "", 1, null); // make sure it is not defined.
Console.WriteLine("onload is undefined? : {0}", result_onload);
// Define Properties Now.
sc.exec(con, this);
con.evaluateString(this, "this.onload= function(){var t1 = 1;};", "", 1, null);
object onloadobjectXYZ = con.evaluateString(this, "this.onload;", "", 1, null); // get function now.
Console.WriteLine("Onload object : {0} is found", onloadobjectXYZ);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
org.mozilla.javascript.Context.exit();
}
private object __onloadFunction;
public object get_onload()
{
Console.WriteLine("get_onload() called!");
return this.__onloadFunction;
}
//[org.mozilla.javascript.annotations.JSSetter]
public void set_onload(object _val)
{
Console.WriteLine("set_onload() called!");
this.__onloadFunction = _val;
}
public override string getClassName()
{
return "Global";
}
}
How can I create FunctionObject which is identical to "onloadobjectXYZ" in pure rhino object operation (not by using script like'strScipt')? It seems that it may be able to create FunctionObject for setter and getter, but I could not find a good example. Does anyone know how to define properties?
Thank you in advance!

defineProperty with java Method setter / getter is slightly different from object.defineProprty()
this.defineProperty("onload", null, javaonloadGetMethod, javaonloadSetMethod, ScriptableObject.PERMANENT);
This works for me as a workaround.