Develop Reference

Unable to pass Array from Angular 2 typescript to Spring Java

I am trying to pass a String array from my typescript
tmp : Array<string> = [];
So I have a function which takes in this array as a parameter input
passValues(test : Array<string>) {
........
// some method to call post method from service
}
So in service
public passingOfValues( test : Array<string> ) : Observable<Array<string>> {
let headers = new Headers({ 'Content-Type': 'application/json'} );
let options = new RequestOptions({ headers: headers);
let response = this.http.post(this.basePath + this.modulePath + '/getArrayValue', {'test' : test }, options)
.map(this.extractData)
.catch(this.handleError);
return response;
}
But I am getting errors such as System property [org.owasp.esapi.devteam] is not set
And I read on other posts that I have to stringify the array before passing to backend.
Is there a reason why I need to stringify / also can I just pass the raw array?
EDIT 1 :
including backend controller codes
public ResponseEntity<?> getArrayValues( ArrayList<String> test ) {
logger.debug("### Test if array has a size ###" + test.size());
}
Apparently size already shows 0 from here.
EDIT 2 :
While debugging, i realised that the SQL at the back is receiving
say
HOME CHARACTER(20 OCTETS)
does this make any difference?
Like passing of string into octets or do I have to do some conversion?
Sorry if I have alot of questions am also working hard on debugging and learning more about it!

Most of the developers like JSON data as request and it's good practice in RESTful apis. why?
JSON format is {key1: value1, key2: value 2,....}
You are passing
this.http.post(this.basePath + this.modulePath + '/getArrayValue',{'test' : YOUR_ACTUAL_ARRAY})
form the front-end. The httpClient.post(url,body,options?) has url and body as mandatory. How can you get it in back-end? Since you have body only,
public ResponseEntity<?> getArrayValues(#RequestBody List<String> test) {
// codes
}
Key of passed parameter from front-end test and variable which
listens in back-end should be in same name. Otherwise
#RequestBody("KEY_NAME") List<String> any_variable
As you asked from comment, you may have two key value pairs. Eg : { "test" : value1, "tmp": value2}. Assume value1 and value2 both are String array.
this.http.post(this.basePath + this.modulePath + '/getArrayValue',{'myJson' : YOUR_JSON})
There are lot of way(Eg : Gson,ObjectMapper etc). I use another way.
Create a class called TestTmpConverter
class TestTmpConverter{
List<String> test;
List<String> tmp;
//No-argument constructors & Argument constructors
//Getters
}
In controller
public ResponseEntity<?> getArrayValues(#RequestBody List<TestTmpConverter> myJson ) {
List<TestTmpConverter> test=myJson.getTest();
List<TestTmpConverter> tmp=myJson.getTmp();
// Do your work
}
I only showed one way.There are a lot of way to pass data to back-end like #RequestParam, #PathVariable etc. I feel now you get something how you can pass the data.

For your client put your data directly on POST's body:
public passingOfValues( test : Array<string> ) : Observable<Array<string>> {
let headers = new Headers({ 'Content-Type': 'application/json'} );
let options = new RequestOptions({ headers: headers);
let response = this.http.post(this.basePath + this.modulePath + '/getArrayValue',
test, options)
.map(this.extractData)
.catch(this.handleError);
return response;
}
On your REST service use the #RequestBody annotation:
public ResponseEntity<?> getArrayValues(#RequestBody String[] test ) {
logger.debug("### Test if array has a size ###" + test.size());
}

Spring data aggregation query elasticsearch

I am trying to make the below elasticsearch query to work with spring data. The intent is to return unique results for the field "serviceName". Just like a SELECT DISTINCT serviceName FROM table would do comparing to a SQL database.
{
"aggregations": {
"serviceNames": {
"terms": {
"field": "serviceName"
}
}
},
"size":0
}
I configured the field as a keyword and it made the query work perfectly in the index_name/_search api as per the response snippet below:
"aggregations": {
"serviceNames": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "service1",
"doc_count": 20
},
{
"key": "service2",
"doc_count": 8
},
{
"key": "service3",
"doc_count": 8
}
]
}
}
My problem is the same query doesn't work in Spring data when I try to run with a StringQuery I get the error below. I am guessing it uses a different api to run queries.
Cannot execute jest action , response code : 400 , error : {"root_cause":[{"type":"parsing_exception","reason":"no [query] registered for [aggregations]","line":2,"col":19}],"type":"parsing_exception","reason":"no [query] registered for [aggregations]","line":2,"col":19} , message : null
I have tried using the SearchQuery type to achieve the same results, no duplicates and no object loading, but I had no luck. The below sinnipet shows how I tried doing it.
final TermsAggregationBuilder aggregation = AggregationBuilders
.terms("serviceName")
.field("serviceName")
.size(1);
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withIndices("index_name")
.withQuery(matchAllQuery())
.addAggregation(aggregation)
.withSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.withSourceFilter(new FetchSourceFilter(new String[] {"serviceName"}, new String[] {""}))
.withPageable(PageRequest.of(0, 10000))
.build();
Would someone know how to achieve no object loading and object property distinct aggregation on spring data?
I tried many things without success to print queries on spring data, but I could not, maybe because I am using the com.github.vanroy.springdata.jest.JestElasticsearchTemplate implementation.
I got the query parts with the below:
logger.info("query:" + searchQuery.getQuery());
logger.info("agregations:" + searchQuery.getAggregations());
logger.info("filter:" + searchQuery.getFilter());
logger.info("search type:" + searchQuery.getSearchType());
It prints:
query:{"match_all":{"boost":1.0}}
agregations:[{"serviceName":{"terms":{"field":"serviceName","size":1,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},{"_key":"asc"}]}}}]
filter:null
search type:DFS_QUERY_THEN_FETCH

I figured out, maybe can help someone. The aggregation don't come with the query results, but in a result for it self and is not mapped to any object. The Objects results that comes apparently are samples of the query elasticsearch did to run your aggregation (not sure, maybe).
I ended up by creating a method which can do a simulation of what would be on the SQL SELECT DISTINCT your_column FROM your_table, but I think this will work only on keyword fields, they have a limitation of 256 characters if I am not wrong. I explained some lines in comments.
Thanks #Val since I was only able to figure it out when debugged into Jest code and check the generated request and raw response.
public List<String> getDistinctField(String fieldName) {
List<String> result = new ArrayList<>();
try {
final String distinctAggregationName = "distinct_field"; //name the aggregation
final TermsAggregationBuilder aggregation = AggregationBuilders
.terms(distinctAggregationName)
.field(fieldName)
.size(10000);//limits the number of aggregation list, mine can be huge, adjust yours
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withIndices("your_index")//maybe can be omitted
.addAggregation(aggregation)
.withSourceFilter(new FetchSourceFilter(new String[] { fieldName }, new String[] { "" }))//filter it to retrieve only the field we ar interested, probably we can take this out.
.withPageable(PageRequest.of(0, 1))//can't be zero, and I don't want to load 10 results every time it runs, will always return one object since I found no "size":0 in query builder
.build();
//had to use the JestResultsExtractor because com.github.vanroy.springdata.jest.JestElasticsearchTemplate don't have an implementation for ResultsExtractor, if you use Spring defaults, you can probably use it.
final JestResultsExtractor<SearchResult> extractor = new JestResultsExtractor<SearchResult>() {
#Override
public SearchResult extract(SearchResult searchResult) {
return searchResult;
}
};
final SearchResult searchResult = ((JestElasticsearchTemplate) elasticsearchOperations).query(searchQuery,
extractor);
final MetricAggregation aggregations = searchResult.getAggregations();
final TermsAggregation termsAggregation = aggregations.getTermsAggregation(distinctAggregationName);//this is where your aggregation results are, in "buckets".
result = termsAggregation.getBuckets().parallelStream().map(TermsAggregation.Entry::getKey)
.collect(Collectors.toList());
} catch (Exception e) {
// threat your error here.
e.printStackTrace();
}
return result;
}

How to read JSON String in Spring data and write to a database MySql

I have the JSON file and I want to persist it into Mysql database can anyone give me the steps to follow with the spring-data to do this, my JSON looks like this
i created a modele with 3 class named like this
Portal.java,Spaces.java,Indicators.java
{
"portalName":"office360",
"spaces": [
{
"spaceName": "Modèle d'Espace collaboratif",
"indicators": [
{
"indicatorName": "Created content",
"indicatorCounter": "0"
},
{
"indicatorName": "answers reader",
"indicatorCounter": "0"
}
]
},
{
"spaceName": "Espace de Travail par Defaut",
"indicators": [
{
"indicatorName": "Created content",
"indicatorCounter": "0"
},
{
"indicatorName": "answers reader",
"indicatorCounter": "0"
}
]
}
]}

If you have file wit JSON type data in it and have in Back-End three classes which should be filled with Json Data so I suggest the following:
use Gson library: https://github.com/google/gson
List item
Create DTO classs (consisting all three classes)
With file read get data from file to String: String fileString = new String(Files.readAllBytes(Paths.get("manifest.mf")), StandardCharsets.UTF_8);
Then combine Gson library, DTO class and this line: Data data = new Gson().fromJson(fileString, Data.class);
Now you will have DTO object filled with data

You need to use the RestTemplate in Spring.
The steps are
If you are too lazy to create the classes that will be mapped to your json object, then you can use this website http://pojo.sodhanalibrary.com/ it will create it for you. Lets assume that the class created is Page.class
Create a Repository class like
import org.springframework.web.client.RestTemplate;
#Repository
public class OfficeRepository{
public Page getSomething() {
RestTemplate restTemplate = new RestTemplate();
Page page = restTemplate.getForObject("http://your-url-goes-here", Page.class);
return page;
}
}
This will return your JSON contents into the Page object created.
This was the hard part. The easy part is to save it to the DB. Please use this guide by spring https://spring.io/blog/2011/02/10/getting-started-with-spring-data-jpa/ in order to save it and if you have any questions feel free to ask.

Iterate a JSON Object of Objects Firebase

I'm using firebase4j (a firebase library for Java, I know it is much better to go with node, I just wanted to try to do it with Java). In my database I need to persist the url of images with a bunch of the picture's information. The thing is that the picture url itself is very deep into the JSON
"users" : {
"aCategory" : {
"aUser" : {
"photos" : {
"photoUid1" : [ {
"value1" : false,
"value2" : "qwerty",
"score" : 40,
"url" : "http://someurl.com"
}
That is why I am trying to create an index for the pictures ordered by score, containing the url pointing to the location of the photo object in the firebase database. Here is where the issue begins. Firebase4j does not let you push, to a list for example, so the index ends up with this format:
{
"-UID1": {
"firebaseImgUrl": "users/aCategory/aUser/photos/photoUid1",
"score": 31
},
"-UID2": {
"firebaseImgUrl": "users/aCategory/aUser/photos/photoUid2",
"score": 30
}
}
I already added the rule ".indexOn" in order for firebase to answer with the right photos when asked for http://firebaseurl.com/users/...?orderBy="score"&limitToFirst=10, which is what I'm doing. I would like to know how should I iterate a JSON object of object as shown in the example above. I'm receiving the data from an Angular 4 client. I've tried a number of methods which haven't worked for me:
result: Photo[] = [];
for(let key in json){
console.log(key); //prints the UIDs
console.log(key.url); //url is not a property of string
//thus
result.push(new Photo(key.url, key.score)); //not working
}

The key is only a string, indicating the keys in your json. You should use it to access your object, like this:
result: Photo[] = [];
for(let key in json){
result.push(new Photo(json[key].firebaseImgUrl, json[key].score));
}

jackson unmarshalling problems

I am trying to deserialize a JSON String using Jackson 2 with RestAssured (java tool for IT tests).
I have a problem. The String I am trying to deserialize is :
{"Medium":{"uuid":"2","estimatedWaitTime":0,"status":"OPEN_AVAILABLE","name":"Chat","type":"CHAT"}}
There is the object type "Medium" at the begining of the String. This cause Jackson failing during deserialization:
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "Medium"
I've set the "IGNORE_ON_UNKNOWN_PROPERTIES" to false and then I got no exception during deserialisation. However, all of my properties are 'null' in java.
Response getAvailability -> {"Medium":{"uuid":"2","estimatedWaitTime":0,"status":"OPEN_AVAILABLE","name":"Chat","type":"CHAT"}}
### MEDIUM name -> null
### MEDIUM uuid -> null
### MEDIUM wait time -> null
### MEDIUM wait time -> null
### MEDIUM status -> null
Does anyone can help me ? (note: I can't change my input JSON string).

{
"Medium": {
"uuid": "2",
"estimatedWaitTime": 0,
"status": "OPEN_AVAILABLE",
"name": "Chat",
"type": "CHAT"
}
}
as you can see uuid and other params are part of medium object , so class in which it can be deserialized is.
class Medium
{
string name;
// specify other params also.
}
class BaseObject
{
Medium Medium;
}
and then use jackson.deserialize('json', BaseObject.class)
above i had given pseudo code

You need to put annotation
#JsonRootName("Medium")
on your bean class and configure object mapper to
mapper.enable(DeserializationFeature.UNWRAP_ROOT_VALUE).

You need a way to remove the Object name that is the part of the input JSON. Since you cannot change the input string, Use this code to change this input string to a tree and get the value of "Medium" node.
ObjectMapper m = new ObjectMapper();
JsonNode root = m.readTree("{\"Medium\":{\"uuid\":\"2\",\"estimatedWaitTime\":0,\"status\":\"OPEN_AVAILABLE\",\"name\":\"Chat\",\"type\":\"CHAT\"}}");
JsonNode obj = root.get("Medium");
Medium medium = m.readValue(obj.asText, Medium.class);