Develop Reference

Spring Data MongoDB - how to update using #Update without setting most fields to null?

After reading Spring's docs at https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongodb.repositories.queries.update ,
I wrote this Repository method:
#Repository
interface TokenRepo: MongoRepository<TokenModel, String> {
#Query("{ authorizationState: ?0 }")
#Update("{ authorizationState: ?0, authorizationCode: ?1 }")
fun updateCode(state: String, code: String): Int
}
Then I use it like this:
#Test fun testUpdate() {
val token = TestTokenModels().makeToken()
val tokenSaved = tokenRepo.save(token)
assertThat(tokenSaved).isNotNull.isEqualTo(token)
assertThat(tokenSaved.requestTimestampMs).isNotNull()
assertThat(tokenRepo.findByState(token.authorizationState)).isNotNull.isEqualTo(token)
tokenRepo.updateCode(token.authorizationState, "someCode")
val tokenUpdated = tokenRepo.findByState(token.authorizationState) // FAILS!
assertThat(tokenUpdated).isNotNull
assertThat(tokenUpdated!!.authorizationCode).isNotNull.isEqualTo("someCode")
}
But this fails when reading back from the database, because almost all fields were set to null:
org.springframework.data.mapping.model.MappingInstantiationException:
Failed to instantiate com.tracker.bl.token.TokenModel using constructor
fun `<init>`(kotlin.String, com.tracker.bl.token.TokenModel.Status, kotlin.String, kotlin.String, kotlin.String, kotlin.Long, kotlin.String, kotlin.String, kotlin.String?, kotlin.String?, com.tracker.rest.Oauth2TokenType, kotlin.String?, kotlin.String?, kotlin.Long?, java.time.ZonedDateTime, kotlin.String?): com.tracker.bl.token.TokenModel with arguments 637e4686ae781b603ac77c12,null,null,null,null,null,null,tokenFlowStateVC8g80BT,null,null,null,null,null,null,null,null,65026,null
at org.springframework.data.mapping.model.
KotlinClassGeneratingEntityInstantiator$DefaultingKotlinClassInstantiatorAdapter
.createInstance(KotlinClassGeneratingEntityInstantiator.java:215)
How am I supposed to use #Update? Or is it only intended for things like $inc and $push? The docs is actually quite brief on this topic. I'm relatively new to MongoDB.

All right that was quick. It was me being new into MongoDB.
Spring Data MongoDB is really just a thin layer, so one needs to follow the MongoDB query language to the extent where update happens through $set { ... }.
So the method is supposed to be like this:
#Query("{ authorizationState: ?0 }")
#Update("{ '\$set': { authorizationCode: ?1 } }")
fun updateCode(state: String, code: String): Int

how to parse postman-collection?

Goal:
I am trying to parse the postman_echo collection json and persist the result into a new json copy on disk, resulting the same file as the original one.
I prefer built-in data structure from the language, but using json library should be good too. Not sure Antlr4 is a better way.
follow-up question
is it possible to allow any valid nested json in the body of post request?
update:
https://github.com/chakpongchung/postman-parser
In the end we come up with this satisfactory solution.

An alternative to what zoran mentioned is to create a case class if the structure is not too dynamic (with Play JSON). This would make it easier to compare the result.
case class MyObject(
queryString: List[KeyValue],
method: String,
url: String,
httpVersion: String
) ... and so on
object MyObject {
implicit val format: Format[MyObject] = Json.format
}
case class KeyValue(name: String, value: String)
object KeyValue {
implicit val format: Format[KeyValue] = Json.format
}
Then, you just need to do:
object JsonParser extends App {
val postman_collections = "./scala_input.json"
val jsonifiedString = scala.io.Source.fromFile(postman_collections).mkString
val myJsonData = Try(Json.parse(jsonifiedString)).map(_.as[MyObject])
myJsonData match {
case Success(myValue) => // compare your case class here
case Failure(err) => println("none")
}
}

I'm not sure if I understand your question well, but if you are trying to iterate over json string, you might try something like this:
import play.api.libs.json.{JsObject, JsValue, Json}
import scala.util.{Failure, Success, Try}
object JsonParser extends App {
val postman_coolections = "./resources/scala_input.json"
val jsonifiedString = scala.io.Source.fromFile(postman_coolections).mkString
val json: JsValue = Try(Json.parse(jsonifiedString)) match {
case Success(js) => js
case Failure(ex) => throw new Exception("Couldn't parse json", ex)
}
json.asInstanceOf[JsObject].fields.foreach{
case (key: String, value: JsValue)=>
println(s"Key:$key value:${value.toString}")
writeFile(s"$key.json", Json.prettyPrint(value))
}
//writing the whole postman input as a single file
writeFile("postmanInputFormatted.json", Json.prettyPrint(json))
writeFile("postmanInput.json", Json.stringify(json))
// To access individual property one option is to use this approach
val lookedValue = json \ "postData" \ "params" \ 1 \ "hello" \ "test"
lookedValue match {
case JsDefined(value) => println(s"Test value is $value")
case JsUndefined() => println("Didn't find test value")
}
// or
val lookedValueAlt = (json \ "postData" \ "params" \ 1 \ "hello" \ "test").getOrElse(throw SomeException)

There are multiple problems in your parser and most of them are that you are trying to use default parser to handle Json object as string. For example, in Request you are handling header as Seq[String] when it's actually Seq of (key, value) pairs. For this particular case, you should change it to something like this:
case class Request(
method: String,
header: Seq[HeaderItem], // header: []
url: Option[Url] = None,
description: String = ""
)
object Request {
implicit val format: Format[Request] = Json.using[Json.WithDefaultValues].format
case class HeaderItem(key: String, value: String)
object HeaderItem {
implicit val format: Format[HeaderItem] = Json.format
}
You can convert header to Seq[String] if you want, but you will have to write custom Read for that.
In the above case you also have cases when description is missing, so you want to handle that with default values.
You have such problems to handle in few other places as well, e.g. "response".
Another problem that I've noticed is a way how you handle "type" property from Json string. Type is reserved keyword, and you can handle it by wrapping it in ``, e.g.
case class Script(
`type`: String,
exec: Seq[String]
)

A satisfactory solution is posted in the github link above in the question.

Problems Sending FormUrlEncoded using Retrofit

I have this request and I need to send it by FormUrlEncoded using Retrofit
{
"clnt_id": "OQW",
"clnt_res": "AA!##$T",
"type": "SCDS",
"len": "ASD"
}
I used this code:
#FormUrlEncoded
#POST("Endpoint")
#Headers("Accept: Application/JSON")
fun connect(
#Field("clnt_id") clnt_id: String,
#Field(value = "clnt_res", encoded = false) clnt_res: String,
#Field("type") type: String,
#Field("len") len: String
): Observable<Token>
First, thing is that the request is not sent as JSON
Second, the value of "clnt_res", encoded by retrofit

You have 2 options to send json request from android using Retrofit.
Create Pojo Model of json request and pass it by setting values of it.
Create HashMap and pass it to request.
Here is solution using 2nd Method:
Create hashmap and put key(parameters) and value:
Map<String,String> requestMap = new HashMap<>();
requestMap.put("clnt_id","your_value");
requestMap.put("clnt_res","your_value");
requestMap.put("type","your_value");
requestMap.put("len","your_value");
Then pass it to your retrofit request using FieldMap:
#FormUrlEncoded
#POST("Endpoint")
#Headers("Accept: Application/JSON")
fun connect(
#FieldMap requestMap:Map<String,String>
): Observable<Token>

I finally get the answer and it was a problem with symbol '$' in 'clnt_res' value "AA!##$T", The problem is in kotlin to escape a special char you need to do this "\$", what I made the IDE didn't tell me that it is wrong is this "$/".

Unable to pass Array from Angular 2 typescript to Spring Java

I am trying to pass a String array from my typescript
tmp : Array<string> = [];
So I have a function which takes in this array as a parameter input
passValues(test : Array<string>) {
........
// some method to call post method from service
}
So in service
public passingOfValues( test : Array<string> ) : Observable<Array<string>> {
let headers = new Headers({ 'Content-Type': 'application/json'} );
let options = new RequestOptions({ headers: headers);
let response = this.http.post(this.basePath + this.modulePath + '/getArrayValue', {'test' : test }, options)
.map(this.extractData)
.catch(this.handleError);
return response;
}
But I am getting errors such as System property [org.owasp.esapi.devteam] is not set
And I read on other posts that I have to stringify the array before passing to backend.
Is there a reason why I need to stringify / also can I just pass the raw array?
EDIT 1 :
including backend controller codes
public ResponseEntity<?> getArrayValues( ArrayList<String> test ) {
logger.debug("### Test if array has a size ###" + test.size());
}
Apparently size already shows 0 from here.
EDIT 2 :
While debugging, i realised that the SQL at the back is receiving
say
HOME CHARACTER(20 OCTETS)
does this make any difference?
Like passing of string into octets or do I have to do some conversion?
Sorry if I have alot of questions am also working hard on debugging and learning more about it!

Most of the developers like JSON data as request and it's good practice in RESTful apis. why?
JSON format is {key1: value1, key2: value 2,....}
You are passing
this.http.post(this.basePath + this.modulePath + '/getArrayValue',{'test' : YOUR_ACTUAL_ARRAY})
form the front-end. The httpClient.post(url,body,options?) has url and body as mandatory. How can you get it in back-end? Since you have body only,
public ResponseEntity<?> getArrayValues(#RequestBody List<String> test) {
// codes
}
Key of passed parameter from front-end test and variable which
listens in back-end should be in same name. Otherwise
#RequestBody("KEY_NAME") List<String> any_variable
As you asked from comment, you may have two key value pairs. Eg : { "test" : value1, "tmp": value2}. Assume value1 and value2 both are String array.
this.http.post(this.basePath + this.modulePath + '/getArrayValue',{'myJson' : YOUR_JSON})
There are lot of way(Eg : Gson,ObjectMapper etc). I use another way.
Create a class called TestTmpConverter
class TestTmpConverter{
List<String> test;
List<String> tmp;
//No-argument constructors & Argument constructors
//Getters
}
In controller
public ResponseEntity<?> getArrayValues(#RequestBody List<TestTmpConverter> myJson ) {
List<TestTmpConverter> test=myJson.getTest();
List<TestTmpConverter> tmp=myJson.getTmp();
// Do your work
}
I only showed one way.There are a lot of way to pass data to back-end like #RequestParam, #PathVariable etc. I feel now you get something how you can pass the data.

For your client put your data directly on POST's body:
public passingOfValues( test : Array<string> ) : Observable<Array<string>> {
let headers = new Headers({ 'Content-Type': 'application/json'} );
let options = new RequestOptions({ headers: headers);
let response = this.http.post(this.basePath + this.modulePath + '/getArrayValue',
test, options)
.map(this.extractData)
.catch(this.handleError);
return response;
}
On your REST service use the #RequestBody annotation:
public ResponseEntity<?> getArrayValues(#RequestBody String[] test ) {
logger.debug("### Test if array has a size ###" + test.size());
}

Retrofit path encode except character?

I use retrofit and my interface below
#GET("{link}")
fun search(#Path(value = "link", encoded = true) link: String?): Call<Any>
Do I need to use encoded for all link except character '?'.
Example:
Link -> /api/search?&query=تست
Encoded link by retrofit -> api/search%3F&query=%D8%AA%D8%B3%D8%AA
I need this link-> api/search?&query=%D8%AA%D8%B3%D8%AA
I need don't convert character '?' to %3F.
do is anyway?

#Url should be used for this instead of #Path
#GET
Call<ResponseClass> list(#Url String url);
It works fine with full URL and a path that is used then with base URL.
Retrofit retrofit = Retrofit.Builder()
.baseUrl("https://website.com/");
.build();
MyService service = retrofit.create(MyService.class);
service.exampleCall("https://another-website.com/example");
service.anotherExampleCall("/only/path/part");

Don't use Path ,you can use #Query in your method,it won't convert ? to %3F .
You can change your method to
#GET("api/serach")
fun search(#Query("query") value: String?): Call<Any>
For example,your value is "aolphn",this code will access
http(s)://xxx/api/search?query=aolphn
? will appended automatically.

I find a solution, must use Interceptor for the request.
class RemoveCharacterInterceptor : Interceptor {
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
val path = request.url().toString()
val string = path.replace("%3F", "?") // replace
val newRequest = request.newBuilder()
.url(string)
.build()
return chain.proceed(newRequest)
}
}
and add to httpClient
val httpClient = OkHttpClient.Builder()
httpClient.addInterceptor(RemoveCharacterInterceptor())