Develop Reference

JsonPath not working properly in Spring Cloud Data Flow filter component

I'm trying to write a simple SCDF flow that reads from Kafka, filters the messages by presence of specific value and pushes data into Mongo. As part of this i had to wrote following #jsonPath
#jsonPath(payload,'$[?(#.metadata!=null)].metadata[?(#.trigger-routing!=
null)].trigger-routing') == {'1'}
And i wrote an example test that will run the SPeL and verify what it returns(Note: I'm intetionally using #EnableIntegration to wire the same profile of SPeL functionality as SCDF, at least that's my theory)
#SpringBootTest(classes = SpelTst.TestConfiguration.class)
public class SpelTst {
#Configuration
#EnableIntegration
public static class TestConfiguration {
}
#Autowired
IntegrationEvaluationContextFactoryBean factory;
#Test
public void test() throws JsonProcessingException {
final StandardEvaluationContext context = factory.getObject();
ExpressionParser parser = new SpelExpressionParser();
Expression exp = parser.parseExpression("#jsonPath(payload,'$[?(#.metadata!=null)].metadata[?(#.trigger-routing!= null)].trigger-routing') == {'1'}");
final PixelHitMessage sampleOne = new PixelHitMessage()
.setMetadata(ImmutableMap.of("trigger-routing", "1"))
.toChildType();
final PixelHitMessage sampleTwo = new PixelHitMessage()
.setMetadata(ImmutableMap.of("trigger-routing", ""))
.toChildType();
final PixelHitMessage sampleThree = new PixelHitMessage()
.setMetadata(Collections.emptyMap())
.toChildType();
final PixelHitMessage sampleFour = new PixelHitMessage()
.toChildType();
System.out.println(resolve(context, exp, sampleOne));
System.out.println(resolve(context, exp, sampleTwo));
System.out.println(resolve(context, exp, sampleThree));
System.out.println(resolve(context, exp, sampleFour));
}
private static Object resolve(StandardEvaluationContext context, Expression exp, PixelHitMessage sampleOne) throws JsonProcessingException {
final ObjectMapper mapper = new ObjectMapper();
final String payload = mapper.writerFor(PixelHitMessage.class).writeValueAsString(sampleOne);
System.out.println(payload);
final Message<String> kafkaMessage = MessageBuilder.withPayload(payload).build();
context.setRootObject(kafkaMessage);
return exp.getValue(context, Object.class);
}
}
When i run this i get a a following output
{"timestamp":"2020-06-26T19:31:38.013Z","level":"INFO","thread":"main","logger":"SpelTst","message":"Started SpelTst in 1.706 seconds (JVM running for 4.352)","context":"default"}
{"eventId":null,"postTime":null,"headers":null,"metadata":{"trigger-routing":"1"}}
true
{"eventId":null,"postTime":null,"headers":null,"metadata":{"trigger-routing":""}}
false
{"eventId":null,"postTime":null,"headers":null,"metadata":{}}
false
{"eventId":null,"postTime":null,"headers":null,"metadata":null}
false
The above is the exact behaviour that i'm seeking to achieve.
But when i use the same SPeL in a filter component in SCDF, i'm getting following exception
Caused by: com.jayway.jsonpath.PathNotFoundException: No results for path: $['metadata']['trigger-routing']
Example of a message that should return false
{"eventId":"acb0afce-7782-4dc6-af09-4d6878fa8fd3","postTime":1593201189799,"headers":{"accept":"*/*","host":"localhost:7071","user-agent":"insomnia/2020.2.2"},"metadata":{}}
Example of a message that should return true
{"eventId":"045698d4-d4dc-41b0-8bab-7c07ad58970a","postTime":1593201492866,"headers":{"accept":"*/*","host":"localhost:7071","user-agent":"insomnia/2020.2.2"},"metadata":{"trigger-routing":"1"}}
In SCDF, the SPeL is working only for the positive scenario, the absence of any of the data on the path causes the above exception. I was thinking about using the Option.DEFAULT_PATH_LEAF_TO_NULL for the JsonPath, but as far as i know there is no way to specify it trough Spring properties(i checked the code for the JsonPathUtils, and they are calling the version of JsonPath logic that uses default context without with default(empty) config.
I also verified whether the filter expression is correctly deployed(screen of the K8 configuration for the pod that is running the filter application), and it appears to be correct.

This kind of solution can be used.
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>2.7.0</version>
</dependency>
Then the validation could be
DocumentContext documentContext = JsonPath.parse(messagePayload);
Map<String, Object> metadata = documentContext.read("$.metadata");
Imports are, for DocumentContext and JsonPath
import com.jayway.jsonpath.DocumentContext;
import com.jayway.jsonpath.JsonPath;
Now for the true scenario
System.out.println(metadata);
Output will be: {trigger-routing=1}
And for the false scenario
System.out.println(metadata);
Output will be: {}
Other JSON Path syntaxes are -
https://support.smartbear.com/alertsite/docs/monitors/api/endpoint/jsonpath.html

Spring Boot R2DBC with MySQL - Exception: Table not found

I'm extremely new to String Boot and backend development (maybe three days or less) and I have the desire to build REST API to consume from different clients.
So I started by a simple demo app that has an endpoint called /register. We post a JSON string with username and password to create a new user if not exist.
I was using JPA with HSQLDB and it worked fine persisting on memory. But recently I wanted to use RxJava since I'm familiar with on Android, so I switched to R2DBC with MySQL.
MySQL server is running fine on port 3306 and the app was tested using PostMan on localhost:8080
The problem occurs when I try to query users table or insert entities and it looks like this:
{
"timestamp": "2020-03-22T11:54:43.466+0000",
"status": 500,
"error": "Internal Server Error",
"message": "execute; bad SQL grammar [UPDATE user_entity SET username = $1, password = $2 WHERE user_entity.id = $3]; nested exception is io.r2dbc.spi.R2dbcBadGrammarException: [42102] [42S02] Table \"USER_ENTITY\" not found; SQL statement:\nUPDATE user_entity SET username = $1, password = $2 WHERE user_entity.id = $3 [42102-200]",
"path": "/register"
}
Here's the full logfile for the exception.
I Have been looking for a solution for hours and I seem like not finding it anywhere, so I hope that I will find it here.
Let's break down the project so it's easier to find the solution:
1. database:
2. application.properties:
logging.level.org.springframework.data.r2dbc=DEBUG
spring.datasource.url=jdbc:mysql://localhost:3306/demodb
spring.datasource.username=root
spring.datasource.password=root
3. DatabaseConfiguration:
#Configuration
#EnableR2dbcRepositories
class DatabaseConfiguration : AbstractR2dbcConfiguration() {
override fun connectionFactory(): ConnectionFactory
= ConnectionFactories.get(
builder().option(DRIVER, "mysql")
.option(HOST, "localhost")
.option(USER, "root")
.option(PASSWORD, "root")
.option(DATABASE, "demodb")
.build()
)
}
4. RegistrationController:
#RequestMapping("/register")
#RestController
class RegistrationController #Autowired constructor(private val userService: UserService) {
#PostMapping
fun login(#RequestBody registrationRequest: RegistrationRequest): Single<ResponseEntity<String>>
= userService.userExists(registrationRequest.username)
.flatMap { exists -> handleUserExistance(exists, registrationRequest) }
private fun handleUserExistance(exists: Boolean, registrationRequest: RegistrationRequest): Single<ResponseEntity<String>>
= if (exists) Single.just(ResponseEntity("Username already exists. Please try an other one", HttpStatus.CONFLICT))
else userService.insert(User(registrationRequest.username, registrationRequest.password)).map { user ->
ResponseEntity("User was successfully created with the id: ${user.id}", HttpStatus.CREATED)
}
}
5. UserService:
#Service
class UserService #Autowired constructor(override val repository: IRxUserRepository) : RxSimpleService<User, UserEntity>(repository) {
override val converter: EntityConverter<User, UserEntity> = UserEntity.Converter
fun userExists(username: String): Single<Boolean>
= repository.existsByUsername(username)
}
6. RxSimpleService:
abstract class RxSimpleService<T, E>(protected open val repository: RxJava2CrudRepository<E, Long>) {
protected abstract val converter: EntityConverter<T, E>
open fun insert(model: T): Single<T>
= repository.save(converter.fromModel(model))
.map(converter::toModel)
open fun get(id: Long): Maybe<T>
= repository.findById(id)
.map(converter::toModel)
open fun getAll(): Single<ArrayList<T>>
= repository.findAll()
.toList()
.map(converter::toModels)
open fun delete(model: T): Completable
= repository.delete(converter.fromModel(model))
}
7. RxUserRepository:
#Repository
interface IRxUserRepository : RxJava2CrudRepository<UserEntity, Long> {
#Query("SELECT CASE WHEN EXISTS ( SELECT * FROM ${UserEntity.TABLE_NAME} WHERE username = :username) THEN CAST(1 AS BIT) ELSE CAST(0 AS BIT) END")
fun existsByUsername(username: String): Single<Boolean>
}
8. And finally, here's my UserEntity
#Table(TABLE_NAME)
data class UserEntity(
#Id
val id: Long,
val username: String,
val password: String
) {
companion object {
const val TABLE_NAME = "user_entity"
}
object Converter : EntityConverter<User, UserEntity> {
override fun fromModel(model: User): UserEntity
= with(model) { UserEntity(id, username, password) }
override fun toModel(entity: UserEntity): User
= with(entity) { User(id, username, password) }
}
}
User and RegistrationRequest are just simple objects with username and password.
What I have missed?
Please leave a comment if you need more code.

I finally managed to solve this mistake!
The problems were so simple yet so sneaky for a beginner:
I was using JDBC in my URL instead of R2DBC
I was using the H2 runtime implementation so it was
expecting an H2 in-memory database
My ConnectionFactory was not very correct
So what I did was the following:
Updated my build.gradle:
Added:
implementation("io.r2dbc:r2dbc-pool") , implementation("dev.miku:r2dbc-mysql:0.8.1.RELEASE") and
runtimeOnly("mysql:mysql-connector-java")
Removed: runtimeOnly("io.r2dbc:r2dbc-h2")
It now looks like this:
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
plugins {
id("org.springframework.boot") version "2.2.5.RELEASE"
id("io.spring.dependency-management") version "1.0.9.RELEASE"
kotlin("jvm") version "1.3.61"
kotlin("plugin.spring") version "1.3.61"
}
group = "com.tamimattafi.backend"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_11
repositories {
mavenCentral()
maven(url = "https://repo.spring.io/milestone")
}
dependencies {
//SPRING BOOT
implementation("org.springframework.boot:spring-boot-starter-security")
implementation("org.springframework.boot:spring-boot-starter-web")
implementation("org.springframework.boot.experimental:spring-boot-starter-data-r2dbc")
//KOTLIN
implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
implementation("org.jetbrains.kotlin:kotlin-reflect")
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
//RX JAVA
implementation("io.reactivex.rxjava2:rxjava:2.2.0")
implementation("io.reactivex:rxjava-reactive-streams:1.2.1")
//MYSQL
implementation("dev.miku:r2dbc-mysql:0.8.1.RELEASE")
implementation("io.r2dbc:r2dbc-pool")
runtimeOnly("mysql:mysql-connector-java")
//TEST
testImplementation("org.springframework.boot:spring-boot-starter-test") {
exclude(group = "org.junit.vintage", module = "junit-vintage-engine")
}
testImplementation("org.springframework.security:spring-security-test")
testImplementation("io.projectreactor:reactor-test")
testImplementation("org.springframework.boot.experimental:spring-boot-test-autoconfigure-r2dbc")
}
dependencyManagement {
imports {
mavenBom("org.springframework.boot.experimental:spring-boot-bom-r2dbc:0.1.0.M3")
}
}
tasks.withType<Test> {
useJUnitPlatform()
}
tasks.withType<KotlinCompile> {
kotlinOptions {
freeCompilerArgs = listOf("-Xjsr305=strict")
jvmTarget = "1.8"
}
}
Updated my application.properties to this:
spring.r2dbc.url=r2dbc:pool:mysql://127.0.0.1:3306/demodb
spring.r2dbc.username=root
spring.r2dbc.password=root
Updated my DatabaseConfiguration to this (Note that I removed the #EnableR2dbcRepositories because it should be elsewhere) :
#Configuration
class DatabaseConfiguration : AbstractR2dbcConfiguration() {
override fun connectionFactory(): ConnectionFactory
= MySqlConnectionFactory.from(
MySqlConnectionConfiguration.builder()
.host("127.0.0.1")
.username("root")
.port(3306)
.password("root")
.database("demodb")
.connectTimeout(Duration.ofSeconds(3))
.useServerPrepareStatement()
.build()
)
}
Updated my Application class (I brought the annotation here):
#SpringBootApplication
#EnableR2dbcRepositories
class DemoApplication
fun main(args: Array<String>) {
runApplication<DemoApplication>(*args)
}
IT WORKS NOW! I hope someone will find this helpful, Happy Coding!

In application.properties you need to set the spring.jpa.hibernate.ddl-auto property.
The options are:
validate: validate the schema, makes no changes to the database.
update: update the schema.
create: creates the schema, destroying previous data.
create-drop: drop the schema when the SessionFactory is closed explicitly, typically when the application is stopped.
none: does nothing with the schema, makes no changes to the database

Initializing Apache Beam Test Pipeline in Scala fails

When I try to run a test pipeline it raise an error
here is the source code to create the test pipeline:
val p: TestPipeline = TestPipeline.create()
and here is the error :
java.lang.IllegalStateException: Is your TestPipeline declaration missing a #Rule annotation? Usage: #Rule public final transient TestPipeline pipeline = TestPipeline.create();

I have solve the problem by following code :
val p: TestPipeline = TestPipeline.create().enableAbandonedNodeEnforcement(false)

Grails: how to call a service and the service name from a string

I want to call services dynamically so the service name will get as a string value, we can list all the services names in the grails project by using the code below.
import org.codehaus.groovy.grails.plugins.metadata.GrailsPlugin
for (type in ['service']) {
for (artifactClass in ctx.grailsApplication."${type}Classes") {
def clazz = artifactClass.clazz
def annotation = clazz.getAnnotation(GrailsPlugin)
if (annotation) {
println "$type $clazz.name from plugin '${annotation.name()}'"
}
else {
println "$type $clazz.name from application"
}
}
}
Here we will get artifactClass of the service.Is there any option to call the service by using this idea.Please help me.

You can get the bean for the service from the applicationContext
//inject application context bean
def applicationContext
//to use
applicationContext."${yourServiceName}".serviceMethod()

You can get the bean of your service this way:
import grails.util.Holders
...
YourService yourService =
(YourService)Holders.grailsApplication.mainContext["yourService"]

Spring Boot can't run single test in IntelliJ

This started happening recently, but I'm not sure what changed to cause it.
When I run all tests from IntelliJ, all is well. Also the gradle build is fine.
When I run a single unit test, all is well.
When I run a single web integration test, it fails because a config class has all null properties.
The config class looks like (Kotlin):
#Component
#ConfigurationProperties(prefix = "api")
public open class ApiConfigImpl : ApiConfig
{
A test looks like:
#RunWith(SpringJUnit4ClassRunner::class)
#ContextConfiguration(classes = arrayOf(ApplicationAssembly::class), loader = SpringApplicationContextLoader::class)
#WebIntegrationTest
open class CandidateProfileControllerTest
{
#Inject lateinit var profileRepo: CandidateProfileRepository
//etc a few more deps used to setup test data
#Test
open fun getById()
{
val greg = CandidateProfile("123", "12312", "Greg", "Jones", dateOfBirth = Date(), gender = Gender.MALE,
biography = "ABC", maxMatchableAge = null, maxMatchableDistance = null)
profileRepo.save(greg)
val auth = given().header("content-type", "application/json")
.body(testCredentials)
.post("/authorization/social").peek().asString()
val accessToken: String = from(auth).get("accessToken")
given().header("Access-Token", accessToken).
header("API-Key", testAPIKey()).
get("/profile/${greg.id}").
peek().then().
body("stageName", notNullValue())
}
I'm not sure what information I can add. Based on the limited information provided:
Is this a known problem with a known solution?

This is a bug, logged in the IntelliJ/Kotlin tracker, with a pending fix.