Develop Reference

Unable to validate query parameters against predefined yaml file using json schema validator

I need to validate query parameter's schema against pre defined yaml file schema, so I using the json schema validator. How ever validation is getting failed.
I am following the below steps:
Populate parameter and corresponding schema.
final List<Parameter> parameters = openAPI.getPaths().get(requestPath).getGet().getParameters()
.stream().filter(parameter -> Objects.nonNull(parameter.getIn()) && parameter.getIn().equalsIgnoreCase("query"))
.collect(Collectors.toList());
final Map<Parameter, JsonNode> parameterAndSchema = parameters.stream().collect(Collectors.toMap(Function.identity(), parameter -> {
JsonNode parameterSchema;
try {
final Schema schema = parameter.getSchema();
parameterSchema = mapper.readTree(objectWriter.writeValueAsString(schema));
return parameterSchema;
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
}));
create queryParameterSchema to validate query parameter against its corresponding schema prepared at step number 1:
Hard coded query parameters for testing
final Map<String,String> queryParameterMap = Map.of("test-parameter", "testValue1");
JsonNode queryParameterSchema = new ObjectMapper()
.readTree(queryParameterMap,JsonNode.class)
Convert step 1 schema (prepared from yaml) into JsonSchema as below:
JsonSchemaFactory schemaFactory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V7);
SchemaValidatorsConfig config = new SchemaValidatorsConfig();
config.setTypeLoose(true);
config.setFailFast(false);
JsonSchema jsonSchema = schemaFactory.getSchema(schema, config);
processingReport = jsonSchema.validate(queryParameterSchema , queryParameterSchema , at);
Sample yaml file:
paths:
/test-instances:
get:
tags:
- Instances (Store)
summary: Test summary
operationId: SearchInstances
parameters:
- name: test-parameter
in: query
description: Names of the services offered
required: false
style: form
explode: false
schema:
type: string
However when I am trying to validate queryParameterSchema against this JsonSchema, TypeValidator is invoked and always returning false as my queryParameterSchema populated at step2 always coming as object node with schema type as OBJECT and validator schema type is coming as String (because its defined as String in yaml),
I think I may have to create queryParameterSchema st step 2 differently but not sure how

Xtext custom warning suppress

I have made my DSL and set some info, warning, and error messages.
For example:
warning('''some message''', obj, package.Literals.LITERAL)
Then I exported project(generator made class Main.xtend) into a some_name.jar
Main.xtend contains:
// Validate the resource
val issues = validator.validate(resource, CheckMode.ALL,CancelIndicator.NullImpl)
if (!issues.empty) {
issues.forEach[System.err.println(it)]
return
}
// Configure and start the generator
fileAccess.outputPath = 'src-gen/'
val context = new GeneratorContext => [cancelIndicator = CancelIndicator.NullImpl]
generator.generate(resource, fileAccess, context)
System.out.println('Code generation finished.')
Now when I run
> *java -jar some_name.jar my_file*
The generator doesn't generate code because these info/warning messages.
How can I suppress specific info/warning/error messages?

Apache Calcite - registering UDFs for use in RelBuilder

The OOTB sample udf junits (UdfTest.java) make use of a dummy jdbc schema and do not show the usage of RelBuilder api.
I am working on registering a simple UDF that returns the length of the input string. I have created the SqlFunction and registered the same in the SqlStdOperatorTable -
SqlFunction length = new SqlFunction("STRLEN",
SqlKind.OTHER_FUNCTION,
ReturnTypes.INTEGER,
null,
OperandTypes.STRING,
SqlFunctionCategory.USER_DEFINED_FUNCTION);
SqlStdOperatorTable sqlStdOperatorTable = SqlStdOperatorTable.instance();
sqlStdOperatorTable.register(length);
And used this for creating the FrameworkConfig -
FrameworkConfig frameworkConfig = Frameworks.newConfigBuilder()
.parserConfig(SqlParser.Config.DEFAULT)
.defaultSchema(connection.getRootSchema().getSubSchema("SYSTEM"))
.programs(Programs.sequence(Programs.ofRules(Programs.RULE_SET), Programs.CALC_PROGRAM))
.operatorTable(sqlStdOperatorTable)
.build();
Now I can use the predefined sql functions like substr, defined in class SqlStringLengthFunction, with following piece-
RelNode udfRelNode = builder
.scan("EMP")
.project(builder.call(new SqlStringLengthFunction(),builder.literal("SampleString"), builder.literal(3))
.build();
PreparedStatement statement = RelRunners.run(udfRelNode);
ResultSet resultSet = statement.executeQuery();
but when I try with the above function 'length' in builder.call, it throws exception -
java.lang.RuntimeException: cannot translate call STRLEN($t3)
The builder fetches the implementation of these functions from a private map in RexImpTable class.
No public/protected apis are exposed in this class for adding values to this map.
Can you please guide how to register any UDF with Calcite and use the same with RelBuilder ?

A comment on calcite jira for my question here answers the question with one approach-
This is because SqlStdOperatorTable.instance() did some initialization work for the registered functions. So an invoke of #register after it would not work as expected. The correct way is to use ListSqlOperatorTable and chained it with the StdSqlOperatorTable with ChainedSqlOperatorTable, the presudo code may like this:
ListSqlOperatorTable listOpTable = new ListSqlOperatorTable();
listOpTable.add(my_udf);
ChainedSqlOperatorTable chainedOpTable = ChainedSqlOperatorTable.of(listOpTable, SqlStdOperatorTable.instance());
// then use this chainedOpTable
// If you want to use a special dialect operators, you can code like this
SqlOperatorTable optable = SqlLibraryOperatorTableFactory.INSTANCE
.getOperatorTable(SqlLibrary.STANDARD, SqlLibrary.POSTGRESQL);
I resolved my issue with following approach -
// methods containing the udf logic
public static class MyUdf1 {
public Integer eval(String a) {
return a.length();
}
}
#Test
public void test1() throws SQLException, ClassNotFoundException {
CalciteConnection connection = MyTests.getCalciteConnection();
final String functionName = "STR_LEN";
final ScalarFunction udfLengthFunction = ScalarFunctionImpl.create(Types.lookupMethod(MyUdf1.class, "eval", String.class));
connection.getRootSchema().getSubSchema("SYSTEM").add(functionName, udfLengthFunction);
FrameworkConfig frameworkConfig = Frameworks.newConfigBuilder()
.parserConfig(SqlParser.Config.DEFAULT)
.defaultSchema(connection.getRootSchema().getSubSchema("SYSTEM"))
.programs(Programs.sequence(Programs.ofRules(Programs.RULE_SET), Programs.CALC_PROGRAM))
.build();
SqlIdentifier udfLengthIdentifier = new SqlIdentifier(Collections.singletonList(functionName), null, SqlParserPos.ZERO, null);
final SqlOperator strLenOperator = new SqlUserDefinedFunction(udfLengthIdentifier, ReturnTypes.INTEGER, null, OperandTypes.STRING, null, udfLengthFunction);
final RelBuilder builder = RelBuilder.create(frameworkConfig);
RelNode udfRelNode = builder
.scan("EMP")
.project(builder.call(strLenOperator, builder.literal("SampleString")))
.build();
ResultSet set = RelRunners.run(udfRelNode).executeQuery();
set.next();
System.out.println(set.getString(1));
}

Does something like assertHTMLEquals exist for unit testing

In my test I check:
assertEquals("<div class=\"action-button\" title=\"actionButton\">"</div>", result);
If someone changes the html (result), putting there SPACE, the HTML still valid, but my test would fail.
Is there some way to compare two html pieces if those are equal as HTML. Like assertHTMLEquals
XML UNIT says that this two lines are equal:
string1:
<ldapConfigurations>
<ldap tenantid="" active="false">
</ldap>
</ldapConfigurations>
string2:
<ldapConfigurations>
<ldapdd tenantid="" active="false">
</ldap>
</ldapConfigurations>
but they are not, as you can see. (see: ldapdd )

This won't necessarily work for your case, but if your HTML happens to be valid XML it will. You can use this tool called xmlunit. With it, you can write an assert method that looks like this:
public static void assertXMLEqual(Reader reader, String xml) {
try {
XMLAssert.assertXMLEqual(reader, new StringReader(xml));
} catch (Exception ex) {
ex.printStackTrace();
XMLAssert.fail();
}
}
If that doesn't work for you, maybe there's some other tool out there meant for HTML comparisons. And if that doesn't work, you may want to end up using a library like jtagsoup (or whatever) and comparing if all the fields it parses are equal.

You can achieve malformed HTML asserting throught the TolerantSaxDocumentBuilder utility of XMLUnit.
TolerantSaxDocumentBuilder tolerantSaxDocumentBuilder =
new TolerantSaxDocumentBuilder(XMLUnit.newTestParser());
HTMLDocumentBuilder htmlDocumentBuilder =
new HTMLDocumentBuilder(tolerantSaxDocumentBuilder);
XMLAssert.assertXMLEqual(htmlDocumentBuilder.parse(expectedHTML),
htmlDocumentBuilder.parse(actualHTML));
To support badly formed HTML (such as elements without closing tags - unthinkable in XML), you must make use of an additional document builder, the TolerantSaxDocumentBuilder, along with the HTMLDocumentBuilder (this one will allow asserting on web pages). After that, assert the documents as usual.
Working code example:
public class TestHTML {
public static void main(String[] args) throws Exception {
String result = "<div class=\"action-button\" title=\"actionButton\"> </div>";
assertHTMLEquals("<div class=\"action-button\" title=\"actionButton\"></div>", result); // ok!
// notice it is badly formed
String expectedHtml = "<html><title>Page Title</title>"
+ "<body><h1>Heads<ul>"
+ "<li id='1'>Some Item<li id='2'>Another item";
String actualMalformedHTML = expectedHtml.replace(" ", " "); // just added some spaces, wont matter
assertHTMLEquals(expectedHtml, actualMalformedHTML); // ok!
actualMalformedHTML = actualMalformedHTML.replace("Heads", "Tails");
assertHTMLEquals(expectedHtml, actualMalformedHTML); // assertion fails
}
public static void assertHTMLEquals(String expectedHTML, String actualHTML) throws Exception {
TolerantSaxDocumentBuilder tolerantSaxDocumentBuilder = new TolerantSaxDocumentBuilder(XMLUnit.newTestParser());
HTMLDocumentBuilder htmlDocumentBuilder = new HTMLDocumentBuilder(tolerantSaxDocumentBuilder);
XMLAssert.assertXMLEqual(htmlDocumentBuilder.parse(expectedHTML), htmlDocumentBuilder.parse(actualHTML));
}
}
Notice that XML functions, such as XPath, will be available to your HTML document as well.
If using Maven, add this to your pom.xml:
<dependency>
<groupId>xmlunit</groupId>
<artifactId>xmlunit</artifactId>
<version>1.4</version>
</dependency>

Unit testing is not working with XSLT tansformer

I have written unit test cases to test the message processors individually in my mule flow.
But the unit test fails with error
org.mule.api.transformer.TransformerMessagingException: Property "xsl-file or xsl-text" not set.
One or more of them must be set (org.mule.api.lifecycle.InitialisationException).
Message payload is of type: String
(org.mule.api.transformer.TransformerMessagingException). Message payload is of type: String
One of the transformers is an XSLT as shown below.
<mule-xml:xslt-transformer maxIdleTransformers="2" maxActiveTransformers="5" xsl-file="C:\EWS\myproj\src\main\resources\xslt\DataAdder.xsl"
name="AdderXSLT" >
</mule-xml:xslt-transformer>
The unit test method looks as below.
MessageProcessor subFlow = muleContext.getRegistry().lookupObject("AdderXSLT");
MuleEvent result = subFlow.process(getTestEvent(getFileAsString("SamplePayloads/input.xml")));
System.out.println("The output from Event is " + result.getMessageAsString());
System.out.println("The converted XML is " + result.getMessage().getPayloadAsString());
assertNotNull(result);
assertNull(result.getMessage().getExceptionPayload());
assertFalse(result.getMessage().getPayload() instanceof NullPayload);
Please help me understand what's going wroong here.

I came across something similar before where you need to initialise the transformer explicitly when you're not executing it within the context of a flow. To test xslt transformers I have used similar to the following is the past:
XsltTransformer xslt = FunctionalTestCase.muleContext.getRegistry()
.lookupObject("SuccessResponseTransformer");
xslt.setReturnDataType(DataType.STRING_DATA_TYPE);
xslt.initialise();
String result = (String) xslt.transform(srcXML);
You could try something like this or try casting to an XsltTransformer to initialise.
I believe this is because when you execute the MP as part of a flow it is part of a MessageProcessorChain that will initialise each MP where appropriate. If you take a look at the following code from AbstractMessageProcessorChain - http://grepcode.com/file/repo1.maven.org/maven2/org.mule/mule-core/3.3.1/org/mule/processor/chain/AbstractMessageProcessorChain.java#AbstractMessageProcessorChain.initialise%28%29 :
public void initialise() throws InitialisationException
{
for (MessageProcessor processor : processors)
{
// MULE-5002 TODO review MP Lifecycle
if (processor instanceof Initialisable /* && !(processor instanceof Transformer) */)
{
((Initialisable) processor).initialise();
}
}
}
Note that (!instanceof Transformer) is commented out. So it will initialise the XsltTransformer for you.
Where as directly referencing the MessageProcessor will not.