Develop Reference

Consuming RESTful pagination information from Link header RFC8288

I am interfacing with Shopify and they use RESTful API. When I request a resource that returns an array of items, they use RFC8288 pagination format.
For example, https://example.com/api/inventory_levels.json?limit=10 returns 10 entities along with the following response header:
Link: <https://example.com/api/inventory_levels.json?limit=10&page_info=eyJs9pZHMiO>;
rel="previous", <https://example.com/api/inventory_levels.json?limit=10&page_info=MiZHeyJs9pO>; rel="next"
Appearantly if I want to retrieve all entites from that resource I need to iterate through the 'next' URL until there's no more 'next' returning. But how am I going to parse these info using JAVA or C# code? I could use a regular expression like <(?<next_url>.*)>; rel="next" to retrieve the 'next_url' from it. But it feels like re-inventing the wheel and not robust.
If this is a well-defined feature, shouldn't there be a readily available library/infrastructure that could be used? I just don't want to be caught by surprise if one day the formatting shows up different (like having an extra space and such) and, whilst abiding to the RFC defination, breaks my hastily scrambled up RegEx solution.
Suggestion welcome for Java or C#.

Parsing non-fixed format binary payload with a custom javascript conversion in Vorto

We are using Vorto now mainly as a normalized format and are starting to look into using the mapping engine for mapping different payload formats to Vorto model as well. I more or less understand how to map functionblock properties from JSON or binary payload using xpath and the conversion functions. However, I'm not clear how to support parsing of non-fixed format binary payload using this method.
For instance we have an off the shelf LoRaWAN sensor which transmits in the following format:
<length><frame type>[<sensor-id><sensor-value>] where length is the total frame length and sensor-id (for eg temperature, humidity, battery, ...) describes how to parse the sensor-value (ie length, datatype). In one frame multiple of these readings may be present in random order.
Parsing this can be done easily in for instance loraserver.io using a small javascript function which iterates over all the bytes en returns the parsed properties. The same way will work in the Ditto payload mapping engine afaik.
However, currently I don't see how to do something similar in Vorto mapping. This is just one specific sensor example of course, but more examples exist on the market using similar dynamic payload format. I know there is already an open issue (#1535) to improve the documentation, but it would already be helpful to know if such flexible parsing would be possible using the mapping DSL.
I tried passing the raw payload as bytearray to the javascript function. In order to test this I duplicated the org.eclipse.vorto.mapping.engine.converter.binary.BinaryMappingTest#testMappingBinaryContaining2DataPoints and adapted the model to use a custom javascript function like this
evaluator.addScriptFunction(new ScriptClassFunction("extractTemperature",
"function extractTemperature(value) { " +
" print(\"parameter of type \" + typeof value + \", value = \" + value);" +
" print(value[1]);" +
"}"));
The output of this function is
parameter of type number, value = 1
undefined
Where the value 1 is the first element of the bytearray used.
So the function does not seem to receive the parameter as bytarray.
The model is configured with .withXPathStereotype("custom:extractTemperature(data)", "demo") so the payload is passed (as BinaryData) in the same way as in the testMappingBinaryContaining2DataPoints test (.withXPathStereotype("custom:convert(vorto_conversion1:byteArrayToInt(data,0,0,0,2))", "demo")). The only difference I see now is that in the testMappingBinaryContaining2DataPoints test is that the byetarray parameter is passed to a Java function instead of a javascript function. Or am I missing something?
Also, I noticed that loop keywords like for and while are not allowed in the javascript code. So even if I can access the bytearray parameter in the javascript function I see no way for now how to iterate over this.
On gitter I received following reply (together with the suggestion to move discussion to SO)
You are right. We restricted the Javascript function usage to very rudimentary set of language keywords excluding for loops as nasty stuff can be implemented there. What you could do Instead is to register a java function In your own namespace to the mapping engine. That function can hold a byte array. Later this function can be contributed to the mapping engine as a standard function to extract a certain value out for other developers to reuse.
I don't think this is solution to the problem however. As mentioned above this is just one example of an off the shelf sensor payload format, and I don't see how this can be generalized enough to include as a generic function in the mapping engine. And I don't think it should be required to implement a sensor specific conversion in Java, since (as an end-user of an IoT platform wanting to deploy a new sensor type) this is more complex to develop and deploy than a little javascript function which can be altered at runtime in the mapping spec. I see a lot of value in being able to do simple mappings in javascript, just like this can be done in for example loraserver.io and Eclipse Ditto.
I think being able to pass a byte array to javascript is a first step. Also I wonder where exactly the risk is in allowing loops in the javascript? For example Ditto also has some restrictions in the javascript sandbox (see here) but this allows loops and only prevents endless looping and recursion.
They state the following:
Using Rhino instead of Nashorn, the newer JavaScript engine shipped with Java, has the benefit that sandboxing can be applied in a better way.
Sandboxing of different payload scripts is required as Ditto is intended to be run as cloud service where multiple connections to different endpoints are managed for different tenants at the same time. This requires the isolation of each single script to avoid interference with other scripts and to protect the JVM executing the script against harmful code execution.
Would using Rhino in Vorto as well allow to control the risks you see and allow loop construct in Vorto mapping?
PS: can someone with enough SO reputation points add the tag eclipse-vorto please?

I created an issue for you request to support this in the Javascript converters: https://github.com/eclipse/vorto/issues/2029
As stated in the issue, as a current workaround, you can register your own custom converter function with Java and re-use this function across your mappings. In these java converter functions, you have all the power of the java language to convert to extract the right property from the arbitrary list.
In order to find out how to implement your own custom converter function with Java, take a look here: https://github.com/eclipse/vorto/tree/master/mapping-engine#Advanced-Usage

Since Eclipse Vorto 0.12.3 release, a fix for your request is available. With this it is possible to pass array object to javascript Converter as well as use for loops inside javascript functions. You might wanna give it a try.
See release notes https://github.com/eclipse/vorto/blob/master/docs/release-notes.md

how to input large amounts of data into a query parameter via REST Assured

I have an API that returns a list of ~30,000 SKUs. I then need to insert each SKU into the query parameter URL of another API to validate the response of this second API.
I know that something like this is possible with Jmeter where you could possibly do this via a CSV file. How can I accomplish this via REST Assured? An example/sample would be greatly appreciated!
Similar question also applies to using outputs from an API to use as input in body content...
Thanks.

Short answer.You can't as it comes to query parameters. You can have no more than 2000 characters in your URL. Explanation here : What is the maximum length of a URL in different browsers?
With respect of POST method you don't have constraints.

If you are open to evaluating alternatives to REST-assured, Karate allows you to easily achieve such a data-driven test and it is based on Cucumber as well.
Disclaimer: I am the dev.
In the demos you will find a number of examples that use dynamic JSON data to drive a loop making an HTTP call. Yes, you can dynamically use the data in HTTP responses also in future steps.

Simple JMeter Test does not work

we are trying to add a simple test using JMeter in a JSF Application. We followed the instructions in:
http://jmeter.apache.org/usermanual/build-adv-web-test-plan.html
It has a simple login page with user name and password and a submit button. You can see from the screenshots that we used a proxy. With the settings in the screenshot we are getting HTTP 500 Error. I am not sure if I placed the question in a right way.. Please ask if you need any clarification.
The error code is:
EDIT:
I think this is going to be the longest question of SO. But images are better than words sometimes. Anyway, what we have done is to sent the data that is equivalent to what we see in the firebug. But still getting 500 error. You can see in the attachments Tomcat log.

HTTP 5xx codes are related to server or application errors. Search log files first.
Your script don't need a "User Defined Variables" component because there's no variable expression that really need to be evaluated per thread/user.
The "Regular Expression Extractor" component suffice to extract the JSF ViewState value.
I suggest you to delete the last part of your expression, " />", and change the regular expression grouping (.+?) to (\w+?) 'cause it will evaluate to a few matches (probably only 2). Change the value of "Match No." field to 1 (no need to use random if all values matched are identical).
I didn't understand why you used both "XPath Extractor" and "Regular Expression Extractor" components to extract the same value. I prefer to use the last one when leading with html. XPath is better when treating with well-formed xml strings/files.
To capture a script from scratch, I suggest you to add a "HTTP Proxy Server" inside Workbench, configure it, start it, configure a browser to use this proxy and navigate those pages using the browser. This way you'll capture all requests made and request headers used by the browser you choose. After this, remove unnecessary requests and change query parameters, like javax.faces.ViewState, to the corresponding variables.
Consider using extractors (Pos-Processors) inside an HTTP Sampler prior to the one that will use the variable in Parameter Values. Ex.: if /EBS request comes first and /EBS/login.xhtml request have a javax.faces.ViewState parameter then, probably, /EBS response will contain a hidden input with the javax.faces.ViewState value.
This is a common make up of JSF application test scripts I use. Providing more information about the cause of the HTTP 500 error should clarify the way to a better solution.

On the Regular Expression Extractor for jsfViewState, add (?s) to the start of the regular expression. So you have:
(?s)<input type="hidden" name="javax\.faces\.ViewState" id="javax\.faces\.ViewState" value="(.+?)" />
This allows the (.+?) to span line break characters.

Your regular expression extractor is in the wrong place. You cannot extract a value from the response to a request and then send it with the same request. The only way to achieve this is to use a time machine, but these don't exist yet and even if they did, it probably wouldn't work.
Typically you get a viewstate in the response to a GET and then you later need it in the POST of the same page. So, put the regular expression extractor as a child of the GET call where the login.xhtml page is first called (as a GET). If your recording does not include this GET call then either add it manually or examine the responses of previous calls before your login POST to find it, eg. maybe the GET homepage.xhtml (or similar) will include it.

Create URL Parameters in GWT

I asked a related question here
How do I programatically write parameters into the URL using GWT? I've learned from my previous question that parameters need to go before the anchor, but how do I do that from GWT?
Here's the code that doesn't work:
Hyperlink pg1 = new Hyperlink("Test", "?testing=abc#pg1");
It results in the following url:
http://localhost:8080/Athena.html#?testing=abc%23pg1
I was thinking about using Window.Location.assign(), but the javadoc says that will loose the state of my application.

The object HyperLink seems to be for linking to internal states, and probably was written so that changing its href is difficult?
I suggest you use this class http://google-web-toolkit.googlecode.com/svn/javadoc/1.5/com/google/gwt/user/client/ui/InlineHTML.html instead - obviously you are generating the href programatically, so it should be easy to generate the element to supply to the InlineHTML object.

You are using this constructor which receives a history token as the second constructor argument, hence you are getting said result.
Use the setHTML() method to set the correct value on the link.