I have to make call to OpenNMS REST service, but it only accepts PUT with application/x-www-form-urlencoded, what is the way using Java to encode non trivial objects into this format.
First, to address getting Java content to application-x-www-form-urlencoded: Java has the URLEncoder which will let you convert a String to the desired format.
To get your data into a useful format to go into the URLEncoder, I recommend a custom method to read the Object and produce the desired String. This is dependent upon the specific call you are trying to make (from a quick review of OpenNMS ReST, they have many different routes and requirements for data associated)).
If making a large number of these calls I recommend making methods to address each of the API calls on the Object that would internally produce the desired String, encode it, and then return the Encoded content for the call.
Related
We are using Vorto now mainly as a normalized format and are starting to look into using the mapping engine for mapping different payload formats to Vorto model as well. I more or less understand how to map functionblock properties from JSON or binary payload using xpath and the conversion functions. However, I'm not clear how to support parsing of non-fixed format binary payload using this method.
For instance we have an off the shelf LoRaWAN sensor which transmits in the following format:
<length><frame type>[<sensor-id><sensor-value>] where length is the total frame length and sensor-id (for eg temperature, humidity, battery, ...) describes how to parse the sensor-value (ie length, datatype). In one frame multiple of these readings may be present in random order.
Parsing this can be done easily in for instance loraserver.io using a small javascript function which iterates over all the bytes en returns the parsed properties. The same way will work in the Ditto payload mapping engine afaik.
However, currently I don't see how to do something similar in Vorto mapping. This is just one specific sensor example of course, but more examples exist on the market using similar dynamic payload format. I know there is already an open issue (#1535) to improve the documentation, but it would already be helpful to know if such flexible parsing would be possible using the mapping DSL.
I tried passing the raw payload as bytearray to the javascript function. In order to test this I duplicated the org.eclipse.vorto.mapping.engine.converter.binary.BinaryMappingTest#testMappingBinaryContaining2DataPoints and adapted the model to use a custom javascript function like this
evaluator.addScriptFunction(new ScriptClassFunction("extractTemperature",
"function extractTemperature(value) { " +
" print(\"parameter of type \" + typeof value + \", value = \" + value);" +
" print(value[1]);" +
"}"));
The output of this function is
parameter of type number, value = 1
undefined
Where the value 1 is the first element of the bytearray used.
So the function does not seem to receive the parameter as bytarray.
The model is configured with .withXPathStereotype("custom:extractTemperature(data)", "demo") so the payload is passed (as BinaryData) in the same way as in the testMappingBinaryContaining2DataPoints test (.withXPathStereotype("custom:convert(vorto_conversion1:byteArrayToInt(data,0,0,0,2))", "demo")). The only difference I see now is that in the testMappingBinaryContaining2DataPoints test is that the byetarray parameter is passed to a Java function instead of a javascript function. Or am I missing something?
Also, I noticed that loop keywords like for and while are not allowed in the javascript code. So even if I can access the bytearray parameter in the javascript function I see no way for now how to iterate over this.
On gitter I received following reply (together with the suggestion to move discussion to SO)
You are right. We restricted the Javascript function usage to very rudimentary set of language keywords excluding for loops as nasty stuff can be implemented there. What you could do Instead is to register a java function In your own namespace to the mapping engine. That function can hold a byte array. Later this function can be contributed to the mapping engine as a standard function to extract a certain value out for other developers to reuse.
I don't think this is solution to the problem however. As mentioned above this is just one example of an off the shelf sensor payload format, and I don't see how this can be generalized enough to include as a generic function in the mapping engine. And I don't think it should be required to implement a sensor specific conversion in Java, since (as an end-user of an IoT platform wanting to deploy a new sensor type) this is more complex to develop and deploy than a little javascript function which can be altered at runtime in the mapping spec. I see a lot of value in being able to do simple mappings in javascript, just like this can be done in for example loraserver.io and Eclipse Ditto.
I think being able to pass a byte array to javascript is a first step. Also I wonder where exactly the risk is in allowing loops in the javascript? For example Ditto also has some restrictions in the javascript sandbox (see here) but this allows loops and only prevents endless looping and recursion.
They state the following:
Using Rhino instead of Nashorn, the newer JavaScript engine shipped with Java, has the benefit that sandboxing can be applied in a better way.
Sandboxing of different payload scripts is required as Ditto is intended to be run as cloud service where multiple connections to different endpoints are managed for different tenants at the same time. This requires the isolation of each single script to avoid interference with other scripts and to protect the JVM executing the script against harmful code execution.
Would using Rhino in Vorto as well allow to control the risks you see and allow loop construct in Vorto mapping?
PS: can someone with enough SO reputation points add the tag eclipse-vorto please?
I created an issue for you request to support this in the Javascript converters: https://github.com/eclipse/vorto/issues/2029
As stated in the issue, as a current workaround, you can register your own custom converter function with Java and re-use this function across your mappings. In these java converter functions, you have all the power of the java language to convert to extract the right property from the arbitrary list.
In order to find out how to implement your own custom converter function with Java, take a look here: https://github.com/eclipse/vorto/tree/master/mapping-engine#Advanced-Usage
Since Eclipse Vorto 0.12.3 release, a fix for your request is available. With this it is possible to pass array object to javascript Converter as well as use for loops inside javascript functions. You might wanna give it a try.
See release notes https://github.com/eclipse/vorto/blob/master/docs/release-notes.md
I have a web service I am trying to call from Java. The XSD for the service defines a field as an xsd:base64Binary. I am using maven jaxb2 plugin to generate Java artifacts. The field becomes a byte[] in the generated Java object. The data that comes back in that field is either CSV or XML data depending on what is passed into the service. SoapUI displays the data perfectly (not encoded). Watching the wire with wireshark I can also see the non encoded data. My question is, how do I grab this data as a string in Java? I want to take this data and later write it into a file.
Response looks something like this:
Service Agreement,Interval Start Time,Interval End Time,Quantity,Unit of Measure .... etc.
Relevant bit of XSD:
Relevant bit of generated java:
protected byte[] greenDoc;
In my client java code I have been trying every possible combination of new String(byte[]), new String(byte[], charset), Base64 decoding, etc. and I just cannot seem to get the data correctly. I know it is not a limitation of the web service because like I said SoapUI can display the data perfectly.
Any pointers on how the client code can take the byte array and convert to string? Thanks!
Programmatically, you can use DatatypeConverter
I am trying to use the google-voice-java API to read texts from a google voice account. I cannot find much documentation on using it other then the code.google.com page. I just need a little help on how to correctly use getSMS();
The google-voice-java API will simply return back raw HTML of your Google Voice data. Thus, you need to parse through the HTML returned by getSMS();, and do with the data as you see fit.
If you'd like to just see what the data returned looks like at a console window, do the following
Voice voice = new Voice(userName, password);
String sms = voice.getSMS();
System.out.println(sms);
Based on that, you can see what the HTML looks like and how to parse through it.
Edit: Probably worth noting that you need to include the additional dependency jars to your build path; that should be more explicitly clear on their GettingStarted!
Edit 2: Based on your comment below, there are the getUnreadSMS() and markAsRead(msgID), but the former appears to return read SMS messages along with the unread ones.
If you notice at the top of the response XML returned by the getSMS() method (and majority of the other API methods), there is JSON data in the <json> element, which seems to have all the necessary information you need (including a isRead variable to indicate if a SMS is marked read, or not).
Pull down a response from the getSMS() method and use this online JSON viewer to better inspect the data in the <json> element, so you have an idea of what's in there. When you paste in the JSON data into the online viewer, omit the leading <![CDATA and trailing ]> inside the <json> element.
I would just setup a JSON parser (maybe even use GSON) and create SMS model objects based on the JSON data, ignoring the HTML completely; for unread messages, you want JSON objects with the isRead field set to false, obviously. You could then use the id field to pass into the markAsRead(msgID) method, to mark them read (I just tested this method and it works).
I am writing a RESTful web service using Java which should be able to take as (POST) input an unknown number of parameter name, value pairs. I am thinking what is the best way to achieve this.
The POST input will be JSON with the expected format being:
{"input" :
{ "x" = 1, "y"="Hello, "z"="1.2.3.4",... }
}
The parameter values are expected to be a mix of int/long/float/string etc. The expected behavior is to save this input to the DB and provide it back on request. The tricky part is with the parameter names being unknown, I cannot write getter and setter methods. Is there a simple way for handling this?
Any pointers in the right direction will be very much appreciated. Can I read this as a List? Do I need to write Custom MessageBodyReaders? Is there some other way of achieving this? The only option I can think of is to have these sent as a file and parse it but am hoping that there are simpler/cleaner solutions. Thank you for any pointers...
If all you need to do is save it and make it available on request, why parse it at all? Simply write the json as a string to your database, and return it as a string while setting the response type to "application/json". The one caveat being you're going to have to have some way to identify which json string you want and associate a unique id that you're going to be able to determine when you're requesting the specific json string in the future.
Something worth noting here would be that when you don't parse something, you don't actually know what it is. Even though a lot of modern libraries/frameworks sanitize against sql injection/xss already, it would be worth it to verify that this is being done in your server if you don't already know for sure.
That being said, if you're still feeling as though you need to parse the json, I would look into gson: http://code.google.com/p/google-gson/
JSON stands for JavaScript Object Notation. But how come languages like php, java, c etc can also communication each other with json.
What I want to know is that, am i correct to say that json is not limited to js only, but served as a protocol for applications to communicate with each other over the network, which is the same purpose as XML?
JSON cannot handle complex data hierarchies like XML can (attributes, namespaces, etc.), but on the other hand you don't get the same overhead with JSON as you get with XML (if you don't need the complex data structures).
Since JSON is plain text with a special notation for JS to interpret, it's an easy protocol to adopt in other languages.
It is easy for a JS script to parse JSON, since it can be done using 'eval' in which the JS enginge can use its full power.
On the other hand, it is more complicated to generate JSON from within JS. Usually one uses the JSON package from www.json.org in which an object can easily be serialised using JSON.stringify, but it is implemented in JS so its not running with optimal performance.
So serialising JSON is about the same complexity using JS as when using Java, PHP or any other server side language.
Therefore, in my opinion, JSON is best suited when there is asymmetry between produce/consumer e.g. a web server that generates a lot of data that is consumed by the web application. Not the other way around.
But! When one choses JSON as data format it should be used in both directions, not XML<>JSON. Except for when simple get requests are used to retrieve JSON data.
yes, JSON is also wildly used as a data exchange protocol much like XML.
Typically a program (not written in JavaScript) needs a JSON library to parse and create JSON objects (although you can probably create them even without one).
Your right - it's a light weight data interchange format -- more details at: http://www.json.org
You are completely correct. JSON definition of how data should be formatted. It is more light weight than XML and therefore well suited to things like AJAX where you want to send data back and forth to the server quickly.