Develop Reference

Scala udf UnsupportedOperationException

I have a dataframe a2 written in scala :
val a3 = a2.select(printme.apply(col(“PlayerReference”)))
the column PlayerReference contains a string.
that calls an udf function :
val printme = udf({
st: String =>
val x = new JustPrint(st)
x.printMe();
})
this udf function calls a java class :
public class JustPrint {
private String ss = null;
public JustPrint(String ss) {
this.ss = ss;
}
public void printMe() {
System.out.println("Value : " + this.ss);
}
}
but i have this error for the udf :
java.lang.UnsupportedOperationException: Schema for type Unit is not supported
The goal of this exercise is to validate the chain of calls.
What should I do to solve this problem ?

The reason you're getting this error is that your UDF doesn't return anything, which, in terms of spark is called Unit.
What you should do depends on what you actually want, but, assuming you just want to track values coming through your UDF you should either change printMe so it returns String, or the UDF.
Like this:
public String printMe() {
System.out.println("Value : " + this.ss);
return this.ss;
}
or like this:
val printme = udf({
st: String =>
val x = new JustPrint(st)
x.printMe();
x
})

having error when represent data to the platform

I get this error:
Message: exception
Details: 'Cannot get property \'data\' on null object
Here’s my code:
import javax.xml.bind.DatatypeConverter;
import com.carriots.sdk.utils.BasicHttp;
def APIKEY = '1c7021dfcc02e4f52a8db39'
//// Data Fetching ////
def inc_data = context.data.data;
def d1 = inc_data[0..1];
//// Custom Rules declaration ////
def device_id = context.data.id;
def devicename = device_id + '23A2B#User.User';
def rssi = context.data.rssi;
def avgSnr = context.data.avgSnr;
def D1 = Long.parseLong(d1, 16)
//// Filter Status ////
if (D1 >= 1) {
fltstatus = "motion detected";
} else {
fltstatus = "No motion";
}
//// Payload ////
// Build Stream to persist.
// I think the error from the data variable but I do not why...
def data = '{"motion detection": "' + fltstatus + '"}'
def payload_data = '{"at": "now", "protocol": "v2", "device": "' +
devicename + '", "data": "' + data +'"}'
def basicHttp = new BasicHttp();
basicHttp.verb = "POST";
basicHttp.payload = payload_data;
basicHttp.url = "http://api.m.om/status/";
basicHttp.headers = [
"Content-type": "application/json",
"Accept": "application/json",
"User-Agent": "Listener-Carriots",
"carriots.apikey": APIKEY
]
basicHttp.send();

Your first use of data as a property is this line:
def inc_data = context.data.data;
^ ^
(1) (2)
Since you use it twice, it’s impossible to know which use is throwing the exception. I recommend breakpointing this line, then using your debugger to determine that:
context is not null
context.data is not null

convert & delimited String to a java class

Is there a method can convert & delimited String to a java class?
Foo foo = Foo.fromString("name1=a&name2=b");
I am coding the twitter api: https://developer.twitter.com/en/docs/basics/authentication/api-reference/access_token. the response is
I need need a function do the blow things for me.
String respnse = "oauth_token=6253282-eWudHldSbIaelX7swmsiHImEL4KinwaGloHANdrY&oauth_token_secret=2EEfA6BG3ly3sR3RjE0IBSnlQu4ZrUzPiYKmrkVU&user_id=6253282&screen_name=twitterapi";
String [] resParas = respnse.split("&");
for(String respara : resParas){
if(respara.indexOf("oauth_token=")>=0){
int index = "oauth_token=".length();
access_token = respara.substring(index);
}else if(respara.indexOf("oauth_token_secret=")>=0){
int index = "oauth_token_secret=".length();
access_token_secret = respara.substring(index);
}else if(respara.indexOf("user_id=")>=0){
int index = "user_id=".length();
user_id = respara.substring(index);
}else if(respara.indexOf("screen_name=")>=0){
int index = "screen_name=".length();
screen_name = respara.substring(index);
}
}

You can use UriComponentsBuilder for this:
MultiValueMap<String, String> parameters = UriComponentsBuilder
.fromUriString("http://twitter.com/oauth?oauth_token=6253282-eWudHldSbIaelX7swmsiHImEL4KinwaGloHANdrY&oauth_token_secret=2EEfA6BG3ly3sR3RjE0IBSnlQu4ZrUzPiYKmrkVU&user_id=6253282&screen_name=twitterapi")
.build()
.getQueryParams();
Will give you a map with all params:
{oauth_token=[6253282-eWudHldSbIaelX7swmsiHImEL4KinwaGloHANdrY], oauth_token_secret=[2EEfA6BG3ly3sR3RjE0IBSnlQu4ZrUzPiYKmrkVU], user_id=[6253282], screen_name=[twitterapi]}

executing javascript code inside Android "java" application

I'm working on an Android app "native written in java"
and I'm getting a response from a server the response is a javascript function
I need to use this function to do some calculations inside my native java code.
any ideas how to do so.
sample response :
function logic_1(surveyString, responseValuesString) {
var survey = eval(surveyString);
var responseValues = eval(responseValuesString);
var target = new Object();
if (isChosen(128133225, responseValues)) {
target.id = 2;
}
if (! target.id) {
target.id = 2;
}
return target;
}

I've previously used Rhino successfully to execute JavaScript code on Android:
http://www.mozilla.org/rhino/

Here's an example of how to return values from a complex type:
String strFunction =
"function add(x,y){ " +
"return { " +
"id:x+y " +
"}; " +
"}";
Context context = Context.enter();
ScriptableObject scope = context.initStandardObjects();
context.evaluateString(scope, strFunction, "test", 1, null);
Function functionAdd = (Function)scope.get("add");
NativeObject untypedResult = (NativeObject)functionAdd.call(context, scope, scope, new Object[] { 1, 2 });
double id = (Double)untypedResult.get("id", untypedResult);
The important part is the last two lines, where we call the JavaScript function, treat the result as a NativeObject, and then retrieve the value of the 'id' property from that object.

Maybe you just need to use a JavaScript auto executing function like this:
(function(x, y){
var result;
result = x + y; // do some calculations
return result;
})(1 , 2); // you can set your parameters from Java
and 1, 2 are just two parameters from Java

Is there a cleaner way to do this Group Query in MongoDB from Groovy?

I'm working on learning MongoDB. Language of choice for the current run at it is Groovy.
Working on Group Queries by trying to answer the question of which pet is the most needy one.
Below is my first attempt and it's awful. Any help cleaning this up (or simply confirming that there isn't a cleaner way to do it) would be much appreciated.
Thanks in advance!
package mongo.pets
import com.gmongo.GMongo
import com.mongodb.BasicDBObject
import com.mongodb.DBObject
class StatsController {
def dbPets = new GMongo().getDB('needsHotel').getCollection('pets')
//FIXME OMG THIS IS AWFUL!!!
def index = {
def petsNeed = 'a walk'
def reduce = 'function(doc, aggregator) { aggregator.needsCount += doc.needs.length }'
def key = new BasicDBObject()
key.put("name", true)
def initial = new BasicDBObject()
initial.put ("needsCount", 0)
def maxNeeds = 0
def needyPets = []
dbPets.group(key, new BasicDBObject(), initial, reduce).each {
if (maxNeeds < it['needsCount']) {
maxNeeds = it['needsCount']
needyPets = []
needyPets += it['name']
} else if (maxNeeds == it['needsCount']) {
needyPets += it['name']
}
}
def needyPet = needyPets
[petsNeedingCount: dbPets.find([needs: petsNeed]).count(), petsNeed: petsNeed, mostNeedyPet: needyPet]
}
}

It should be possible to be change the whole method to this (but I don't have MongoDB to test it)
def index = {
def petsNeed = 'a walk'
def reduce = 'function(doc, aggregator) { aggregator.needsCount += doc.needs.length }'
def key = [ name: true ] as BasicDBObject
def initial = [ needsCount: 0 ] as BasicDBObject
def allPets = dbPets.group( key, new BasicDBObject(), initial, reduce )
def maxNeeds = allPets*.needsCount.collect { it as Integer }.max()
def needyPet = allPets.findAll { maxNeeds == it.needsCount as Integer }.name
[petsNeedingCount: dbPets.find([needs: petsNeed]).count(), petsNeed: petsNeed, mostNeedyPet: needyPet]
}