Develop Reference

Format a string w/ Java MaskFormatter in Databricks/Scala

I create a function based on Java MaskFormatter function in Databricks/Scala.
But when I call it from sparksql, I received error message
Error in SQL statement: AnalysisException: Undefined function:
formatAccount. This function is neither a built-in/temporary function,
nor a persistent function that is qualified as
spark_catalog.default.formataccount.; line 1 pos 32
Here is my function
import javax.swing.text.MaskFormatter
def formatAccount(account: String, mask:String ) : String =
{
val formatter = new MaskFormatter(mask.replace("X", "A"))
formatter.setValueContainsLiteralCharacters(false)
val formatAccount = formatter.valueToString(account)
formatAccount
}
Here is the query code which received the error message
sql("""select java_method(emitToKafka ,formatAccount("1222233334", "X-XXXX-XXXX-X"))""")
However if I run below code, it works fine.
formatAccount("1222233334", "X-XXXX-XXXX-X")
res0: String = 1-2222-3333-4
what could be missed?

No signature of method java.util.LinkedHashMap.getBytes() Is applicable for argument types (String) values: [UTF-8] in Groovy

I am working on a Groovy script that should send an http post request, but once it is triggered I get the following error: "Script action failed on line #16: groovy.lang.MissingMethodException: no signature of method: java.util.LinkedHashMap.getBytes() is applicable for argument types: (String) values: [UTF-8]
Here is my code:
def post = new URL("https://someUrl.com").openConnection();
def transcript = actionJson;
def session_id = _.getSessionId();
def requestBody =
['Session ID': session_id,
'Transcript': transcript]
post.setRequestMethod("POST")
post.setDoOutput(true)
post.setRequestProperty("Content-Type", "application/json")
post.getOutputStream().write(requestBody.getBytes("UTF-8"));
post.connect();
def postRC = post.getResponseCode();
println(postRC);
if (postRC.equals(200)) {
println(post.getInputStream().getText());
}
I am very new to groovy and I am not exactly sure why I am hitting this problem.
Any ideas?

Sending PythonJavaClass into Java function with pyjnius

I'm trying to send a PythonJavaClass (NokeServiceListener) into mNokeService.registerNokeListener() but it keeps erroring out there and when printing the NokeServiceListener object I'm getting nokeLock.NokeServiceListener object at 0x97389b70 which doesn't look like a java object and then I get an error JNI DETECTED ERROR IN APPLICATION: use of invalid jobject 0x6e617278
from jnius import autoclass,PythonJavaClass,cast,java_method
Context = autoclass('android.content.Context')
Parcelable = autoclass('android.os.Parcelable')
Intent = autoclass('android.content.Intent')
Uri = autoclass('android.net.Uri')
PythonActivity = autoclass('org.kivy.android.PythonActivity')
NokeDeviceManagerService = autoclass('com.noke.nokemobilelibrary.NokeDeviceManagerService')
#LocalBinder = autoclass('com.noke.nokemobilelibrary.NokeDeviceManagerService$LocalBinder')
NokeDevice = autoclass('com.noke.nokemobilelibrary.NokeDevice')
NokeMobileError = autoclass('com.noke.nokemobilelibrary.NokeMobileError')
#NokeServiceListener =autoclass('com.noke.nokemobilelibrary.NokeServiceListener')
global mNokeService
##run_on_ui_thread
class NokeApi():
def __init__(self):
self.python_activity = PythonActivity.mActivity
self.service_connection = ServiceConnection()
def initiateNokeService(self):
currentActivity = cast('android.app.Activity', self.python_activity)
context = cast('android.content.Context', currentActivity.getApplicationContext())
nokeIntent = Intent()
nokeIntent.setClassName(context, 'com.noke.nokemobilelibrary.NokeDeviceManagerService')
self.python_activity.bindService(nokeIntent,self.service_connection,Context.BIND_AUTO_CREATE)
def onCreate(self):
self.initiateNokeService()
##run_on_ui_thread
class ServiceConnection(PythonJavaClass):
__javainterfaces__ = ['android.content.ServiceConnection']
__javacontext__ = 'app'
#java_method('(Landroid/content/ComponentName;Landroid/os/IBinder;)V')
def onServiceConnected(self,className, rawBinder):
print 'debug1'
#nokeDeviceManagerService = NokeDeviceManagerService()
#localBinder = LocalBinder()
nokeService = cast('com.noke.nokemobilelibrary.NokeDeviceManagerService$LocalBinder',rawBinder)
global mNokeService
mNokeService = nokeService.getService()
print mNokeService
#mNokeService = ((NokeDeviceManagerService.LocalBinder)rawBinder).getService()
print 'debug2'
mNokeServiceListener = NokeServiceListener()
#mNokeServiceListener = cast('com.noke.nokemobilelibrary.NokeServiceListener',nokeServiceListener)
print mNokeServiceListener
print 'debug2.5'
mNokeService.registerNokeListener(mNokeServiceListener)
print 'debug3'
noke1 = NokeDevice("NOKE3P", "F7:F3:F1:2C:66:25")
print 'debug4'
mNokeService.addNokeDevice(noke1)
print 'debug5'
mNokeService.setUploadUrl("https://coreapi-sandbox.appspot.com/upload/")
print 'debug6'
mNokeService.startScanningForNokeDevices()
print "Scanning for devices"
if not mNokeService.initialize():
print "Unable to initialize Bluetooth"
class NokeServiceListener(PythonJavaClass):
__javainterfaces__ = ['com.noke.nokemobilelibrary.NokeServiceListener']
__javacontext__ = 'app'
def __init__(self):
pass
#java_method('(Lcom/noke/nokemobilelibrary/NokeDevice;)V')
def onNokeDiscovered(self,noke):
print "Connecting to Noke"
mNokeService.connectToNoke(self,noke)
#java_method('(Lcom/noke/nokemobilelibrary/NokeDevice;)V')
def onNokeConnecting(self,noke):
print "Connecting"
#java_method('(Lcom/noke/nokemobilelibrary/NokeDevice;)V')
def onNokeConnected(self,noke):
print "Noke Connected"
self.requestUnlock(noke)
#java_method('(Lcom/noke/nokemobilelibrary/NokeDevice;)V')
def onNokeSyncing(self,noke):
print "NOKE SYNCING"
#java_method('(Lcom/noke/nokemobilelibrary/NokeDevice;)V')
def onNokeUnlocked(self,noke):
print 'Noke Unlocked'
#java_method('(Lcom/noke/nokemobilelibrary/NokeDevice;)V')
def onNokeDisconnected(self,noke):
print "Noke Disconnected"
self.mNokeService.uploadData()
self.mNokeService.startScanningForNokeDevices()
#java_method('(I)V')
def onBluetoothStatusChanged(self,bluetoothStatus):
pass
#java_method('(Lcom/noke/nokemobilelibrary/NokeDevice;ILjava/lang/String;)V')
def onError(self,noke, error, message):
pass
def requestUnlock(self,noke):
msg = '{"function":"Noke_Unlock","session":"%s","mac":"%s"}' % (noke.getSession(),noke.getMac())
rsp = connectToServer(_host, _port, msg)
if rsp['result'] == "success":
commandStr = rsp["commands"]
noke.sendCommands(commandStr)
else:
print "Access Denied"
I feel like there error is because I'm passing in a non java object into a java function but I've already tried to cast the object as a 'com.noke.nokemobilelibrary.NokeServiceListener' and that didn't work either. The code always errors out after debug 2.5 so something is going wrong between those two steps but I can't figure out what.

class NokeServiceListener(PythonJavaClass):
def __init__(self, callback):
super().__init__()
self.callback = callback
It will work because python is not java / C#. base class init will not called unless you explicitly write it.

Adding
class NokeServiceListener(PythonJavaClass):
def __init__(self, callback):
super(NokeServiceListener, self).__init__()
self.callback = callback
seemed to fix this issue... no idea why though.

Strange scala java interoperability behavior

I ran into the following Java compiler warning issue (using IntelliJ 14):
I have the following class in my Scala library:
object ScalatoJava {
def asJava[A,B](map: Map[A,B]): java.util.Map[A,B] = { map.asJava }
def asJava[A](list: List[A]): java.util.Collection[A] = {list.asJavaCollection}
}
I use it to convert few collections coming from Scala in my Java program.
Now in my Java program I have the following code for which IntelliJ is underlying in red stating that it can't resolve things:
private final java.util.Map<String, FieldSettings> poolpartyfieldsettings =
ScalatoJava.asJava(poolpartyConnectorSettings.fieldsSettingsMap());
It says it can not resolve the method asJava(scala.collection.immutable.Map<String, org.....FieldSettings>).
While if I write either
private final java.util.Map <String, FieldSettings> poolpartyfieldsettings =
ScalatoJava.asJava((scala.collection.immutable.Map<String, FieldSettings>)poolpartyConnectorSettings.fieldsSettingsMap());
or
scala.collection.immutable.Map<String, FieldSettings> poolpartyfieldsettings =
poolpartyConnectorSettings.fieldsSettingsMap();
ScalatoJava.asJava(poolpartyfieldsettings)
the compiler does not complain.
Here is the PoolpartyConnectorSettings Scala class:
trait DspacePoolPartyConnectorSettings {
def poolpartyServerSettings : PoolpartySettings
def fieldsSettingsList : List[FieldSettings]
def fieldsSettingsMap : Map[String, FieldSettings]
}
case class DspaceDspacePoolPartyConnectorSettingImpl (configUri: String) extends DspacePoolPartyConnectorSettings {
private val config = ConfigFactory.parseURL(new URL(configUri)).resolve()
private val configFieldSettinglist = config.getConfigList("PoolPartyConnectorSettings.FieldSettings").asScala.toList
private val configPoolPartySetting = config.getConfig("PoolPartyConnectorSettings.PoolPartySettings")
val poolpartyServerSettings = PoolpartySettings( configPoolPartySetting.getString("apirootEndpoint"),
configPoolPartySetting.getString("thesaurusapiEndpoint"),
configPoolPartySetting.getString("extratorapiEndpoint"),
configPoolPartySetting.getString("coreProjectId"),
configPoolPartySetting.getString("coreThesaurusUri"),
configPoolPartySetting.getString("jelProjectId"),
configPoolPartySetting.getString("jelThesaurusUri"))
val fieldsSettingsList = configFieldSettinglist map { e =>
FieldSettings(e.getString("fieldname"), e.getBoolean("treeBrowser"),
e.getBoolean("multilanguage"), e.getStringList("languages").asScala.toList,
e.getBoolean("closed"), e.getString("scheme"), e.getString("poolpartyProjectId"))
}
val fieldsSettingsMap = Map(fieldsSettingsList map {e => (e.fieldName, e)}: _*)
}
Any idea what the problem might be?

converting java.io.File to java.sql.Blob in Scala

I am trying to transform a java.io.File to a java.sql.Blob. More specifically I have a model in Scala Play that needs to store a file inside of a database, here is what I have so far.
case class Complication(complicationId: Long,
vitalSignId: Long,
complication: String, definition: String, reason: String,
treatment: String, treatmentImage: Option[java.io.File], notes: String,
weblinks: String, upperlower: String)
object ComplicationDAO extends Table[Complication]("complications") with GasGuruDatabase {
def complicationId = column[Long]("complication_id", O.PrimaryKey, O.AutoInc)
def vitalSignId = column[Long]("vital_sign_id")
def complication = column[String]("complication")
def definition = column[String]("definition", O.DBType("varchar(4000)"))
def reason = column[String]("reason", O.DBType("varchar(4000)"))
def treatment = column[String]("treatment", O.DBType("varchar(4000)"))
def treatmentImage = column[Option[Blob]]("treatment_image")
def notes = column[String]("notes", O.DBType("varchar(4000)"))
def weblinks = column[String]("weblinks")
def upperlower = column[String]("upperlower")
def * = complicationId ~ vitalSignId ~ complication ~ definition ~ reason ~ treatment ~ treatmentImage ~ notes ~ weblinks ~ upperlower <> (Complication, Complication.unapply _)
The error is happening in between the mapping from a java.io.File to a Blob to store the file inside of the database, how can I do this?
Thanks!

You have to actually read the file into something compatible with blob before trying to store it, java.io.File is just a descriptor
From Slick's documentation
LOB types: java.sql.Blob, java.sql.Clob, Array[Byte]
case class Complication(complicationId: Long,
vitalSignId: Long,
complication: String, definition: String, reason: String,
treatment: String, treatmentImage: **Option[Array[Byte]]**, notes: String,
weblinks: String, upperlower: String)