Develop Reference

passing extra variables in Spark UDF JAVA

I have written a spark UDF in JAVA to encrypt particular columns in a dataframe. It is type 1 UDF and only accepts a string that needs to be encrypted or decrypted at a time. I want to pass corresponding password as well. I tried the currying approach but was not able to write the function properly. Can anyone suggest me any solution ?
public class EncryptString implements UDF1<String, String> {
#Override
public String call(String s) throws Exception {
return Aes256.encrypt(s);
//Aes.encrypt needs to have another variable password.
//So that while calling the UDF we can pass the required password.
}
}

You can pass the password - as well as any other parameters - as constructor parameter to the EncryptString class:
public static class EncryptString implements UDF1<String, String> {
private final String password;
public EncryptString(String password) {
this.password = password;
}
public String call(String s) throws Exception {
return Aes256.encrypt(s, password);
}
}
When instantiating the udf, you can pass the actual password:
spark.sqlContext().udf().register("EncryptUdf", new EncryptString("secret"), DataTypes.StringType);
[...]
spark.sql("select EncryptUdf(_c2) from df").show();

calling methods from objects (Java)

I'm new(ish) to java and I'm making a program for fun that allows you to create different users, login to those users, and make notes. I'm stuck on the user creation part. There is this one line of code that won't work. I have an array called userarr that holds user objects. Inside the object is the user creation method. This is the line of code that takes the variables you type in for the username and password and plugs it into the usercreation method:
userarr[userarr.length+1] = new user.usercreation(username,password);
It says it can't find usercreation method inside the class. But I don't know how to use the usercreation method outside the object and be able to create different named objects.
Here is the entire class:
public class TextGame {
static Scanner scan = new Scanner(System.in);
class user extends TextGame {
String username;
int password;
String[] notes;
public void usercreation(String username, int password) {
this.username = username;
this.password = password;
}
public void login(int password) {
this.password = password;
System.out.println("please type the password to proceed.");
if (scan.nextLine().equals(this.password)) {
System.out.println("logged in. type 'note' to access notes, or 'logoff' to log off this user.");
}
}
}
static user[] userarr;
public static void newuser() {
System.out.println("\n\nType the username for this user.");
String username = scan.nextLine();
System.out.println("Username is now " + username + ". is this what you want? type 'yes' to proceed, or 'no' to enter username again.");
if (scan.nextLine().equals("no")) {
newuser();
}
else {
System.out.println("\n\n type the password for this user. (numbers only.)");
int password = scan.nextInt();
System.out.println("user is " + username + " and password is " + password + " is this what you want?");
if (scan.nextLine().equals("no")) {
newuser();
} else {
userarr[userarr.length + 1] = new user.usercreation(username, password);
}
}
}
public static void main(String[] args) {
System.out.println("Welcome to LINCOLN COMP console OS. Type 'new' to create a new user, type 'log' to log in to an existing user, or type 'exit' to leave.\nif you are asked a yes or no question, if you type something ether than yes or no, it will default to yes.");
String ch1 = scan.nextLine();
switch (ch1) {
case "new":
System.out.println("Initializing user creation method:");
newuser();
break;
case "log":
break;
case "exit":
System.out.println("Goodbye!");
System.exit(0);
break;
}
}
}

Okay, so starting with new user.usercreation(username,password);
usercreation is not a static inner class of user, so you can't create it
usercreation is a method of user which returns void, so you returns nothing, so you can't assign it to anything.
You could...
Create the instance of user, apply the properties and assign it to the array as separate actions
user newUser = user();
newUser.usercreation(username,password);
userarr[userarr.length+1] = newUser;
Equally, you could make usercreation a factory methhod, but I'm trying to keep it simply.
You could...
Based on what you seem to be trying to do, is make the usercreation method into a class constructor, which would make more sense...
class user extends TextGame {
String username;
int password;
String[] notes;
public user(String username, int password) {
this.username=username;
this.password=password;
}
public void login(int password) {
this.password=password;
System.out.println("please type the password to proceed.");
if (scan.nextLine().equals(this.password)) {
System.out.println("logged in. type 'note' to access notes, or 'logoff' to log off this user.");
}
}
}
Then you could just do...
userarr[userarr.length+1] = new user(username,password);
You're also haven't create an instance of the userarr, so you're going to hit a NullPointerException
You should do something like...
userarr = new user[10];
before you try and use it.
This will allow you to maintain ten instances of the user class. You should also check to ensure that you've not exceeded the number of available elements in the array before you try and add new elements.
Have a look at the Arrays Trail for more details
I'd encourage you to have a look at Code Conventions for Java, which make it easier for other people to read your code and make it easier for you to read others

Before you use the usercreation method from a different class, you must first instantiate an object of usercreation class to gain access to it's properties(methods, variables) in a different class.
Example:
UserCreation us = new UserCreation (default constructor parameters);
userarr[userarr.length+1] = us.usercreation(username,password);

You should really be using a constructor:
public user(String username, int password) { // should be named User
this.username = username;
this.password = password;
}
Then you could add a user to an array like so:
userarr[userarr.length+1] = new User("username", 1); // doesn't work!
...Except, this code will fail because you didn't instantiate the array. And even if you had, you can't assign an element to an array like this. Arrays do not automatically re-size themselves, so when you try to add an element like this you will get an ArrayIndexOutOfBoundsException and your program will exit.

Firstly, I'd like to state that you haven't instantiated the userarr array yet. So, I recommend to do that first.
Secondly,new user.usercreation(username,password); usercreation is not a static inner class of user, so you can't create it.
Thirdly, usercreation is a method which returns void in which case you can't assign it to anything.
it seems that you want to invoke this method:
public void usercreation(String username, int password) {
this.username=username;
this.password=password;
}
In order to invoke that method, you must create an object of type user first in order to invoke it.
Example:
user user1 = new user();
user1.usercreation(username,password);
userarr[userarr.length+1] = user1;
However, the easiest way would be to make a constructor to initialise the username and password like below:
public user(String username, int password) {
this.username=username;
this.password=password;
}
Hence you can easily do this:
userarr[userarr.length+1] = new user(username,password);

char[] password getters and setters to a String

I have a typical char[] password variable with getters and setters. I have done this so that my password variable is never shown in plain text.
private String passwordAsString;
private char[] password;
public char[] getPassword() { return password; }
public void setPassword(char[] password) { this.password = password; }
I want to send my password to a database, but I believe it needs to be sent as a String type. So, I decided to create a new variable called PasswordAsString. Is there a better approach? This seems sloppy.
Secondly my getPasswordAsString is not returning the password as a String. What I have so far is:
public void setPasswordAsString(String passwordAsString) {
this.passwordAsString = passwordAsString;
}
public String getPasswordAsString() {
return password.toString();
}

The getter/setter for the password as a String could be implemented like this:
public void setPasswordAsString(String passwordAsString) {
this.password = passwordAsString.toCharArray();
}
public String getPasswordAsString() {
return new String(password);
}
These get and set the char[] password field directly – no need for a passwordAsString field. Convert from char[] to String with new String(char[]) and from String to char[] with String.toCharArray().
I'm pretty sure that passing around the password as a String like this defeats any security purpose for using a char array, though.

A char array's toString() method is not going to get you what you want. Instead, construct a new String:
public String getPasswordAsString() {
return new String(password);
}
Also, I would probably consider not including a setter for the passwordAsString field (or having a second field at all, for that matter), as this goes against what you are attempting to accomplish here.

The method Registration(String, String, String, String) is undefined for the type Client

if (type == "REGISTRATION"){
String name = json.getString("name");
String Location = json.getString("loc");
Client.Registration(username, password, name, Location); //error
DatabaseController.registerUser(Pobj, userObj);
}
Client.java
public static boolean Registration(String username, String password, String name, String loc){
clientUsername = username;
clientPassword = password;
clientname = name;
clientlocation = loc;
}
Registration function is defined here
it gives me error like:
method Registration(String, String, String, String) is undefined for the type Client

In java (and many other programming languages), your methods (or functions) have to have a return type. In your case, you declared the return type of your function to be boolean. This however means that this method must return a boolean. In your code, you have no return statement.
To solve the problem: you could either add a return statement, or change the return type to void, meaning it doesn't return anything.
Considering that you aren't returning anything in your function, I suggest using the second option, as follows:
public static void Registration(String username, String password, String name, String loc)
{ ... }
Also, as #Peadar Ó Duinnín mentioned, Java methods should be written in camel case, meaning the first word is not capitalized, but all the words after are, i.e. myFunctionThatDoesSomething(). This means your method should become registration(...)

Your Registration method (which should be registration. Method/Functions are camelCase in Java.) should be in your Client class as follows. You should also be returning a boolean or change the method signature to public static void registration(...
public class Client {
public static boolean registration(String username, String password, String name, String loc) {
clientUsername = username;
clientPassword = password;
clientName = name;
clientLocation = loc;
}
}

Java overriding static method

I have found myself in the need to override a static method, simply because it makes most sense, but I also know this is not possible.
The superclass, Entity.java:
abstract public class Entity<T> {
public Entity() {
//set up database connection
}
abstract public static Map<Object, T> getAll();
abstract public void insert();
abstract public void update();
protected void getData(final String query) {
//get data via database
}
protected void executeQuery(final String query) {
//execute sql query on database
}
}
One of the many concrete implementations, Account.java:
public class Account extends Entity<Account> {
private final static String ALL_QUERY = "SELECT * FROM accounts";
private final static String INSERT_QUERY = "INSERT INTO accounts (username, password) VALUES(?, ?)";
private final static String UPDATE_QUERY = "UPDATE accounts SET password=? WHERE username=?";
private String username;
private String password;
public Account(final String username, final String password) {
this.username = username;
this.password = password;
}
public String getUsername() {
return username;
}
public void setUsername(final String username) {
this.username = username;
}
public String getPassword() {
return password;
}
public void setPassword(final String password) {
this.password = password;
}
#Override
public static Map<Object, Account> getAll() {
//return a map using the ALL_QUERY string, calls getData(string);
}
#Override
public void insert() {
//insert this using INSERT_QUERY, calls executeQuery(string);
}
#Override
public void update() {
//update this using UPDATE_QUERY, calls executeQuery(string);
}
}
I haven't been going in depth explaining the code, but any general feedback on it would also be appreciated, I hope the comments explain enough.
So basically I think we can all agree that using Account.getAll() makes more sense over new Account().getAll() (if I would introduce a dummy syntax for it).
However I do want to have it extend the Entity class, currently it is only for convienience, but later on I may have to use sets/lists/multisets of Entity and perform an update() action on all of them, for example if I would build some queue that performances all updates every minute.
So well, is there a way to construct getAll() correctly?
Regards.

You could have separate classes for operations on all elements:
abstract public class Collection<T extends Entity<T>> {
abstract public static List<T> getAll();
public void printAll() {
// Print all entries of List obtained from getAll()
}
}
Which you could use as:
public class Accounts extends Collection<Account> {
#Override
public List<Account> getAll() {
//return a list using the ALL_QUERY string, calls getData(string);
}
}

It doesn't seems to me that it is really "simply because it makes most sense".
Tying persistence at your entity is not a good idea. There are already lots of patterns that give an appropriate design on this problem.
For example, in Domain Driven Design, "Persistence Ignorance" is what people trying to achieve. Consider making a Repository for each of your entity:
interface Repository<T> {
List<T> findAll();
void insert(T);
void update(T);
}
so you can override it by whatever way you want:
interface UserRepository extends Repository<User> {
// some other methods which is meaningful for User
User findByLoginName(String loginName);
}
class UserRepositoryImpl implements UserRepository {
List<User> findAll() {
// call whatever query
}
void insert(T){...}
void update(T){...}
User findByLoginName(String loginName) {...}
}
With a proper design and a component to handle the retrieval/storage of entity, you can have a less-persistence-coupled entity, and with repository that can perform proper "overriding".