Pig: UDF not returning expected resultset - java

This is the sample data on which i was working:
Peter Wilkerson 27 M
James Owen 26 M
Matt Wo 30 M
Kenny Chen 28 M
I created a simple UDF for filtering the age like this:
public class IsApplicable extends FilterFunc {
#Override
public Boolean exec(Tuple tuple) throws IOException {
if(tuple == null || tuple.size() > 0){
return false;
}
try {
Object object = tuple.get(0);
if(object == null){
return false;
}
int age = (Integer)object;
return age > 28;
} catch (Exception e) {
throw new IOException(e);
}
}
}
This is the Script I used for using this UDF:
records = LOAD '~/Documents/data.txt' AS (firstname:chararray,lastname:chararray,age:int,gender:chararray);
filtered_records = FILTER records BY com.udf.IsApplicable(age);
dump filtered_records;
Dumping does not display any record. Please let me know where I missed.

tuple.size() > 0 condition is always true in the if stmt, so it will never go to the try block(ie filtering logic), that is the reason you are getting empty result. Can you change the if condition like this?
System.out.println("TupleSize="+tuple.size());
if(tuple == null || tuple.size() < 0){
return false;
}
Sample debug output in console:
2015-02-13 07:40:46,994 [Thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map - Aliases being processed per job phase (AliasName[line,offset]): M: records[3,10],records[-1,-1],filtered_records[4,19] C: R:
TupleSize=1
TupleSize=1
TupleSize=1

This is returning false for all of the rows:
if (tuple == null || tuple.size() > 0) {
return false;
}
This is fetching the userName and not age:
Object object = tuple.get(0);

Related

I am having trouble when I run the java application and it can't execute as I expect

My application cannot find the roommate I added to the room, see sequence:
Please select one of the following options:
Add a room in the apartment[1]
Search for a room in the apartment[2]
Add a roommate to an existing room[3]
Check if a room has a roommate[4]
Count the number of existing rooms that have roommates living in them[5]
3
What is the name of this roommate?
kack
What is the surname of thid roommata?
han
What is the age of this roommate?
20
Which room you want to add this roommate in?[A]/[B]/[C]
b
The roommate was added to the room!
Do you want to continue using the App? (Y/N)
y
Please select one of the following options:
Add a room in the apartment[1]
Search for a room in the apartment[2]
Add a roommate to an existing room[3]
Check if a room has a roommate[4]
Count the number of existing rooms that have roommates living in them[5]
4
What is the ID of the room?[A]/[B]/[C]
b
Sorry! There is no roommate in this room.
I am not sure where I have to rewrite. I want to know which part is wrong and how I should rewrite my code. The code is following, and the first part is to add roommate and the second part is to check if there is a roommate in the room:
public boolean addRoommate(String pName, String pSurname, int pAge, char pID) {
boolean response = false;
if (roomA != null && roomA.getID() == (pID)) {
Roommate newRoommate = new Roommate(pName, pSurname, pAge);
response = true;
} else if (roomB != null && roomB.getID() == (pID)) {
Roommate newRoommate = new Roommate(pName, pSurname, pAge);
response = true;
} else if (roomC != null && roomC.getID() == (pID)) {
Roommate newRoommate = new Roommate(pName, pSurname, pAge);
response = true;
}
return response;
}
Here is the second part of the code, which is where I check for roommate:
public Roommate checkRoommate(char pID) {
Roommate response = null;
if (roomA != null && roomA.getID() == (pID) && roomA.getRoommate() != null) {
response = roomA.getRoommate();
} else if (roomB != null && roomB.getID() == (pID) && roomB.getRoommate() != null) {
response = roomB.getRoommate();
} else if (roomC != null && roomC.getID() == (pID) && roomC.getRoommate() != null) {
response = roomC.getRoommate();
}
return response;
}
You need to put the rommate into a rool, for example this way:
roomA.setRoomMate(newRoommate)

Best way to check specific result of 2 methods in Java

I have the following method
public Message JavaMethod(String id1, String id2)
In which I need to call a Dao class's method to verify that an user with the provided Id exist, and if it does not, create a message detailing the Id that couldn't be found on the database with the following method:
createMessage("Message string",Enum.TYPE,IdofMissingUser);
At first I thought of doing it like this:
public Message JavaMethod(String id1, String id2) {
if(Dao.findUser(id1) == null || Dao.findUser(id2) == null){
return createMessage("Error",Enum.Error,id1);
}else{
//do some other stuff
}
}
But obviously this way I won't know which of the ids has not been found.
So I went ahead and created an ugly if else cycle:
public Message JavaMethod(String id1, String id2) {
if (Dao.findUser(id1) == null) {
return createMessage("Error", Enum.Error, id1);
} else if (Dao.findUser(id2) == null) {
return createMessage("Error", Enum.Error, id2);
} else {
// Do stuff after veryfing users exists
return createMessage("All OK", Enum.OK, messageData);
}
}
But I'm not feeling really confident that this is the best solution for this basic issue.
What would you guys recommend in this case?
You could wrap the ids in a list and use a for loop:
public Message someMethod(String id1, String id2) {
for (String id: Arrays.asList(id1, id2)) {
if (Dao.findUser(id) == null) {
return createMessage("Error", Enum.Error, id);
}
}
// Do stuff after verifying users exists
return createMessage("All OK", Enum.OK, messageData);
}
If you're only ever going to have two IDs, you could deal with a shorthand boolean. Question is whether that makes it less readable though. E.g.
public Message JavaMethod(String id1, String id2) {
User user1 = Dao.findUser(id1);
User user2 = Dao.findUser(id2);
if(user1 == null || user2 == null){
return createMessage("Error",Enum.Error,user1 == null ? id1 : id2);
}else{
//do some other stuff
}
}
This also doesn't deal with if both of the IDs were null, for that you could extend it:
public Message JavaMethod(String id1, String id2) {
User user1 = Dao.findUser(id1);
User user2 = Dao.findUser(id2);
if(user1 == null || user2 == null){
return createMessage("Error",Enum.Error,user1 == null && user2 == null? both : user1 == null ? id1 : id2);
}else{
//do some other stuff
}
}
You'd need to define what you would return for the both variable
More details on the shorthand boolean annotation can be found here

How build sql query with many parameters from java code?

I have sql select with parameters:
SELECT * FROM tbl t WHERE t.name = ? AND t.age = ? AND t.number = ? AND ... AND t.last_parameter = ? order by t.some desc //many parameterss
I get parameters from form's fields and some fields may be empty. I build sql string:
String sqlStatementText;
MessageFormat sqlStatementTextTemplate = new MessageFormat(Queries.WAR_GET_REPORT_COUNT);
List<Object> parametrs = new ArrayList<>();
if (null == subscriberMSISDN || subscriberMSISDN.length() == 0) {
parametrs.add("");
} else {
parametrs.add(Queries.WAR_REPORT_CALLING_NUMBER);
}
if (null == operatorID || operatorID.length() == 0) {
parametrs.add("");
} else {
parametrs.add(Queries.WAR_REPORT_OPERATOR_AVAYA_ID);
}
if (null == operatorNickname || operatorNickname.length() == 0) {
parametrs.add("");
} else {
parametrs.add(Queries.WAR_REPORT_NICKNAME);
}
if (null == msg1 || msg1.length() == 0) {
parametrs.add("");
} else {
parametrs.add(Queries.WAR_REPORT_MSG1);
}
if (null == msg2 || msg2.length() == 0) {
parametrs.add("");
} else {
parametrs.add(Queries.WAR_REPORT_MSG2);
}
sqlStatementText = sqlStatementTextTemplate.format(parametrs.toArray());
ant them i do it:
try (Connection sqlConnection = connectionPool.getConnection();
PreparedStatement sqlStatment = sqlConnection.prepareStatement(sqlStatementText)) {
int paramID = 1;
sqlStatment.setInt(paramID++, 1);
sqlStatment.setDate(paramID++, new java.sql.Date(fromDate.getTime()));
sqlStatment.setDate(paramID++, new java.sql.Date(toDate.getTime()));
if (null != subscriberMSISDN && subscriberMSISDN.length() != 0) {
sqlStatment.setString(paramID++, subscriberMSISDN);
}
if (null != operatorID && operatorID.length() != 0) {
sqlStatment.setString(paramID++, operatorID);
}
if (null != operatorNickname && operatorNickname.length() != 0) {
sqlStatment.setString(paramID++, operatorNickname);
}
if (null != msg1 && msg1.length() != 0) {
sqlStatment.setString(paramID++, msg1);
}
if (null != msg2 && msg2.length() != 0) {
sqlStatment.setString(paramID++, msg2);
}
try (ResultSet resultSet = sqlStatment.executeQuery()) {
while (resultSet.next()) {
count = resultSet.getInt(1);
}
resultSet.close();
sqlStatment.close();
sqlConnection.close();
}
But i thig it not correctly. But I dont know how build sql query with many paramaters and if some parameters maybe empty.
Switch to an ORM. They will have some form of criteria-like object.
Use the param is null or column = param SQL syntax. select x from y where (? is null OR column1 = ?)
You need to set the value of the param twice, and the input value can not legitimately be null.
There is no way to do it, given the SQL statement you have.
You need to change the SQL statement WHERE conditions from things like t.name = ? to t.name = nvl(?, t.name). Then, you can bind a NULL there and the condition will always evaluate to true (so it's not acting as a filter -- which is what you want when the user leaves the field blank).
Or -- a better approach if you can do it, it's even better to use conditions like you've got them (e.g., t.name= ?), but build the conditions dynamically based on what fields the user give you. That is, for example, if the user leaves the "name" parameter blank, just omit the t.name = ? condition entirely.
That leaves you with a shorter SQL statement that makes the Oracle optimizer's job a little bit easier. With the t.name = nvl(?, t.name) approach I gave you above, you're relying on some pretty advanced optimizer features to get the best performance, because it's not immediately clear whether, say, it would be good or bad for the optimizer to use an index on t.name.

userobjs cannot be resolved to a variable

I am getting this error.Return type is mismatched
public List<EmpRecord> Empdata(String uname) throws Exception {
System.out.println("Inside into service class2");
try {
#SuppressWarnings("unchecked")
List<EmpRecord> userObjs = hibernateTemplate.find("from EmpRecord u where u.uname=? ",uname);
if(userObjs.size() != 0) {
System.out.println(" Employee Name : " + userObjs.get(0).getEmpName());
}
return userobjs;
Typing error. Thats all there is to it. Use a capital O in userObjs in the return statement
change return userobjs; to return return userObjs;
I would also change if(userObjs.size() != 0) to if(userObjs!=null && userObjs.size() != 0) for safety reason.

Ormlite Where issue with parenthesis, building meta-layer query library

I've read the other posts and the docs about how to use the "Where" clause to "create" parenthesis statements.
My requirement is simple:
... WHERE companyID=1 AND (director=true OR officer=true) ;
I'm writing a routine that takes an array of Object, which are then parsed into an Ormlite Where call. a typical call looks like this:
.., "companyID", 1, Q.AND, Q.Bracket, "director", true, Q.OR, "officer", true, Q.Bracket)
The intent is to speed up simple queries. There is no desire to replace Ormlite's querying tools. This is a simple meta-layer on top.
Everything works fine for simple queries, since the parameters are processed sequentially and the where clause is built incrementally.
For parenthesis I am postponing the processing until the bracket is closed.
This is where I am having a problem. The example from the docs I am using shows this:
-- From the OrmLite docs...
Where<Account, String> where = queryBuilder.where();
where.or(
where.and(
where.eq(Account.NAME_FIELD_NAME, "foo"),
where.eq(Account.PASSWORD_FIELD_NAME, "_secret")),
where.and(
where.eq(Account.NAME_FIELD_NAME, "bar"),
where.eq(Account.PASSWORD_FIELD_NAME, "qwerty")));
This produces the following approximate SQL:
SELECT * FROM account
WHERE ((name = 'foo' AND password = '_secret')
OR (name = 'bar' AND password = 'qwerty'))
The key thing I understand from the docs example, is that the same where instance is used in the nested and(...) call. This is precisely what I'm doing but I'm still getting a "Did you forget an AND or an OR" message.
The code implementing the "delayed" processing looks like this:
#SuppressWarnings("unchecked")
private void processWhere(Where<?, ?> where, Q q, List<QValue> list)
{
if (null == list || list.size() < 2)
{
System.err.println("Invalid where passed: " + list);
return;
}
if (q.equals(Q.AND))
where.and(getCondition(where, list.get(0)), getCondition(where, list.get(1)));
else
where.or(getCondition(where, list.get(0)), getCondition(where, list.get(1)));
}
The "QValue" item is just a "holder" for column, condition and value data.
The "getCondition" method is as follows:
#SuppressWarnings("rawtypes")
protected Where getCondition(Where<?, ?> where, QValue qv)
{
if (null != where && null != qv)
return getCondition(where, qv.getType(), qv.getColumn(), qv.getValue(), qv.getValue2());
else
return null;
}
#SuppressWarnings("rawtypes")
protected Where getCondition(Where<?, ?> where, Q cond, String key, Object val, Object val2)
{
if (null == where || null == cond || null == key || null == val)
return null;
SelectArg arg = new SelectArg();
arg.setValue(val);
try
{
switch (cond)
{
case NotNull:
where.isNotNull(key);
break;
case IsNull:
where.isNull(key);
break;
case Equals:
where.eq(key, arg);
break;
case NotEqual:
where.ne(key, arg);
break;
case GreaterThan:
where.gt(key, arg);
break;
case LessThan:
where.lt(key, arg);
break;
case Like:
arg.setValue("%" + val + "%");
where.like(key, arg);
break;
case LikeStart:
arg.setValue("" + val + "%");
where.like(key, arg);
break;
case LikeEnd:
arg.setValue("%" + val);
where.like(key, arg);
break;
case Between:
if (null != val && null != val2)
where.between(key, val, val2);
break;
default:
where.eq(key, arg);
break;
}
}
catch (SQLException e)
{
GlobalConfig.log(e, true);
return null;
}
return where;
}
As far as I can tell, I'm using the Where object correctly, but I am still getting a:
"Did you forget an AND or OR?" message.
I've tried creating "new" Where clauses with QueryBuilder:
Where w1 = qb.where() ;
//process w1 conditions...
return where.query() ;
Which also fails or generates incorrect SQL in the various combinations I've tried. Any suggestions on how to get the and(...) and or(...) methods working properly will be greatly appreciated.
BTW once the library is working properly, I'll put it up as Open Source or donate it to Gray, or both.
Thanks in advance.
Anthony
I faced the same issue and solved it like this:
where.eq("companyID", 1);
where.and(where, where.or(where.eq("director", true), where.eq("officer", true)));
or
where.and(where.eq("companyID", 1), where.or(where.eq("director", true), where.eq("officer", true)));
which in SQL gives us:
((`companyID` = 1 AND `director` = 1 ) OR (`companyID` = 1 AND `officer` = 1 ))
It's not identical to your example clause WHERE companyID=1 AND (director=true OR officer=true) but has the same meaning.

Categories

Resources