Develop Reference

Getting No data when accessing SAP table with jCo(3.x))

Here is my sample code.In this example has only elementary types,no structure types has to set.But in the output no data exists in the table.
When I check the records in SAP it contains multiple records for this particular id.Can someone explain this to me?
public void invokeRFC(JCoDestination destination) {
JCoFunction function=null;
try
{
JCoFunctionTemplate functionTemplate = destination.getRepository().getFunctionTemplate("RFC_METHOD");
if (functionTemplate != null) {
function = functionTemplate.getFunction();
}
if (function == null)
throw new RuntimeException("Not found in SAP.");
//to fill elementary types and structures
configureImportParameters(function,"xxx", "abc");
//to fill table type parameters
configureTableParameters(function, "tblName",1,"100");
function.execute(destination);
} catch (JCoException e)
{
e.printStackTrace();
}
}
public void configureTableParameters(JCoFunction function, String table_name, int index, String id) {
JCoTable table = function.getTableParameterList().getTable("table_name");
table.appendRow();
table.setRow(index);
table.setValue("Partner", "100");
}
private void exportTable(JCoFunction jCoFunction, String tblName) {
JCoTable resultTable = jCoFunction.getTableParameterList().getTable(tblName);
int value = resultTable.getNumRows();
System.out.println(value);
}
private void configureImportParameters(JCoFunction function, String param1, String param2) {
JCoParameterList parameterList =
function.getImportParameterList();
parameterList.setValue("field1", param1);
parameterList.setValue("field2", param2);
}
UPDATED the code.

multiple problem can cause this.
if you setting "" or " " to fields. (when you set values better set if those has some values
if it says partner does not exist and if you sure its exist this mean your data does not pass properly. add debug points to where you set the data and make sure you pass correct name and correct values.
also you do not need to add(index) you can just table.appendRow(); // but this will not impact on your case
also when you setValue make sure its int filed. (normally not) in your given example its int
eg:
private void configureTableParameters(JCoParameterList tableParameters){
JCoTable jCoTable=tableParameters.getTable(key);
jCoTable.appendRow();
if(value!=null)
jCoTable.setValue(fieldKey,String.valueOf(value));
}
this is just psuda code and will not work

Test your ABAP remote function module with an SAP GUI via transaction code SE37 first.
If this test is successful and you get a different result if called from JCo with using the same parameter values, then I recommend to study SAP note 206068 for possible reasons.
Also check your method configureTableParameters. I guess, index shall be a field index and not a row count. Your implementation will create far too many unnecessary rows. I assume you wanted to call table.appendRow(); instead of table.appendRows(index);. Moreover, you maybe intended to fill the first field in the row with the value "100", for which you would have to pass the index value 0 instead of 1 in this case.

How to write large id's in SQL In ('','') format?

I have a method which returns record from a database based on a id's. Now I need to fetch records for 1000 records and I feel like it is not a good practice to write all id's in the in clause manually. Is there a tool where I paste the id's and it gives me like this id in ('123', '456', and so on)?
I can't write all the id's by my self manually.Business provides the id's in excel and they are very large in number. is there a way to accomplish this?
My method
#Override
public List<NYProgramTO> getLatestNYData() throws Exception {
String query = "SELECT REQ_XMl, SESSIONID, EXPIRATION_DATE, QUOTE_DATE, POLICY_EFFECTIVE_DATE, TARGET_CREATED, RATING_TRANSACTION_ID, SOURCE_LASTMODIFIED FROM dbo.XML_SESSIONS with (nolock) WHERE XML_SESSIONS.LOB = 'PersonalAuto' AND XML_SESSIONS.RATING_STATE = 'NY' AND XML_SESSIONS.ID IN ('72742212', '71289432') ORDER BY XML_SESSIONS.SOURCE_LASTMODIFIED DESC";
return this.sourceJdbcTemplate.query(query, (rs, rowNum) -> {
NYProgramTO to = new NYProgramTO();
to.setRequestXML(rs.getString("REQ_XML"));
to.setSessionId(rs.getString("SESSIONID"));
to.setExpirationDate(rs.getDate("EXPIRATION_DATE"));
to.setQuoteDate(rs.getString("QUOTE_DATE"));
to.setEffectiveDate(rs.getDate("POLICY_EFFECTIVE_DATE"));
to.setCreatedDate(rs.getDate("TARGET_CREATED"));
to.setRatingTransactionID(rs.getString("RATING_TRANSACTION_ID"));
to.setSourceLastModified(rs.getTimestamp("SOURCE_LASTMODIFIED"));
return to;
});
}
Thanks

Correct me if I understood wrong.
If you are looking for something which can give you concatenated value of a column separated with comma, then u should use excel.
There you can paste your data in a column and apply concatenation function available in excel to get you result.
You can refer This link

Sounds like a rather simple java solution. Why don't you write a java method that takes the ids as a string parameter and returns a String formatted the way you need it.
Given you have all ids in one long character sequence/string you can simply do this:
public String toSQLIds(String pRawIds){
String[] ids = pRawIds.split(Pattern.quote("SEPARATOR"));
StringBuilder builder = new StringBuilder();
builder.append("(");
for(int i = 0; i < ids.length; i++){
builder.append("'").append(ids[i]).append("'");
if(i < ids.length-1){
builder.append(", ");
}
}
builder.append(")");
return builder.toString();
}
Something like that. Of course you need the correct separator to successfully split your ids up and format it the way you need it. Run this once with your ids and paste it into your sql.

How Do You Check For DELETE/UPDATE Without A WHERE Clause

I currently have a listener that we use to do a few different monitoring-type activities (like log a warning if a query takes more than 5 seconds), but it also watches for and kills "silly bugs" -- especially UPDATE and DELETE queries that are missing a WHERE clause.
In the past we did the following (note that we are using com.foundationdb.sql):
/**
* Hook into the query execution lifecycle before rendering queries. We are checking for silly mistakes,
* pure SQL, etc.
*/
#Override
public void renderStart(final #NotNull ExecuteContext ctx) {
if (ctx.type() != ExecuteType.WRITE)
return;
String queryString = ctx.sql();
try (final Query query = ctx.query()) {
// Is our Query object empty? If not, let's run through it
if (!ValidationUtils.isEmpty(query)) {
queryString = query.getSQL(ParamType.INLINED);
final SQLParser parser = new SQLParser();
try {
final StatementNode tokens = parser.parseStatement(query.getSQL());
final Method method = tokens.getClass().getDeclaredMethod("getStatementType");
method.setAccessible(true);
switch (((Integer) method.invoke(tokens)).intValue()) {
case StatementType.UPDATE:
SelectNode snode = ConversionUtils.as(SelectNode.class,
((DMLStatementNode) tokens).getResultSetNode());
// check if we are a mass delete/update (which we don't allow)
if ((Objects.isNull(snode)) || (Objects.isNull(snode.getWhereClause())))
throw new RuntimeException("A mass update has been detected (and prevented): "
+ DatabaseManager.getBuilder().renderInlined(ctx.query()));
break;
case StatementType.DELETE:
snode = ConversionUtils.as(SelectNode.class,
((DMLStatementNode) tokens).getResultSetNode());
// check if we are a mass delete/update (which we don't allow)
if ((Objects.isNull(snode)) || (Objects.isNull(snode.getWhereClause())))
throw new RuntimeException("A mass delete has been detected (and prevented): "
+ DatabaseManager.getBuilder().renderInlined(ctx.query()));
break;
default:
if (__logger.isDebugEnabled()) {
__logger
.debug("Skipping query because we don't need to do anything with it :-): {}", queryString);
}
}
} catch (#NotNull StandardException | IllegalAccessException
| IllegalArgumentException | InvocationTargetException | NoSuchMethodException
| SecurityException e) {
// logger.error(e.getMessage(), e);
}
}
// If the query object is empty AND the SQL string is empty, there's something wrong
else if (ValidationUtils.isEmpty(queryString)) {
__logger.error(
"The ctx.sql and ctx.query.getSQL were empty");
} else
throw new RuntimeException(
"Someone is trying to send pure SQL queries... we don't allow that anymore (use jOOQ): "
+ queryString);
}
}
I really don't want to use yet another tool -- especially since most SQL parsers can't handle UPSERTs or the wide variety of queries that jOOQ can, so a lot just get cut out -- and would love to use jOOQ's constructs, but I'm having trouble. Ideally I could just check the query class and if it's an Update or Delete (or a subclass), I would just scream if it isn't an instance of UpdateConditionStep or DeleteConditionStep, but that doesn't work because the queries are coming back as UpdateQueryImpl... and without crazy reflection, I can't see if there is a condition in use.
So... right now I'm doing:
/**
* Hook into the query execution lifecycle before rendering queries. We are checking for silly mistakes, pure SQL,
* etc.
*/
#Override
public void renderStart(final #NotNull ExecuteContext ctx) {
if (ctx.type() != ExecuteType.WRITE)
return;
try (final Query query = ctx.query()) {
// Is our Query object empty? If not, let's run through it
if (!ValidationUtils.isEmpty(query)) {
// Get rid of nulls
query.getParams().entrySet().stream().filter(entry -> Objects.nonNull(entry.getValue()))
.filter(entry -> CharSequence.class.isAssignableFrom(entry.getValue().getDataType().getType()))
.filter(entry -> NULL_CHARACTER.matcher((CharSequence) entry.getValue().getValue()).find())
.forEach(entry -> query.bind(entry.getKey(),
NULL_CHARACTER.matcher((CharSequence) entry.getValue().getValue()).replaceAll("")));
if (Update.class.isInstance(query)) {
if (!UpdateConditionStep.class.isInstance(query)) {
if (!WHERE_CLAUSE.matcher(query.getSQL(ParamType.INDEXED)).find()) {
final String queryString = query.getSQL(ParamType.INLINED);
throw new RuntimeException(
"Someone is trying to run an UPDATE query without a WHERE clause: " + queryString);
}
}
} else if (Delete.class.isInstance(query)) {
if (!DeleteConditionStep.class.isInstance(query)) {
if (!WHERE_CLAUSE.matcher(query.getSQL(ParamType.INDEXED)).find()) {
final String queryString = query.getSQL(ParamType.INLINED);
throw new RuntimeException(
"Someone is trying to run a DELETE query without a WHERE clause: " + queryString);
}
}
}
} else
throw new RuntimeException(
"Someone is trying to send pure SQL queries... we don't allow that anymore (use jOOQ): "
+ ctx.sql());
}
}
This let's me get rid of the third party SQL parser, but now I'm using a regular expression on the non-inlined query looking for \\s[wW][hH][eE][rR][eE]\\s, which isn't ideal, either.
Is there a way to use jOOQ to tell me if an UPDATE, DELETE, has a WHERE clause?
Similarly, is there a way that let's me see what table the query is acting against (so that I can limit the tables someone can perform mutable actions against -- obviously that one wouldn't check if it's UPDATE or DELETE, instead using the ExecuteType)?

That's an interesting idea and approach. One problem I can see with it is performance. Rendering the SQL string a second time and then parsing it again sounds like a bit of overhead. Perhaps, this ExecuteListener should be active in development and integration test environments only, not in production.
Regarding your questions
Is there a way to use jOOQ to tell me if an UPDATE, DELETE, has a WHERE clause?
Since you seem to be open to use reflection to access a third party library's internals, well of course, you could check if the ctx.query() is of type org.jooq.impl.UpdateQueryImpl or org.jooq.impl.DeleteQueryImpl. In version 3.10.1, both of them have a private condition member, which you could check.
This will obviously break any time the internals are changed, but it might be a pragmatic solution for now.
Similarly, is there a way that let's me see what table the query is acting against
A more general and more robust approach would be to implement a VisitListener, which is jOOQ's callback that is called during expression tree traversal. You can hook into the generation of the SQL string and the collection of bind variables, and throw your errors as soon as you encounter:
An UPDATE or DELETE statement
... without a WHERE clause
... updating a table from a specific set of tables
You "just" have to implement a stack machine that remembers all of the above things prior to throwing the exception. An example of how VisitListener can be implemented is given here:
https://blog.jooq.org/2015/06/17/implementing-client-side-row-level-security-with-jooq
New feature in the future
This kind of feature has been discussed a couple of times on the mailing list as well. It's a low hanging fruit to support by jOOQ natively. I've created a feature request for jOOQ 3.11, for this:
https://github.com/jOOQ/jOOQ/issues/6771

How to get all of the subjects of a Jena Query?

Suppose I have some jena query object :
String query = "SELECT * WHERE{ ?s <some_uri> ?o ...etc. }";
Query q = QueryFactory.create(query, Syntax.syntaxARQ);
What would be the best way to get all of the subjects of the triples in the query? Preferably without having to do any string parsing/manipulation manually.
For example, given a query
SELECT * WHERE {
?s ?p ?o;
?p2 ?o2.
?s2 ?p3 ?o3.
?s3 ?p4 ?o4.
<http://example.com> ?p5 ?o5.
}
I would hope to have returned some list which looks like
[?s, ?s2, ?s3, <http://example.com>]
In other words, I want the list of all subjects in a query. Even having only those subjects which were variables or those which were literals/uris would be useful, but I'd like to find a list of all of the subjects in the query.
I know there are methods to return the result variables (Query.getResultVars) and some other information (see http://jena.apache.org/documentation/javadoc/arq/com/hp/hpl/jena/query/Query.html), but I can't seem to find anything which will get specifically the subjects of the query (a list of all result variables would return the predicates and objects as well).
Any help appreciated.

Interesting question. What you need to do is go through the query, and for each block of triples iterate through and look at the first part.
The most robust way to do this is via an element walker which will go through each part of the query. It might seem over the top in your case, but queries can contain all sorts of things, including FILTERs, OPTIONALs, and nested SELECTs. Using the walker means that you can ignore that stuff and focus on only what you want:
Query q = QueryFactory.create(query); // SPARQL 1.1
// Remember distinct subjects in this
final Set<Node> subjects = new HashSet<Node>();
// This will walk through all parts of the query
ElementWalker.walk(q.getQueryPattern(),
// For each element...
new ElementVisitorBase() {
// ...when it's a block of triples...
public void visit(ElementPathBlock el) {
// ...go through all the triples...
Iterator<TriplePath> triples = el.patternElts();
while (triples.hasNext()) {
// ...and grab the subject
subjects.add(triples.next().getSubject());
}
}
}
);

It might be too late but another way is to make use of Jena ARQ libraries and create Algebra of the given query. Once the algebra is created, it can be compiled and you can traverse through all the triples (given in the where clause). Here is the code, I hope it helps:
Query query = qExec.getQuery(); //qExec is an object of QueryExecutionFactory
// Generate algebra of the query
Op op = Algebra.compile(query);
CustomOpVisitorBase opVisitorBase = new CustomOpVisitorBase();
opVisitorBase.opVisitorWalker(op);
List<Triple> queryTriples = opVisitorBase.triples;
CustomOpVisitor class is given below:
public class CustomOpVisitorBase extends OpVisitorBase {
List<Triple> triples = null;
void opVisitorWalker(Op op) {
OpWalker.walk(op, this);
}
#Override
public void visit(final OpBGP opBGP) {
triples = opBGP.getPattern().getList();
}
}
Traverse through the list of Triples and make use of given property functions such as triple.getSubject() etc etc.

How to get distinct results in hibernate with joins and row-based limiting (paging)?

I'm trying to implement paging using row-based limiting (for example: setFirstResult(5) and setMaxResults(10)) on a Hibernate Criteria query that has joins to other tables.
Understandably, data is getting cut off randomly; and the reason for that is explained here.
As a solution, the page suggests using a "second sql select" instead of a join.
How can I convert my existing criteria query (which has joins using createAlias()) to use a nested select instead?

You can achieve the desired result by requesting a list of distinct ids instead of a list of distinct hydrated objects.
Simply add this to your criteria:
criteria.setProjection(Projections.distinct(Projections.property("id")));
Now you'll get the correct number of results according to your row-based limiting. The reason this works is because the projection will perform the distinctness check as part of the sql query, instead of what a ResultTransformer does which is to filter the results for distinctness after the sql query has been performed.
Worth noting is that instead of getting a list of objects, you will now get a list of ids, which you can use to hydrate objects from hibernate later.

I am using this one with my code.
Simply add this to your criteria:
criteria.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY);
that code will be like the select distinct * from table of the native sql.

A slight improvement building on FishBoy's suggestion.
It is possible to do this kind of query in one hit, rather than in two separate stages. i.e. the single query below will page distinct results correctly, and also return entities instead of just IDs.
Simply use a DetachedCriteria with an id projection as a subquery, and then add paging values on the main Criteria object.
It will look something like this:
DetachedCriteria idsOnlyCriteria = DetachedCriteria.forClass(MyClass.class);
//add other joins and query params here
idsOnlyCriteria.setProjection(Projections.distinct(Projections.id()));
Criteria criteria = getSession().createCriteria(myClass);
criteria.add(Subqueries.propertyIn("id", idsOnlyCriteria));
criteria.setFirstResult(0).setMaxResults(50);
return criteria.list();

A small improvement to #FishBoy's suggestion is to use the id projection, so you don't have to hard-code the identifier property name.
criteria.setProjection(Projections.distinct(Projections.id()));

The solution:
criteria.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY);
works very well.

session = (Session) getEntityManager().getDelegate();
Criteria criteria = session.createCriteria(ComputedProdDaily.class);
ProjectionList projList = Projections.projectionList();
projList.add(Projections.property("user.id"), "userid");
projList.add(Projections.property("loanState"), "state");
criteria.setProjection(Projections.distinct(projList));
criteria.add(Restrictions.isNotNull("this.loanState"));
criteria.setResultTransformer(Transformers.aliasToBean(UserStateTransformer.class));
This helped me :D

if you want to use ORDER BY, just add:
criteria.setProjection(
Projections.distinct(
Projections.projectionList()
.add(Projections.id())
.add(Projections.property("the property that you want to ordered by"))
)
);

I will now explain a different solution, where you can use the normal query and pagination method without having the problem of possibly duplicates or suppressed items.
This Solution has the advance that it is:
faster than the PK id solution mentioned in this article
preserves the Ordering and don’t use the 'in clause' on a possibly large Dataset of PK’s
The complete Article can be found on my blog
Hibernate gives the possibility to define the association fetching method not only at design time but also at runtime by a query execution. So we use this aproach in conjunction with a simple relfection stuff and can also automate the process of changing the query property fetching algorithm only for collection properties.
First we create a method which resolves all collection properties from the Entity Class:
public static List<String> resolveCollectionProperties(Class<?> type) {
List<String> ret = new ArrayList<String>();
try {
BeanInfo beanInfo = Introspector.getBeanInfo(type);
for (PropertyDescriptor pd : beanInfo.getPropertyDescriptors()) {
if (Collection.class.isAssignableFrom(pd.getPropertyType()))
ret.add(pd.getName());
}
} catch (IntrospectionException e) {
e.printStackTrace();
}
return ret;
}
After doing that you can use this little helper method do advise your criteria object to change the FetchMode to SELECT on that query.
Criteria criteria = …
// … add your expression here …
// set fetchmode for every Collection Property to SELECT
for (String property : ReflectUtil.resolveCollectionProperties(YourEntity.class)) {
criteria.setFetchMode(property, org.hibernate.FetchMode.SELECT);
}
criteria.setFirstResult(firstResult);
criteria.setMaxResults(maxResults);
criteria.list();
Doing that is different from define the FetchMode of your entities at design time. So you can use the normal join association fetching on paging algorithms in you UI, because this is most of the time not the critical part and it is more important to have your results as quick as possible.

Below is the way we can do Multiple projection to perform Distinct
package org.hibernate.criterion;
import org.hibernate.Criteria;
import org.hibernate.Hibernate;
import org.hibernate.HibernateException;
import org.hibernate.type.Type;
/**
* A count for style : count (distinct (a || b || c))
*/
public class MultipleCountProjection extends AggregateProjection {
private boolean distinct;
protected MultipleCountProjection(String prop) {
super("count", prop);
}
public String toString() {
if(distinct) {
return "distinct " + super.toString();
} else {
return super.toString();
}
}
public Type[] getTypes(Criteria criteria, CriteriaQuery criteriaQuery)
throws HibernateException {
return new Type[] { Hibernate.INTEGER };
}
public String toSqlString(Criteria criteria, int position, CriteriaQuery criteriaQuery)
throws HibernateException {
StringBuffer buf = new StringBuffer();
buf.append("count(");
if (distinct) buf.append("distinct ");
String[] properties = propertyName.split(";");
for (int i = 0; i < properties.length; i++) {
buf.append( criteriaQuery.getColumn(criteria, properties[i]) );
if(i != properties.length - 1)
buf.append(" || ");
}
buf.append(") as y");
buf.append(position);
buf.append('_');
return buf.toString();
}
public MultipleCountProjection setDistinct() {
distinct = true;
return this;
}
}
ExtraProjections.java
package org.hibernate.criterion;
public final class ExtraProjections
{
public static MultipleCountProjection countMultipleDistinct(String propertyNames) {
return new MultipleCountProjection(propertyNames).setDistinct();
}
}
Sample Usage:
String propertyNames = "titleName;titleDescr;titleVersion"
criteria countCriteria = ....
countCriteria.setProjection(ExtraProjections.countMultipleDistinct(propertyNames);
Referenced from https://forum.hibernate.org/viewtopic.php?t=964506

NullPointerException in some cases!
Without criteria.setProjection(Projections.distinct(Projections.property("id")))
all query goes well!
This solution is bad!
Another way is use SQLQuery. In my case following code works fine:
List result = getSession().createSQLQuery(
"SELECT distinct u.id as usrId, b.currentBillingAccountType as oldUser_type,"
+ " r.accountTypeWhenRegister as newUser_type, count(r.accountTypeWhenRegister) as numOfRegUsers"
+ " FROM recommendations r, users u, billing_accounts b WHERE "
+ " r.user_fk = u.id and"
+ " b.user_fk = u.id and"
+ " r.activated = true and"
+ " r.audit_CD > :monthAgo and"
+ " r.bonusExceeded is null and"
+ " group by u.id, r.accountTypeWhenRegister")
.addScalar("usrId", Hibernate.LONG)
.addScalar("oldUser_type", Hibernate.INTEGER)
.addScalar("newUser_type", Hibernate.INTEGER)
.addScalar("numOfRegUsers", Hibernate.BIG_INTEGER)
.setParameter("monthAgo", monthAgo)
.setMaxResults(20)
.list();
Distinction is done in data base! In opposite to:
criteria.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY);
where distinction is done in memory, after load entities!