Is any possibility to execute parametrized N1QL query with turned off adhoc flag using annotation #Query?
I.e. for given query:
#Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter} and author = $author")
List<Comment> getCommentsByAuthor(#Param("author") String author);
If not, is any other way to force couchbase to use secondary index (in this example index on author field) in annotation queries?
You seem to mix a few things:
Parametrized Query will work with the #Query annotation as in your snippet. The documentation mentions it (below the 1st warning block in this section)
N1QL secondary indexes should be automatically picked up by N1QL, depending on the statement. The subtlety here is that n1ql.filter. SDC needs that to limit such queries to the correct set of documents in an heterogeneous bucket.
adhoc is something else: it is about prepared statements. SDC doesn't use that feature, and it will only produce N1QL query with the default value for adhoc (which is still true as far as I know).
If you've logged the query produced by this annotation and run an EXPLAIN on it to effectively see that the index is not picked up, maybe try inverting the two expressions in the WHERE clause?
Related
I'm extending on my last question I asked about jOOQ. In the Hibernate models the #Filter annotation gets used, and I want to apply this same 'default filter' to the jOOQ queries. As I'm passing a jOOQ query to the nativeQuery(org.jooq.Query query, Class<E> type) I was wondering if it's possible to extract the table (TableImpl<?,?>) used from the FROM clause in the jOOQ query (org.jooq.Query).
This is what I've tried:
private static <E> SelectConditionStep<Record> applyDefaultFilters(Class<E> type, SelectConditionStep<Record> query)
{
if (BaseOrganizationModel.class.isAssignableFrom(type)) {
query
.getQuery()
.addConditions(
query
.getQuery()
.asTable()
.field("organization_id", Long.class)
.eq(currentOrganization().id));
if (SoftDeletableModel.class.isAssignableFrom(type)) {
query
.getQuery()
.addConditions(query.getQuery().asTable().field("deleted", Boolean.class).eq(false));
}
}
return query;
}
The result is this SQL, which is not what I want. I want it to filter the corresponding table.
select distinct `EventGroup`.*
from `EventGroup`
where (
...
and `alias_100341773`.`organization_id` = ?
and `alias_17045196`.`deleted` = ?
)
I want this
select distinct `EventGroup`.*
from `EventGroup`
where (
...
and `EventGroup`.`organization_id` = ?
and `EventGroup`.`deleted` = ?
)
Is this possible at all? And if not, what possible other routes are there? (aside from the obvious passing the table to the function)
Using jOOQ 3.16 query object model API
jOOQ 3.16 introduced a new, experimental (as of 3.16) query object model API, which can be traversed.
On any Select, just call Select.$from() to access an unmodifiable view of the contained table list.
An alternative, dynamic SQL approach for the ad-hoc case
Every time you're trying to mutate an existing query, ask yourself, is there a more elegant way to do this using a more functional, immutable approach do dynamic SQL? Rather than appending your additional predicates to the query, why not produce predicates from a function?
private static Condition defaultFilters(Class<?> type, Table<?> table) {
Condition result = noCondition();
if (BaseOrganizationModel.class.isAssignableFrom(type)) {
result = result.and(table.field("organization_id", Long.class)
.eq(currentOrganization().id));
if (SoftDeletableModel.class.isAssignableFrom(type))
result = result.and(not(table.field("deleted", Boolean.class)))
}
return result;
}
And now, when you construct your query, you can add the filters:
ctx.select(T.A, T.B)
.from(T)
.where(T.X.eq(1))
.and(defaultFilters(myType, T))
.fetch();
A generic way to transform your SQL
If you really want to mutate your query (e.g. in a utility for all queries), then a transformation approach might be better suited. There are different ways to approach this.
Using views
Some RDBMS can access session variables in views. In Oracle, you'd be setting some SYS_CONTEXT variable to your organization_id inside of a view, and then query only the (possibly updatable) views instead of the tables directly. MySQL unfortunately can't do the equivalent thing, see Is there any equivalent to ORACLE SYS_CONTEXT('USERENV', 'OS_USER') in MYSQL?
I've described this approach here in this blog post. The advantage of this approach is that you will never forget to set the predicate (you can validate your view source code with CI/CD tests), and if you ever forget to set the session context variable, the view will just not return any data, so it's quite a secure approach.
Together with the WITH CHECK OPTION clause, you can even prevent insertions into wrong organization_id, which improves security.
Using a VisitListener in jOOQ
This is the most powerful approach to do this in jOOQ, and exactly what you want, but also quite a tricky one to get right for all edge cases. See this post about implementing row level security in jOOQ. Starting from jOOQ 3.16, there will be better ways to transform your SQL via https://github.com/jOOQ/jOOQ/issues/12425.
Note, it won't work for plain SQL templates that do not use any jOOQ query parts, nor for JDBC based queries or other queries that you may have in your system, so be careful with this approach as you might leak data from other organisations.
Of course, you could implement this step also on the JDBC layer, using jOOQ's ParsingConnection or ParsingDataSource, that way you can intercept also third party SQL and append your predicates.
This can work for all DML statements, including UPDATE, DELETE. It's a bit harder for INSERT, as you'd have to transform INSERT .. VALUES into INSERT .. SELECT, or throw an exception if someone wants to insert into the wrong organization_id.
Using a ExecuteListener in jOOQ
A bit more hackish than the above VisitListener approach, but generally easier to get right, just regex-replace the WHERE clause of all your statements by WHERE organization_id = ... AND in an ExecuteListener.
To play it safe, you could reject all queries without a WHERE clause, or do some additional trickery to add the WHERE clause at the right place in case there isn't already one.
Using jOOQ's equivalent of Hibernate's #Filter
jOOQ's equivalent of Hibernate's #Filter is the Table.where(Condition) clause. It's not an exact equivalent, you'd have to prevent direct access to T in your code base and make sure users access T only via a method that replaces T by T.where(defaultFilters(myType, T)) instead.
This approach currently loses type safety of the T table, see: https://github.com/jOOQ/jOOQ/issues/8012
Is there any difference when using Spring Data JPA keywords between:
List<SomeEntity> findBySomeCondition();
and
List<SomeEntity> findAllBySomeCondition();
No, there is no difference between them, they will execute exactly the same query, the All part is ignored by Spring Data when deriving the query from the method name. The only important bit is the By keyword, anything following it is treated as a field name (with the exception of other keywords like OrderBy which incidentially can lead to some strange looking method names like findAllByOrderByIdAsc).
This means something like this is perfectly valid:
List<SomeEntity> findAnythingYouWantToPutHereBySomeCondition();
And will execute exactly the same SQL query as:
List<SomeEntity> findBySomeCondition();
or
List<SomeEntity> findAllBySomeCondition();
The documentation for the 2.3.6 release of Spring Data discusses this feature:
Any text between find (or other introducing keywords) and By is considered to be descriptive unless using one of the result-limiting keywords such as a Distinct to set a distinct flag on the query to be created or Top/First to limit query results.
The purpose of feature was explained in a blog post about the then-upcoming 2.0 release of Spring Data:
Spring Data’s method parsing uses prefix keywords like find, exists, count, and delete and a terminating By keyword. Everything you put in between find and By makes your method name more expressive and does not affect query derivation.
To illustrate the difference lets look at the two functions:
1. Set<Policy> findAllByRoleIn(Iterable<Role> role);
2. Set<Policy> findByRoleIn(Iterable<Role> role);
The query generated by 1st function:
1. select policy.id, policy.role from policy where (policy.role in (? , ? , ? , ?))
The query generated by 2nd function:
2. select policy.id, policy.role from policy where (policy.role in (? , ? , ? , ?))
Conclusion: Clearly, if we look at the queries generated by both functions. We can clearly see, there is no difference between the two function definitions, they execute exactly the same query.
one difference is that with findAllBy Hibernate filters (#Filters from org.hibernate.annotations) are applied and so a different sql.
Actually, the difference between findallBy and findby, is that :
findAllBy returns a Collection but
findBy returns Optional.
so it's preferable to write List findAllBy instead of writing List findBy (but it will work also :p).
and to write Optional findBy instead of Optional findAllBy.
check this doc https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#repositories.core-concepts
findBy method is used if we want to find by name or some other criteria like findByFirstName(String firstName);
findAll methods generally finds by providing specification
List<T> findAll(Specification<T> spec);
Please see docs below for more clarity:
http://docs.spring.io/spring-data/jpa/docs/1.4.3.RELEASE/reference/html/jpa.repositories.html
While building a query using Hibernate, I noticed something rather odd. If I use sequential named parameters for the ORDER BY clause, Hibernate throws a QuerySyntaxException (the colon prefix being an unexpected token):
createQuery("FROM MyEntity ORDER BY :orderProperty :orderDirection");
However, when this is done with a plain SQL query the query is created without a problem:
createSQLQuery("SELECT * FROM my_entity_table ORDER BY :orderProperty :orderDirection");
I know Hibernate is doing more String evaluation for the HQL query, which is probably why the SQL query is created without an error. I am just wondering why Hibernate would care that there are two sequential named parameters.
This isn't a huge issue since it is simple to work around (can just append the asc or desc String value to the HQL instead of using a named paramater for it), but it struck my curiosity why Hibernate is preventing it (perhaps simply because 99% of the time sequential named parameters like this result in invalid SQL/HQL).
I've been testing this in my local, and I can't get your desired outcome to work with HQL.
Here is quote from the post I linked:
You can't bind a column name as a parameter. Only a column value. This name has to be known when the execution plan is computed, before binding parameter values and executing the query. If you really want to have such a dynamic query, use the Criteria API, or some other way of dynamically creating a query.
Criteria API looks to be the more useful tool for your purposes.
Here is an example:
Criteria criteria = session.createCriteria(MyEntity.class);
if (orderDirection.equals("desc")) {
criteria.addOrder(Order.desc(orderProperty));
}
else {
criteria.addOrder(Order.asc(orderProperty));
}
According to the answer accepted in this question, you can only define parameters in WHERE and HAVING clauses.
The same answer also gives you some ways to have a workaround for your problem, however I will add one more way to do this:
Use the CASE - WHEN clause in your ORDER BY, this would work by the following way:
SELECT u FROM User u
ORDER BY
CASE WHEN '**someinputhere**' = :orderProperty
AND '**someotherinput**' = :orderDirection
THEN yourColumn asc
ELSE yourColumn desc END
Please, note that in this approach would required you to write all the possible inputs for ordering. Not really beautiful but really useful, especially because you would not need to write multiple queries with different orderings, plus with this approach you can use NamedQueries, which would be possible by writing the query dinamically using string concats.
Hope this can solve your problem, good luck!
We are examining 2 different methods to do our entities updates:
"Standard" updates - Fetch from DB, set Fields, persist.
We have a Query for each entity that looks like this:
String sqlQuery = "update Entity set entity.field1 = entity.field1, entity.field2 = entity.field2, entity.field3 = entity.field3, .... entity.fieldn = entity.fieldn"
We receive the fields that changed (and their new values) and we replace the string fields (only those required) with the new values. i.e. something like :
for each field : fields {
sqlQuery.replace(field.fieldName, getNewValue(field));
}
executeQuery(sqlQuery);
Any ideas if these options differ in performance to a large extent? Is the 2nd option even viable? what are the pros / cons?
Please ignore the SQL Injection vulnerability, the example above is only an example, we do use PreparedStatements for the query building.
And build a mix solution?
First, create a hasmap .
Second, create a new query for a PreparedStament using before hasmap (for avoid SQL injection)
Third, set all parameters.
The O(x)=n, where n is the number of parameters.
The first solution is much more flexible You can rely on Hibernate dirty checking mechanism for generating the right updates. That's one good reason why an ORM tool is great when writing data.
The second approach is no way way better because it might generate different update plans, hence you can't reuse the PreparedStatement statement cache across various column combinations. Instead of using string based templates (vulnerable to SQL injections) you could use JOOQ instead. JOOQ allows you to reference your table columns in Java, so you can build the UPDATE query in a type-safe fashion.
Let's depict the following use case: I have a JPQL Query which on the fly creates data objects using the new keyword. In the SELECT clause I would like to inject an attribute which is not known to the database but to the layer which queries it.
This could look like
EntityManager em; // Got it from somewhere
boolean editable = false; // Value might change, e.g. depending on current date
Query q = em.createQuery("SELECT new foo.bar.MyDTO(o, :editable) FROM MyObject o")
.setParameter("editable", editable);
List<MyDTO> results = (List<MyDTO>) q.getResultList();
Any ideas how this kind of attribute or parameter injection into the SELECT clause might work in JPQL? Both JPA and JPA 2.0 solutions are applicable.
Edit: Performance does not play a key role, but clarity and cleanness of code.
Have you measured a performance problem when simply iterating over the list of results and call a setter on each of the elements. I would guess that compared to
the time it takes to execute the query over the database (inter-process call, network communication)
the time it takes to transform each row into a MyObject instance using reflection
the time it takes to transform each MyObject instance into a MyDTO using reflection
your loop will be very fast.
If you're so concerned about performance, you should construct your MyDTO instances manually from the returned MyObject instances instead of relying on Hibernate and reflection to do it.
Keep is simple, safe, readable and maintainable first. Then, if you have a performance problem, measure to detect where it comes from. Then and only then, optimize.
It will not work without possible vendor extensions, because according specification:
4.6.4 Input Parameters
...
Input parameters can only be used in the
WHERE clause or HAVING clause of a query.