I have used javax.jdo.Query like here JDO for Google App Engine: escaping quotes. Yet, my query string with single quote (') keep getting exploded.
Query query = pm.newQuery("select from " + Book.class.getName() + " where mArtist== '"+ artist + "' && mTitle=='" + title + "'");
Here is the exception
javax.jdo.JDOUserException: Portion of expression could not be parsed: 't Give Up'
org.datanucleus.store.query.QueryCompilerSyntaxException: Portion of expression could not be parsed: 't Give Up'
Here is this query.toString()
SELECT FROM com.example.Book WHERE mArtist== 'Famous Writer' && mTitle=='We Won''t Give Up'
Yeh, I have even escaped the single quote(') with double single quote per appengine docs
a str literal, as a single-quoted string. Single-quote characters in the string must be escaped as ''. For example: 'Joe''s Diner'
Building a query by string concatenation is almost always a risky thing to do, even when SQL Injection attacks aren't possible. (They aren't with GAE.)
See http://code.google.com/appengine/docs/java/datastore/jdo/queries.html#Introducing_Queries and note the bit on "parameter substitution".
The example code in the document only cover a single parameter substitution. Here is a bit more.
Query query = pm.newQuery(Book.class);
query.setFilter("mArtist == artist && mTitle == title");
query.declareParameters("String artist,String title");
List<Book> list = (List<Book>) query.execute("Famous Writer","We Won't Give Up");
Some SO questions worth reading :
How to dynamically build JDO Queries on multiple parameters
Google Datastore problem with query on *User* type
Related
I have a search functionality that is implemented on my current project, I am using HQL for the repo method - Everything is okay except searching for special characters for some reason HQL is not escaping it.
#Query("SELECT f FROM Item f WHERE f.id=:id AND f.itemNm LIKE %:itemNm %")
Item findByItemNm(#Param("itemNm ") String itemNm, #Param("id") Long id);
The String search I am sending is
abc_xyz(YYYYMM[t-2]-YYYYMM[t-1])_[xyz]HU.xlsx
Debugging Steps
Before sending to the Repo - I did replace all the special characters with the following -newsearchString is the one thatt is being passed to the repo
String newsearchString = searchString.replaceAll("(?=[]\\[+&|!(){}^\"~*?:\\\\_]) ", "/");
I have also tried it without replacing the special characters- It is not being returned
How do you escape characters in HQL?
Here is the solution for your problem...It should work for your need.
1) Find and replace all your escape characte, and replaced by "\\"
Example: [hi]_mysearch should be replaced like \\[hi\\]_mysearch
2) Change your repo query like below
#Query("SELECT f FROM Item f WHERE f.id=:id AND f.itemNm LIKE %:itemNm % escape '\\'")
Item findByItemNm(#Param("itemNm ") String itemNm, #Param("id") Long id);
HQL does not have regular expressions out of the box. You'd have to modify the Dialect for your specific database to add that functionality. (Oracle example)
If you're just trying use like with the % wildcard, then you can do the following:
String itemNm = "abc_xyz(YYYYMM[t-2]-YYYYMM[t-1])_[xyz]HU.xlsx";
Query q = session.createQuery("from Item where itemNm like :itemNm ";
q.setString("itemNm","%"+itemNm+"%");
I am using Mockrunner to mock Sql DB for my unit tests. Following is my query:-
"select * from table where userId in (" + userIds + ")"
Now my userIds is state dependent. I don't need my test cases dependent on the arrangement inside the list - userIds. So I don't need exact match but regex matching. I have already enabled regex matching by below code:-
StatementResultSetHandler statementHandler = connection.getStatementResultSetHandler();
usersResult = statementHandler.createResultSet("users");
statementHandler.setUseRegularExpressions(true);
//How to write this regex query?
statementHandler.prepareResultSet("select * from table where userId in .*", campaignsResult);
But as it is noted, I have no idea about the regex syntax supported by Mockrunner.
Edit: I unable to match queries like "Select * from tables" with "Select * from tab .*". So It has to do something with the way I using regex with Mockrunner
There are some helpful examples available here. For instance:
public void testCorrectSQL() throws Exception {
MockResultSet result = getStatementResultSetHandler().createResultSet();
getStatementResultSetHandler().prepareResultSet("select.*isbn,.*quantity.*", result);
List orderList = new ArrayList();
orderList.add("1234567890");
orderList.add("1111111111");
Bookstore.order(getJDBCMockObjectFactory().getMockConnection(), orderList);
verifySQLStatementExecuted("select.*isbn,.*quantity.*\\(isbn='1234567890'.*or.*isbn='1111111111'\\)");
}
From this, I surmise that it's using standard Java regex syntax. In which case, you probably want:
prepareResultSet("select \\* from table where userId in \\(.*\\)", campaignsResult);
...or perhaps more succinctly (and depending upon exactly how fine-grained your tests need to be):
prepareResultSet("select .* from table where userId in .*", campaignsResult);
The main caveat to be aware of when enabling the regex matching is that any literal special characters that you want in your query (such as *, (, and ) literals) need to be escaped in your regex before it will work properly.
I ran into some unexpected behavior when trying to combine whitespaces in a Directory Service API search query with a wildcard character:
Code
Directory ds = new GoogleDirectoryServiceManager().getDirectoryService("admin#randomdomain.com");
Directory.Users usersClient = ds.users();
String lastNameBuffer = term;
StringBuilder sb = new StringBuilder();
if(term.contains(" ")){
sb.append("\'");
sb.append(lastNameBuffer);
sb.append("*\'");
}else{
sb.append(lastNameBuffer);
sb.append("*");
}
queryString = "familyName:" + sb.toString();
users = usersClient.list()
.setDomain("randomdomain.com")
.setQuery(queryString)
.setMaxResults(MAX_DIRECTORY_RESULTS)
.setFields("users(name,primaryEmail,thumbnailPhotoUrl)")
.execute();
I added the single quotes to search through the directory with the spaces. Following is an extract from the Google Directory API:
Surround with single quotes ' if the query contains whitespace. Escape single quotes in queries with \', for example 'Valentine\'s Day'.
Input
"johns" --> Returns all users with Johnson name (familyName: johns*) GOOD
"van bur" --> Does not return any users (familyName: 'van bur*') NOT GOOD
"van buren" --> Returns all users with Van Buren name (familyName: 'van buren*') GOOD
tl;dr
Is it not possible to combine a search query with spaces with a wildcard? Or does the wildcard symbol (*) have to be escaped?
What query do I need to use to successfully return a list of users in case #2?
During my attempts with the API explorer embedded in the documentation, I have noted that the following query, the same as you without the single quotes, would work as expected on my test domain. Can you test it ?
familyName:van bur
To prevent from SQL injections, OWASP encodes characters received.Below is the code implemented for org.owasp.esapi.codecs.OracleCodec.java class
//Default implementation that should be overridden in specific codecs. Encodes ' to '' Encodes ' to '' (according to doc)
public String encodeCharacter( char[] immune, Character c ) {
if ( c.charValue() == '\'' )
return "\'\'";
return ""+c;
}
How does above help for the prevention of SQL injection?Please explain.
Using the guidelines at OWASP, multiple test cases can be found here.
The snippet of code you're looking at here defends against someone trying to escape out of the query to run their own arbitrary command.
if ( c.charValue() == '\'' )
If the input value is equal to ASCII char value 0x27 (a single quote)
return "\'\'";
Escape the single quote.
Oracle escaping is here.
Lets say your query is "select * from users where id = \'" + request.getParameter("id")
By not escaping single-quotes, an input like this:
request.setParameter("id", "\' OR 1=1;"); would result in returning all the information in that table by changing the final, non-Java formatted query to select * from users where id = '' OR 1=1;
I highly recommend you download the WebGoat program, and follow its lessons. It will teach you how to use SQL injection, and many other basic web attacks. And the ESAPI swingset will help you learn how to mitigate them.
Here explains very well for oracle and others DBMS:
https://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheet
But rule n. 1 to prevent SQL injection is not to use query concatenation but instead prepared statements! With prepared statements there is no need to encode any parameter (set by sql api) and there are also DB performance optimizations.
Hello guys I am having some problems with exact matches while doing a NamedQuery.
I am currently using something like this:
#NamedQuery(name = MyClass.GET_ENTRY_BY_NAME, query = "select e from Entry e where e.name =:"+ Entry.NAME )
...
Query query = em.createNamedQuery(MyClass.GET_ENTRY_BY_NAME);
query.setParameter(Entry.NAME, myEntry.getName());
It works for most cases, however I noticed that in case the user pass the file name with an space at the end, the namedQuery ignores that character. For example:
Query query = em.createNamedQuery(MyClass.GET_ENTRY_BY_NAME);
query.setParameter(Entry.NAME, myEntry.getName()+ " ");
Will return the same result as the query before. Bypassing my 'valid entry' validation. In other words I'd like the query to return no entry at all and treat the error later on.
One workaround I could think of, is to put single quotes surrounding my parameter in the namedQuery, like this:
#NamedQuery(name = MyClass.GET_ENTRY_BY_NAME, query = "select e from entry e where e.name =':"+ Entry.NAME "'")
However it will trash my code in case the String contains single quotes in it...
Any ideas guys?
I guess this happens because your database field is declared as CHAR(...), and therefore stored values are padded with whitespaces which are not taken into account by = operation.
So, you may either declare your database field as VARCHAR(...) or use a built-in trim function:
query = "select e from Entry e where trim(trailing from e.name) =:"+ Entry.NAME
I did some research in JPA and found out that it does some automatic trimming for CHARs, I am not sure if this behaves the same with Strings, but since it is happening to me... I believe so. The only way to bypass it is by setting some attribute within the session DatabaseLogin object (see http://www.eclipse.org/eclipselink/api/1.1/org/eclipse/persistence/sessions/DatabaseLogin.html#setShouldTrimStrings) .
Well I didn't want to be messing up with the session properties so I decided to make some sort of check and throwing the same exception as the NoResultException catch does in my code.
I basically took the result from the database and compared the field with the String I used:
query.setParameter(Entry.NAME, myEntry.getName());
...
if(!StringUtils.equals(result.getName(), myEntry.getName()){
do a cool throw just like NoResultException Catch
}
I also had to include the Trim function axtavt! This is just to make sure that if the database has a column with trailing spaces and it matches the parameter given by the user, it will be included as a valid answer. For example:
Database entry: Name = "Flavio " - Trimmed with Function = "Flavio".
Parameter passed: Name = "Flavio " - Trimmed by JPA automatic function = "Flavio".
If it isnt trimmed at all it will just Compare "Flavio " with "Flavio", returning NoResult when it was supposed to return that Entry.
Nasty workaround, but as long as there is no other way to stop the auto-trimming we will have to just make use of this sort of things.
Thanks for all the other answers!!