I am trying to connect to postgresql with ORMlite from a Java client.
the DB gets generated perfectly, but when I try to insert something into a table that is using an autoincrement id I get the following error:
org.postgresql.util.PSQLException: ERROR: relation "commandusage_id_seq" does not exist
When I check the DB I can see a "commandUsage_id_seq" sequence has been created. With a Capital U.
How can I configure ORMLite to use all the same casing for both creating and interacting with the DB ? I couldn't find this in the documentation
Thanks in advace.
update:
When explicitly setting the sequenceId I can circumvent the issue
generatedIdSequence = "commandusage_id_seq"
but still I would like to know if this is possible by setting some config for ORMLite instead of setting this per DBObject class
update2:
URL of the package to prevent confusion: ORMLite
update3:
Below a snippet of the code and how it works. Again I would like to know if ORMLite is capable of doing the to lowercase conversion automatically instead of me doing it explicitly.
#DatabaseTable(tableName = "commandusage", daoClass = CommandUsageDaoImpl.class)
public class CommandUsage {
#DatabaseField(columnName = "id", generatedIdSequence = "commandusage_id_seq")
private transient int identifier;
When I check the DB I can see a "commandUsage_id_seq" sequence has been created. With a Capital U.
Interesting. After some initial confusion on my part, this seems like a bug a in ORMLite. The pattern is if you force a table name with mixed case (typical is to downcase the name) and then ask for a sequence-id on it in Postgresql:
#DatabaseTable(tableName = "TableNameCaseWithSeqeuence")
private static class TableNameCaseWithSeqeuence {
#DatabaseField(generatedId = true)
public int id;
...
}
I've got a fix in trunk but it's going to take a bit to spin a release.
The workaround right now is to extend the PostgresDatabaseType and inject it into your ConnectionSource. It should do something like:
public OurPostgresDatabaseType extends PostgresDatabaseType {
// constructors ...
#Override
public String generateIdSequenceName(String tableName, FieldType idType) {
String name = tableName + DEFAULT_SEQUENCE_SUFFIX;
return downCaseString(name, true);
}
}
I want to run queries on my SQLite database that have been generated at runtime (instead of the standard compiletime queries in the #Dao). For example I might want to search a TEXT column in the SQLite db, to see if it contains all words in a list of N length. In raw SQLITE, a query where N is 3 would look like this :
SELECT * FROM table
WHERE textValue LIKE %queryTerm1%
AND textValue LIKE %queryTerm2%"
AND textValue LIKE %queryTerm3%"
I have tried generating, and passing the end of the query, instead of just passing variables. For example :
String generatedQuery = "textValue LIKE %queryTerm1% AND textValue LIKE %queryTerm2% AND textValue LIKE %queryTerm3%";
tableDao.find(generatedQuery);
and in the #Dao:
#Query("SELECT * FROM tableName WHERE :endQuery")
List<POJO> find(String endQuery);
This doesn't seem to work for me. Do you have any idea how to get runtime generated queries working with Room?
PS:
I have debugged the Dao implementation and looked at the statement it is running. This confirms that the generated query information, and the query are being passed correctly. I assume this is an issue with SQL injection prevention (aka more of an SQLITE problem, than a Room problem)
Update: latest release 1.1.1 of Room now uses SupportSQLiteQuery instead of String.
A query with typed bindings. It is better to use this API instead of
rawQuery(String, String[]) because it allows binding type safe
parameters.
New Answer:
#Dao
interface RawDao {
#RawQuery(observedEntities = User.class)
LiveData<List<User>> getUsers(SupportSQLiteQuery query);
}
Usage:
LiveData<List<User>> liveUsers = rawDao.getUsers( new
SimpleSQLiteQuery("SELECT * FROM User ORDER BY name DESC"));
Update your gradle to 1.1.1 (or whatever the current version is)
implementation 'android.arch.persistence.room:runtime:1.1.1'
implementation 'android.arch.lifecycle:extensions:1.1.1'
annotationProcessor "android.arch.persistence.room:compiler:1.1.1"
The problem is you want to pass a part of SQL statement, but Room treats it like a query parameter.
If you want you can try to use Kripton Persistence Library, an open source library written (by me :) ) that drastically simplify SQLite's management code for Android platform and support situations like this.
Kripton works with DAO pattern too, so concept are quite similar. Just to write an example that fit your needs:
Given a model class:
#BindType
public class User {
public long id;
public String name;
public String username;
public String email;
public Address address;
public String phone;
public String website;
public Company company;
}
a DAO definition:
#BindDao(User.class)
public interface UserDao {
#BindSqlInsert
void insert(User bean);
#BindSqlSelect
List<User> selectDynamic(#BindSqlDynamicWhere String where, #BindSqlDynamicWhereParams String[] args);
}
and a data source definition:
#BindDataSource(daoSet={UserDao.class}, fileName = "kripton.quickstart.db", generateAsyncTask = true)
public interface QuickStartDataSource {
}
Kripton will generate at compile time all code is need to work with database. So to accomplish your task with Kripton you have to write a code similar to:
BindQuickStartDataSource ds = BindQuickStartDataSource.instance();
// execute operation in a transaction
ds.execute(new BindQuickStartDataSource.SimpleTransaction() {
#Override
public boolean onExecute(BindQuickStartDaoFactory daoFactory) throws Throwable
{
UserDaoImpl dao = daoFactory.getUserDao();
String[] p={"hello"};
dao.selectDynamic("name=?",p);
return true;
}
});
In logcat when code above is executed you will see the generated log:
database OPEN READ_AND_WRITE_OPENED (connections: 1)
UserDaoImpl, selectDynamic (line 352): SELECT id, name, username, email, address, phone, website, company FROM user WHERE name=?
selectDynamic (line 357): ==> param0: 'hello'
Rows found: 0
database CLOSED (READ_AND_WRITE_OPENED) (connections: 0)
Kripton obviously supports static where conditions too and many other features (i start to develop it in 2015).
For more information about Kripton Persistence Library:
https://github.com/xcesco/kripton
http://abubusoft.com/
https://github.com/xcesco/kripton/wiki
I have this query using native query in JPA2 to search for parcial text independent of case or accents used (based on http://www.guj.com.br/java/212706-accent-insensitive-hibernate):
public List<Hipotesis> findHipotesisByText(String srchtext) {
EntityManager em = getEntityManager();
String textNormalized =
Normalizer.normalize(srchtext, Normalizer.Form.NFD).replaceAll("[^\\p{ASCII}]", "").toUpperCase();
Query query =
em.createNativeQuery(
"select * from HIPOTESIS where ( UPPER(TRANSLATE( TEXTFIELD,'ÀÁÂÃáàâãÉÈÊéèêÍíÓÒÔÕóòôõÚÜúü','AAAAaaaaEEEeeeIiOOOOooooUUuu' ) ) like '%" + textNormalized + "%'",
Hipotesis.class);
#SuppressWarnings("unchecked")
List<Hipotesis> results = query.getResultList();
return results;
}
The search text is normalized to strip accents and is converted to upper case.
The native query uses TRANSLATE to convert accents to pure text and UPPER converts the result to upper case.
So the search text eMeRgÊ will be normalized to EMERGE and will match any occurence in the database like emergencia, emergência, Emergência.
Although TRANSLATE is specified by SQL99 it is not supported or implemented exactly like the standard.
The question: is there any way to implement this query without using TRANSLATE? or without using native query?
The most elegant solution, in my personal opinion is to actually duplicate the data and convert it to a normalized form. You're using a LIKE condition in your query, which basically precludes any normal (short of full-text) indexing mechanism. This means that the TRANSLATE query will most likely turn out to be inefficient and difficult to optimize.
Using JPA, you can make use of entity lifecycle events to manage the normalized forms in a fairly convenient manner:
#Entity
public class Whatever implements Serializable {
private static final long serialVersionUID = 0L;
private String string;
private String normalizedString;
// getters and setters
#PreUpdate
#PrePersist
protected void normalize() {
normalizedString = yourNormalizationMethod(string);
}
}
I believe this to be the cleanest, most elegant, and most database-agnostic way to address this type of issue.
I need to create a JOOQ SELECT query dynamically based on the set of parameters. I dont know how to append it dynamically.
Please help
Thanks in advance.
jOOQ has two types of APIs to construct queries.
The DSL API that allows for creating inline SQL statements in your Java code, e.g.
create.select(T.A, T.B).from(T).where(T.X.eq(3).and(T.Y.eq(5)));
The "model" API that allows for incremental SQL building. At any time, you can access the "model" API through the getQuery() method on a DSL query object
An example of what you want to do is given in the manual here:
https://www.jooq.org/doc/latest/manual/sql-building/sql-statements/dsl-and-non-dsl/
For instance, optionally adding a join:
DSLContext create = DSL.using(configuration);
SelectQuery query = create.selectQuery();
query.addFrom(AUTHOR);
// Join books only under certain circumstances
if (join)
query.addJoin(BOOK, BOOK.AUTHOR_ID.equal(AUTHOR.ID));
Result<?> result = query.fetch();
Or, optinally adding conditions / predicates:
query.addConditions(BOOK.TITLE.like("%Java%"));
query.addConditions(BOOK.LANGUAGE_CD.eq("en"));
UPDATE: Given your comments, that's what you're looking for:
// Retrieve search strings from your user input (just an example)
String titleSearchString = userInput.get("TITLE");
String languageSearchString = userInput.get("LANGUAGE");
boolean lookingForTitles = titleSearchString != null;
boolean lookingForLanguages = languageSearchString != null;
// Add only those conditions that the user actually provided:
if (lookingForTitles)
query.addConditions(BOOK.TITLE.like("%" + titleSearchString + "%"));
else if (lookingForLanguages)
query.addConditions(BOOK.LANGUAGE_CD.eq(languageSearchString));
Note, you can also use the Field.compare(Comparator, Object) methods:
// Initialise your dynamic arguments
Field<String> field = BOOK.TITLE;
Comparator comparator = Comparator.LIKE;
String value = "%" + titleSearchString + "%";
// Pass them to the field.compare() method
query.addConditions(field.compare(comparator, value));
For more info, consider the org.jooq.SelectQuery Javadoc
We'd like to use only annotations with MyBatis; we're really trying to avoid xml. We're trying to use an "IN" clause:
#Select("SELECT * FROM blog WHERE id IN (#{ids})")
List<Blog> selectBlogs(int[] ids);
MyBatis doesn't seem able to pick out the array of ints and put those into the resulting query. It seems to "fail softly" and we get no results back.
It looks like we could accomplish this using XML mappings, but we'd really like to avoid that. Is there a correct annotation syntax for this?
I believe the answer is the same as is given in this question. You can use myBatis Dynamic SQL in your annotations by doing the following:
#Select({"<script>",
"SELECT *",
"FROM blog",
"WHERE id IN",
"<foreach item='item' index='index' collection='list'",
"open='(' separator=',' close=')'>",
"#{item}",
"</foreach>",
"</script>"})
List<Blog> selectBlogs(#Param("list") int[] ids);
The <script> element enables dynamic SQL parsing and execution for the annotation. It must be very first content of the query string. Nothing must be in front of it, not even white space.
Note that the variables that you can use in the various XML script tags follow the same naming conventions as regular queries, so if you want to refer to your method arguments using names other than "param1", "param2", etc... you need to prefix each argument with an #Param annotation.
I believe this is a nuance of jdbc's prepared statements and not MyBatis. There is a link here that explains this problem and offers various solutions. Unfortunately, none of these solutions are viable for your application, however, its still a good read to understand the limitations of prepared statements with regards to an "IN" clause. A solution (maybe suboptimal) can be found on the DB-specific side of things. For example, in postgresql, one could use:
"SELECT * FROM blog WHERE id=ANY(#{blogIds}::int[])"
"ANY" is the same as "IN" and "::int[]" is type casting the argument into an array of ints. The argument that is fed into the statement should look something like:
"{1,2,3,4}"
Had some research on this topic.
one of official solution from mybatis is to put your dynamic sql in #Select("<script>...</script>"). However, writing xml in java annotation is quite ungraceful. think about this #Select("<script>select name from sometable where id in <foreach collection=\"items\" item=\"item\" seperator=\",\" open=\"(\" close=\")\">${item}</script>")
#SelectProvider works fine. But it's a little complicated to read.
PreparedStatement not allow you set list of integer. pstm.setString(index, "1,2,3,4") will let your SQL like this select name from sometable where id in ('1,2,3,4'). Mysql will convert chars '1,2,3,4' to number 1.
FIND_IN_SET don't works with mysql index.
Look in to mybatis dynamic sql mechanism, it has been implemented by SqlNode.apply(DynamicContext). However, #Select without <script></script> annotation will not pass parameter via DynamicContext
see also
org.apache.ibatis.scripting.xmltags.XMLLanguageDriver
org.apache.ibatis.scripting.xmltags.DynamicSqlSource
org.apache.ibatis.scripting.xmltags.RawSqlSource
So,
Solution 1: Use #SelectProvider
Solution 2: Extend LanguageDriver which will always compile sql to DynamicSqlSource. However, you still have to write \" everywhere.
Solution 3: Extend LanguageDriver which can convert your own grammar to mybatis one.
Solution 4: Write your own LanguageDriver which compile SQL with some template renderer, just like mybatis-velocity project does. In this way, you can even integrate groovy.
My project take solution 3 and here's the code:
public class MybatisExtendedLanguageDriver extends XMLLanguageDriver
implements LanguageDriver {
private final Pattern inPattern = Pattern.compile("\\(#\\{(\\w+)\\}\\)");
public SqlSource createSqlSource(Configuration configuration, String script, Class<?> parameterType) {
Matcher matcher = inPattern.matcher(script);
if (matcher.find()) {
script = matcher.replaceAll("(<foreach collection=\"$1\" item=\"__item\" separator=\",\" >#{__item}</foreach>)");
}
script = "<script>" + script + "</script>";
return super.createSqlSource(configuration, script, parameterType);
}
}
And the usage:
#Lang(MybatisExtendedLanguageDriver.class)
#Select("SELECT " + COLUMNS + " FROM sometable where id IN (#{ids})")
List<SomeItem> loadByIds(#Param("ids") List<Integer> ids);
I've made a small trick in my code.
public class MyHandler implements TypeHandler {
public void setParameter(PreparedStatement ps, int i, Object parameter, JdbcType jdbcType) throws SQLException {
Integer[] arrParam = (Integer[]) parameter;
String inString = "";
for(Integer element : arrParam){
inString = "," + element;
}
inString = inString.substring(1);
ps.setString(i,inString);
}
And I used this MyHandler in SqlMapper :
#Select("select id from tmo where id_parent in (#{ids, typeHandler=ru.transsys.test.MyHandler})")
public List<Double> getSubObjects(#Param("ids") Integer[] ids) throws SQLException;
It works now :)
I hope this will help someone.
Evgeny
Other option can be
public class Test
{
#SuppressWarnings("unchecked")
public static String getTestQuery(Map<String, Object> params)
{
List<String> idList = (List<String>) params.get("idList");
StringBuilder sql = new StringBuilder();
sql.append("SELECT * FROM blog WHERE id in (");
for (String id : idList)
{
if (idList.indexOf(id) > 0)
sql.append(",");
sql.append("'").append(id).append("'");
}
sql.append(")");
return sql.toString();
}
public interface TestMapper
{
#SelectProvider(type = Test.class, method = "getTestQuery")
List<Blog> selectBlogs(#Param("idList") int[] ids);
}
}
In my project, we are already using Google Guava, so a quick shortcut is.
public class ListTypeHandler implements TypeHandler {
#Override
public void setParameter(PreparedStatement ps, int i, Object parameter, JdbcType jdbcType) throws SQLException {
ps.setString(i, Joiner.on(",").join((Collection) parameter));
}
}
In Oracle, I use a variant of Tom Kyte's tokenizer to handle unknown list sizes (given Oracle's 1k limit on an IN clause and the aggravation of doing multiple INs to get around it). This is for varchar2, but it can be tailored for numbers (or you could just rely on Oracle knowing that '1' = 1 /shudder).
Assuming you pass or perform myBatis incantations to get ids as a String, to use it:
select #Select("SELECT * FROM blog WHERE id IN (select * from table(string_tokenizer(#{ids}))")
The code:
create or replace function string_tokenizer(p_string in varchar2, p_separator in varchar2 := ',') return sys.dbms_debug_vc2coll is
return_value SYS.DBMS_DEBUG_VC2COLL;
pattern varchar2(250);
begin
pattern := '[^(''' || p_separator || ''')]+' ;
select
trim(regexp_substr(p_string, pattern, 1, level)) token
bulk collect into
return_value
from
dual
where
regexp_substr(p_string, pattern, 1, level) is not null
connect by
regexp_instr(p_string, pattern, 1, level) > 0;
return return_value;
end string_tokenizer;
You could use a custom type handler to do this. For example:
public class InClauseParams extends ArrayList<String> {
//...
// marker class for easier type handling, and avoid potential conflict with other list handlers
}
Register the following type handler in your MyBatis config (or specify in your annotation):
public class InClauseTypeHandler extends BaseTypeHandler<InClauseParams> {
#Override
public void setNonNullParameter(final PreparedStatement ps, final int i, final InClauseParams parameter, final JdbcType jdbcType) throws SQLException {
// MySQL driver does not support this :/
Array array = ps.getConnection().createArrayOf( "VARCHAR", parameter.toArray() );
ps.setArray( i, array );
}
// other required methods omitted for brevity, just add a NOOP implementation
}
You can then use them like this
#Select("SELECT * FROM foo WHERE id IN (#{list})"
List<Bar> select(#Param("list") InClauseParams params)
However, this will not work for MySQL, because the MySQL connector does not support setArray() for prepared statements.
A possible workaround for MySQL is to use FIND_IN_SET instead of IN:
#Select("SELECT * FROM foo WHERE FIND_IN_SET(id, #{list}) > 0")
List<Bar> select(#Param("list") InClauseParams params)
And your type handler becomes:
#Override
public void setNonNullParameter(final PreparedStatement ps, final int i, final InClauseParams parameter, final JdbcType jdbcType) throws SQLException {
// note: using Guava Joiner!
ps.setString( i, Joiner.on( ',' ).join( parameter ) );
}
Note: I don't know the performance of FIND_IN_SET, so test this if it is important
I had done this with postgresql.
#Update('''
UPDATE sample_table
SET start = null, finish = null
WHERE id=ANY(#{id});
''')
int resetData(#Param("id") String[] id)
ANY works like the IN.
Code above is using groovy but can be converted into java by replacing the single quotes into double.