Develop Reference

Rolling back a Postgres database using Liquibase in Java

I'm trying to roll back the changes to a postgres table inbetween component tests so each one has a clean db to work with.
I'm using liquibase to set up postgres (the changelog xml to describe the setup and then the liquibase-core Kotlin/Java library to apply it). I'm also using Hibernate to interact with postgres directly. The test framework I'm using is Kotest, using the beforeXXX methods to make sure all the setup happens before the tests run. The database is set up once before everything runs and the idea is to rollback after each test.
From looking in the docs I've found tagDatabase and rollback seem to be what I need, however when running them they don't seem to actually roll anything back.
The code is roughly as follows (this is just test code to see if it works at all, mind - code would ideally be segmented as I descirbed above):
// 1 - (Pre-all-tests) Postgres Setup
liquibase = Liquibase(
"/db/changelog/changelog-master.xml",
ClassLoaderResourceAccessor(),
DatabaseFactory.getInstance().findCorrectDatabaseImplementation(JdbcConnection(connection))
)
liquibase.update(Contexts(), LabelExpression())
liquibase.tag("initialised")
// 2 - Something is inserted
val newEntity = ThingEntity()
entityManager.persist(
entity
)
entityManager.transaction.commit()
entityManager.clear()
// 3 - Cleanup
liquibase.rollback("initialised", Contexts())
// 4 - Fetching
entityManager.find(ThingEntity::class.java, id)
Thing is, after running liquibase.rollback the newEntity I persisted earlier is still present. The tag has dissapeared - if I run the doesTagExist method it returns true and then false after the rollback so the tag is being removed at least.
Given I'm clearing the entity manager after the commit I don't think it's because it's being cached and as I said the tag is being removed - just not the data.
Can anyone tell my why the actual transactions (i.e. the persist) aren't being erased?
Thanks!

Looks like you are using liquibase in a wrong way. What you are trying to have (rollback of data that is added in unit-test) is something close to what is described here: Rollback transaction after #Test
And when you are asking liquibase to rollback to some tag it just executes rollback scripts (if any provided) for changesets that were applied after changeset with tag: https://docs.liquibase.com/commands/community/rollbackbytag.html

Backslash in Jooq/PostgreSQL INSERT clause

I am trying to use Jooq to do an INSERT into a PostgreSQL database. The query fails if the String includes a backslash character with SQL state code: 42601 which means SYNTAX ERROR.
Jooq: 3.4.4
postgresql driver: 8.4-702.jdbc4
PostgreSQL: "PostgreSQL
8.4.20 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4), 64-bit"
JDK 1.8.0_25
Spring Tool Suite 3.6.0.RELEASE
Database:
CREATE TABLE datahub.test (
body TEXT NOT NULL
);
Jooq code generated using maven:
jooq-codegen-maven version 3.4.4
generator.name: org.jooq.util.DefaultGenerator
generator.database.name: org.jooq.util.postgres.PostgresDatabase
Unit test
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {"/spring-config.xml"})
public class BatchExceptionJooqTest {
private static Logger log = LogManager.getLogger(BatchExceptionJooqTest.class);
#Autowired
private DSLContext db;
#Test
public void runBasicJooqTest(){
try{
final List<InsertQuery<TestRecord>> batchUpdate = Lists.newLinkedList();
InsertQuery<TestRecord> insertQuery = db.insertQuery(TEST);
insertQuery.addValue(TEST.BODY, "It's a bit more complicated than just doing copy and paste... :\\");
batchUpdate.add(insertQuery);
db.batch(batchUpdate).execute();
}catch(Exception e){
log.error(e);
}
}
}
Problem
The test fails with an exception:
2014-12-26 17:11:16,490 [main] ERROR BatchExceptionJooqTest:36 :runBasicJooqTest - org.jooq.exception.DataAccessException: SQL [null]; Batch entry 0 insert into "datahub"."test" ("body") values ('It''s a bit more complicated than just doing copy and paste... :\') was aborted. Call getNextException to see the cause.
The test passes, if instead of String: "It's a bit more complicated than just doing copy and paste... :\\" I use String: "It's a bit more complicated than just doing copy and paste... :\\\\". This seems a bit inconsistent when compared to what is happening to the the single quote during the operation. It is correctly doubled so as to get through the SQL parser. Not so with the backslash.
I read somewhere that escaping a backslash with another backslash is not part of the SQL standard and Postgre has changed its default behavior lately. However I am not clear on the meaning of the manual p 4.1.2.2 - it seems to indicate that double backslashes should work and there is not really any reason for jooq not to do it.
So.. could someone please explain if the described situation in Jooq:
Is desired behavior and there is no workaround besides doubling all incoming backslashes my application is processing?
Is desired behavior but there is a configuration change I can do to make Jooq process the backslashes in a similar manner to the single quotes?
Is it a bug?
What am I doing incorrectly?
Thank you

You are using PostgreSQL 8.x. In that version, the system defaulted to accepting backslash escaped string literals even without the preceding E.
To avoid this, you should set the server configuration variable standard_conforming_strings to ON.
It is, of course, strongly recommended that you migrate to a version of PostgreSQL higher than 8.x, as the 8.x versions have reached end-of-life and are no longer supported.

jOOQ 3.5 has introduced org.jooq.conf.Settings.backslashEscaping (https://github.com/jOOQ/jOOQ/issues/3000). This was mainly introduced for MySQL, which still today defaults to non-standards compliant string literal escaping using backslashes.
Note that this setting affects only inlined bind values, so it will not escape backslashes when binding values to a PreparedStatement.
I agree with RealSkeptic's answer, which suggests you change the database behaviour or upgrade to a newer PostgreSQL version.

ORA-01460: unimplemented or unreasonable conversion requested using Hibernate #Lob

I have a byte[] that I am persisting to a Lob as follows:
#Basic(fetch = FetchType.LAZY)
#Column(name = "ABF", length = Integer.MAX_VALUE)
#Lob
private byte[] abf;
Seems simple enough, but when I attempt to store anything sizable in it (more than 4000 characters) I get the following exception when I try to commit:
java.sql.SQLException: ORA-01460: unimplemented or unreasonable conversion requested
None of the files I am attempting to store are anywhere near 32,000 characters. Is there some other gotcha here?

See this post.
Nutshell:
<property name="hibernate.connection.SetBigStringTryClob">true</property>
<property name="hibernate.jdbc.batch_size">0</property>
It can also be:
Old Oracle JDBC driver (although I think then the limit was 2k)
Driver/DB version mismatch
Wrong Oracle dialect specified in Hibernate config
For DB stuff it's always helpful to supply driver and DB version info :)

i just updated the oracle driver and it worked fine.
it is mainly due to the oracle driver mismatch.
If you have right version of jdbc driver corresponding to you oracle version, it should not be an issue.

Sometimes it helps to do things in that order:
insert the new entity with an empty lob;
commit;
populate the lob on the newly created entity;
update and commit.

#Dave Newton set me on the right path. The answer involved a few things. As Dave pointed out, I added these lines to hibernate.cfg.xml:
<property name="hibernate.connection.SetBigStringTryClob">true</property>
<property name="hibernate.jdbc.batch_size">0</property>
I was previously using hsqldb-2.0.0.jar. I updated this to the current version (hsqldb-2.2.5.jar). I think this was the main culprit, and I swear I've noticed a database performance increase since doing this.
I also updated to the current version of ojdbc14.jar (10.2.0.5). I was previously on some older version, but I don't know exactly which one. It should be noted that even after updating to version 10.2.0.5 the problem did not go away. It wasn't until I updated the hsqldb.jar version that the problem was resolved.

EclipseLink can't retrieve entities inserted manually

I'm having some trouble with EclipseLink. My program has to interact with a database (representing a building). I've written a little input-testmode where I can manually insert stuff through the console.
My problem: a normal getByID-operation works just fine if I try to retrieve an entity I previously inserted through EclipseLink itself (by commit()), but throws a NoResultException when trying to select a row manually inserted via SQL-script (building -> lots of rooms -> script).
This (oversemplified) works fine:
main() {
MyRoom r = new MyRoom();
r.setID("floor1-roomnr4");
em.commit(r); //entity manager
DAO.getRoomByID("floor1-roomnr4"); // works
}
and the combination of generation-script + simply getRoomByID() throws an exception.
If I try it in SQL Developer I get the result I want for the exact select statement which just threw a NoResultException. I also only get this problem in the input-mode, otherwise selecting the generated rows works also fine.
Does EclipseLink have some cache-mechanism I'm unaware of which is causing some problem?

Are you sure EclipseLink and SQL Developer are connected to the same Database? Please verify the connection information for both. Is the generation-script committing the changes with the "commit" command?

If EclipseLink works similarly to Hibernate then yes there is a cache. The "first level cache" guaranties that you get the exact same instance within one transaction which makes sense. If you know EclipseLink/transactions then try to evict all loaded instances or commit the transaction and then try your DAO again. This would force EclipseLink to fetch the data from the database again
See Answer to similar question

Problem persisting a java.util.Date into MySql using Hibernate

I've been debugging this problem for the last couple of hours with no success and figured I'd throw it out to SO and see where that goes.
I'm developing a Java program that persists data into a MySql database using Hibernate and the DAO/DTO pattern. In my database, I have a memberprofile table with a firstLoginDate column. In the database, the SQL type of that column is a DateTime. The corresponding section of the Hibernate XML file is
<property name="firstLoginDate" type="timestamp">
<column name="firstLoginDate" sql-type="DATETIME"/>
</property>
However, when I try to save a Date into that table (in Java), the "date" (year/month/day) part is persisted correctly, but the "time of day" part (hours:minutes:seconds) is not. For instance, if I try to save a Java date representing 2009-09-01 14:02:23, what ends up in the database is instead 2009-09-01 00:00:00.
I've already confirmed that my own code isn't stomping on the time component; as far as I can see source code (while debugging) the time component remains correct. However, after committing changes, I can examine the relevant row using the MySql Query Browser (or just grabbing back out from the database in my Java code), and indeed the time component is missing. Any ideas?
I did try persisting a java.sql.Timestamp instead of a java.util.Date, but the problem remained. Also, I have a very similar column in another table that does not exhibit this behavior at all.
I expect you guys will have questions, so I'll edit this as needed. Thanks!
Edit #Nate:
...
MemberProfile mp = ...
Date now = new Date();
mp.setFirstLoginDate(now);
...
MemberProfile is pretty much a wrapper class for the DTO; setting the first login date sets a field of the DTO and then commits the changes.
Edit 2: It seems to only occur on my machine. I've already tried rebuilding the table schema and wiping out all of my local source and re-checking-out from CVS, with no improvement. Now I'm really stumped.
Edit 3: Completely wiping my MySql installation, reinstalling it, and restoring the database from a known good copy also did not fix the problem.

I have a similar setup to you (except mine works), and my mapping file looks like this:
<property name="firstLoginDate" type="timestamp">
<column name="firstLoginDate" length="19"/>
</property>
My database shows the column definition as datetime.
Edit:
Some more things to check...
Check that the mysql driver the same
on your local as on the working machines.
Try dropping the table, and have
hibernate recreate it for you. If
that works, then there's a problem in
the mapping.

This may or may not be your problem, but we have had serious problems with date/time info - if your database server is on a different time zone than the machine submitting the data, you can have inconsistencies in the data saved.
Beyond that, with our annotation configuration, it looks something like the following:
#Column(name="COLUMN_NAME", length=11)

If it is viable for you, consider using the JodaTime DateTime class which is much nicer than the built in classes and you can also persist them using Hibernate with their Hibernate Support
Using them I mark my fields or getters with the annotation for custom Hibernate Types as:
#org.hibernate.annotations.Type(type = "org.joda.time.contrib.hibernate.PersistentDateTime")
#Column(name = "date")
This works fine for me and it also generates correct schema generation sql
This works fine in MySQL

Use TemporalType.TIMESTAMP beside your Temporal annonation.
Please check the example below.
#Temporal(TemporalType.TIMESTAMP)
public Date getCreated() {
return this.created;
}