no values persisted in database when running spring boot JPA test - java

I was looking at solutions of mapping column default value in spring-data-jpa which lead me to documentation of #ColumnDefault annotation; and I wanted to try it out. So I updated an entity where we needed default values (today this is handled via a schema.sql where we define additional column behaviours) like so:
#Entity
#Getter
#Setter
#ToString
#DynamicInsert
#NoArgsConstructor
#Relation(collectionRelation = "customers")
public class Customer extends Organisation implements Identifiable<Long>, Serializable {
private static final long serialVersionUID = 8101819808147191270L;
#Column(nullable = false, updatable = false, length = 3)
#ColumnDefault("'INR'")
private String currency;
#Column(scale = 2)
#ColumnDefault("0.10")
private Double tds;
#Column(nullable = false, updatable = false, length = 3)
#ColumnDefault("'INV'")
private String invoicePrefix;
Now since I'm using the #ColumnDefault I expect that default values are configured for the column, which they are when I look at the database. However, when I run a test to insert values; the defaults fields are null. I configured Postgresql for the test (as production will be in Postgresql); While the DDL for table in Postgresql shows default values are added to
table definition
CREATE TABLE data_api_it.customer
(
currency character varying(3) COLLATE pg_catalog."default" NOT NULL DEFAULT 'INR'::character varying,
invoice_prefix character varying(3) COLLATE pg_catalog."default" NOT NULL DEFAULT 'INV'::character varying,
tds double precision DEFAULT 0.10,
id bigint NOT NULL,
CONSTRAINT customer_pkey PRIMARY KEY (id),
CONSTRAINT fk3afgab8nfy6ykn6b70uuh9v59 FOREIGN KEY (id)
REFERENCES data_api.organisation (id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION
)
Here's the test that fails
#Slf4j
#RunWith(SpringRunner.class)
#DataJpaTest(showSql = false)
#ContextConfiguration(classes = { DataApiJpaConfiguration.class })
#TestPropertySource(locations = {"classpath:application-it.properties","classpath:application-test.properties"})
#AutoConfigureTestDatabase(replace = Replace.NONE)
public class CustomerRepositoryIntegrationTest {
#Autowired
private CustomerRepository customerRepository;
#Test
public void testPrePersistAddsMandatoryFields() {
Customer bhau = new Customer();
bhau.setName("Bhau & Sons Pvt. Ltd.");
bhau.setDomain(RandomStringUtils.randomAlphanumeric(8));
bhau = customerRepository.saveAndFlush(bhau);
Customer badaBhau = customerRepository.findById(bhau.getId()).get();
assertThat(badaBhau.getTds().doubleValue()).isEqualTo(0.10);
assertThat(badaBhau.getInvoicePrefix()).isEqualTo("INV");
assertThat(badaBhau.getCurrency()).isEqualTo("INR");
}
}
The failure is a NullPointerException at assertThat(badaBhau.getTds().doubleValue()).isEqualTo(0.10); when I run on debug I notice that badaBhau indeed doesn't have the default values set when queried from the database. I then paused after the line when it persists bhau; browsed test database configured and noticed that the bhau entity isn't even saved to the database in the first place.
I also ran the app with Postgesql Dev DB trying to hit the API url for saving customer with the JSON data
{
"name":"Minty and Sons Pvt. Ltd.",
"pan": "AASONAL123",
"domain": "xy123456"
}
this call succeeds in creating the record of course with default values INR, INV, 0.10 for currency, invoice_prefix and tds respectively.
So while I understand and like how #ColumnDefault solves the default value issue for me; I'm utterly confused as to what why the test fails (or what am I doing wrong)

Due to Hibernate's first-level cache, bhau and badaBhau will be the same instance, viz. there will be no database lookup triggered by the following: instead the customer with the specified ID will be retrieved from the first level cache. You can verify this by enabling SQL logging.
Customer badaBhau = customerRepository.findById(bhau.getId()); //no db lookup
To have the value can force a database lookup by clearing the persistence context or by refreshing the persistent instance.
public class MyTestClass {
#PersistenceContext
private EntityManager em;
#Test
public void testDefaultFieldsArePopulated() {
Customer bhau = new Customer();
bhau.setName("Bhau & Sons Pvt. Ltd.");
bhau.setDomain(RandomStringUtils.randomAlphanumeric(8));
bhau = customerRepository.saveAndFlush(bhau);
em.clear(); //db lookup will now happen
Customer badaBhau = customerRepository.findById(bhau.getId());
assertThat(badaBhau.getTds().doubleValue()).isEqualTo(0.10);
assertThat(badaBhau.getInvoicePrefix()).isEqualTo("INV");
assertThat(badaBhau.getCurrency()).isEqualTo("INR");
}
}
https://howtodoinjava.com/hibernate/understanding-hibernate-first-level-cache-with-example/
Crizzis also raises a valid point in his answer and as the test passes it looks like Postgres is then replacing nulls with the column's default value. As you may not be able to depend on this across different database engines then you could also look at using Hibernates #DynamicInsert annotation on your entity which would create an insert statement with only non-null fields set.
https://docs.jboss.org/hibernate/orm/5.2/javadocs/org/hibernate/annotations/DynamicInsert.html
For inserting, should this entity use dynamic sql generation where
only non-null columns get referenced in the prepared sql statement?

Part of my answer was going to be what #AlanHay already mentioned.
Even when you follow his advice though, I'd still expect your test to fail. The purpose of #ColumnDefault is to use the specified default value with the column definition when executing DLL, no more, no less. The default value means: 'whenever the value for the column is not specified in an INSERT statement, use the following value instead'.
The thing is, you are specifying a value for the column, since your column is not excluded from the INSERT statement using insertable = false. Even if you didn't set a value to the property, Hibernate will send an explicit NULL. Your approach may work for currency and indexPrefix, since you marked them as nullable = false (this is DB-specific, and you'd have to consult the docs to see how Postgres handles such a situation). I highly doubt it will work for the tds column, since NULL is a valid value in that case.

Related

Hibernate #Lob on byte[] causes "Bad value for type long"

I'm trying to lazily fetch single byte[] content java property using Hibernate under Spring Boot, accessing PostgreSQL database. So I pulled together testing app for testing different solutions. One of them required me to use #Lob annotation on said property, so I did. Now reading entity from the database leads to very curious error, precisely:
Bad value for type long : \x454545454545445455
The value \x45... is value of bytea column not the bigint one, why is it trying to force it into the long even though it's wrong column? Why annotation on one column somehow affects another one?
As for fix, removing #Lob seems to work (at least in my stack) but the problem remains unexplained to me and I would like to know what is going rather than just blindly moving on. Is it bug or I am misunderstanding something completely?
Entity:
#Entity
#Table(name = "blobentity")
public class BlobEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Lob //this annotation breaks code
#Column(name = "content")
#Basic(fetch = FetchType.LAZY)
private byte[] content;
#Column(name = "name")
private String name;
//getters/setters
}
Repository:
#Repository
public interface BlobRepo extends JpaRepository<BlobEntity, Long> {
}
Calling code:
#Autowired
BlobRepo blobrepo;
#GetMapping("lazyBlob")
public String blob () {
var t = blobrepo.findAll().get(0);
var name = t.getName();
var dataAccessedIfLazy = t.getContent();
return t.getName();
}
Postgres DDL:
CREATE TABLE test.blobentity (
id bigserial NOT NULL DEFAULT nextval('test.blobentity_id_seq'::regclass),
"name" varchar NULL,
"content" bytea NULL,
CONSTRAINT blobentity_pk PRIMARY KEY (id)
);
Select result:
Used version:
PostgreSQL 10.4; springframework.boot 2.4.2; hibernate version that comes with this spring boot version
The bytea type is inlined into the table whereas other types are chunked into a separate table which is called TOAST on PostgreSQL. To access these values, database have a concept often referred to as a LOB locator which essentially is just an id for doing the lookup. Some drivers/databases just work either way but others might need to match the actual physical representation. In your case, using #Lob is just wrong because AFAIK bytea is inlined up to a certain size and de-TOASTed i.e. materialized automatically behind the scenes if necessary. If you were using the varbinary/blob type or something like that, you would have to use #Lob as in that case, the main table only contains this LOB locator which is a long. The driver then knows when you ask for the value by using getBlob that it has to execute some select get_lob(?) query to retrieve the actual contents.

Spring JPA does not detect deleted tables

I deleted my tables to let them be recreated by Spring JPA, but spring does not create them. Instead, I'm getting the following exception:
Unable to create unique key constraint (guild_id, setting_key) on table guild_setting: database column 'guild_id' not found. Make sure that you use the correct column name which depends on the naming strategy in use (it may not be the same as the property name in the entity, especially for relational types)
I don't know why I get this error, but before I deleted the tables in the database, the column name was guild_id, so what JPA says is not right.
This is an excerpt of the Entity:
#Entity
#Table(uniqueConstraints=#UniqueConstraint(columnNames={"guild_id", "setting_key"}))
public class GuildSetting extends Setting {
#Column(nullable = false)
private long guildId;
The following properties are set with the spring.datasource properties:
spring.jpa.database-platform=org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.generate-ddl=true
spring.jpa.hibernate.ddl-auto=update
If you don't want to use #Column(name="guild_id")
You should use #UniqueConstraint(columnNames={"guildId", ...}
The generated table will contain the (correct) column, named guild_id
#Entity
#Table(uniqueConstraints=#UniqueConstraint(columnNames={"guildId", "setting_key"}))
public class GuildSetting extends Setting {
#Column(nullable = false)
private long guildId;
Agreed, this looks like a bug...
Note: You did not mention you have problems with the setting_key unique constraint.
Do you use #Column(name="setting_key") or private String setting_key?

Default value declared in JPA #Column( columnDefinition ... ) not set when persisted

I have attribute in my Java entity like this:
#Basic(fetch = FetchType.LAZY) //I tried without this as well
#Column(name = "value_x", columnDefinition = "bigint default 0")
private Long valueX;
In table definition in pgAdmin I see:
value_x bigint DEFAULT 0,
but when object is inserted (with this attribute as null), column is empty/null not having 0 value inserted.
Anyone would know why it does not insert the default value? Using EclipseLink.
Null value is because JPA explicitly inserts a null value in to that column if no special handling is implemented. The columnDefinition does make the column creation with DEFAULT but it does not make JPA aware of/or obey it afterwards.
And it is a nullable column so there is no error. See what happens in plain SQL, think of this table:
create table example (
col1 bigint default 42,
col2 bigint default 99
);
Then make an insert like:
insert into example (col1) values (null);
then selecting:
select * from example;
would show result like:
col1 | col2
------+------
(null) | 99
If you need to have default values managed in the java side you need some special stuff in the Java side.
See for example this question and note that the accepted answer is not the working one but this answer. So setting the value when class is instantiated:
private Long valueX = 0;
Another way is as this answer for different question suggests, using #PrePersist:
#PrePersist
void prePersist() {
if (this.valueX == null)
this.valueX = 0;
}
I found another way to resolve the same problem, because when I create my own object and persist in database and didn´t respect the DDL with default value.
So I looked at my console, and the SQL generated, and saw that insert came with all fields, but only one propertie in my object has the value changed.
So I put in the model class this annotation.
#DynamicInsert
When is inserting data, the framework not insert null values or values that are not modified, making the insert shorter.
Also has #DynamicUpdate annotation.

Conditional insert with Spring JPA / Hibernate

I'm working on a project that runs in a clustered environment, where there are many nodes and a single database. The project uses Spring-data-JPA (1.9.0) and Hibernate (5.0.1). I'm having trouble resolving how to prevent duplicate row issues.
For sake of example, here's a simple table
#Entity
#Table(name = "scheduled_updates")
public class ScheduledUpdateData {
public enum UpdateType {
TYPE_A,
TYPE_B
}
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "id")
private UUID id;
#Column(name = "type", nullable = false)
#Enumerated(EnumType.STRING)
private UpdateType type;
#Column(name = "source", nullable = false)
private UUID source;
}
The important part is that there is a UNIQUE(type, source) constraint.
And of course, matching example repository:
#Repository
public class ScheduledUpdateRepository implements JpaRepository<ScheduledUpdateData, UUID> {
ScheduledUpdateData findOneByTypeAndSource(final UpdateType type, final UUID source);
//...
}
The idea for this example is that parts of the system can insert rows to be schedule for something that runs periodically, any number of times between said runs. When whatever that something is actually runs, it doesn't have to worry about operating on the same thing twice.
How can I write a service method that would conditionally insert into this table? A few things I've tried that don't work are:
Find > Act - The service method would use the repository to see if a entry already exists, and then either update the found entry or save a new one as needed. This does not work.
Try insert > Update if fail - The service method would try to insert, catch the exception due to the unique constraint, and then do an update instead. This does not work since the transaction will already be in a rolled-back state and no further operations can be done in it.
Native query with "INSERT INTO ... WHERE NOT EXISTS ..."* - The repository has a new native query:
#Repository
public class ScheduledUpdateRepository implements JpaRepository<ScheduledUpdateData, UUID> {
// ...
#Modifying
#Query(nativeQuery = true, value = "INSERT INTO scheduled_updates (type, source)" +
" SELECT :type, :src" +
" WHERE NOT EXISTS (SELECT * FROM scheduled_updates WHERE type = :type AND source = :src)")
void insertUniquely(#Param("type") final String type, #Param("src") final UUID source);
}
This unfortunately also does not work, as Hibernate appears to perform the SELECT used by the WHERE clause on its own first - which means in the end multiple inserts are tried, causing a unique constraint violation.
I definitely don't know a lot of the finer points of JTA, JPA, or Hibernate. Any suggestions on how insert into tables with unique constraints (beyond just the primary key) across multiple JVMs?
Edit 2016-02-02
With Postgres (2.3) as a database, tried using Isolation level SERIALIZABLE - sadly by itself this still caused constraint violation exceptions.
You are trying to ensure that only 1 node can perform this operation at a time.
The best (or at least most DB-agnostic) way to do this is with a 'lock' table.
This table will have a single row, and will act as a semaphore to ensure serial access.
Make sure that this method is wrapped in a transaction
// this line will block if any other thread already has a lock
// until that thread's transaction commits
Lock lock = entityManager.find(Lock.class, Lock.ID, LockModeType.PESSIMISTIC_WRITE);
// just some change to the row, it doesn't matter what
lock.setDateUpdated(new Timestamp(System.currentTimeMillis()));
entityManager.merge(lock);
entityManager.flush();
// find your entity by unique constraint
// if it exists, update it
// if it doesn't, insert it
Hibernate and its query language offer support for an insert statement. So you can actually write that query with HQL. See here for more information. http://docs.jboss.org/hibernate/orm/5.0/userguide/html_single/Hibernate_User_Guide.html#_hql_syntax_for_insert
It sounds like an upsert case, that can be handled as suggested here.
Find > Act - The service method would use the repository to see if a entry already exists, and then either update the found entry or save a new one as needed. This does not work.
Why does this not work?
Have you considered "optimistic locking"?
These two posts may help:
https://www.baeldung.com/jpa-optimistic-locking
https://www.baeldung.com/java-jpa-transaction-locks

Primitive type in JPA mapping. What if the database column may be NULL?

Given this class mapped with JPA (using JPA 1.0 and Hibernate):
#Entity
public class Foo {
private int bar;
/* ... */
}
What happens if I try to load a record which has the BAR column set to NULL?
Can I specify how to set the bar field when the corresponding column is NULL?
Notes
I know this is not a best practice. The question is more out of curiosity and it is inspired by this situation:
The database table is a staging table: adding a NOT NULL constraint is impractical. Bad data is expected, and the point of my code is to validate, clean up and/or reject data before loading it into the "real" database.
I have acceptable default values for some fields. For example, a boolean flag which should default to false.
I would rather use objects if a column may contain null value because Matheus's idea introduces false data. NULL <> 0!
Exception
2.
#Column(name = “bar”, nullable = false, columnDefinition = “bigint(20) default 0″)
private int bar;
it solves your problem.

Categories

Resources