I have a spring boot service that should persist several entities of two types in a transaction to an Oracle DB. The table for the first entity type is huge (3 Mio. entries/day, partitioned,...) and I have the issue that I need to react on duplicates. There are some fields I use to create a hash on and I have a unique constraint in the database on that field. I thought it is a clever idea to just saveAndFlush entity by entity and react on the ConstraintViolationException. Based on the result of saving the list of first entities, I need to create the second entity and save that as well, but it rolls back everything.
My question now would be if this approach is generally wrong, or ok and there is some small issue? If it is generally wrong, how should I do this duplicate check then (a select upfront is not an option)?
Here is some pseudo-code to get a better idea
#Entity
public class Foo{
public String uniqueHash;
// couple of other properties that will be used to calculate the hash
}
#Entity
public class Bar{
private List goodIds;
private List badIds;
public Bar(List goodIds, List badIds){
this.goodIds = goodIds;
this.badIds = badIds;
}
}
#Repository
#Transactional(noRollbackFor = PersistenceException.class)
public interface FooRepository extends JpaRepository<Foo, String> {
Foo saveAndFlush(Foo f) throws PersistenceException;
}
#Repository
#Transactional(noRollbackFor = PersistenceException.class)
public interface BarRepository extends JpaRepository<Bar, String> {
Bar saveAndFlush(Bar b) throws PersistenceException;
}
SomeService
#Transactional(noRollbackFor = PersistenceException.class)
public void doSomething(List<Foo> foos){
List<String> goodIds = new ArrayList();
List<String> badIds = new ArrayList();
for (Foo foo : foos) {
try {
fooRepository.saveAndFlush(foo);
goodIds.add(foo.getId());
} catch (PersistenceException e) {
if (e.getCause() instanceof ConstraintViolationException) {
badIds.add(foo.getId);
} else {
throw e;
}
}
}
barRepository.saveAndFlush(new Bar(goodIds, badIds));
}
Finally, I found a way to achieve the expected behavior, and even better, I was able to get rid of these "noRollBackFor" attributes. I only restructured the process and try to save everything in a transaction, if it fails, the Exception is caught on the calling method, the input is "cleaned" and the transactional method is called again (recursively). These duplicates are rare situations (happens every 10k Foo instance), so from a performance perspective, it's fine to have these subsequent transactions. Here is the changed pseudo-code again
#Entity
public class Foo{
public String uniqueHash;
// couple of other properties that will be used to calculate the hash
}
#Entity
public class Bar{
private List goodIds;
private List badIds;
public Bar(List goodIds, List badIds){
this.goodIds = goodIds;
this.badIds = badIds;
}
public List getGoodIds(){
return goodIds;
}
public List getBadIds(){
return badIds;
}
}
#Repository
public interface FooRepository extends JpaRepository<Foo, String> {
}
#Repository
public interface BarRepository extends JpaRepository<Bar, String> {
}
public class FooException extends RuntimeException {
private final Foo foo;
public FooException(String message, Foo foo) {
super(message);
this.foo = foo;
}
public getFoo(){
return foo;
}
}
SomeService
public void doSomething(List<Foo> foos, Bar bar){
try{
doSomethingTransactional(foos,bar);
}
catch (FooException e) {
bar.getBadIds().add(e.getFoo().getId());
foos.remove(foo);
doSomething(foos, bar);
}
}
#Transactional
public void doSomethingTransactional(List<Foo> foos, Bar bar){
for (Foo foo : foos) {
try {
fooRepository.saveAndFlush(foo);
bar.getGoodIds.add(foo.getId());
} catch(DataAccessException e) {
if (e.getCause() instanceof ConstraintViolationException
&& ((ConstraintViolationException) e.getCause()).getConstraintName().contains("Some DB Message")) {
throw new FooException("Foo already exists", foo);
} else {
throw e;
}
}
}
barRepository.saveAndFlush(bar);
}
You might be able to use a custom #SQLInsert to make use of Oracles MERGE statement for this purpose. Also see https://stackoverflow.com/a/64764412/412446
Related
I have faced this problem a few times in the past, but haven't really found a good solution/design for it.
The below example code will generate PDF doc from Entity (Company or Article)
public class Entity
{
int id;
}
public class Company extends Entity
{
private String HQ;
}
public class Article extends Entity
{
private String title;
}
public interface EntityPDFGenerator
{
void generate(Entity entity);
}
public class ArticlePDFGenerator implements EntityPDFGenerator
{
public void generate(Entity entity)
{
Article article = (Article) entity;
// create Article related PDF from entity
}
}
public class CompanyPDFGenerator implements EntityPDFGenerator
{
public void generate(Entity entity)
{
Company company = (Company) entity;
// create Company related PDF
}
}
Main class:
public class PDFGenerator
{
public void generate(Entity entity)
{
EntityPDFGenerator pdfGenerator = getConcretePDFGenerator(entity);
pdfGenerator.generate(entity);
}
// lets make the factory task simple for now
EntityPDFGenerator getConcretePDFGenerator(Entity entity)
{
if(entity instanceof Article){
return new ArticlePDFGenerator();
}else{
return new CompanyPDFGenerator();
}
}
}
In the above approach the problem is with the casting the Entity to the concrete type (casting can be dangerous in later stage of the code). I tried to make it with generics, but then I get the warning
Unchecked call to 'generate(T)'
Can I improve this code?
Here, you go with the suggested changes:
public interface EntityPDFGenerator<T extends Entity> {
void generate(T entity);
}
public class ArticlePDFGenerator implements EntityPDFGenerator<Article> {
public void generate(Article entity)
{
// create Article related PDF from entity
}
}
public class CompanyPDFGenerator implements EntityPDFGenerator<Company> {
public void generate(Company entity)
{
// create Company related PDF
}
}
Short answer
Generics is not the right tool here. You can make the casting explicit:
public class CompanyPDFGenerator implements EntityPDFGenerator
{
public void generate(Entity entity)
{
if (! (entity instanceof Company)) {
throw new IllegalArgumentException("CompanyPDFGenerator works with Company object. You provided " + (entity == null ? "null" : entity.getClass().getName()));
}
Company company = (Company) entity;
System.out.println(company);
// create Company related PDF
}
}
Or you can define some sort of data structure in the entity class and use only that in the printer:
public abstract class Entity
{
int id;
public abstract EntityPdfData getPdfData();
}
// ...
public class CompanyPDFGenerator implements EntityPDFGenerator
{
public void generate(Entity entity)
{
EntityPdfData entityPdfData = entity.getPdfData();
// create Company related PDF
}
}
Long answer
Generics is useful if you know the types at compile-time. I.e. if you can write into your program that actual type. For lists it looks so simple:
// now you know at compile time that you need a list of integers
List<Integer> list = new ArrayList<>();
In your example you don't know that:
public void generate(Entity entity)
{
// either Article or Company can come it. It's a general method
EntityPDFGenerator pdfGenerator = getConcretePDFGenerator(entity);
pdfGenerator.generate(entity);
}
Suppose you want to add type to the EntityPDFGenerator , like this:
public static interface EntityPDFGenerator<T extends Entity>
{
void generate(T entity);
}
public static class ArticlePDFGenerator implements EntityPDFGenerator<Article>
{
public void generate(Article entity)
{
Article article = (Article) entity;
// create Article related PDF from entity
}
}
public static class CompanyPDFGenerator implements EntityPDFGenerator<Company>
{
public void generate(Company entity)
{
Company company = (Company) entity;
// create Company related PDF
}
}
This looks nice. However, getting the right generator will be tricky. Java generics is invariant. Even ArrayList<Integer> is not a subclass of ArrayList<Number>. So, ArticlePdfGenerator is not a subclass of EntityPDFGenerator<T extends Entity>. I.e. this will not compile:
<T extends Entity> EntityPDFGenerator<T> getConcretePDFGenerator(T entity, Class<T> classToken)
{
if(entity instanceof Article){
return new ArticlePDFGenerator();
}else{
return new CompanyPDFGenerator();
}
}
I would suggest to move the getGenerator() method in the Entity class and override it in the Company and Article classes.
Unless, of course, there is a good reason not to.
Is there a better way to deal with an instanciation of an object (Product) which depends upon another object type (Condition) than using if-else paired with instanceof as the following code shows?
import java.util.ArrayList;
import java.util.List;
abstract class AbstractProduct {
private AbstractCondition condition;
public AbstractProduct(AbstractCondition condition) {
this.condition = condition;
}
public abstract void doSomething();
}
class ProductA extends AbstractProduct {
AbstractCondition condition;
public ProductA(AbstractCondition condition) {
super(condition);
}
#Override
public void doSomething() {
System.out.println("I'm Product A");
}
}
class ProductB extends AbstractProduct {
public ProductB(AbstractCondition condition) {
super(condition);
}
#Override
public void doSomething() {
System.out.println("I'm Product B");
}
}
class AbstractCondition { }
class ConditionA extends AbstractCondition { }
class ConditionB extends AbstractCondition { }
public class Try {
public static void main(String[] args) {
List<AbstractCondition> conditions = new ArrayList<AbstractCondition>();
List<AbstractProduct> products = new ArrayList<AbstractProduct>();
conditions.add(new ConditionA());
conditions.add(new ConditionB());
conditions.add(new ConditionB());
conditions.add(new ConditionA());
for (AbstractCondition c : conditions) {
tryDoSomething(c);
}
}
public static void tryDoSomething(AbstractCondition condition) {
AbstractProduct product = null;
if (condition instanceof ConditionA) {
product = new ProductA(condition);
} else if (condition instanceof ConditionB) {
product = new ProductB(condition);
}
product.doSomething();
}
}
The difference with the code above of my real code is: I have NO direct control over AbstractCondition and its subtypes (as they are in a library), but the creation of a concrete subtype of AbstractProduct depends on the concrete condition.
My goal being: try to avoid the if-else code smell in tryDoSomething().
I would also like to avoid reflection because it feels like cheating and I do think it's not an elegant, clean and readable solution.
In other words, I would like to tackle the problem just with good OOP principles (e.g. exploiting polymorphism) and pheraps some design patterns (which apparently I don't know in this specific case).
Since you can't edit the original objects, you need to create a static map from condition type to product type:
private static HashMap< Class<? extends AbstractCondition>,
Class<? extends AbstractProduct>
> conditionToProduct;`
Fill it in static initialization with the pairs of Condition,Product:
static {
conditionToProduct.put(ConditionA.class, ProductA.class);
...
}
and in runtime just query the map:
Class<? extends AbstractProduct> productClass = conditionToProduct.get(condition.getClass());
productClass.newInstance();
AbstractCondition needs to know either the type or how to construct a product.
So add one of the following functions to AbstractCondition
Class<? extends AbstractProduct> getProductClass()
or
AbstractProduct createProduct()
You should create a Factory class to help you with that then.
interface IFactoryProduct{
AbstractProduct getProduct(AbstractCondition condition) throws Exception;
}
This will be your interface, just need to implement it like this.
class FactoryProduct implements IFactoryProduct{
public AbstractProduct getProduct(AbstractCondition condition) throws Exception{
return (AbstractProduct)getClass().getMethod("getProduct", condition.getClass()).invoke(this, condition);
}
public ProductA getProduct(ConditionA condition){
return new ProductA();
}
public ProductB getProduct(ConditionB condition){
return new ProductB();
}
}
Using the reflexion to redirect with the correct method will do the trick. this is upgradable for subclassed if you want.
EDIT:
Some example :
List<AbstractCondition> list = new ArrayList<AbstractCondition>();
list.add(new ConditionA());
list.add(new ConditionB());
for(AbstractCondition c : list){
try {
System.out.println(f.getProduct(c));
} catch (Exception ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
}
labo.ProductA#c17164
labo.ProductB#1fb8ee3
A more complexe reflexion version allowing a subclass to be received :
public AbstractProduct getProduct(AbstractCondition condition) throws Exception{
Method m = getMethodFor(condition.getClass());
if(m == null )
throw new Exception("No method for this condition " + condition.getClass().getSimpleName());
else
return (AbstractProduct) m.invoke(this, condition);
}
private Method getMethodFor(Class<? extends AbstractCondition> clazz ) throws Exception{
try {
return getClass().getMethod("getProduct", clazz);
} catch (NoSuchMethodException ex) {
if(clazz.getSuperclass() != AbstractCondition.class){
return getMethodFor((Class<? extends AbstractCondition>)clazz.getSuperclass());
}
return null;
}
}
This allows me to send ConditionC extending ConditionB to build the same product has ConditionB would. Interesting for complexe heritage.
I have some helper classes for testing with the following structure:
public class EntitiesForTest {
public static Entity firstEntity() {
return new Entity(/*some dummy data*/)
}
public static Entity secondEntity() {...}
...
public static Entity nthEntity() {...}
public static List<Entity> allEntities() {???}
}
The purpose of this classes is to have some objects to test the upper layers of my system, like having some JSON data to test a REST service. This technique is not mine but from an online course I'm taking and is pretty cool.
I'd like to know if there is a way to construct a List<Entity> based on the static inherited methods of the class and with the Collection framework. I can do Arrays.asList(/*call the methods one by one comma-separated*/) but there must be a smarter, functional and reusable way of doing this.
Thanks in advance for your answers.
"Modern" Way
public static List<Entity> getEntities() {
return Arrays.stream(Foo.class.getMethods()).
filter(method -> method.getReturnType() == Entity.class &&
Modifier.isStatic(method.getModifiers())).
map(method -> {
try {
return (Entity)method.invoke(null);
} catch (IllegalAccessException | InvocationTargetException e) {
throw new RuntimeException(e); // exceptions in streams... lol
}
}).
collect(Collectors.toList());
}
I wish I knew a way to avoid the cast, but that's not immediately clear to me.
Here is the samele code for invoking specified methods of EntitiesForTest class and collect the return objects:
public static List<Entity> allEntities() {
ArrayList<Entity> list = new ArrayList<Entity>();
Method[] ma = EntitiesForTest.class.getMethods();
Object[] emptyObject = new Object[0];
for (int i = 0; i < ma.length; i++) {
if (ma[i].getReturnType().equals(Entity.class) &&
ma[i].getParameterTypes().length == 0 &&
Modifier.isStatic(ma[i].getModifiers())) {
try {
Entity e = (Entity)(ma[i].invoke(null, emptyObject));
list.add(e);
} catch (Exception e) {
e.printStackTrace();
}
}
}
return list;
}
This answer makes several assumptions:
You don't need the individual methods (firstEntity, secondEntity, etc.)
The entities only hold data and are not expensive to create and keep around
You don't need to modify them, which means you wouldn't call your methods multiple times.
These might not hold, because we have no definition of Entity or how it's used.
All in all, I'm just removing your methods.
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
public class EntitiesForTest {
private static final List<Entity> entities = Arrays.asList(
new Entity(),
new Entity(),
new Entity()
);
public static Entity allEntities(int n) {
return entities.get(n);
}
public static List<Entity> allEntities() {
return Collections.unmodifiableList(entities);
}
}
Maybe using a functional way (in java 8 ) ?
public class Entities {
static class Entity{
private String x;
Entity( String x){
this.x = x;
}
public String getX(){
return x;
}
}
public static Entity firstEntity() {
return new Entity("first Entity");
}
public static Entity secondEntity() {
return new Entity("second Entity");
}
public static Entity nthEntity() {
return new Entity("nth Entity");}
#FunctionalInterface interface GetEntity{
public Entity getEntity();
}
public static List<GetEntity> allEntities =
Arrays.asList(Entities::firstEntity,
Entities::secondEntity,
Entities::nthEntity);
public static void main(String ...p){
allEntities
.stream()
.forEach(x->{System.out.println(x.getEntity().getX());});
}
}
Could you guys please help me find where I made a mistake ?
I switched from SimpleBeanEditorDriver to RequestFactoryEditorDriver and my code no longer saves full graph even though with() method is called. But it correctly loads full graph in the constructor.
Could it be caused by circular reference between OrganizationProxy and PersonProxy ? I don't know what else to think :( It worked with SimpleBeanEditorDriver though.
Below is my client code. Let me know if you want me to add sources of proxies to this question (or you can see them here).
public class NewOrderView extends Composite
{
interface Binder extends UiBinder<Widget, NewOrderView> {}
private static Binder uiBinder = GWT.create(Binder.class);
interface Driver extends RequestFactoryEditorDriver<OrganizationProxy, OrganizationEditor> {}
Driver driver = GWT.create(Driver.class);
#UiField
Button save;
#UiField
OrganizationEditor orgEditor;
AdminRequestFactory requestFactory;
AdminRequestFactory.OrderRequestContext requestContext;
OrganizationProxy organization;
public NewOrderView()
{
initWidget(uiBinder.createAndBindUi(this));
requestFactory = createFactory();
requestContext = requestFactory.contextOrder();
driver.initialize(requestFactory, orgEditor);
String[] paths = driver.getPaths();
createFactory().contextOrder().findOrganizationById(1).with(paths).fire(new Receiver<OrganizationProxy>()
{
#Override
public void onSuccess(OrganizationProxy response)
{
if (response == null)
{
organization = requestContext.create(OrganizationProxy.class);
organization.setContactPerson(requestContext.create(PersonProxy.class));
} else
organization = requestContext.edit(response);
driver.edit(organization, requestContext);
}
#Override
public void onFailure(ServerFailure error)
{
createConfirmationDialogBox(error.getMessage()).center();
}
});
}
private static AdminRequestFactory createFactory()
{
AdminRequestFactory factory = GWT.create(AdminRequestFactory.class);
factory.initialize(new SimpleEventBus());
return factory;
}
#UiHandler("save")
void buttonClick(ClickEvent e)
{
e.stopPropagation();
save.setEnabled(false);
try
{
AdminRequestFactory.OrderRequestContext ctx = (AdminRequestFactory.OrderRequestContext) driver.flush();
if (!driver.hasErrors())
{
// Link to each other
PersonProxy contactPerson = organization.getContactPerson();
contactPerson.setOrganization(organization);
String[] paths = driver.getPaths();
ctx.saveOrganization(organization).with(paths).fire(new Receiver<Void>()
{
#Override
public void onSuccess(Void arg0)
{
createConfirmationDialogBox("Saved!").center();
}
#Override
public void onFailure(ServerFailure error)
{
createConfirmationDialogBox(error.getMessage()).center();
}
});
}
} finally
{
save.setEnabled(true);
}
}
}
with() is only used for retrieval of information, so your with() use with a void return type is useless (but harmless).
Whether a full graph is persisted is entirely up to your server-side code, which is intimately bound to your persistence API (JPA, JDO, etc.)
First, check that the Organization object you receive in your save() method on the server-side is correctly populated. If it's not the case, check your Locators (and/or static findXxx methods) ; otherwise, check your save() method's code.
Judging from the code above, I can't see a reason why it wouldn't work.
It took me some time to realize that the problem was the composite id of Person entity.
Below is the code snippet of PojoLocator that is used by my proxy entities.
public class PojoLocator extends Locator<DatastoreObject, Long>
{
#Override
public DatastoreObject find(Class<? extends DatastoreObject> clazz, Long id)
{
}
#Override
public Long getId(DatastoreObject domainObject)
{
}
}
In order to fetch child entity from DataStore you need to have id of a parent class. In order to achieve that I switched "ID class" for Locator<> to String which represents textual form of Objectify's Key<> class.
Here is how to looks now:
public class PojoLocator extends Locator<DatastoreObject, String>
{
#Override
public DatastoreObject find(Class<? extends DatastoreObject> clazz, String id)
{
Key<DatastoreObject> key = Key.create(id);
return ofy.load(key);
}
#Override
public String getId(DatastoreObject domainObject)
{
if (domainObject.getId() != null)
{
Key<DatastoreObject> key = ofy.fact().getKey(domainObject);
return key.getString();
} else
return null;
}
}
Please note that your implementation may slightly differ because I'm using Objectify4.
I want to create a Factory which returns dao-instance, depending on the Class clazz
Teammember, Scene and Equipment are my Model Classes.
My DAO's look like this:
public class JDBCTeammemberDAO implements JdbcDAO<Teammember>
my Factory looks like this:
public class DAOFactory {
JdbcDAO createDAO(Class clazz) {
if(clazz.equals(Teammember.class)) {
return new JDBCTeammemberDAO();
}
if(clazz.equals(Scene.class)) {
return new JDBCSceneDAO();
}
if(clazz.equals(Equipment.class)) {
return new JDBCEquipmentDAO();
}
return null;
}
}
I was thinking about switch and polymorphism, but I couldn't figure out how.
Basically I want to find the Implementation "SomeClass implements JdbcDAO"
My first approach was:
String name = clazz.getName().substring(6); // model.Teammember
Class<?> forName;
try {
forName = Class.forName("dao.jdbc.JDBC" + name + "DAO");
return (JdbcDAO) forName.newInstance();
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}
but I don't feel good with handling this with String method. Besides, it doesn't work, if I have different Model and Dao names (like: JDBCMemberDAO instead of JDBCTeammemberDAO)
I was in a similar situation and decided to use a Dao registry to handle the issue. Using the generic dao pattern #Perception mentioned:
public interface JdbcDAO<T> {
T find(Long id) ;
T create(T entity);
T update(T entity);
void delete(T entity);
}
public class JdbcDAOImpl<T> {
this.clazz = clazz;
DaoRegistry.register (clazz, this);
}
Then you can have your
public class JDBCTeammemberDAO extend JdbcDAOImpl<TeamMember> {
public class JDBCTeammemberDAO () {
super(TeamMember.class);
}
}
DaoRegistry would look something like this:
public class DaoRegistry {
private Map<Class, JdbcDAO> daoMap;
public synchronized void register (Class type, JdbcDao dao) {
if (!daoMap.containsKey(type))
daoMap.put (type, dao);
else
logger.error ("Something is really wrong because you are creating another dao for this class.", e);
}
public JdbcDAO get(Class type) {return daoMap.get(type);
}
This is just the jest of it, you will need to make sure it is thread-safe. Hope this helps.
Sormula works as you describe. It provides a method to get the "DAO" for a row/record class. See database.getTable(Inventory.class); in example 1. You don't need to write any DAO's.
If you don't mind a slight redesign, this problem is easy enough to solve with a little bit of Generics and Polymorphism:
public interface JdbcDAO<T> {
T find(Long id) ;
T create(T entity);
T update(T entity);
void delete(T entity);
// Other common definitions
}
public class JdbcDAOImpl<T> {
private Class<T> clazz;
public JdbcDAOImpl() {
super();
}
protected JdbcDAOImpl(Class<T> clazz) {
super();
this.clazz = clazz;
}
// Common implementation here
}
public class EquipmentDAO extends JdbcDAOImpl<Equipment> {
public EquipmentDAO() {
super(Equipment.class);
}
// Subclass specific implementation here
}
Rinse and repeat the specific impl for each of your Models and instantiate them directly (without use of a factory).