Develop Reference

Neo4j Query : Find the Average of values found after fetching the latest record (By Timestamp) for each row

"Neo4j Query"
Find the Average of values found after fetching the latest record (By Timestamp) for each row of a Java object(Node) which contains a List of another Java object(Node). Please have a look at below classes :
Class Train contains a List of Attributes object.
#Node
#Data
#NoArgsConstructor
#SuperBuilder
#AllArgsConstructor
public class Train {
#Id
#GeneratedValue
private Long id;
private String trainId;
#Relationship(type = "HAS", direction = Relationship.Direction.OUTGOING)
private List<Attributes> attributes;
private String timestamp;
}
Attributes class is as below.
#Node
#Data
#NoArgsConstructor
#SuperBuilder
#AllArgsConstructor
public class Attributes {
#Id
#GeneratedValue
private Long id;
private String name;
private String value;
}
The records in DB look like below :
{"id" : "123", "trainId": "train1","timestamp": "2022-05-27T10:10:10+00:00","attributes": [{"name":"fuelPercentLeft","value": "48"},{"name": "engineTemp","value": "56"}]},
**{"id" : "456", "trainId": "train1","timestamp": "2022-05-27T10:49:10+00:00","attributes": [{"name":"fuelPercentLeft","value": "39"},{"name": "engineTemp","value": "59"}]}**,
{"id" : "789", "trainId": "train2","timestamp": "2022-05-27T14:10:10+00:00","attributes": [{"name":"fuelPercentLeft","value": "88"},{"name": "engineTemp","value": "73"}]},
**{"id" : "983", "trainId": "train2","timestamp": "2022-05-27T16:22:10+00:00","attributes": [{"name":"fuelPercentLeft","value": "65"},{"name": "engineTemp","value": "71"}]}**,
{"id" : "553", "trainId": "train3","timestamp": "2022-05-27T23:10:10+00:00","attributes": [{"name":"fuelPercentLeft","value": "22"},{"name": "engineTemp","value": "44"}]},
**{"id" : "801", "trainId": "train3","timestamp": "2022-05-27T23:52:10+00:00","attributes": [{"name":"fuelPercentLeft","value": "20"},{"name": "engineTemp","value": "46"}]}**
Note : Database stores the historical data, that is trainId are repeated. Class Train contains the List of different attributes. The Bold records are the latest entries available in the database.
We need to write a Neo4j Cypher query which will fetch the latest record for every Distinct trainID. For these distinct trainIds we to find the average of fuelPercentLeft of all the fetched trains.
e.g. The output of above mentioned sample data should be 41.333. Because the the distinct records we found by applying latest Timestamp are as below :
{"id" : "456", "trainId": "train1","timestamp": "2022-05-27T10:49:10+00:00","attributes": [{**"name":"fuelPercentLeft","value": "39"**},{"name": "engineTemp","value": "59"}]},
{"id" : "983", "trainId": "train2","timestamp": "2022-05-27T16:22:10+00:00","attributes": [{**"name":"fuelPercentLeft","value": "65"**},{"name": "engineTemp","value": "71"}]},
{"id" : "801", "trainId": "train3","timestamp": "2022-05-27T23:52:10+00:00","attributes": [{**"name":"fuelPercentLeft","value": "20"**},{"name": "engineTemp","value": "46"}]}
Note the highlighted fields above. Average of fuelPercentLeft in all three trains is 41.333 (AVG = (39+65+20)/3).
Requirement : Query should fetch all the latest train records by timestamp available in DB and should return the AVERAGE value of their fuelPercentLeft. So the output should be 41.3333 in above case.
Below query is not working as expected :
MATCH (n:Train)-[:HAS]->(m:Attributes{name:'fuelPercentLeft'}) return n.trainId , MAX(n.timestamp) , m.value
Kindly help! Thank you.

Try this:
MATCH (t:Train)
WITH t.trainId AS trainid, MAX(t.timestamp) AS maxTimeStamp
MATCH (n:Train{trainId: trainid, timestamp: maxTimeStamp})-[:HAS]->(m:Attributes{name:'fuelPercentLeft'})
WITH count(n) AS trainCount, SUM(m.value) AS totalFuel
RETURN (toFloat(totalFuel) / trainCount) AS averageFuel
Here, we are finding the maxTimeStampfor each trainId, and then we fuel values for those trains, finally, we calculate the average.

Creating an entity with a ManyToOne annotation

I'm creating an entity which has a ManyToOne relation to another one, it being multiple customers can belong to one company. Now I have my Customer entity defined as follows:
#Entity
#Table(name = "Customers")
data class Customer(
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
var customerId: Long? = null,
var firstName: String,
var lastName: String,
var gender: String,
#Column(insertable = false, updatable = false)
var companyId: Long,
#ManyToOne(targetEntity = Company::class, fetch = FetchType.LAZY)
#JoinColumn(name = "companyId", referencedColumnName = "companyId")
var company: Company? = null,
var profilePicture: String,
var email: String,
var phone: String,
var birthDay: String,
var bio: String,
var notifyByPhone: Boolean,
var notifyByEmail: Boolean,
var notifyBySms: Boolean,
#UpdateTimestamp
var updatedAt: LocalDateTime,
#CreationTimestamp
var createdAt: LocalDateTime
)
I use the following function from CustomerResource as an endpoint to persist the Customer:
#Transactional
#POST
fun post(#Valid entity: Customer): Response = try {
repository.persist(entity)
created(entity)
} catch (exc: Exception) {
serverError(exc)
}
Using Postman, I use the following JSON object to fire a request to the endpoint:
{
"firstName": "Ricardo",
"lastName": "de Vries",
"gender": "male",
"companyId": "200",
"profilePicture": "test",
"email": "mail#gmail.com",
"phone": "0612536263",
"birthDay": "28-12-1995",
"bio": "Nothing",
"notifyByPhone": true,
"notifyByEmail": true,
"notifyBySms": true
}
I have a property called "company" which is being mapped to the Company the user belongs to, based on the companyId. I have a separate companyId field which is being mapped to the companyId field in the database.
When I want to create a new Customer, I'm including the companyId in the request. This succeeds and the Customer is being successfully created.
Now when I try to fetch that specific customer, I get the following
error: "org.hibernate.PropertyAccessException: Null value was assigned to a property [class org.acme.domains.core.models.entities.Customer.companyId] of primitive type setter of org.acme.domains.core.models.entities.Customer.companyId".
I don't really get why this error occurs. I would think that the Company which belongs to the Customer gets added afterwards whenever I fetch a specific Customer.
Does anybody know how to be able to add an entity this way?

Do you have the proper getter and setters generated? It seems like the first request is failing to set the fields properly. I suggest you put a breakpoint to the controller that creates your customer and check if the field is being set like it should. Maybe you are passing the data in a wrong format etc. I can't say that from this information but you should debug that.
You can also put #NotNull validation on your fields to see if it fails to be set in object creation.

You have mapped companyId as not insertable and not updatable, so it won't appear in the query.
Try change it to:
#Column(insertable = true, updatable = true)
Long companyId
What's happening, I think, it's that Hibernate ORM is creating the insert query with all the other values, except for this one.
I would recommend enabling the log to check. You can do it in the configuration by setting:
hibernate.show_sql = true
hibernate.format_sql = true

Java - Attach extra column from #ManyToMany relashionship to entity model

Scenario:
I have 3 tables on MySQL server thats interacting with #ManyToMany, everything is working as well, but I need to retrieve another column rather than the column named on #ManyToMany annotation, like:
Midias
|
|_ midiasId
|
|...
|#ManyToMany...
|_ veiculos
Veiculos
|
|_ veiculosId
|
|...
|
|#ManyToMany
|- midias
|...
|
|- bonus <- I like this variable here fetched from MidiaVeiculos table and attached to Veiculos entity*
MidiaVeiculos
|
|- midiaCodigo
|- veiculoCodigo
|- bonus
I don't know if my code is fine, but I believe that have some other method more efficient, like:
veiculos.getBonus();
My code:
private List<MidiaResponseBonus> listarMidiasPorFiltros(BuscarMidiaRequest request) {
List<MidiaModel> midias = this.mRepository.findAll(request.listarMidiasPorFiltros());
List<MidiaResponseBonus> mb = midias.stream()
.map(m -> {
List<VeiculoResponseComBonus> veiculos = m.getVeiculos().stream().map(v -> new VeiculoResponseComBonus(
v.getCodigo(),
v.getNumeroOrdem(),
v.getAnoFabricacao(),
v.getEmpresa(),
v.getRota(),
v.getOperacional(),
this.mvRepository.findByMidiaCodigoAndVeiculoCodigo(m.getCodigo(), v.getCodigo()).getBonus()
)).collect(Collectors.toList());
veiculos.forEach(v -> System.out.println(v.getBonus()));
return new MidiaResponseBonus(
m.getCodigo(),
m.getCliente(),
m.getCampanha(),
m.getProduto(),
m.getPedidoInsercao(),
m.getPedidoProducao(),
m.getInicioPeriodo(),
m.getTerminoPeriodo(),
m.getInicioFotos(),
m.getTerminoFotos(),
veiculos
);
}).collect(Collectors.toList());
return mb;
}
I need to retrieve the bonus column on the midiaVeiculos table
Can I get some way with #ManyToOne and #OneToMany?
Thank you!

To my knowledge, #ManyToMany, #OneToMany, etc are all relations between Entities. An Entity A could have a relation to Entity B through a specific attribute.
If you wanted your Veiculos to have knowledge about bonus, then you will need a #OneToOne relation to MidiaVeiculos:
#Entity
public class Veiculos{
#OneToOne
MidiVeiculous midi;
}
veiculos.getMidi().getBonus();
https://docs.jboss.org/hibernate/jpa/2.1/api/javax/persistence/OneToOne.html
Another option which could work is using SecondaryTable:
#Entity
#Table(name = "Veiculos")
#SecondaryTable(name = "MidiaVeiculos", pkJoinColumns = #PrimaryKeyJoinColumn(name = "MidiaVeiculos_id"))
class Veiculos{
#Column(name = "bonus", table = "MidiaVeiculos")
String bonus;
}
Still however, there must be a relation between the two tables. Like a PK-FK of some kind.
See: Mapping one entity to several tables
https://docs.jboss.org/hibernate/core/3.6/reference/en-US/html_single/#mapping-declaration-join

Disable warning "entity was modified, but it won't be updated because the property is immutable" when saving entity with fields marked updatable=false

I have some fields marked with #Column(name = "column1", updatable = false) in entities in jpa repository, and it works as intended but when saving the entity a warning message like this is show:
2020-04-23 18:48:34.358 WARN 1112 --- [nio-8080-exec-6] o.h.p.entity.AbstractEntityPersister : HHH000502: The [column1] property of the [com.nodobanka.core.data.model.Entity1] entity was modified, but it won't be updated because the property is immutable.
I just want to know How Can I stop this warning from printing in log?.

You can set logging level for org.hibernate.persister.entity package to ERROR. That way it'll only display logs with ERROR and FATAL levels.
You can do this in Spring by adding following line in application.properties file:
logging.level.org.hibernate.persister.entity: ERROR

You can use relation for updates (instead of FK field).
Kotlin example:
data class ... (
...
) {
#Column(name = "market_id", nullable = false, insertable = false, updatable = false)
var marketId: Long = 0
/**
* This field is used for updates instead of [marketId] to avoid warning about immutability
*/
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "market_id")
lateinit var market: NiceHashMarket
}
...
entity.market = Market(...)

JPA double relation with the same Entity

I have these Entities:
#Entity
public class Content extends AbstractEntity
{
#NotNull
#OneToOne(optional = false)
#JoinColumn(name = "CURRENT_CONTENT_REVISION_ID")
private ContentRevision current;
#OneToMany(mappedBy = "content", cascade = CascadeType.ALL, orphanRemoval = true)
private List<ContentRevision> revisionList = new ArrayList<>();
}
#Entity
public class ContentRevision extends AbstractEntity
{
#NotNull
#ManyToOne(optional = false)
#JoinColumn(name = "CONTENT_ID")
private Content content;
#Column(name = "TEXT_DATA")
private String textData;
#Temporal(TIMESTAMP)
#Column(name = "REG_DATE")
private Date registrationDate;
}
and this is the db mapping:
CONTENT
+-----------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------------------+--------------+------+-----+---------+----------------+
| ID | bigint(20) | NO | PRI | NULL | auto_increment |
| CURRENT_CONTENT_REVISION_ID | bigint(20) | NO | MUL | NULL | |
+-----------------------------+--------------+------+-----+---------+----------------+
CONTENT_REVISION
+-----------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------------------+--------------+------+-----+---------+----------------+
| ID | bigint(20) | NO | PRI | NULL | auto_increment |
| REG_DATE | datetime | YES | | NULL | |
| TEXT_DATA | longtext | YES | | NULL | |
| CONTENT_ID | bigint(20) | NO | MUL | NULL | |
+-----------------------------+--------------+------+-----+---------+----------------+
I have also these requirements:
Content.current is always a member of Content.revisionList (think about Content.current as a "pointer").
Users can add a new ContentRevision to an existing Content
Users can add a new Content with an initial ContentRevision (cascade persist)
Users can change Content.current (move the "pointer")
Users can modify Content.current.textData, but saves Content (cascade merge)
Users can delete ContentRevision
Users can delete Content (cascade remove to ContentRevision)
Now, my questions are:
Is this the best approach? Any best practice?
Is it safe to cascade merge when the same entity is referenced twice? (Content.current is also Content.revisionList[i])
Are Content.current and Content.revisionList[i] the same instance? (Content.current == Content.revisionList[i] ?)
Thanks
#jabu.10245 I'm very grateful for your effort. Thank you, really.
However, there's a problematic (missing) case from your tests: when you run it inside a container using CMT:
#RunWith(Arquillian.class)
public class ArquillianTest
{
#PersistenceContext
private EntityManager em;
#Resource
private UserTransaction utx;
#Deployment
public static WebArchive createDeployment()
{
// Create deploy file
WebArchive war = ShrinkWrap.create(WebArchive.class, "test.war");
war.addPackages(...);
war.addAsResource("persistence-arquillian.xml", "META-INF/persistence.xml");
war.addAsManifestResource(EmptyAsset.INSTANCE, "beans.xml");
// Show the deploy structure
System.out.println(war.toString(true));
return war;
}
#Test
public void testDetached()
{
// find a document
Document doc = em.find(Document.class, 1L);
System.out.println("doc: " + doc); // Document#1342067286
// get first content
Content content = doc.getContentList().stream().findFirst().get();
System.out.println("content: " + content); // Content#511063871
// get current revision
ContentRevision currentRevision = content.getCurrentRevision();
System.out.println("currentRevision: " + currentRevision); // ContentRevision#1777954561
// get last revision
ContentRevision lastRevision = content.getRevisionList().stream().reduce((prev, curr) -> curr).get();
System.out.println("lastRevision: " + lastRevision); // ContentRevision#430639650
// test equality
boolean equals = Objects.equals(currentRevision, lastRevision);
System.out.println("1. equals? " + equals); // true
// test identity
boolean same = currentRevision == lastRevision;
System.out.println("1. same? " + same); // false!!!!!!!!!!
// since they are not the same, the rest makes little sense...
// make it dirty
currentRevision.setTextData("CHANGED " + System.currentTimeMillis());
// perform merge in CMT transaction
utx.begin();
doc = em.merge(doc);
utx.commit(); // --> ERROR!!!
// get first content
content = doc.getContentList().stream().findFirst().get();
// get current revision
currentRevision = content.getCurrentRevision();
System.out.println("currentRevision: " + currentRevision);
// get last revision
lastRevision = content.getRevisionList().stream().reduce((prev, curr) -> curr).get();
System.out.println("lastRevision: " + lastRevision);
// test equality
equals = Objects.equals(currentRevision, lastRevision);
System.out.println("2. equals? " + equals);
// test identity
same = currentRevision == lastRevision;
System.out.println("2. same? " + same);
}
}
since they are not the same:
if I enable cascading on both properties, an Exception is thrown
java.lang.IllegalStateException:
Multiple representations of the same entity [it.shape.edea2.jpa.ContentRevision#1] are being merged.
Detached: [ContentRevision#430639650];
Detached: [ContentRevision#1777954561]
if I disable cascade on current, the change get lost.
the strange thing is that running this test outside the container results in successful execution.
Maybe it's lazy loading (hibernate.enable_lazy_load_no_trans=true), maybe something else, but it's definitely NOT SAFE.
I wonder if there's a way to get the same instance.

Is it safe to cascade merge when the same entity is referenced twice?
Yes. If you manage an instance of Content, then it's Content.revisionList and Content.current are managed as well. Changes in any of those will be persisted when flushing the entity manager. You don't have to call EntityManager.merge(...) manually, unless you're dealing with transient objects that need to be merged.
If you create a new ContentRevision, then call persist(...) instead of merge(...) with that new instance and make sure it has a managed reference to the parent Content, also add it to the content's list.
Are Content.current and Content.revisionList[i] the same instance?
Yes, should be. Test it to be sure.
Content.current is always a member of Content.revisionList (think about Content.current as a "pointer").
You could do that check in in SQL with a check constraint; or in Java, although you'd have to be sure the revisionList is fetched. By default it's lazy fetched, meaning Hibernate will run another query for this list if you access the getRevisionList() method. And for that you need a running transaction, otherwise you'll be getting a LazyInitializationException.
You could instead load the list eagerly, if that's what you want. Or you could define a entity graph to be able to support both strategies in different queries.
Users can modify Content.current.textData, but saves Content (cascade merge)
See my first paragraph above, Hibernate should save changes on any managed entity automatically.
Users can delete ContentRevision
if (content.getRevisionList().remove(revision))
entityManager.remove(revision);
if (revision.equals(content.getCurrentRevision())
content.setCurrentRevision(/* to something else */);
Users can delete Content (cascade remove to ContentRevision)
Here I'd prefer to ensure that in the database schema, for instance
FOREIGN KEY (content_id) REFERENCES content (id) ON DELETE CASCADE;
UPDATE
As requested, I wrote a test. See this gist for the implementations of Content and ContentRevision I used.
I had to make one important change though: Content.current cannot really be #NotNull, especially not the DB field, because if it were, then we couldn't persist a content and revision at the same time, since both have no ID yet. Hence the field must be allowed to be NULL initially.
As a workaround I added the following method to Content:
#Transient // ignored in JPA
#AssertTrue // javax.validation
public boolean isCurrentRevisionInList() {
return current != null && getRevisionList().contains(current);
}
Here the validator ensures that the there is always a non-null current revision and that it is contained in the revision list.
Now here are my tests.
This one proves that the references are the same (Question 3) and that it is enough to persist content where current and revisionList[0] is referencing the same instance (question 2):
#Test #InSequence(0)
public void shouldCreateContentAndRevision() throws Exception {
// create java objects, unmanaged:
Content content = Content.create("My first test");
assertNotNull("content should have current revision", content.getCurrent());
assertSame("content should be same as revision's parent", content, content.getCurrent().getContent());
assertEquals("content should have 1 revision", 1, content.getRevisionList().size());
assertSame("the list should contain same reference", content.getCurrent(), content.getRevisionList().get(0));
// persist the content, along with the revision:
transaction.begin();
entityManager.joinTransaction();
entityManager.persist(content);
transaction.commit();
// verify:
assertEquals("content should have ID 1", Long.valueOf(1), content.getId());
assertEquals("content should have one revision", 1, content.getRevisionList().size());
assertNotNull("content should have current revision", content.getCurrent());
assertEquals("revision should have ID 1", Long.valueOf(1), content.getCurrent().getId());
assertSame("current revision should be same reference", content.getCurrent(), content.getRevisionList().get(0));
}
The next ensures that it's still true after loading the entity:
#Test #InSequence(1)
public void shouldLoadContentAndRevision() throws Exception {
Content content = entityManager.find(Content.class, Long.valueOf(1));
assertNotNull("should have found content #1", content);
// same checks as before:
assertNotNull("content should have current revision", content.getCurrent());
assertSame("content should be same as revision's parent", content, content.getCurrent().getContent());
assertEquals("content should have 1 revision", 1, content.getRevisionList().size());
assertSame("the list should contain same reference", content.getCurrent(), content.getRevisionList().get(0));
}
And even when updating it:
#Test #InSequence(2)
public void shouldAddAnotherRevision() throws Exception {
transaction.begin();
entityManager.joinTransaction();
Content content = entityManager.find(Content.class, Long.valueOf(1));
ContentRevision revision = content.addRevision("My second revision");
entityManager.persist(revision);
content.setCurrent(revision);
transaction.commit();
// re-load and validate:
content = entityManager.find(Content.class, Long.valueOf(1));
// same checks as before:
assertNotNull("content should have current revision", content.getCurrent());
assertSame("content should be same as revision's parent", content, content.getCurrent().getContent());
assertEquals("content should have 2 revisions", 2, content.getRevisionList().size());
assertSame("the list should contain same reference", content.getCurrent(), content.getRevisionList().get(1));
}
SELECT * FROM content;
id | version | current_content_revision_id
----+---------+-----------------------------
1 | 2 | 2
UPDATE 2
It was hard to reproduce that situation on my machine, but I got it to work. Here is what I've done so far:
I changed all #OneToMany relations to use lazy fetching (the default) and rerun the following test case:
#Test #InSequence(3)
public void shouldChangeCurrentRevision() throws Exception {
transaction.begin();
entityManager.joinTransaction();
Document document = entityManager.find(Document.class, Long.valueOf(1));
assertNotNull(document);
assertEquals(1, document.getContentList().size());
Content content = document.getContentList().get(0);
assertNotNull(content);
ContentRevision revision = content.getCurrent();
assertNotNull(revision);
assertEquals(2, content.getRevisionList().size());
assertSame(revision, content.getRevisionList().get(1));
revision.setTextData("CHANGED");
document = entityManager.merge(document);
content = document.getContentList().get(0);
revision = content.getCurrent();
assertSame(revision, content.getRevisionList().get(1));
assertEquals("CHANGED", revision.getTextData());
transaction.commit();
}
The test passed with lazy fetching. Note that lazy fetching requires it to be executed within a transaction.
For some reason the content revision instance you're editing is not the same as the one in the one-to-many list. To reproduce that I've modified my test as follows:
#Test #InSequence(4)
public void shouldChangeCurrentRevision2() throws Exception {
transaction.begin();
Document document = entityManager.find(Document.class, Long.valueOf(1));
assertNotNull(document);
assertEquals(1, document.getContentList().size());
Content content = document.getContentList().get(0);
assertNotNull(content);
ContentRevision revision = content.getCurrent();
assertNotNull(revision);
assertEquals(2, content.getRevisionList().size());
assertSame(revision, content.getRevisionList().get(1));
transaction.commit();
// load another instance, different from the one in the list:
revision = entityManager.find(ContentRevision.class, revision.getId());
revision.setTextData("CHANGED2");
// start another TX, replace the "current revision" but not the one
// in the list:
transaction.begin();
document.getContentList().get(0).setCurrent(revision);
document = entityManager.merge(document); // here's your error!!!
transaction.commit();
content = document.getContentList().get(0);
revision = content.getCurrent();
assertSame(revision, content.getRevisionList().get(1));
assertEquals("CHANGED2", revision.getTextData());
}
And there, I got exactly your error. Then I modified the cascading setting on the #OneToMany mapping:
#OneToMany(mappedBy = "content", cascade = { PERSIST, REFRESH, REMOVE }, orphanRemoval = true)
private List<ContentRevision> revisionList;
And the error disappeared :-) ... because I removed CascadeType.MERGE.