I am using Spring Boot 2.1.6 and Elasticsearch 6.2.2
EDIT to better clarify my question:
When I let Spring generate a query for me by using the following method in my repository:
Account findTop1ByAccountIdOrderByCreatedDesc(final String accountId);
I imagine it means it will select from the index filtering by accountId, then ordering the results by created descending, and finally it will return only the first (latest) result.
But I only have two entries in the index, identical ones minus the created date, and that query returns both results. I think it means it does not translate to what I have in mind but rather it will pick all accounts with that ID (since that is a key, all are "top"), ordered descending.
This would more closely match my query, but it is not legal naming:
Account findTop1OrderByCreatedDescByAccountId(final String accountId);
org.springframework.data.mapping.PropertyReferenceException: No property descByAccountId found for type LocalDateTime! Traversed path: Account.created.
And this one as well:
Account findTop1OrderByCreatedDescAndAccountIdEquals(final String accountId);
org.springframework.data.mapping.PropertyReferenceException: No property desc found for type LocalDateTime! Traversed path: Account.created.
So how do I translate, if possible at all, my query to Spring repository magic?
/EDIT
Original question:
I have a POJO declared as such (trimmed version):
#Document(indexName = "config.account", type = "account")
public class Account{
#Id
private String id;
#Field(type = FieldType.Text)
#JsonProperty("ilmAccountIdentifier")
private String accountId;
#Field(type = FieldType.Text)
private String endOfBusinessDay;
#Field(type = FieldType.Date)
private LocalDateTime created;
}
And I would like to query the index to retrieve the latest entry (created = max) for a given accountId.
I know I can use the query builder, but I was wondering if there was some magic that does that for me by using the spring named queries, currently I was trying with (couldn't find other wording combination that are valid):
Account findTop1ByAccountIdOrderByCreatedDesc(final String accountId);
But it returns null. I see the data is in the index (trimmed version):
Account(id=W83u0GsBEjwDhWt1-Whn,
accountId=testfindByAccountIdReturnsLatestVersion,
endOfBusinessDay=17:00:00,
created=2019-07-08T09:34)
But the query Spring generated is quite strange and not what I would expect:
SearchRequest{
searchType=QUERY_THEN_FETCH,
indices=[config.account],
indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false],
types=[account],
routing='null',
preference='null',
requestCache=null,
scroll=null,
maxConcurrentShardRequests=0,
batchedReduceSize=512,
preFilterShardSize=128,
allowPartialSearchResults=null,
source={
"from":0,
"query":{
"bool":{
"must":[{
"query_string":
{"query":
"testfindByAccountIdReturnsLatestVersion",
"fields": ["accountId^1.0"],
"type":"best_fields",
"default_operator":"and",
"max_determinized_states":10000,
"enable_position_increments":true,
"fuzziness":"AUTO",
"fuzzy_prefix_length":0,
"fuzzy_max_expansions":50,
"phrase_slop":0,
"escape":false,
"auto_generate_synonyms_phrase_query":true,
"fuzzy_transpositions":true,
"boost":1.0}
}],
"adjust_pure_negative":true,"boost":1.0}
}
,"version":true}}
What I would like instead is the equivalent of this SQL query:
select *
from(
select *
from account
where accountId = ?
order by created desc
)
where rownum = 1
Is it possible at all to do with the Spring magic or must I use QueryBuilder or my own logic for it?
Thanks and cheers
EDIT
Unrelated, but I realized that spring Repository magic doesn't work if a field is renamed during mapping with #JsonProperty. Assuming I do NOT do that renaming, the question remains the same. Currently I worked around this by implementing my own logic with:
#Repository
public interface AccountRepository extends ElasticsearchRepository<Account, String>, AccountRepositoryCustom {
List<Account> findByIlmAccountIdentifierOrderByCreatedDesc(final String accountId);
}
public interface AccountRepositoryCustom {
Account findLatestByAccountId(final String accountId);
}
#Repository
public class AccountRepositoryImpl implements AccountRepositoryCustom {
#Autowired
#Lazy
private AccountRepository accountRepository;
public Account findLatestByAccountId(final String accountId){
return accountRepository.findByIlmAccountIdentifierOrderByCreatedDesc(accountId).get(0);
}
}
For the queries that are built from method names you must use the property names from your Document class, so here findByAccountId is correct.
The problem indeed is the renaming of this property in the index with the #JsonProperty annotation. This annotation was used for writing the mapping to the index, and as far as I recall as well when writing data with the Template classes but not with the Repository classes. So the repository query searches for a field accountId in the index, whereas the data is stored in the field ilmAccountIdentifier.
The version 3.2.0 of Spring Data Elasticsearch (currently RC1, RC2 will be released later this month and GA should be out in the beginning of september) has a new Mapper implementation available with which this is working without the #JsonPropertyjust by using the #Field annotation:
#Field(name="ilmAccountIdentifier", type = FieldType.Text)
private String accountId;
But 3.2.x will not work with Elasticsearch 6.2, you would need to update to 6.7.
As a workaround, can you rename the accountId property to ilmAccountIdentifier?
Edit 23.07.2019:
I just found out that limiting the result with topN not working is an old bug in Spring Data Elasticsearch, seems it has never been implemented in this sub-module. So please vote for this issue.
Note: spring-data-elasticsearch is a community driven module, so we live from contributions!
Edit 28.11.2019:
I implemented topN in August 2019, it will be in the Spring Data Elasticsearch Neumann release (version 4)
I got a bunch of DTO's which are not commented at all. However, there are comments in the SQL-Database. I can get these comments by sending a query and then retrieving the ResultSet.
My task is to create a javadoc-API (as HTML) with the comments from the SQL-Database in order to make the codebase better understandable.
After asking about this task already HERE, I tried to looked into creating my own doclet. I then wrote my own doclet by rewriting the Standard-, Abstract- and HtmlDoclet from Java.Tools. My results are working fine and I can create javadoc html pages WITH the comments from the database.
HOWEVER its a massive hack imho. There are two main tasks that need to be done in order to get the Database comments.
know the table name
know the column name
How it should be done: (which is what I want to ask - How do I implement it like this?)
For 1. : Find the #Table annotation. Read name = "tablename".
For 2. : For each variable:
Is there a #Column annotation ? return "columnName" : return ""
How I do it right now:
For 1. : I read the RootDoc.name() variable and then read the String char by char. Find a capital letter. Insert '_'. And at the end, turn everything .toUpperCase(). So "testFile" turns into "TEST_FILE".
This sometimes does not work. If you read carefully in the example class. Its name is "SaklTAdrkla" but the Databasetable name is SAKL_T_ADRKLAS. Parsing the name from RootDoc.name() would result in "SAKL_T_ADRKLA" which is missing the character 'S' at the end, therefore it wont find the table in the database.
For 2. : I get all Fields from the ClassDoc. I then parse Field.name() the same way I parsed the RootDoc.name() variable.
This wont work for the same reason as 1.; but also because some fieldnames are not the same as their mapped names. In the example Class - field sakgTAklgrpAklAkgid is mapped in the database as AKL_AKGID
I am able to find the Annotation itselfe by calling FieldDoc.annotations(). But thats ONLY the annotation without the String (name = "xyz") which is the most important part for me!
I have found the Jax-Doclet, which can parse the javax annotations. However after downloading the jar-source file and implementing the java files, there are numerous dependency issues which are not resolvable because the referenced classes no longer exist in java 8 tools.jar.
Is there another solution, that is capable of reading the javax annotations?
Can I implement something into my doclet so it can read the javax annotations?
Edit:
I found out you can call .elementValues() on the AnnotationDesc class which I can get from FieldDoc.annotations(). However I always get a com.sun.jdi.ClassNotLoadedException Type has not been loaded occurred while retrieving component type of array. To fix it I manually load the classes AnnotationDesc and AnnotationDesc.ElementValuePair by calling Class.forName(). However now the Array with the elementValuePairs is empty..?
Example class:
/**
* The persistent class for the SAKL_T_ADRKLAS database table.
*/
#Entity
#IdClass(SaklTAdrklaPK.class)
#Table(name = "SAKL_T_ADRKLAS")
#NamedQuery(name = "SaklTAdrkla.findAll", query = "SELECT s FROM SaklTAdrkla s")
public class SaklTAdrkla implements Serializable, IModelEntity {
private static final long serialVersionUID = 1L;
#Id #Column(name = "AKL_AKLID") private String aklAklid;
#Id
// uni-directional many-to-one association to SakgTAklgrp
#JsonBackReference(value = "sakgTAklgrpAklAkgid") #ManyToOne #JoinColumn(name = "AKL_AKGID") private SakgTAklgrp sakgTAklgrpAklAkgid;
#Temporal(TemporalType.TIMESTAMP) #Column(name = "AKL_AEND") private Date aklAend;
#Column(name = "AKL_DEFLT") private BigDecimal aklDeflt;
#Column(name = "AKL_SPERRE") private BigDecimal aklSperre;
#Column(name = "AKL_T_BEZ") private String aklTBez;
#Column(name = "AKL_USRID") private String aklUsrid;
public SaklTAdrkla() {
}
It took me quite a while to figure this out now, but I finnally did.
The Problem was that my doclet could find all the annotations, which it displayed in the console as errors.
error: cannot find symbol #Column(name = "TST_USER") private
String tstUser;
What I also found was this message in the lot of errors that got thrown:
error: package javax.persistence does not exist import
javax.persistence.*;
So I imported javax.persistance.jar into my project.
I also added com.fasterxml.jaxkson.annotations.jar into the project since it would also not work without it.
Surprise Surprise! IT WORKS!
I can get all the annotations and annotation values by using annotation.elementValues().
I no longer get an empty Array nor do I get an ClassNotLoadedException.
I'm working on a Spring JPA Application, using MySQL as database. I ensured that all spring-jpa libraries, hibernate and mysql-connector-java is loaded.
I'm running a mysql 5 instance. Here is a excerpt of my application.properties file:
spring.jpa.show-sql=false
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.database-platform=org.hibernate.dialect.MySQL5Dialect
spring.datasource.url=jdbc:mysql://localhost/mydatabase
spring.datasource.username=myuser
spring.datasource.password=SUPERSECRET
spring.datasource.driverClassName=com.mysql.jdbc.Driver
When executing an integration test, spring startsup properly but fails on creating the hibernate SessionFactory, with the exception:
org.hibernate.MappingException: No Dialect mapping for JDBC type: 1111
I think my dialects should be Mysql5Dialect, I also tried the one explicitly stating InnoDB, and the two dialect options which don't indicate the version 5. But I always end up with the same 'No Dialect mapping for JDBC type: 1111' message.
My application.properties file resides in the test/resources source folder. It is recognized by the JUnit Test runner (I previously got an exception because of an typo in it).
Are the properties I'm setting wrong? I couldn't find some official documentation on these property names but found a hint in this stackoverflow answer: https://stackoverflow.com/a/25941616/1735497
Looking forward for your answers, thanks!
BTW The application is already using spring boot.
I got the same error because my query returned a UUID column. To fix that I returned the UUID column as varchar type through the query like "cast(columnName as varchar)", then it worked.
Example:
public interface StudRepository extends JpaRepository<Mark, UUID> {
#Modifying
#Query(value = "SELECT Cast(stuid as varchar) id, SUM(marks) as marks FROM studs where group by stuid", nativeQuery = true)
List<Student> findMarkGroupByStuid();
public static interface Student(){
private String getId();
private String getMarks();
}
}
Here the answer based on the comment from SubOptimal:
The error message actually says that one column type cannot be mapped to a database type by hibernate.
In my case it was the java.util.UUID type I use as primary key in some of my entities. Just apply the annotation #Type(type="uuid-char") (for postgres #Type(type="pg-uuid"))
There is also another common use-case throwing this exception. Calling function which returns void. For more info and solution go here.
I got the same error, the problem here is UUID stored in DB is not converting to object.
I tried applying these annotations #Type(type="uuid-char") (for postgres #Type(type="pg-uuid") but it didn't work for me.
This worked for me. Suppose you want id and name from a table with a native query in JPA. Create one entity class like 'User' with fields id and name and then try converting object[] to entity we want. Here this matched data is list of array of object we are getting from query.
#Query( value = "SELECT CAST(id as varchar) id, name from users ", nativeQuery = true)
public List<Object[]> search();
public class User{
private UUID id;
private String name;
}
List<User> userList=new ArrayList<>();
for(Object[] data:matchedData){
userList.add(new User(UUID.fromString(String.valueOf(data[0])),
String.valueOf(data[1])));
}
Suppose this is the entity we have
Please Check if some Column return many have unknow Type in Query .
eg : '1' as column_name can have type unknown
and 1 as column_name is Integer is correct One .
This thing worked for me.
Finding the column that triggered the issue
First, you didn't provide the entity mapping so that we could tell what column generated this problem. For instance, it could be a UUID or a JSON column.
Now, you are using a very old Hibernate Dialect. The MySQL5Dialect is meant for MySQL 5. Most likely you are using a newer MySQL version.
So, try to use the MySQL8Dialect instead:
spring.jpa.database-platform=org.hibernate.dialect.MySQL8Dialect
Adding non-standard types
In case you got the issue because you are using a JSON column type, try to provide a custom Hibernate Dialect that supports the non-standard Type:
public class MySQL8JsonDialect
extends MySQL8Dialect{
public MySQL8JsonDialect() {
super();
this.registerHibernateType(
Types.OTHER, JsonStringType.class.getName()
);
}
}
Ans use the custom Hibernate Dialect:
<property
name="hibernate.dialect"
value="com.vladmihalcea.book.hpjp.hibernate.type.json.MySQL8JsonDialect"
/>
If you get this exception when executing SQL native queries, then you need to pass the type via addScalar:
JsonNode properties = (JsonNode) entityManager
.createNativeQuery(
"SELECT properties " +
"FROM book " +
"WHERE isbn = :isbn")
.setParameter("isbn", "978-9730228236")
.unwrap(org.hibernate.query.NativeQuery.class)
.addScalar("properties", JsonStringType.INSTANCE)
.getSingleResult();
assertEquals(
"High-Performance Java Persistence",
properties.get("title").asText()
);
Sometimes when you call sql procedure/function it might be required to return something. You can try returning void: RETURN; or string (this one worked for me): RETURN 'OK'
If you have native SQL query then fix it by adding a cast to the query.
Example:
CAST('yourString' AS varchar(50)) as anyColumnName
In my case it worked for me.
In my case, the issue was Hibernate not knowing how to deal with an UUID column. If you are using Postgres, try adding this to your resources/application.properties:
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL9Dialect
Another simple explanation might be that you're fetching a complex Type (Entity/POJO) but do not specify the Entity to map to:
String sql = "select yourentity.* from {h-schema}Yourentity yourentity";
return entityManager.createNativeQuery(sql).getResultList();
simply add the class to map to in the createNativeQuery method:
return entityManager.createNativeQuery(sql, Yourentity.class).getResultList();
In my case the problem was that, I forgot to add resultClasses attribute when I setup my stored procedure in my User class.
#NamedStoredProcedureQuery(name = "find_email",
procedureName = "find_email", resultClasses = User.class, //<--I forgot that.
parameters = {
#StoredProcedureParameter(mode = ParameterMode.IN, name = "param_email", type = String.class)
}),
This also happens when you are using Hibernate and returning a void function. AT least w/ postgres. It doesnt know how to handle the void. I ended up having to change my void to a return int.
If you are using Postgres, check that you don't have a column of type Abstime. Abstime is an internal Postgres datatype not recognized by JPA. In this case, converting to Text using TO_CHAR could help if permitted by your business requirements.
if using Postgres
public class CustomPostgreSqlDialect extends PostgreSQL94Dialect{
#Override
public SqlTypeDescriptor remapSqlTypeDescriptor(SqlTypeDescriptor sqlTypeDescriptor)
{
switch (sqlTypeDescriptor.getSqlType())
{
case Types.CLOB:
return VarcharTypeDescriptor.INSTANCE;
case Types.BLOB:
return VarcharTypeDescriptor.INSTANCE;
case 1111://1111 should be json of pgsql
return VarcharTypeDescriptor.INSTANCE;
}
return super.remapSqlTypeDescriptor(sqlTypeDescriptor);
}
public CustomPostgreSqlDialect() {
super();
registerHibernateType(1111, "string");
}}
and use
<prop key="hibernate.dialect">com.abc.CustomPostgreSqlDialect</prop>
For anybody getting this error with an old hibernate (3.x) version:
do not write the return type in capital letters. hibernate type implementation mapping uses lowercase return types and does not convert them:
CREATE OR REPLACE FUNCTION do_something(param varchar)
RETURNS integer AS
$BODY$
...
This is for Hibernate (5.x) version
Calling database function which return JSON string/object
For this use unwrap(org.hibernate.query.NativeQuery.class).addScalar() methods for the same.
Example as below (Spring & Hibernate):
#PersistenceContext
EntityManager em;
#Override
public String getJson(String strLayerName) {
String *nativeQuery* = "select fn_layer_attributes(:layername)";
return em.createNativeQuery(*nativeQuery*).setParameter("layername", strLayerName).**unwrap(org.hibernate.query.NativeQuery.class).addScalar**("fn_layer_attributes", **new JsonNodeBinaryType()**) .getSingleResult().toString();
}
Function or procedure returning void cause some issue with JPA/Hibernate, so changing it with return integer and calling return 1 at the end of procedure may solved the problem.
SQL Type 1111 represents String.
If you are calling EntityManager.createNativeQuery(), be sure to include the resulting java class in the second parameter:
return em.createNativeQuery(sql, MyRecord.class).getResultList()
After trying many proposed solutions, including:
https://stackoverflow.com/a/59754570/349169 which is one of the solutions proposed here
https://vladmihalcea.com/hibernate-no-dialect-mapping-for-jdbc-type/
it was finally this one that fixed everything with the least amount of changes:
https://gist.github.com/agrawald/adad25d28bf6c56a7e4618fe95ee5a39
The trick is to not have #TypeDef on your class, but instead have 2 different #TypeDef in 2 different package-info.java files. One inside your production code package for your production DB, and one inside your test package for your test H2 DB.
I'm running a cypher query using
org.neo4j.ogm.session.Session#query(java.lang.Class<T>, java.lang.String, java.util.Map<java.lang.String,?>)
The Class is a POJO which I have annotated using #QueryResult
#QueryResult
public class Neo4jQueryResultClip {
private String clipUuid;
private String postTitle;
private Date clipCreatedAt;
//getters and setters
}
My query cypher goes something like this
match (c:Clip) where (:User{uuid:{uuidParam}})-[:USER_FOLLOWS_USER]->(:User)-[:CLIP_BY_USER]->(c) OR (:User{uuid:{uuidParam}})-[:CLIP_BY_USER]->(c)match (c)<-[:CLIP_PRODUCT|:CLIP_INSPIRATION]-(post) optional match (c)<-[cp:CLIP_PRODUCT]-(post) return c.uuid as clipUuid,c.createdAt as clipCreatedAt,post.title as postTitle order by c.createdAt DESC
However the iterator of results returned is empty
If I run the same query using
org.neo4j.ogm.session.Session#query(java.lang.String, java.util.Map<java.lang.String,?>)
I get proper results encapsulated in the
org.neo4j.ogm.session.result.Result
object.
Is there something I am missing here?
I have verified that the class Neo4jQueryResultClip is getting scanned by neo4j spring configuration
I am using following versions
spring-data-neo4j (4.0.0.RELEASE) and neo4j-ogm library (1.1.4)
#QueryResult used on in Spring Data Neo4j repositories (see http://docs.spring.io/spring-data/neo4j/docs/4.0.0.RELEASE/reference/html/#reference_programming-model_mapresult) so that's why it isn't mapped.
If you do this instead inside a repository
#Query("match (c:Clip) where (:User{uuid:{uuidParam}})-[:USER_FOLLOWS_USER]->(:User)-[:CLIP_BY_USER]->(c) OR (:User{uuid:{uuidParam}})-[:CLIP_BY_USER]->(c)match (c)<-[:CLIP_PRODUCT|:CLIP_INSPIRATION]-(post) optional match (c)<-[cp:CLIP_PRODUCT]-(post) return c.uuid as clipUuid,c.createdAt as clipCreatedAt,post.title as postTitle order by c.createdAt DESC")
Neo4jQueryResultClip getClip(...);
then it should work just fine.
I am using spring and the MongoTemplate and trying to write an equivalent query SQL LIKE statement.. I am have not seen a satisfactory answer, code below:
#Document
public class Lake {
#Id
private String oid;
#Indexed (sparse = true)
private String name;
private String state;
}
public List<Lake> listLakesLike(String likename) {
try {
Query filter = new Query(Criteria.where("name").regex("lakename","i"));
List<Lake> lakes = mongoTemplate.find(filter, Lake.class);
return lakes
}
I saw this as an example that DOES NOT work, no lakes returned.
How do I write a mongoTemplate.find that results in matching LIKE lake names based on the passed in value likename?
Thank you in advance.. This is driving me crazy.. Or if you can point me to an example.
Spring MongoDB syntax
Query filter = new Query(Criteria.where("name").regex("lakename","i"));
Is equivalent to MongoDB shell command
db.lake.find({name:/lakename/i})
Notice in spring leading and trailing slashes are not required. Try the command first in your shell to check you have all the data correct, if it doesn't work I'm pretty sure your problem lies elsewhere (eg: your Spring MongoDB is pointing to a wrong host / database.collection name)
Also notice Spring MongoDB implicitly converts the class name Lake into collection name lake (with lowercase). If you need to specify explicit collection name you can do so using #Document(collection = "Lake")