Hibernate lucene only searches in 60 first records

Hibernate lucene only searches in 60 first records - java

I have more than 500 records in Table but Hibernate lucene only search in 60 top records.
I use the hibernate session instead of the entity manager.
How can I search in all records.
This is my code:
My hibernate.cfg
<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.driver_class">com.microsoft.sqlserver.jdbc.SQLServerDriver</property>
<property name="hibernate.dialect">org.hibernate.dialect.SQLServer2012Dialect</property>
<property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>
<property name="show_sql">true</property>
<property name="hibernate.current_session_context_class">thread</property>
<property name="hibernate.globally_quoted_identifiers">true</property>
<property name="hibernate.show_sql">true</property>
<property name="hibernate.connection.release_mode">auto</property>
<property name="hibernate.connection.autoReconnect">true</property>
<property name="hibernate.transaction.auto_close_session">true</property>
<property name="hibernate.id.new_generator_mappings">true</property>
<property
name="hibernate.transaction.flush_before_completion">true</property>
<property name="hibernate.search.default.indexBase">D:/tvc/indexes</property>
<mapping class="com.hellojob.entities.WebsiteOrderContract" />
</session-factory>
</hibernate-configuration>
My Entity:
#Entity
#Indexed
#AnalyzerDef(name = "customanalyzer",
charFilters = {
#CharFilterDef(factory = MappingCharFilterFactory.class
//, params = {#Parameter(name = "mapping", value = "org/hibernate/search/test/analyzer/mapping-chars.properties")}
)
},
tokenizer = #TokenizerDef(factory = StandardTokenizerFactory.class),
filters = {
#TokenFilterDef(factory = ASCIIFoldingFilterFactory.class),
#TokenFilterDef(factory = LowerCaseFilterFactory.class), // #TokenFilterDef(factory = StopFilterFactory.class, params = {#Parameter(name="words", value= "org/hibernate/search/test/analyzer/stoplist.properties" ),#Parameter(name="ignoreCase", value="true")})
})
#Table(name = "Website_OrderContract")
public class WebsiteOrderContract implements java.io.Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "ID", insertable = false, updatable = false)
private Integer id;
#Field(index = Index.YES, analyze = Analyze.YES, store = Store.NO, analyzer = #Analyzer(definition = "customanalyzer"))
#Column(name = "Name")
private String name;
}
My DAO:
FullTextSession fullTextSession = Search.getFullTextSession(session);
QueryBuilder qb = fullTextSession.getSearchFactory().buildQueryBuilder()
.forEntity(WebsiteOrderContract.class).get();
org.apache.lucene.search.Query query = qb
.all()
.createQuery();
FullTextQuery hibQuery = fullTextSession.createFullTextQuery(query, WebsiteOrderContract.class);
hibQuery.setFirstResult(0);
hibQuery.setMaxResults(10);
rs = hibQuery.list();
System.out.println(hibQuery.getResultSize()); // 60 results, must be 590
System.out.println(rs.size()); // 10 result
Thanking you.

The most likely explanation is that your entities haven't been indexed.
Did you take care of indexing pre-existing data, for example using a Mass indexer?
Did you check the logs to see if there are indexing errors?

Related

Entity not mapped, although mapped

I have 2 projects:
1 for the entities
1 for a web service
While using Hibernate 4.3.11.Final everything was working properly. I have a hibernate.cfg.xml where I have all the classes so selecting the entities from the DB was ok using Query
Then I changed to Hibernate 5.2.6.Final and I had to change Query for TypedQuery<Entity> and now it complains that the entity is not mapped.
A test I made calls this code.
TypedQuery<Employee> query = getSession().createQuery(
"SELECT e from Employee e where e.username = :username");
query.setParameter("username", employee.getUsername());
return query.getSingleResult();
And it complains that the entity I'm selecting in the query "SELECT e from Employee..." (not the one defined in the TypedQuery) is the one that's not mapped.
Is there a new way to map the entities in 5.2.6? How could I solve this?
The Employee entity:
#Entity
#Table(name = "employee")
#XmlRootElement
public class Employee implements Serializable
{
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Basic(optional = false)
#Column(name = "id")
private Long id;
#Basic(optional = false)
#Column(name = "username")
private String username;
// Getters & Setters...
}
The entry in the hibernate.cfg.xml file is like this:
<hibernate-configuration>
<session-factory>
<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.connection.driver_class">com.mysql.cj.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/db</property>
<property name="hibernate.connection.username">user</property>
<property name="hibernate.connection.password">pass</property>
<mapping class="com.entities.Employee"/>
<mapping class="all other entities"/>
If I change the dependency back from 5.2.6.Final to 4.3.11.Final
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>4.3.11.Final</version>
</dependency>
I have no problem and everything works as it should be.

Hibernate #GeneratedValue(name=???): ID gets incremented by five not by one

I am using Hibernate 4 orm and search for my twitter application. The problem is when I start persisting tweets into MariaDB table, id (primary key) gets incremented by five fold:
2
7
12
17
22
27
I tried all Generation Types which are available : SEQUENCE, IDENTITY, TABLE and AUTO. But none of this provided increment by one. I guess I am missing some configuration, otherwise it should provide increment by one.
Here is my Entity class and hibernate.cfg.xml file:
#Entity
#Indexed
#Table(name="tweets_table")
public class TweetPOJO implements Serializable{
private static final long serialVersionUID = 1123211L;
#Id
#GeneratedValue (???)
#Column(name = "raw_id" )
private int raw_id;
#Column(name = "tweet_id")
private String tweet_id;
#DateBridge(resolution=Resolution.DAY)
#Column(name = "posted_time")
private String timestamp;
#Field(index = Index.YES, analyze = Analyze.YES, store = Store.NO)
#Column(name = "cleaned_text")
private String tweet_text;
#Column(name = "hashtags")
#Field(index=Index.YES, analyze=Analyze.NO, store=Store.YES)
private String hashtags;
#Column(name = "media_url")
private String media_url;
#Column(name = "media_text")
private String media_text;
#Column(name = "media_type")
private String media_type;
#Column(name = "location")
private String location;
...
...
public void setUser_frind_count(int user_frind_count) {
this.user_frind_count = user_frind_count;
}
#Override
public String toString() {
return tweet_id+" : "+tweet_text;
}
}
Here is hibernate.cfg.xml
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE hibernate-configuration SYSTEM "http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="hibernate.bytecode.use_reflection_optimizer"> true </property>
<property name="hibernate.dialect"> org.hibernate.dialect.MySQLDialect </property>
<property name="hibernate.connection.driver_class"> com.mysql.jdbc.Driver </property>
<!-- Assume test is the database name -->
<property name="hibernate.connection.url"> jdbc:mysql://*******/clouddata </property>
<property name="hibernate.connection.username"> root </property>
<property name="hibernate.connection.password"> qwe123 </property>
<property name="connection.pool_size"> 1 </property>
<property name="hibernate.search.default.directory_provider"> filesystem </property>
<property name="hibernate.search.default.indexBase"> E:\lucene_index </property>
<!--
whether the schema will be created or just updated, every time the sessionFactory is created.
This is configured in the hibernate.hbm2ddl.auto property, which is set to update. So the schema
is only updated. If this property is set to create, then every time we run our application, the
schema will be re-created, thus deleting previous data.
-->
<property name="hibernate.hbm2ddl.auto">update</property>
<mapping class="twitter_crawling_modul.TweetPOJO"/>
</session-factory>

You can try to use sequence generator
#SequenceGenerator(name = "seq", sequenceName = "seq_name", allocationSize = 1)
#GeneratedValue (strategy = GenerationType.SEQUENCE, generator = "seq")

Unable to load class declared as < mapping class="Package.Class" /> in the configuration

hibernate.cfg.xml
?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.CharSet">utf8</property>
<property name="hibernate.connection.characterEncoding">utf8</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/UsoSurvey</property>
<property name="hibernate.connection.username">UsoSurvey</property>
<property name="hibernate.connection.password">UsoSurvey</property>
<property name="hibernate.hbm2ddl.auto">update</property>
<property name="hibernate.show_sql">true</property>
<property name="hbm2ddl.auto">update</property>
<property name="hibernate.transaction.factory_class">org.hibernate.transaction.JDBCTransactionFactory</property>
<property name="hibernate.current_session_context_class">thread</property>
<!-- configuration pool via c3p0-->
<property name="c3p0.acquire_increment">1</property>
<property name="c3p0.idle_test_period">100</property> <!-- seconds -->
<property name="c3p0.max_size">10</property>
<property name="c3p0.max_statements">0</property>
<property name="c3p0.min_size">1</property>
<property name="c3p0.timeout">100</property> <!-- seconds -->
<mapping class="Modelo.Grpusuario"/>
<mapping class="Modelo.Menu"/>
<mapping class="Modelo.Encuesta"/>
...
In iReport
Tools->Options->iReport->Classpath
C:\Users\Administrador\Documents\NetBeansProjects\UsoSurvey\src\java\Util -> I*n this folder is the mapping class*
C:\Users\Administrador\Documents\NetBeansProjects\UsoSurvey\src\java\Modelo -> In this folder are the model classes
Grpusuario.java
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
package Modelo;
import java.io.Serializable;
import java.util.Collection;
import javax.persistence.*;
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Size;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlTransient;
/**
*
* #author Administrador
*/
#Entity
#Table(name = "grpusuarios", catalog = "UsoSurvey", schema = "")
#XmlRootElement
#NamedQueries({
#NamedQuery(name = "Grpusuario.findAll", query = "SELECT g FROM Grpusuario g"),
#NamedQuery(name = "Grpusuario.findByGrupo", query = "SELECT g FROM Grpusuario g WHERE g.grupo = :grupo"),
#NamedQuery(name = "Grpusuario.findByDescripcion", query = "SELECT g FROM Grpusuario g WHERE g.descripcion = :descripcion")})
public class Grpusuario implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#Basic(optional = false)
#NotNull
#Column(name = "grupo")
private Integer grupo;
#Basic(optional = false)
#NotNull
#Size(min = 1, max = 64)
#Column(name = "descripcion")
private String descripcion;
#OneToMany(cascade = CascadeType.ALL, mappedBy = "secLevel")
private Collection<Menu> menuCollection;
#OneToMany(cascade = CascadeType.ALL, mappedBy = "grupo", orphanRemoval = true)
private Collection<Usuario> usuarioCollection;
public Grpusuario() {
}
....
but when trying to create the connection I get the following error: unable to load class declared as < mapping class="Modelo.Grpusuario" /> in the configuration

Hibernate Criteria creates a '?' in the generated SQL

I have created a sample web application. I am using MS SQL Server 2008 as database and hibernate + annotations to access the database. My hibernate configuration xml is as below.
The problem is, the Criteria.list() returns an empty list, and also I am seeing a '?' in the generated HSQL instead of the parameter I am passing in the Criteria.
<session-factory name="">
<property name="hibernate.connection.driver_class">sun.jdbc.odbc.JdbcOdbcDriver</property>
<property name="hibernate.dialect">org.hibernate.dialect.SQLServerDialect</property>
<property name="hibernate.connection.url">jdbc:odbc:dbname</property>
<property name="connection.pool_size">10</property>
<property name="current_session_context_class">thread</property>
<!-- Disable the second-level cache -->
<property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>
<!-- Echo all executed SQL to stdout -->
<property name="show_sql">true</property>
<property name="hibernate.connection.username"></property>
<property name="hibernate.connection.password"></property>
<mapping class="com.demo.Person" />
</session-factory>
This is my annotated bean
#Entity
#Table(name = "person")
public class Person implements Serializable {
public Person(){
}
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
#Basic(optional = false)
#Column(name = "personid")
private Integer personid;
#Basic(optional = false)
#Column(name = "firstname")
private String firstname;
#Column(name = "lastname")
private String lastname;
#Basic(optional = false)
#Column(name = "phone")
private String phone;
#Column(name = "mobile")
private String mobile;
#Column(name = "street")
private String street;
#Basic(optional = false)
#Column(name = "city")
private String city;
#Basic(optional = false)
#Column(name = "country")
private String country;
#Basic(optional = false)
#Column(name = "bussinessowner")
private int bussinessowner;
#OneToMany(cascade = CascadeType.ALL, mappedBy = "resultid1")
private Collection<Recent> recentCollection;
//setters & getters
}
And the code I am running is
Session session;
List list = new ArrayList();
try{
session = HibernateUtil.getSessionFactory().openSession();
Criteria criteria = session.createCriteria(Person.class);
criteria.add(Restrictions.like("firstname", name, MatchMode.START));
list= criteria.list();
System.out.println(list.size());
}catch (Exception e) {
e.printStackTrace();
}
Generated HSQL :
Hibernate: select this_.personid as personid2_0_, this_.city as city2_0_, this_.country as country2_0_, this_.firstname as firstname2_0_, this_.lastname as lastname2_0_ from person this_ where this_.firstname like ?
Other than this I am not getting any exception either. Can you please help me out with this.
Thanks !

Does changing it to criteria.add(Restrictions.like("firstname", name + "%")); help?
And, by the way, ? is correct parameter placeholder for jbdc statements.

'?' is the place holder for parameters in JDBC statements. You can't see its actual value, that's all. There is no problem with it, just the tracing with hibernate remains incomplete.
If you want to see the actual value for '?', then you have to use an extra product like log4jdbc.

#Rakesh Try below code:
criteria.add(Restrictions.ilike("firstname", name +"%"));

Instead of this line
criteria.add(Restrictions.like("firstname", name, MatchMode.START));
try
criteria.add(Restrictions.like("firstname", name, MatchMode.ANYWHERE));
OR
criteria.add(Restrictions.ilike("firstname", name, MatchMode.ANYWHERE));
It will gerenerate SQL as firstname like '%abcd%' , if you pass firstname as "abcd"
Can you simply try this
session = HibernateUtil.getSessionFactory().openSession();
List<Person> personList = session.createQuery("Select * from Person p where p.firstname like 'rake%'").list();
System.out.println("Person result :" + personList .size());

Spring, Hibernate, Blob lazy loading

I need help with lazy blob loading in Hibernate.
I have in my web application these servers and frameworks: MySQL, Tomcat, Spring and Hibernate.
The part of database config.
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="user" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="driverClass" value="${jdbc.driverClassName}"/>
<property name="jdbcUrl" value="${jdbc.url}"/>
<property name="initialPoolSize">
<value>${jdbc.initialPoolSize}</value>
</property>
<property name="minPoolSize">
<value>${jdbc.minPoolSize}</value>
</property>
<property name="maxPoolSize">
<value>${jdbc.maxPoolSize}</value>
</property>
<property name="acquireRetryAttempts">
<value>${jdbc.acquireRetryAttempts}</value>
</property>
<property name="acquireIncrement">
<value>${jdbc.acquireIncrement}</value>
</property>
<property name="idleConnectionTestPeriod">
<value>${jdbc.idleConnectionTestPeriod}</value>
</property>
<property name="maxIdleTime">
<value>${jdbc.maxIdleTime}</value>
</property>
<property name="maxConnectionAge">
<value>${jdbc.maxConnectionAge}</value>
</property>
<property name="preferredTestQuery">
<value>${jdbc.preferredTestQuery}</value>
</property>
<property name="testConnectionOnCheckin">
<value>${jdbc.testConnectionOnCheckin}</value>
</property>
</bean>
<bean id="lobHandler" class="org.springframework.jdbc.support.lob.DefaultLobHandler" />
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="configLocation" value="/WEB-INF/hibernate.cfg.xml" />
<property name="configurationClass" value="org.hibernate.cfg.AnnotationConfiguration" />
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">${hibernate.dialect}</prop>
</props>
</property>
<property name="lobHandler" ref="lobHandler" />
</bean>
<tx:annotation-driven transaction-manager="txManager" />
<bean id="txManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
The part of entity class
#Lob
#Basic(fetch=FetchType.LAZY)
#Column(name = "BlobField", columnDefinition = "LONGBLOB")
#Type(type = "org.springframework.orm.hibernate3.support.BlobByteArrayType")
private byte[] blobField;
The problem description. I'm trying to display on a web page database records related to files, which was saved in MySQL database. All works fine if a volume of data is small. But the volume of data is big I'm recieving an error java.lang.OutOfMemoryError: Java heap space
I've tried to write in blobFields null values on each row of table. In this case, application works fine, memory doesn't go out of. I have a conclusion that the blob field which is marked as lazy (#Basic(fetch=FetchType.LAZY)) isn't lazy, actually!

I'm confused. Emmanuel Bernard wrote in ANN-418 that #Lob are lazy by default (i.e. you don't even need to use the #Basic(fetch = FetchType.LAZY) annotation).
Some users report that lazy loading of a #Lob doesn't work with all drivers/database.
Some users report that it works when using bytecode instrumentation (javassit? cglib?).
But I can't find any clear reference of all this in the documentation.
At the end, the recommended workaround is to use a "fake" one-to-one mappings instead of properties. Remove the LOB fields from your existing class, create new classes referring to the same table, same primary key, and only the necessary LOB fields as properties. Specify the mappings as one-to-one, fetch="select", lazy="true". So long as your parent object is still in your session, you should get exactly what you want. (just transpose this to annotations).

I would suggest you to use inheritance to handle this scenario. Have a base class without the blob and a derived class containing the byte array. You would use the derived class only when you need to display the blob on the UI.

Of course you could extract that value and put it into a new table with a "#OneToOne" relation that is lazy, however in our application the LOBs are loaded lazily on demand using just this configuration
#Lob
#Fetch(FetchMode.SELECT)
#Type(type="org.hibernate.type.PrimitiveByteArrayBlobType")
byte[] myBlob;
This is tested in our project simultaneously on PostgreSQL, MySQL, SQLServer and Oracle, so it should work for u

Lazy property loading requires buildtime bytecode instrumentation.
Hibernate docs: Using lazy property fetching
If you want to avoid bytecode instrumentation one option is to to create two entities that use same table, one with the blob one without. Then only use the entity with blob when you need the blob.

I had the same issue and this was my fix:
My Entity:
#Entity
#Table(name = "file")
public class FileEntity {
#Id
#GeneratedValue
private UUID id;
#NotNull
private String filename;
#NotNull
#Lob #Basic(fetch = FetchType.LAZY)
private byte[] content;
...
Added plugin to pom.xml:
<plugin>
<groupId>org.hibernate.orm.tooling</groupId>
<artifactId>hibernate-enhance-maven-plugin</artifactId>
<executions>
<execution>
<phase>compile</phase>
<configuration>
<failOnError>true</failOnError>
<enableLazyInitialization>true</enableLazyInitialization>
</configuration>
<goals>
<goal>enhance</goal>
</goals>
</execution>
</executions>
</plugin>

For me lazy load only worked by compiling and then running it, didn't work on eclipse or intellij for example.
I'm using gradle then I did the following to get it working
Annotate entity
Setup Hibernate gradle plugin
build.gradle
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath "org.hibernate:hibernate-gradle-plugin:5.4.0.Final"
}
}
apply plugin: 'java'
apply plugin: 'application'
apply plugin: 'org.hibernate.orm'
hibernate {
enhance {
enableLazyInitialization = true
enableDirtyTracking = true
enableAssociationManagement = true
}
}
Entity.java
#Entity
public class Person {
#Id
#GeneratedValue
private Integer id;
#Lob
#Basic(fetch = FetchType.LAZY)
#Column(length = 255, nullable = false)
private String name;
Testing
./gradlew run
Full working example

Lazy loading works for me if I use Blob type instead of byte[].
#Column(name = "BlobField", nullable = false)
#Lob
#Basic(fetch = FetchType.LAZY)
private Blob blobField;
This one gets lazily loaded and if you need to retrieve its value access this field:
String value = IOUtils.toByteArray(entity.getBlobField().getBinaryStream());

A simple workarround using #OneTone notation based on the response of #MohammadReza Alagheband (Why does #Basic(fetch=lazy) doesn't work in my case?) but without the requirement of create a new table for each required lazy attribute is the following:
#Getter
#Setter
#Entity
#Table(name = "document")
#AllArgsConstructor
#NoArgsConstructor
public class DocumentBody implements java.io.Serializable{
#Column(name = "id", insertable = false)
#ReadOnlyProperty
#Id
#PrimaryKeyJoinColumn
private Integer id;
#Column(name = "body", unique = true, nullable = false, length = 254)
#JsonView({JSONViews.Simple.class, JSONViews.Complete.class})
private String content;
}
#Getter
#Entity
#Setter
#Table(name = "document")
#AllArgsConstructor
#NoArgsConstructor
public class DocumentTitle implements java.io.Serializable{
#Column(name = "id", insertable = false)
#ReadOnlyProperty
#Id
private Integer id;
#Column(name = "title", unique = true, nullable = false, length = 254)
#JsonView({JSONViews.Simple.class, JSONViews.Complete.class})
private String content;
}
public class Document implements java.io.Serializable {
#Id
#GeneratedValue(strategy = IDENTITY)
#Column(name = "id", unique = true, nullable = false)
#JsonView({JSONViews.Simple.class, JSONViews.Complete.class})
private Integer id;
//Also it is posssible to prove with #ManyToOne
#OneToOne(fetch = FetchType.LAZY, optional = false, cascade = {CascadeType.MERGE, CascadeType.PERSIST, CascadeType.REFRESH})
#JoinColumn(name = "id", referencedColumnName = "id", nullable = false)
#JsonView({JSONViews.Simple.class, JSONViews.Complete.class})
private DocumentTitle title;
#OneToOne(fetch = FetchType.LAZY, optional = false, cascade = {CascadeType.MERGE, CascadeType.PERSIST, CascadeType.REFRESH})
#JoinColumn(name = "id", referencedColumnName = "id", nullable = false)
#JsonView({JSONViews.Simple.class, JSONViews.Complete.class})
private DocumentBody body;
}

Develop Reference

Java is a programming language and computing platform first released by Sun Microsystems in 1995.

Hibernate lucene only searches in 60 first records - java

The most likely explanation is that your entities haven't been indexed. Did you take care of indexing pre-existing data, for example using a Mass indexer? Did you check the logs to see if there are indexing errors?

Related

Entity not mapped, although mapped

Hibernate #GeneratedValue(name=???): ID gets incremented by five not by one

Unable to load class declared as < mapping class="Package.Class" /> in the configuration

Hibernate Criteria creates a '?' in the generated SQL

Spring, Hibernate, Blob lazy loading

Categories

Resources