I've searched high and low for an answer and cant seem to find anything, so apologies if this has been asked before.
I have a simple Spring boot API which allows students to add their college projects. These projects are stored in ES which I'm trying to make searchable.
FWIW, From a use case POV, a user can search for a project with a query string which is based on a project Name or a hashtag for the project i.e. "Java".
Right now with the code below, if I search for projects which have a given hashtag I can retrieve them no problem. I get exactly the results I'm looking for.
However...once I try to add a second query whereby I want to limit the results to only students who have a userId contained in a ArrayList my search fails. By Fails I mean it returns everything from the index.
Can anyone spot what I'm doing wrong?
Many thanks
private Iterable<UserProjects> searchCollegeProjects(Integer page, Integer size, List<String> hashtags, List<String> studentUserIds) {
PageRequest pageRequest = PageRequest.of(page, size);
QueryBuilder projectQueryBuilder = null;
if (hashtags != null) {
projectQueryBuilder =
QueryBuilders
.multiMatchQuery(hashtags.toString(), "projectName", "projectHashtag")
.fuzziness(Fuzziness.AUTO);
}
BoolQueryBuilder userIdQueryBuilder = null;
if(studentUserIds!=null) {
userIdQueryBuilder = QueryBuilders.boolQuery()
.must(QueryBuilders.termsQuery("userId.keyword", studentUserIds));
}
Query searchQuery = new NativeSearchQueryBuilder()
.withQuery(projectQueryBuilder)
.withQuery(userIdQueryBuilder)
.withFilter(boolQuery()
.mustNot(QueryBuilders.termsQuery("isDraft", true)))
.withPageable(PageRequest.of(page, size))
.build();
SearchHits<UserProjects> projectHits;
try {
projectHits =
elasticsearchOperations
.search(searchQuery, UserProjects.class,
IndexCoordinates.of(elasticProjectsIndex));
} catch (Exception ex) {
logger.error("Error unable to perform search query" + ex);
throw new GeneralSearchException(ex.toString());
}
List<UserProjects> projectMatches = new ArrayList<>();
projectHits.forEach(srchHit -> {
projectMatches.add(srchHit.getContent());
});
long totalCount = projectHits.getTotalHits();
Page<UserProjects> resultPage = PageableExecutionUtils.getPage(
projectMatches,
pageRequest,
() -> totalCount);
return resultPage;
}
Also below is my POJO for UserProjects:
#Getter
#Setter
#Document(indexName = "college.userprojects")
#JsonIgnoreProperties(ignoreUnknown = true)
public class UserProjects {
#Id
#Field(type = FieldType.Auto, name ="_id")
private String projectId;
#Field(type = FieldType.Text, name = "userId")
private String userId;
#Field(type = FieldType.Text, analyzer = "autocomplete_index", searchAnalyzer = "lowercase" ,name = "projectName")
private String projectName;
#Field(type = FieldType.Auto, name = "projectHashTag")
private List<String> projectHashTag;
#Field(type = FieldType.Boolean, name = "isDraft")
private Boolean isDraft;
}
EDIT: Just to be clear, if I remove this line from the native query:
.withQuery(userIdQueryBuilder)
I get back projects (Correctly) which contain the query text in either their projectName or projectHashtag fields. Putting the above line back in will cause the search to return everything.
Related
I am using hibernate-generic-dao for a searching function. Since I only need to show one of the records if they have same value on a column field. But I am not sure how to achieve this by the search / filter functions.
package com.googlecode.genericdao.search;
PersonContact domain object:
...
#Column(name = "group_key", length = 20)
public String getGroupKey() {
return groupKey;
}
#Formula(value = "(SELECT status from person_contact m " +
" WHERE m.case = case AND m.movement_id = movement_id )")
public String getActiveRecord() {
return activeRecord;
}
...
Search search = new Search();
search.addFilterNotNull("groupKey"); //groupKey is the field I want to use "group by / unqiue" with it
search.addFilterEqual("type","C");
search.addFilterCustom("{activeRecord} != 'I' ");
search.setMaxResults(limit);//for paging
search.setFirstResult(startIdx);
SearchResult<PersonContact> resultObj = PersonContactDAO.searchAndCount(search);
You should probably ask this question by opening an issue in the repository for that project here: https://github.com/vincentruan/hibernate-generic-dao
It seems though as if the project is abandoned, so unless you feel like digging into the details, you should probably try to get away from it.
I am fetching a param list from my jsp which I need to identify according to the prefix so that I can set the values in my entity class.
The parameter names looks like the below:
List<String> reqParamNames = Arrays.asList("1_component_role", "2_component_role", "3_component_role",
"4_component_role", "1_survey_wt", "2_survey_wt", "3_survey_wt", "4_survey_wt", "1dynaGroup1", "1component_role1", "1wt1", "2dynaGroup1", "2component_role1", "2wt1", "3dynaGroup1",
"3component_role1", "3wt1", "4dynaGroup1", "4component_role1", "4wt1");
Now, from the above list, I need to get the param according to the prefix, i.e.1,2,3,4 etc. Once grouped correctly, I would need to set it to my Entity class so that I can save the parameters in my table using Hibernate.
I am unable to set the values for the dynamic table.
#RequestMapping(value = { "dynamicSettings/persist" }, method = RequestMethod.POST)
public String saveComponents(HttpServletRequest request, HttpServletResponse response,
Model model) {
LOG.debug("Entering persist area :: ");
Locale locale = LocaleUtil.getLocale();
// ToDo: validation for form
//For dynamic tables
List<String> reqParamNames = (List<String>) Collections.list((Enumeration<String>)request.getParameterNames());
for(int i =0; i < reqParamNames.size(); i++){
System.out.println("Param names are {} ::"+ reqParamNames.get(i));
String paramName = reqParamNames.get(i);
Matcher m = Pattern.compile("[^0-9]*([0-9]+).*").matcher(paramName);
if (m.matches()) {
System.out.println("Number ::" +m.group(1)); // Need to comment/remove this post development
}
System.out.println("ParamNumber ::" +""+m.group(1));
String attributeValue = request.getParameter(paramName);
System.out.println("Param Name ::"+paramName+"::: Attribute value ::"+attributeValue);
DynamicComponentSettings dynamicSettings = new DynamicComponentSettings();
if( i == paramNumber){
String group_type = request.getParameter("groupType"+i);
String component_role = request.getParameter("component_role"+i);
String survey_weight = request.getParameter("wt"+i);
System.out.println("Group Type ::"+group_type+ "::Component Role::" +component_role+ "::Survey Weight::"+survey_weight);
if(!StringUtils.isEmpty(group_type) && StringUtils.isEmpty(component_role)&&!StringUtils.isEmpty(survey_weight)){
Double survey_wt = Double.parseDouble(survey_weight);
dynamicSettings.setSurvey_wt(survey_wt);
dynamicSettings.setGroup_type(group_type);
dynamicSettings.setComponent_role(component_role);
}
}
dynamicComponentService.saveDynamicComponents(dynamicSettings);
}
**//For concrete table**
List<DynamicComponentSettings> resultList = dynamicComponentService.loadAllDynamicComponents();
for(DynamicComponentSettings component : resultList)
{
String _survey_wt = request.getParameter(component.getPk1().toString() + "survey_wt");
String _groupRoleType = request.getParameter(component.getPk1().toString() + "group_type");
String _componentRole = request.getParameter(component.getPk1().toString() + "component_role");
if(!StringUtils.isEmpty(_survey_wt) && StringUtils.isEmpty(_groupRoleType)&&!StringUtils.isEmpty(_componentRole)){
Double survey_wt = Double.parseDouble(_survey_wt);
component.setSurvey_wt(survey_wt);
component.setGroup_type(_groupRoleType);
component.setComponent_role(_componentRole);
}
dynamicComponentService.saveDynamicComponents(component);
}
return "redirect:" + "some url";
}
The concrete table works correctly, i.e. saving values correctly.
Entity class
<package declaration>
<imports>
#Entity
#Table(name = "dynamic_components")
public class DynamicComponentSettings {
/** The pk1. */
#Id
#SequenceGenerator(name = "dynamic_components_seq", sequenceName = "dynamic_components_seq", allocationSize = 1)
#GeneratedValue(strategy = GenerationType.AUTO, generator = "dynamic_components_seq")
private Long pk1;
private String group_type;
private String component_role;
private Double survey_wt;
<getters and setters>
}
Please provide your inputs and provide guidance as how to save the dynamic table values.
I managed to figure out the problems and fix it. There are multiple problems in that code sample. For eg. the loops are incorrect, the approach to match with the main loop was horribly wrong.
I had to get the prefix and suffix for each item, loop correctly[i.e. first get the suffix and prefix of the table and then based on suffix match, iterate through the prefix like rows] and put each dynamic table data with suffix being the key and prefix+data+suffix being the values into a Map containing linked lists. After that, I had to loop again through the Map containing the lists to set the values to my entity class correctly.
Please pardon me for not posting the code again as the method is quite long and may not be of much help. In any case, if someone does want to see how this was solved, please let me know.
Thanks for your time.
I want a CSV format for Order objects. My Order Object will have order details, order line details and item details. Please find the java object below:
Order {
OrderNo, OrderName, Price,
OrderLine {
OrderLineNo, OrderLinePrice,
Item{
ItemNo, ItemName, Item Description
}
}
}
Can anyone please guide me to create csv format for this.
Have a POJO class for your Object for which you want to create CSV file and use java.io.FileWriter to write/append values in csv file. This Java-code-geek Link will help you with this.
If you are feeling adventurous, I'm building support for nested elements in CSV in uniVocity-parsers.
The 2.0.0-SNAPSHOT version supports parsing nested beans with annotations. We are planning to release the final version in a couple of weeks. Writing support has not been implemented yet, so that part you'll have to do manually (should be fairly easy with the current API).
Parsing this sort of structure is more complex, but the parser seems to be working fine for most cases. Have a look at that test case:
Input CSV:
1,Foo
Account,23234,HSBC,123433-000,HSBCAUS
Account,11234,HSBC,222343-130,HSBCCAD
2,BAR
Account,1234,CITI,213343-130,CITICAD
Note that the first column of each row identifies which bean will be read. As "Client" in the CSV matches the class name, you don't need to annotate
Pojos
enum ClientType {
PERSONAL(2),
BUSINESS(1);
int typeCode;
ClientType(int typeCode) {
this.typeCode = typeCode;
}
}
public static class Client {
#EnumOptions(customElement = "typeCode", selectors = { EnumSelector.CUSTOM_FIELD })
#Parsed(index = 0)
private ClientType type;
#Parsed(index = 1)
private String name;
#Nested(identityValue = "Account", identityIndex = 0, instanceOf = ArrayList.class, componentType = ClientAccount.class)
private List<ClientAccount> accounts;
}
public static class ClientAccount {
#Parsed(index = 1)
private BigDecimal balance;
#Parsed(index = 2)
private String bank;
#Parsed(index = 3)
private String number;
#Parsed(index = 4)
private String swift;
}
Code to parse the input
public void parseCsvToBeanWithList() {
final BeanListProcessor<Client> clientProcessor = new BeanListProcessor<Client>(Client.class);
CsvParserSettings settings = new CsvParserSettings();
settings.getFormat().setLineSeparator("\n");
settings.setRowProcessor(clientProcessor);
CsvParser parser = new CsvParser(settings);
parser.parse(new StringReader(CSV_INPUT));
List<Client> rows = clientProcessor.getBeans();
}
If you find any issue using the parser, please send update this issue
I have a csvfile
id|name
1|PC
2|Activation
3|USB
public class TESTResult
{
private Long id;
private String name;
private Float score;
// with setters & getters
}
public class TEST
{
private Long id;
private String name;
// with setters & getters
}
public class JobTESTTagger {
private static Version VERSION;
private static CharArraySet STOPWORDS;
private static RewriteMethod REWRITEMETHOD;
private static Float MINSCORE = 0.0001F;
static {
BooleanQuery.setMaxClauseCount(100000);
VERSION = Version.LUCENE_44;
STOPWORDS = StopAnalyzer.ENGLISH_STOP_WORDS_SET;
REWRITEMETHOD = MultiTermQuery.CONSTANT_SCORE_FILTER_REWRITE;
}
public static ArrayList<TESTResult> searchText(String text, String keyId,
List<TEST> TESTs) {
ArrayList<TESTResult> results = new ArrayList<TESTResult>();
MemoryIndex index = new MemoryIndex();
EnglishAnalyzer englishAnalyzer = new EnglishAnalyzer(VERSION,STOPWORDS);
QueryParser parser = new QueryParser(VERSION, "text", englishAnalyzer);
parser.setMultiTermRewriteMethod(REWRITEMETHOD);
index.addField("text", text, englishAnalyzer);
for (int i = 0; i < TESTs.size(); i++) {
TEST TEST = TESTs.get(i);
String criteria = "\"" + TEST.getName().trim() + "\"";
if (criteria == null || criteria.isEmpty())
continue;
criteria = criteria.replaceAll("\r", " ");
criteria = criteria.replaceAll("\n", " ");
try {
Query query = parser.parse(criteria);
Float score = index.search(query);
if (score > MINSCORE) {
int result = new TESTResult(TEST.getId(), TEST.getName(),score);
results.add(result);
}
} catch (ParseException e) {
System.out.println("Could not parse article.");
}
}
return results;
}
public static void main(String[] args) {
ArrayList<TESTResult> testresults = searchText(text, keyId, iths);
CsvReader reader = new CsvReader("C:\a.csv");
reader.setDelimiter('|');
reader.readHeaders();
List<TEST> result = new ArrayList<TEST>();
while (reader.readRecord()) {
Long id = Long.valueOf(reader.get("id").trim());
String name = reader.get("name").trim();
TEST concept = new TEST(id, name);
result.add(concept);
}
String text = "These activities are good. I have a good PC in my house.";
}
I am matching 'activities' to Activation. How is it possible. Can anybody tell me how Lucene matches the words.
Thanks
R
EnglishAnalyzer, along with most language-specific analyzers, uses a stemmer. This means that it reduces terms to a stem (or root) of the term, in order to attempt to match more loosely. Mostly this works well, removing suffixes and matching up derived words to a common root. So when I search for "fish", I also find "fished", "fishing" and "fishes".
In this case though, both "activities" and "activation" both reduce to the root of "activ", resulting in the match you are seeing. Another example: "organ", "organic" and "organize" all have the common stem "organ".
You can stem or not, neither approach is perfect. If you don't stem you'll miss relevant results. If you do, you'll hit some odd irrelevant results.
To deal with specific problematic cases, you can define a stemmer exclusion set in EnglishAnalyzer to prevent stemming just on those specific problematic terms. In this case, I would think of "activation" as the probable term to prevent stemming on, though you could go either way. So I could do something like:
CharArraySet stemExclusionSet = new CharArraySet(VERSION, 1, true);
stemExclusionSet.add("activation");
EnglishAnalyzer englishAnalyzer = new EnglishAnalyzer(VERSION, STOPWORDS, stemExclusionSet);
I am new to hibernate and am having difficulty trying to get it to work for anything other than a direct table mapping scenario. Basically, I have a user defined Java class called: BookInvoice. This class is a non-persistent (not a db table) and uses columns from previously defined and Annotated) db tables. Every time I try to label it as an #Entity it tells me I cant because it is not an existing db table. How do I map it so that I dont get the
Unknown entity: com.acentia.training.project15.model.BookInvoice
error message that I have been experiencing.
My sql queries are good and I am able to get the info from the db; however, they come back as class Object and I am not permitted to cast them into my desired BookInvoice class in order to send it back to the calling method. Below pls find snipets of my work thus far . . .
Please note all of my regular classes that conform to existing db tables queries work fine, it is just the ones that are non-persistent that I am having issues with.
PurchaseOrderInvoiceDAO:
List<BookInvoice> bInvoiceList = null;
final String bookInvoiceQuery =
"SELECT Books.ID, PO_Details.QUANTITY, Books.ISBN, Books.TITLE, Books.AUTHOR, Author.Name, Books.PUBLISHED, Books.COVER, Books.SERIES, Books.SERIES_NO,\n" +
" Books.SUBJECT_ID,Books.PRICE\n" +
" FROM Purchase_Order, PO_Details, Books, Author\n" +
" WHERE Purchase_Order.ID=?\n" +
" AND Purchase_Order.ID=PO_Details.PO_ID\n" +
" AND PO_Details.Book_ID=Books.ID\n" +
" AND Books.AUTHOR=Author.ID";
Query bookInvoicQ = getSession().createSQLQuery(bookInfoQuery).addEntity(BookInvoice.class);
bookInvoicQ.setInteger(0, id);
bList = (List<Books>) bookInvoicQ.list();
BookInvoice class:
public class BookInvoice {
Integer id = null;
Integer quantity = null;
String isbn = null;
String title = null;
Integer authorId = null;
Date publishedDate = null;
String cover = null;
String series = null;
Integer seriesNo = null;
Integer subjectId = null;
Double price = null;
public BookInvoice(final Integer id, final Integer quantity, final String isbn, final String title,
final Integer authorId, final Date publishedDate, final String cover,
final String series, final Integer seriesNo, final Integer subjectId, final Double price) {
this.id = id;
this.quantity = quantity;
this.isbn = isbn;
this.title = title;
this.authorId = authorId;
this.publishedDate = publishedDate;
this.cover = cover;
this.series = series;
this.seriesNo = seriesNo;
this.subjectId = subjectId;
this.price = price;
}
public BookInvoice(){}
public Integer getId() {
return id;
}
etc. . . .
Stack Trace Snippet:
Struts Problem Report
Struts has detected an unhandled exception:
Messages:
Unknown entity: com.acentia.training.project15.model.BookInvoice
File: org/hibernate/impl/SessionFactoryImpl.java
Line number: 693
Stacktraces
org.hibernate.MappingException: Unknown entity:
com.acentia.training.project15.model.BookInvoice
org.hibernate.impl.SessionFactoryImpl.getEntityPersister(SessionFactoryImpl.java:693)
org.hibernate.loader.custom.sql.SQLQueryReturnProcessor.getSQLLoadable(SQLQueryReturnProcessor.java:335)
org.hibernate.loader.custom.sql.SQLQueryReturnProcessor.processRootReturn(SQLQueryReturnProcessor.java:376)
org.hibernate.loader.custom.sql.SQLQueryReturnProcessor.processReturn(SQLQueryReturnProcessor.java:355)
org.hibernate.loader.custom.sql.SQLQueryReturnProcessor.process(SQLQueryReturnProcessor.java:171)
org.hibernate.loader.custom.sql.SQLCustomQuery.(SQLCustomQuery.java:87)
org.hibernate.engine.query.NativeSQLQueryPlan.(NativeSQLQueryPlan.java:67)
org.hibernate.engine.query.QueryPlanCache.getNativeSQLQueryPlan(QueryPlanCache.java:166)
org.hibernate.impl.AbstractSessionImpl.getNativeSQLQueryPlan(AbstractSessionImpl.java:160)
org.hibernate.impl.AbstractSessionImpl.list(AbstractSessionImpl.java:165)
org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl.java:157)
com.acentia.training.project15.bo.PurchaseOrderInvoiceBO$PurchaseOrderInvoiceDAO.getById(PurchaseOrderInvoiceBO.java:196)
...
Ok! Finally broke down and talked to my supervisor about this. He explained that I am doing like waaaaaay to much extra work on this.
Basically, if I set the #Entity class/DB mappings up correctly then they will get all of the right information (ie. #OneToMany, etc.) from the mappings of the classes that correspond directly to the DB tables. Basically, Hibernate will go down as many levels (PO_Details ->Payments->Books, etc) they would give me all the additional information that I need and I wouldn't need to create my own custom classes.