Twitter4j how to get tweets from certain time - java

Query query = new Query("Apple");
query.lang("en");
query.setCount(100);
query.setSince("2018-12-03");
query.setUntil("2018-12-04");
QueryResult result = twitter.search(query);
SentiWordNetDemoCode sentiwordnet = new SentiWordNetDemoCode();
for (Status tweet : result.getTweets()){
System.out.println(tweet.getCreatedAt());
}
When testing this, all the tweets are from 7:59 SRET. Is there any way to get a tweet from a time other than this?

According to the API reference on twitter, there are some restrictions to the query. There is no (more) since parameter, only a since_id which takes a status id. Then you can only search in the last 7 days. So the until field should not be older than that.
https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets.html

Related

Wrong result metadata when using pagination with Balze-Persistence

When I fetch all results without pagination result is OK but when use pagination appear problem in pagination metadata
Page p=PageImpl(result.getResults(), PageRequest.of(pageNo, pageSize), result.getTotal())
p.getTotalElements() // wrong result
p.result.getTotalPages() // also is wrong
code snippet:-
QOrderAppliedTax qTax = QOrderAppliedTax.orderAppliedTax;
QOrderDetails qOrderDetails = QOrderDetails.orderDetails;
NumberExpression<Integer> groupBy = qOrderDetails.createdDate.yearWeek();
StringPath name = qTax.name;
NumberExpression<Double> totalTaxValue = qTax.value.sum();
NumberExpression<Long> ordersCount = qOrderDetails.order.countDistinct();
DateTimeExpression<Date> fromDate = qOrderDetails.createdDate.min();
DateTimeExpression<Date> toDate = qOrderDetails.createdDate.max();
CriteriaBuilderFactory cbf = Criteria.getDefault().createCriteriaBuilderFactory(entityManagerFactory);
QueryResults<TaxDto> result= new BlazeJPAQuery<>(em, cbf)
.select(Projections.constructor(TaxDto.class, fromDate, toDate, groupBy, name, totalTaxValue,
ordersCount))
.from(qTax).innerJoin(qOrderDetails).on(qTax.orderDetails.eq(qOrderDetails))
.groupBy(taxName, groupBy).offset(pageNo * pageSize).limit(pageSize).fetchResults();
the wrong result appears even I used
orderBy(groupBy.asc(), taxName.asc())
exactly the two fields in order by is unique together.
This issue turned out to be a bug which will be resolved in Blaze-Persistence 1.6.3 (expected soon). The patch has already landed in 1.6.3-SNAPSHOT.

How to query Marklogic with punctuation-sensitve terms using JAVA?

I have the following info stored in Marklogic for the json files as follows.
1.json>> "dateSubmitted" : "2017/10/11 09:15:14"
2.json>> "dateSubmitted" : "2017/10/11 10:13:14"
3.json>> "dateSubmitted" : "2017/10/14 11:12:13"
My query term is:
String dateQuery = "2017/10/11";
I tried 2 methods and none seems to be working.
Method 1:
StructuredQueryBuilder qb = new StructuredQueryBuilder();
QueryDefinition queryDef = qb.and(qb.word((qb.jsonProperty("dateSubmitted"),dateQuery)));
queryDef.setDirectory(DIRECTORY);
SearchHandle resultsHandle = new SearchHandle();
queryManager.search(queryDef, resultsHandle, start);
Method 2:
StructuredQueryBuilder qb = new StructuredQueryBuilder();
String[] wordQueryOptions = {"punctuation-sensitive", "space-sensitive"};
QueryDefinition queryDef = qb.and(qb.word((qb.jsonProperty("dateSubmitted"),
FragmentScope.DOCUMENTS,
wordQueryOptions,100.0,dateQuery)));
queryDef.setDirectory(DIRECTORY);
SearchHandle resultsHandle = new SearchHandle();
queryManager.search(queryDef, resultsHandle, start);
The expected result is to return only 1.json and 2.json.
However 3.json was also returned.
Is there some settings I'm missing in my Marklogic admin to activate options or punctuation-sensitive?
Working with dates is often easier and more powerful if you index the property as a date. That way, you can do before and after matches on the date as well as sort on the date.
To index a property as a date, you can create a range index on the date. You can then use a range query on the date.
In MarkLogic 9, you can also use TDE to project rows from the documents with a column for the dates.
Hoping that helps,

String Parser in Lucene

I am new in Lucene Search,
I merged DrillDownQuery and Boolean Query in such a way like
BooleanQuery inner=new BooleanQuery();
inner.add(NumericRangeQuery.newIntRange("year",null,null, false, false));
DrillDownQuery q = new DrillDownQuery(config);
q.add("book","lucene in action","lissa" );
inner.add(q,Occur.MUST)
DrillDownQuery q1 = new DrillDownQuery(config);
q1.add("book","java programming","hills publications" );
inner.add(q1,Occur.MUST)
i just want to store the above Query in Database as a string
so that I can use it
String str=inner.toString();
the str variable contains this type of value
++ConstantScore($facets:booklucene in actionlissa)^0.0 ++ConstantScore($facets:bookjava programminghills publications)^0.0 +year:{* TO *}
as in future whenever any user clicks on link, I just want to fetch this string query from database and get parse into Lucene Query
for this purpose i used lots of parser but am not able to parse such type of query. Some of the parser I am using is here
QueryParser qs=new QueryParser(null,new WhitespaceAnalyzer());
Query q= qs.parse(str)
I have also use ComplexPhraseQueryParser,SimpleParser but I am not getting proper result yet
any kind of help would be heartily appreciated

How to reduce the query quota in App engine?

The user will browse (paging) sorted entities in Datastore 12 at time.
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
//get url parameter
int next = Integer.parseInt(request.getParameter("next") );
//query with sort
Query query = new Query("page").addSort("importance", SortDirection.DESCENDING );
PreparedQuery pq = datastore.prepare(query);
//get 12 entity from query result from the index (next)
FetchOptions options = FetchOptions.Builder.withLimit(12).chunkSize(12).offset(next);
for (Entity result : pq.asIterable(options)) {
Text text = (Text)result.getProperty("content");
Document doc = Jsoup.parse(text.getValue());
//display the content
.....
}
The problem is that when the next variable increase the quota consume increase faster!.
For example when the next is 6000, the quota consumed by 40%, while when the next is 10 the quota consumed by less than 1%.
If you you use the Google App Engine cursors to facilitate your paging then your queries will be optimized. It is not recommended to use large offsets. The recommended way to do paging in GAE is with cursors.

Hibernate search with Criteria restriction returning incorrect count

The result list is perfect but the getResultSize() is incorrect.
I've knocked up some code to illustrate.
Criteria criteria2 = this.getSession().createCriteria(Film.class);
Criterion genre = Restrictions.eq("genreAlias.genreName", details.getSearch().getGenreName());
criteria2.createAlias("genres", "genreAlias", CriteriaSpecification.INNER_JOIN);
criteria2.add(genre);
criteria2.setMaxResults(details.getMaxRows())
.setFirstResult(details.getStartResult());
FullTextEntityManager fullTextEntityManager = org.hibernate.search.jpa.Search.createFullTextEntityManager(entityManager);
org.apache.lucene.queryParser.QueryParser parser2 = new QueryParser("title", new StopAnalyzer() );
org.apache.lucene.search.Query luceneQuery2 = parser2.parse( "title:"+details.getSearch()");
FullTextQuery fullTextQuery = fullTextEntityManager.createFullTextQuery( luceneQuery2, Film.class);
fullTextQuery.setCriteriaQuery(criteria2);
fullTextQuery.getResultList()); // Returns the correctly filtered list
fullTextQuery.getResultSize()); // Returns the retsult size without the genre resrtiction
From http://docs.jboss.org/hibernate/search/3.3/api/org/hibernate/search/jpa/FullTextQuery.html
int getResultSize()
Returns the number of hits for this search Caution: The number of results might be slightly different from getResultList().size() because getResultList() may be not in sync with the database at the time of query.
You should try to use some of the more specialized queries like this one:
Query query = new FuzzyQuery(new Term("title", q));
FullTextQuery fullTextQuery = fullTextSession.createFullTextQuery(query, Film.class);
int filmCount = fullTextQuery.getResultSize();
and this is how you do pagination requests (I'm guessing you have improperly implemented your paggination):
FullTextQuery hits = Search.getFullTextSession(getSession()).createFullTextQuery(query, Film.class)
.setFirstResult((pageNumber - 1) * perPageItems).setMaxResults(perPageItems);
The above works for me every time. You should keep in mind that the result of getResultSize() more of estimate. I use pagination a lot and I have experienced the number changing between pages. So you should say "about xxxx" results.

Categories

Resources