I am currently developing a Java application based on Google Custom Search API, using their Java libraries.
According to Google's documentation, they associate a date to each indexed Web page:
Page Dates: Google estimates the date for a page based the URL, title, byline date and other features. This date can be used with the sort operator using the special structured data type date, as in &sort=date.
I want to retrieve the date associated to all the results returned for a given request. However, I didn't find anything related to this task in Google's documentation: there are parameters one can use to sort the results by date, or focus on a certain period of time, but nothing regarding retrieving the precise dates themselves. And I couldn't find any reference to this problem on the Web neither.
So, I am turning to SO to ask these questions:
Is it even possible to do that through Google's API? How?
Otherwise, is there a workaround?
Related
In Grails(which provide complete set of plugins) also provide searchable plugin which is sufficient to do indexing and searching job. Now since we are moving to AWS cloudsearch we have achieved how to search(based on lucene) any object. Now I was getting error while uploading the documents(which is actually array of objects)?
What is the best way to upload any array of object(java) which maps to the domain of the cloudsearch? This is to achieve the bulkupload from grails.
CloudSearch does not support objects. It supports the following types, as well as arrays of each type (except latlon).
date
double
int
latlon
literal
text
Reference
My team is switching to AWS Elasticsearch service for this and other reasons. CloudSearch hasn't seen an update since January, 2013 and there's no indication that they plan to make any updates or improvements to it.
If I want to get recent tweets from a particular user I can do that only using the username in twitter4j
twitter.getUserTimeline(username)
I can also look globally for all tweets that match a search criteria:
twitter.search(new Query(keyword))
Is there a way I can search within all of the tweets from one user's timeline for a particular keyword?
Yes, it's possible using the Twitter Search API but the caveat is that it will only return tweets going back 7 days.
The Twitter Search API is part of Twitter’s REST API. It allows queries against the indices of recent or popular Tweets and behaves similarly to, but not exactly like the Search feature available in Twitter mobile or web clients, such as Twitter.com search. The Twitter Search API searches against a sampling of recent Tweets published in the past 7 days.
Before getting involved, it’s important to know that the Search API is focused on relevance and not completeness. This means that some Tweets and users may be missing from search results. If you want to match for completeness you should consider using a Streaming API instead.
To use the Twitter4J API search(query) the query would be from:<username> <keyword>.
The Streaming API can be used for search but would not really apply as it's designed to return large amounts of tweets from many users at that moment.
The thorough approach would be to go through every tweet on a user's timeline and do the search yourself. This would be much more involved but not that hard actually.
You would use the GET statuses/user_timeline API (getUserTimeline() in Twitter4J) and then search your keyword in the text field.
My primary need is to get DAU, MAU, Crash percent, Availability, Rating etc., for any custom time period. (Eg: last 2days, 1week, Date1 - Date2 etc.,) So far I have been using the data from Crash Trends page in dashboard, by setting custom date values and getting the data/values manually.
So, I wanted to automate this, and started implementing the Rest API. The documentation seemed pretty vague, and I only found the endpoint "apps" in the API to be returning something related to what I am looking for (but it only provides very limited details, and no way to set custom dates)
API I request used : https://developers.crittercism.com:443/v1.0/apps?attributes=appName,crashPercent,mau,rating
Am I missing something in the documentations??
Can someone tell me how I can get the details I want from via the Rest API??
Mainly the crash trends details like AVAILABILITY/CRASH PERCENT/DAU/MAU etc., for custom date intervals (not exceeding more than a month). Thanks!
I am a product manager at Crittercism.
The developers.crittercism.com/v1.0/apps endpoint gives you a snapshot of the app data along with some other properties ( link to the apps tore, icon url etc)
For your requirement you should use this endpoint
developers.crittercism.com/v1.0/errorMonitoring/graph
You use this to get retrieve the following metrics
dau
mau
rating
crashes
crashPercen
appLoads
affectedUsers
affectedUserPercent
You can query for two time ranges 1 day (1440 mins) and 1 month (43200 mins)
Here is the documentation for this http://docs.crittercism.com/api/api.html#!/errorMonitoring/graph
Hope this helps.
I need to build a web service that analyzises SEO. The service will show how often the site was updated. I need to figure out how to get the posted date or update frequency from the HTML of the website.
For example on http://googletesting.blogspot.com/ I can get date from the tag <span>Wednesday, June 04, 2014</span>. Other websites don't use the same tags and date format so I can't us the same code to detect those dates.
(Dates can have very different formats in different locales. Also, month names can be written as text or as number. I need to match as much dates as possible.Sometime,date format isn't posted date but it's just words in articles.
My Algorithm about this
I attempt to get "posted date" from all posted then calculate update frequency.
Such as Fist posted at 30May 2012, Second posted at 29May2012, Third posted at 28May2012
So I will get result that this website was updated dairly
In the end, I want to know if each website updates:
Yearly
Monthly
Weekly
Daily
How do I reliably get this from any website?
Instead of parsing the dates in the page, you could download the home page and store it. Then you could come back every day and download the homepage again to see if it changed. This approach would work even for sites that don't publish any dates on their homepage. It would take longer to get your answer though.
Another approach would be to download the RSS feed for the site if it has one. The example site you give one has an XML feed: http://feeds.feedburner.com/blogspot/RLXA?format=xml RSS feeds are meant to be machine readable and the dates are in a consistent format.
You also say that you are using Java. I've found that Java's date parsing libraries are not very flexible. They force you to know the exact format of the date before you parse it. I have written a free, open source flexible date time parser in Java that you could try: http://ostermiller.org/utils/DateTimeParse.html Once you found dates on the page (maybe for looking at what comes after "posted on"), you could use my flexible parser to parse dates in a variety of formats.
I'm currently working with a team on a project that will serve as a campus-wide event calendar for my school. We're designing it to be a web application using JSP having a java back end and connected to a relational database located on a server. The database will store events and produce a calendar on the web page based on the events.
Users will also be able to conduct searches and we would like to return a calendar based on the search results (such as activities occurring during a particular a time frame). Potentially we would be creating 100's of calendars at a time to accommodate multiple user requests.
We don't want users to need any special account to use the site (except maybe an account with us). The users will not be editing the events and changing anything but we want a nice GUI interface for them.
Is this a possible task to achieve using the Google Calendar API?
Just to clarify, we will be performing sql queries to construct a list of "events" in a separate section of our application. With this in mind, we do NOT want a calendar that queries our database on its own. We would like a API that allows us to input this list of events, and would output a calendar GUI that provides a user with access to multiple views (daily, weekly, monthly, etc) in an easy-to-use format.
thanks!
It sounds like a decent use of the Google Calendar API to me. After browsing through the API docs for Java, it looks like you can create a calendar, add whatever events to it you wish, and pass a link to that calendar back to the user. In fact, the API page I linked mentions that "you can generate a public calendar for Google Calendar to display, based on your organization's event database". This sounds like exactly like what you are wanting to do. Try out some of the sample code there and see if it looks like it will meet your needs.
I totally agree with bta and have an additional idea:
You said:
The database will store events and
produce a calendar on the web page
based on the events.
You would benefit even more from using Google Calendar in this case. You wouldn't need a database to store the events which has many pros:
You would eliminate a possible bottleneck, because as you said there would be 100s of calendars generated at the same time,
You could have non-tech-savvy people to manage calendars (I believe Google's interface is pretty simple, compared to the backend you would have to develop)
You would eliminate the need for backend (or at least the part responsible for event CRUDs)
You can always "wrap" Google Galendar using its API, so the GUI would be completely up to you,
These are just some of my thoughts, because I believe that simple is better. I hope this will be helpful.
Good luck developing your app!
P.S. If you could, please tell us which method you used and how its working :)