I'm trying to write a simple Java desktop app to query Google and get the results. I've read so many articles in the past few days. I know that I have to do it through the GWT (Google Web Toolkit), the API that Google offers to programmatically query its search engine. My problem is that the GWT seems to be useful only to construct AJAX applications. I just want to write a simple desktop app. I am not interested in Javascript, XML or any server side application. Isn't there a way to do it?
Sure, take a look at this approach.
The general idea is that you make an HTTP fetch (using vanilla java.net.HttpUrlConnection or Apache HTTP Client). The magic is in the forming of the URL with the search term, and in the processing of what you get back from Google.
For the exact details on how to do this, see Google's REST search API documentation, including this section.
You can always get a license for the webservice and get the results returned through SOAP, however you did say that you were uninterested in XML. Shame, thats the quickest way. There is always doing a direct request to google through the query URL, since it uses HTTP get.
Related
How can I use ember js without ember-cli with PHP or java? How can I interact with back end data? Any example will be much appreciated.
The first thing you'll need to understand, is that ember-cli is a command line interface. It acts as a blueprint generator and an asset pipeline and glues all the ember stuff together in a wonderful productive package. It's just standard now, and you wouldn't want to use Ember without CLI. They are basically now the same thing unless you were using Ember since pre 1.13 and can't upgrade.
Ember is a JavaScript framework for front-end/client-side user interface. It doesn't deal with server style data storage like a traditional Apache/PHP. In a PHP setup, each page is rendered on the server side and then delivered to the browser in a complete form. Ember is more like a shell that you fill with data from somewhere else, but also has the ability to manipulate that data in the browsers and persist it back to the data-source.
The data source could really be something as simple as local storage. Maybe a simple game that just stores your userScore in the built-in local storage.
The data source could be something like the WordPress API, where ember pulls in posts or page data to display on the screen. Maybe you even have a form that persists new posts back to the server.
You could use parse or firebase for real-time back-end as a service. Or you could build a server of any sort that was able to generate an API that Ember could consume.
Currently, rails is popular for the server. Also, node frameworks like hapi.js and sails. Elixr and Pheonix are gaining interest and have proven to be a powerful backend team with Ember.
Ember uses ember-data and adapters to take in API data and serialize into a simple set of conventions.
If you have API endpoints coming from PHP or Java, you could use them - however, if you are starting a new project - I wouldn't think that would be an ideal route.
You can follow the Ember guides tutorial to learn more about how Ember works. Good luck!.
I have the following questions which I have researched a lot on Google as well as in SO but found nothing::
I have to create a Native Android Application of a Magento based Site, and I have to use the same database of Magento Site. Now, I have found that we can't Access the PHP files of the Magento project as the access is denied. So how to call a Magento Login Function from the Android Client?
I have come to know that we have to call the function through using XML-Connect file in Magento. Please any one can give me a Example how to call that file and how?
Last Parsing. Which parser would be more handy? (DOM, SAX, SIMPLE XML Serialization parser?)
How to manage the Sessions with the Magento Store So that the user can add to cart items and could do Online Transaction.
Anyone who has worked on Magento Site's App Development provide some direction to move further. I am totally lost and getting nothing from the Online Research.
Any help would be highly Appreciated.
And Regards.
I've not worked with Magento site. But I've worked with similar websites to connect to the mobile applications.
Short Answer: XML-Connect is the best way to minimize your work. And its easy to use API than connecting to the database using php files. For mobile applications, it is always advisable to use a simple xml format, however, in your case, you have to stick to xml-connect's protocol.
For some more information, see this link http://inchoo.net/ecommerce/magento/develop-your-own-magento-mobile-application/
does any one has idea how to get google page index in Java?
I was googleing since last 2-3 days but helpless, can any one refer me API for that or give some suggestion for how to do that
Lots of thanks in advance
For example if we search for facebook in google, we get around 22,980,000,000 results. So I want to fetch this number using JAVA
make a corresponding HTTP request from Java to Google, then parse the replied HTML code. There is a div with the ID resultStats. This div contains the number of results.
Not sure what your real requirement is, what kind of index do you want? Google export fairly a bit amount of APIs via RESTful service, some of them are packaged with JavaScript lib like Google MAP API. There are also Java client library for OAUTH authentication
The custom search API information could be found at http://code.google.com/apis/customsearch/v1/overview.html. A comprehensive list of google APIs could be accessed at https://code.google.com/apis/console
Does google provide a Java server side api **(NOT java script).**I dont want the ajax api which works at clint side.
What i want is that the result returned for a keyword should return me search result in some specific data structure.Like List or set data structure.Then i want to manipulate the result according to my need in java code.
I had used such a java server site api for youtube.
They used to but is discontinued. Most likely because you can remove their ads ;-)
From Google's terms of service
You may not send automated queries of any sort to Google's system
without express permission in advance from Google.
So, no - use their AJAX API. The SAOP API is discoutinued, but I think you should be able to use it, unless it requires a key, in which case you are tied to the AJAX API
Technically, the statement quoted above doesn't mean you can't use some sort of server-side API - there are examples of that if you google around. It means you shouldn't do it, because sooner or later you will be blacklisted (banned), as violating the terms.
What you would be looking for now is the CustomSearch API:
They used to have a different API developers could use but it has been depreciated (Nov. 2009 I think) so this is a for cost service now. I am not aware how long it might have been a free service. The new API allows 100 free searches a day, but you have to be signed up for billing else anything above 100 searches will fail, here are the details:
http://code.google.com/apis/customsearch/v1/overview.html
And sorry yes, this would still be using Java Script but you could use this:
http://www.json.org/java/
There is at least a SOAP API that I'm aware of: Google SOAP Search API
I don't think Google wants 3rd parties to use their search engine for their own services/applications. You would get "we think you are a robot" error page as a result if Google thinks you are not a real person.
You can however try Google Custom Search
I need to screen scrape some data from a website, because it isn't available via their web service. When I've needed to do this previously, I've written the Java code myself using Apache's HTTP client library to make the relevant HTTP calls to download the data. I figured out the relevant calls I needed to make by clicking through the relevant screens in a browser while using the Charles web proxy to log the corresponding HTTP calls.
As you can imagine this is a fairly tedious process, and I'm wodering if there's a tool that can actually generate the Java code that corresponds to a browser session. I expect the generated code wouldn't be as pretty as code written manually, but I could always tidy it up afterwards. Does anyone know if such a tool exists? Selenium is one possibility I'm aware of, though I'm not sure if it supports this exact use case.
Thanks,
Don
I would also add +1 for HtmlUnit since its functionality is very powerful: if you are needing behaviour 'as though a real browser was scraping and using the page' that's definitely the best option available. HtmlUnit executes (if you want it to) the Javascript in the page.
It currently has full featured support for all the main Javascript libraries and will execute JS code using them. Corresponding with that you can get handles to the Javascript objects in page programmatically within your test.
If however the scope of what you are trying to do is less, more along the lines of reading some of the HTML elements and where you dont much care about Javascript, then using NekoHTML should suffice. Its similar to JDom giving programmatic - rather than XPath - access to the tree. You would probably need to use Apache's HttpClient to retrieve pages.
The manageability.org blog has an entry which lists a whole bunch of web page scraping tools for Java. However, I do not seem to be able to reach it right now, but I did find a text only representation in Google's cache here.
You should take a look at HtmlUnit - it was designed for testing websites but works great for screen scraping and navigating through multiple pages. It takes care of cookies and other session-related stuff.
I would say I personally like to use HtmlUnit and Selenium as my 2 favorite tools for Screen Scraping.
A tool called The Grinder allows you to script a session to a site by going through its proxy. The output is Python (runnable in Jython).