I am working on a Spring MVC and I want to insert javascript into the html output for analytics purpose. I am only partially familiar with serialization but I figured it does the job nicely rather than manually constructing a string containing javascript.
Would it be possible to generate something the following snippet? Any pointers would be great!
"emd" : new Date('6/6/2014')
Update:
I need to output a javascript object which has many fields which may be complex. Hence, on the backend I am gathering all the data into java beans with all the information and I plan to use Jackson mapper to convert to string that I can just output through JSP.
Generating the above snippet does not seem straightforward though, not sure if it is even possible. For context, the rest of that javascript looks something like this.
Analytics.items["item_123"] = {
//ratings and reviews
"rat" : a.b, //the decimal value for the rating
"rev" : xxxx, //integer
//list of flags that indicate how the product was displayed to the customer
//add as needed...tracking code will pick up flags as needed when they are available
"dec" : ["mbe", "green", "recycled"],
//delivery messaging
"delivery" : {
"cd" : new Date() //current date
"offers" : [{
"type" : "abcd"
"emd" : new Date('6/6/2014'),
"weekend" : true
}
]
},
};
JSON.stringify should do the trick. It will be built into your browser, unless you're using a very old browser, in which case you can use a polyfill.
I know you can link out from a Google Chart based on this:
How to add links in google chart api
However, I have parameter that I have stored in the "f" property of the Javascript Literal Notation that I want to use to send to a dynamic link. For example, my data looks like:
"Load Time":
{"cols":[{"id":"date","label":"Date","type":"string"},
{"id":"First View","label":"First View","type":"number"},
"rows":[{"c":[{"f":"130303_67_C", "v":"2013-02-26"},{"v":3.565},{"v":2.495}]},
{"c":[{"f":"130304_R6_C", "v":"2013-02-27"},{"v":3.6550000000000002},{"v":2.63}]}]},
My question is how can I retrieve the value in "f" to use as a parameter for my link which will look like http://www.example.com?testId=
The following function will return the formatted value you require:
data.getFormattedValue(chart.getSelection()[0].row, 0)
More about this function can be read at these links:
https://developers.google.com/chart/interactive/docs/events
https://developers.google.com/chart/interactive/docs/reference#DataTable_getFormattedValue
I learn about jSoup recently and would like to dive more into it. However, I have met obstacle handling webpages with javascript (I have no knowledge in js, yet :/).
I have read that htmlunit would be the correct tool to perform webbrowser actions, but I figured out that I would need no knowledge in js if I can find out the JSON object obtained in the webpage using the javascript.
For example, this page:
among the source files, one of them is tooltips.js. In this file, variable rgNeededFeeds is generated and called in method LoadHeropediaData(), which is the method to generate the whole URL link for getting the json object.
URL = URL + 'jsfeed/heropediadata?feeds='+strFeeds+'&v=3633666222511362823&l=english';
I could not get my mind on what is actually strFeeds. I have tried various combinations but it doesn't work (it returned an empty array...). Or, my guess is totally off?
What I actually need is the data it displays on top when you click on one of the "items". The info in the "hover" would do too, but it lack the "recepi" info. And I'm presuming that by getting the json object from the full URL above, well, basically all data infos should be in that json.
Anyways, this is only based on what I understand from staring at those source files for hours. Do correct me if I'm wrong. (I'm in Java by the way)
**p/s: I would also like to take this opportunity to express my thanks to Balusc, he has been everywhere when I have doubts on jSoup. :>*
strFeeds is nothing but one of these two strings : itemdata or abilitydata
You can find this in tooltips.js at line 38-45
var rgNeededFeeds = [];
$.each( [ 'item', 'ability' ],
function( i, ttType ){
icons = GetIconCollection( ttType );
if ( icons.length ){
rgNeededFeeds.push( ttType+'data' );
//..............
}
}
)
ttType is the value of an iteration over the array [ 'item', 'ability' ] which concatenated with the string data is pushed into the array rgNeededFeeds
The function LoadHeropediaData is called at the end of the function above with rgNeededFeeds as parameter :
LoadHeropediaData( rgNeededFeeds );
Aside note : If you begin to start scraping websites, learning javascript will be MANDATORY.
NOTE : you're right, the JSON contains all the information needed...
I'm looking for access to financial data from Google services.
I found this URL that gets the stock data for Microsoft.
What are all the possible parameters that Google allows for this kind of HTTP request? I'd like to see all the different information that I could get.
The Google Finance Gadget API has been officially deprecated since October 2012, but as of April 2014, it's still active. It is completely dead as of March 2022.
http://www.google.com/finance/info?q=NASDAQ:GOOG
http://www.google.com/finance/info?q=CURRENCY:GBPUSD
http://finance.google.com/finance/info?client=ig&q=AAPL,YHOO
You can also get charts: https://www.google.com/finance/getchart?q=YELP
Note that if your application is for public consumption, using the Google Finance API is against Google's terms of service.
Check google-finance-get-stock-quote-realtime for the complete code in python
There's a whole API for managing portfolios. *Link removed. Google no longer provides a developer API for this.
Getting stock quotes is a little harder. I found one article where someone got stock quotes using Google Spreadsheets.
You can also use the gadgets but I guess that's not what you're after.
The API you mention is interesting but doesn't seem to be documented (as far as I've been able to find anyway).
Here is some information on historical prices, just for reference sake.
I found this site helpful.
http://benjisimon.blogspot.com/2009/01/truly-simple-stock-api.html
It links to an API yahoo seems to offer that is very simple and useful.
For instance:
http://finance.yahoo.com/d/quotes.csv?s=GOOG+AAPL&f=snl1
Full details here:
http://www.gummy-stuff.org/Yahoo-data.htm
Edit: the api call has been removed by google. so it is no longer functioning.
Agree with Pareshkumar's answer. Now there is a python wrapper googlefinance for the url call.
Install googlefinance
$pip install googlefinance
It is easy to get current stock price:
>>> from googlefinance import getQuotes
>>> import json
>>> print json.dumps(getQuotes('AAPL'), indent=2)
[
{
"Index": "NASDAQ",
"LastTradeWithCurrency": "129.09",
"LastTradeDateTime": "2015-03-02T16:04:29Z",
"LastTradePrice": "129.09",
"Yield": "1.46",
"LastTradeTime": "4:04PM EST",
"LastTradeDateTimeLong": "Mar 2, 4:04PM EST",
"Dividend": "0.47",
"StockSymbol": "AAPL",
"ID": "22144"
}
]
Google finance is a source that provides real-time stock data. There are also other APIs from yahoo, such as yahoo-finance, but they are delayed by 15min for NYSE and NASDAQ stocks.
The problem with Yahoo and Google data is that it violates terms of service if you're using it for commercial use. When your site/app is still small it's not biggie, but as soon as you grow a little you start getting cease and desists from the exchanges.
A licensed solution example is FinancialContent: http://www.financialcontent.com/json.php
or Xignite
You can also pull data from Google Fiance directly in Google Sheets via GOOGLEFINANCE() function for both current and historical data:
GOOGLEFINANCE("NASDAQ:GOOGL", "price", DATE(2014,1,1), DATE(2014,12,31), "DAILY")
Another way is to use Yahoo finance instead via yfinance package. Or with such query which will return a JSON:
https://query1.finance.yahoo.com/v8/finance/chart/MSFT
Code to parse price and panel on the right, and example in the online IDE:
from bs4 import BeautifulSoup
import requests, lxml, json
from itertools import zip_longest
def scrape_google_finance(ticker: str):
# https://docs.python-requests.org/en/master/user/quickstart/#passing-parameters-in-urls
params = {
"hl": "en"
}
# https://docs.python-requests.org/en/master/user/quickstart/#custom-headers
# https://www.whatismybrowser.com/detect/what-is-my-user-agent
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.88 Safari/537.36",
}
html = requests.get(f"https://www.google.com/finance/quote/{ticker}", params=params, headers=headers, timeout=30)
soup = BeautifulSoup(html.text, "lxml")
# describe empty dict where data will be appended
ticker_data = {
"ticker_data": {},
"about_panel": {}
}
ticker_data["ticker_data"]["current_price"] = soup.select_one(".AHmHk .fxKbKc").text
ticker_data["ticker_data"]["quote"] = soup.select_one(".PdOqHc").text.replace(" • ",":")
ticker_data["ticker_data"]["title"] = soup.select_one(".zzDege").text
right_panel_keys = soup.select(".gyFHrc .mfs7Fc")
right_panel_values = soup.select(".gyFHrc .P6K39c")
for key, value in zip_longest(right_panel_keys, right_panel_values):
key_value = key.text.lower().replace(" ", "_")
ticker_data["about_panel"][key_value] = value.text
return ticker_data
data = scrape_google_finance(ticker="GOOGL:NASDAQ")
print(json.dumps(data, indent=2))
JSON output:
{
"ticker_data": {
"current_price": "$2,534.60",
"quote": "GOOGL:NASDAQ",
"title": "Alphabet Inc Class A"
},
"about_panel": {
"previous_close": "$2,597.88",
"day_range": "$2,532.02 - $2,609.59",
"year_range": "$2,193.62 - $3,030.93",
"market_cap": "1.68T USD",
"volume": "1.56M",
"p/e_ratio": "22.59",
"dividend_yield": "-",
"primary_exchange": "NASDAQ",
"ceo": "Sundar Pichai",
"founded": "Oct 2, 2015",
"headquarters": "Mountain View, CaliforniaUnited States",
"website": "abc.xyz",
"employees": "156,500"
}
}
Out of scope of your question. If there's a need to parse the whole Google Finance Ticker page, there's a line-by-line scrape Google Finance Ticker Quote Data in Python blog post about it at SerpApi.
Perhaps of interest, the Google Finance API documentaton includes a section detailing how to access different parameters via JavaScript.
I suppose the JavaScript API might be a wrapper to the JSON request you mention above... perhaps you could check which HTTP requests are being sent.
This is no longer an active API for google, you can try Xignite, although they charge: http://www.xignite.com
The simplest way as you have explained is this link this is for
'Dow Jones Industrial Average'
Link 2 is for 'NASDAQ-100'
and for all related to NASDAQ link 3
I think this should be it, else you want same in JSON notations the same as Microsoft
Please refer this old post I think this will help,
Update:
To know the details of volume and other details,
I have created a vbscript that is using IE object to fetch details from the link, and alerts the content in the particular id(Create a .vbs file and run it..
Set IE = CreateObject("InternetExplorer.Application")
while IE.readyState = 4: WScript.Sleep 10: wend
IE.Navigate "https://www.google.com/finance?q=INDEXNASDAQ%3ANDX&sq=NASDAQ&sp=2&ei=B3UoUsiIH5DIlgPEsQE"
IE.visible = true
while IE.readyState = 4: WScript.Sleep 10: wend
dim ht
ht= IE.document.getElementById("market-data-div").innerText
msgBox ht
IE.quit
this will alert the values from page
like this
3,124.54 0.00 (0.00%)
Sep 4 - Close
INDEXNASDAQ real-time data - Disclaimer
Range -
52 week 2,494.38 - 3,149.24
Open -
Vol. 0.00
I am sure this will help..
Here is an example that you can use. Havent got Google Finance yet, but Here is the Yahoo Example. You will need the HTMLAgilityPack , Which is awesome. Happy Symbol Hunting.
Call the procedure by using YahooStockRequest(string Symbols);
Where Symbols = a comma-delimited string of symbols, or just one symbol
public string YahooStockRequest(string Symbols,bool UseYahoo=true)
{
{
string StockQuoteUrl = string.Empty;
try
{
// Use Yahoo finance service to download stock data from Yahoo
if (UseYahoo)
{
string YahooSymbolString = Symbols.Replace(",","+");
StockQuoteUrl = #"http://finance.yahoo.com/q?s=" + YahooSymbolString + "&ql=1";
}
else
{
//Going to Put Google Finance here when I Figure it out.
}
// Initialize a new WebRequest.
HttpWebRequest webreq = (HttpWebRequest)WebRequest.Create(StockQuoteUrl);
// Get the response from the Internet resource.
HttpWebResponse webresp = (HttpWebResponse)webreq.GetResponse();
// Read the body of the response from the server.
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
string pageSource;
using (StreamReader sr = new StreamReader(webresp.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
doc.LoadHtml(pageSource.ToString());
if (UseYahoo)
{
string Results=string.Empty;
//loop through each Symbol that you provided with a "," delimiter
foreach (string SplitSymbol in Symbols.Split(new char[] { ',' }))
{
Results+=SplitSymbol + " : " + doc.GetElementbyId("yfs_l10_" + SplitSymbol).InnerText + Environment.NewLine;
}
return (Results);
}
else
{
return (doc.GetElementbyId("ref_14135_l").InnerText);
}
}
catch (WebException Webex)
{
return("SYSTEM ERROR DOWNLOADING SYMBOL: " + Webex.ToString());
}
}
}
Building upon the shoulders of giants...here's a one-liner I wrote to zap all of Google's current stock data into local Bash shell variables:
stock=$1
# Fetch from Google Finance API, put into local variables
eval $(curl -s "http://www.google.com/ig/api?stock=$stock"|sed 's/</\n</g' |sed '/data=/!d; s/ data=/=/g; s/\/>/; /g; s/</GF_/g' |tee /tmp/stockprice.tmp.log)
echo "$stock,$(date +%Y-%m-%d),$GF_open,$GF_high,$GF_low,$GF_last,$GF_volume"
Then you will have variables like $GF_last $GF_open $GF_volume etc. readily available. Run env or see inside /tmp/stockprice.tmp.log
http://www.google.com/ig/api?stock=TVIX&output=csv by itself returns:
<?xml version="1.0"?>
<xml_api_reply version="1">
<finance module_id="0" tab_id="0" mobile_row="0" mobile_zipped="1" row="0" section="0" >
<symbol data="TVIX"/>
<pretty_symbol data="TVIX"/>
<symbol_lookup_url data="/finance?client=ig&q=TVIX"/>
<company data="VelocityShares Daily 2x VIX Short Term ETN"/>
<exchange data="AMEX"/>
<exchange_timezone data="ET"/>
<exchange_utc_offset data="+05:00"/>
<exchange_closing data="960"/>
<divisor data="2"/>
<currency data="USD"/>
<last data="57.45"/>
<high data="59.70"/>
<low data="56.85"/>
etc.
So for stock="FBM" /tmp/stockprice.tmp.log (and your environment) will contain:
GF_symbol="FBM";
GF_pretty_symbol="FBM";
GF_symbol_lookup_url="/finance?client=ig&q=FBM";
GF_company="Focus Morningstar Basic Materials Index ETF";
GF_exchange="NYSEARCA";
GF_exchange_timezone="";
GF_exchange_utc_offset="";
GF_exchange_closing="";
GF_divisor="2";
GF_currency="USD";
GF_last="22.82";
GF_high="22.82";
GF_low="22.82";
GF_volume="100";
GF_avg_volume="";
GF_market_cap="4.56";
GF_open="22.82";
GF_y_close="22.80";
GF_change="+0.02";
GF_perc_change="0.09";
GF_delay="0";
GF_trade_timestamp="8 hours ago";
GF_trade_date_utc="20120228";
GF_trade_time_utc="184541";
GF_current_date_utc="20120229";
GF_current_time_utc="033534";
GF_symbol_url="/finance?client=ig&q=FBM";
GF_chart_url="/finance/chart?q=NYSEARCA:FBM&tlf=12";
GF_disclaimer_url="/help/stock_disclaimer.html";
GF_ecn_url="";
GF_isld_last="";
GF_isld_trade_date_utc="";
GF_isld_trade_time_utc="";
GF_brut_last="";
GF_brut_trade_date_utc="";
GF_brut_trade_time_utc="";
GF_daylight_savings="false";
The Google stock quote API has gone away. However, Investor's Exchange offers an API that's very easy to use for quote data.
I have personally built an app for stock data and fundamentals with Intrinio Two years ago but abandoned the project because I was beaten to market by a competitor.
I built it in Java but they support multiple stacks. Back then, You could access their api for free for testing purposes, but I think they build packages based on your needs now.
In any case, they were exceptionally helpful and charge low fees from what I remember, and their library is well documented so pulling data in json is very straightforward.
In order to find chart data using the financial data API of Google, one must simply go to Google as if looking for a search term, type finance into the search engine, and a link to Google finance will appear. Once at the Google finance search engine, type the ticker name into the financial data API engine and the result will be displayed. However, it should be noted that all Google finance charts are delayed by 15 minutes, and at most can be used for a better understanding of the ticker's past history, rather than current price.
A solution to the delayed chart information is to obtain a real-time financial data API. An example of one would be the barchartondemand interface that has real-time quote information, along with other detailed features that make it simpler to find the exact chart you're looking for. With fully customizable features, and specific programming tools for the precise trading information you need, barchartondemand's tools outdo Google finance by a wide margin.
Try with this:
http://finance.google.com/finance/info?client=ig&q=NASDAQ:GOOGL
It will return you all available details about the mentioned stock.
e.g. out put would look like below:
// [ {
"id": "694653"
,"t" : "GOOGL"
,"e" : "NASDAQ"
,"l" : "528.08"
,"l_fix" : "528.08"
,"l_cur" : "528.08"
,"s": "0"
,"ltt":"4:00PM EST"
,"lt" : "Dec 5, 4:00PM EST"
,"lt_dts" : "2014-12-05T16:00:14Z"
,"c" : "-14.50"
,"c_fix" : "-14.50"
,"cp" : "-2.67"
,"cp_fix" : "-2.67"
,"ccol" : "chr"
,"pcls_fix" : "542.58"
}
]
You can have your company stock symbol at the end of this URL to get its details:
http://finance.google.com/finance/info?client=ig&q=<YOUR COMPANY STOCK SYMBOL>