I'm trying to get some financial data from yahoo finance using java / spring RestTemplate.
Suppose I need the data for IBM, the link would be something like this:
https://query1.finance.yahoo.com/v10/finance/quoteSummary/IBM?formatted=true&modules=incomeStatementHistory%2CcashflowStatementHistory%2CbalanceSheetHistory%2CincomeStatementHistoryQuarterly%2CcashflowStatementHistoryQuarterly%2CbalanceSheetHistoryQuarterly%2Cearnings
which is a nice formatted JSON with all the data I need:
{
"quoteSummary": {
"result": [
{
"cashflowStatementHistoryQuarterly": {
"cashflowStatements": [
{
"maxAge": 1,
"endDate": {
"raw": 1459382400,
"fmt": "2016-03-31"
},
"netIncome": {
"raw": -5490000,
"fmt": "-5.49M",
"longFmt": "-5,490,000"
},
"depreciation": {
"raw": 16849000,
"fmt": "16.85M", ... etc
But when I try to open the same link using RestTemplate:
using a code like:
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<String>response = restTemplate.getForEntity(url, String.class);
The result I'm getting back in response is very different, it looks like this:
<200 OK,{"quoteSummary":{"result":[{}],"error":null}},{X-YQL-Host=[pprd3-node7511-lh2.manhattan.ir2.yahoo.com], X-Content-Type-Options=[nosniff], X-Yahoo-Request-Id=[6s0s3nlbqhlls], Cache-Control=[max-age=1, stale-while-revalidate=15], Content-Type=[application/json; charset=utf-8], Vary=[Origin], Content-Length=[45], Date=[Mon, 08 Aug 2016 19:00:12 GMT], Age=[2], Server=[ATS], Via=[http/1.1 media-ncache-api14.prod.media.ir2.yahoo.com (ApacheTrafficServer [cMsSfW]), http/1.1 media-ncache-api15.prod.media.ir2.yahoo.com (ApacheTrafficServer [cMsSf ]), http/1.1 media-router-api3.prod.media.ir2.yahoo.com (ApacheTrafficServer [cMsSfW]), https/1.1 r07.ycpi.loa.yahoo.net (ApacheTrafficServer [cMsSfW])], Strict-Transport-Security=[max-age=0], Connection=[keep-alive], Y-Trace=[BAEAQAAAAACPmeCQTRIz3QAAAAAAAAAAOEHjlsK9d9kAAAAAAAAAAAAFOZQGmKa_AAU5lAap7rIUoHdnAAAAAA--]}>
Any idea how can I get the same result I'm getting in the browser?
Thank you
Related
im using karate framework to test some site, but one call to the endpoints is like this: https://urlbase/index?id=60&filters[start_date]=Fri%20Nov%2018%202022%2010:14:59%20GMT-0300&filters[end_date]=Sat%20Nov%2019%202022%2023:59:59%20GMT-0300
And i have problems with the filters[...] i don't know how to do that :(
I have this query def:
* def query = {id: 60, filters[start_date]:'Fri%20Nov%2018%202022%2010:14:59%20GMT-0300', filters[end_date]:'Sat%20Nov%2019%202022%2023:59:59%20GMT-0300'}
But the request url shows: GET https://urlbase/index?id=60&filters=%7Bstart_date%3D%2C+end_date%3D%7D
`
with no brackets.
And i want my GET url looks like this: https://urlbase/index?id=60&filters[start_date]=Fri%20Nov%2018%202022%2010:14:59%20GMT-0300&filters[end_date]=Sat%20Nov%2019%202022%2023:59:59%20GMT-0300
I tried this and it seems to work fine:
* url 'https://httpbin.org/anything'
* param id = 60
* param filters[start_date] = 'Fri Nov 18 2022 10:14:59 GMT-0300'
* param filters[end_date] = 'Sat Nov 19 2022 23:59:59 GMT-0300'
* method get
You can look at the response and confirm the server handles the request correctly:
{
"args": {
"filters[end_date]": "Sat Nov 19 2022 23:59:59 GMT-0300",
"filters[start_date]": "Fri Nov 18 2022 10:14:59 GMT-0300",
"id": "60"
},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept-Encoding": "gzip,deflate",
"Host": "httpbin.org",
"User-Agent": "Apache-HttpClient/4.5.13 (Java/17.0.4.1)",
"X-Amzn-Trace-Id": "Root=1-6377b983-6bfc4696374f7d08239de823"
},
"json": null,
"method": "GET",
"origin": "X.X.X.X",
"url": "https://httpbin.org/anything?filters[start_date]=Fri+Nov+18+2022+10%3A14%3A59+GMT-0300&filters[end_date]=Sat+Nov+19+2022+23%3A59%3A59+GMT-0300&id=60"
}
I am trying to use the bitbucket server rest API to fetch file content(.json file) from a bitbucket repository in a Java spring boot Application but the response is not as expected.
Content in the file meta.json
{
"key1": "value1",
"key2": "value2"
}
API used
https://bitbucket.domain.com/rest/api/1.0/projects/my-project/repos/my-repo/browse/path/to/meta.json
Java snippet used to get file content
String url = "https://bitbucket.domain.com/rest/api/1.0/projects/my-project/repos/my-repo/browse/path/to/meta.json";
JsonNode bucketData = restTemplate.exchange(url, HttpMethod.GET, entity, JsonNode.class).getBody();
Actual output:
{
"lines": [
{
"text": "{"
},
{
"text": " \"key1\": \"value1\","
},
{
"text": " \"key2\": \"value2\""
},
{
"text": "}"
}
],
"start": 0,
"size": 4,
"isLastPage": true
}
Whereas the output I want is the following
{
"key1": "value1",
"key2": "value2"
}
So I can convert it to JsonNode and use it.
Observation
While hitting the repo url with ?raw param (https://bitbucket.domain.com/projects/my-project/repos/my-repo/browse/path/to/meta.json?raw) I was getting the excepted output in text format, not json format. I also tried using the same url in my spring boot application but that didnt work either.
Any suggestion on how can I get the expected output?
I was able fix this issue, by using Unirest
The following syntax was used
String url = "https://bitbucket.domain.com/rest/api/1.0/projects/my-project/repos/my-repo/browse/path/to/meta.json";
HttpResponse<T> response = Unirest.get(url).headers(headers).asObject(responseClass);
JsonNode bucketData = response.getBody();
I have the following example URI:
localhost/users
and this JSON coming as response when I send a request to it:
[
{
"id": 1,
"name": "Joe"
,
{
"id": 3,
"name": "Ben"
,
{
"id": 4,
"name": "Jim"
}
}
]
How do I verify with RestAssured, for example that there are no user with id = 2?
Would it be something along the lines of this semi-pseudo:
given().spec(requestSpecification)
.when().get("/users")
.and().body("id==2", ????);
Any advice/guidance will be appreciated, thank you.
try this:
given().spec(requestSpecification)
.when().get("/users")
.and().body("id", is(not(equalTo(2))));
Usage rest-assured documentation section Use the response to verify other parts of the response provides more detail.
is(not(Matcher)) is provided by hamcrest
How can I use Solrj to query Solr using the following api:
http://localhost:8983/solr/admin/zookeeper?detail=true&path=%2Fconfigs%2Fmy-search%2Fdataimport.properties
the above api gives the content of the dataimport.properties file.
{
"znode": {
"path": "/configs/my-search/dataimport.properties",
"prop": {
"version": 186,
"aversion": 0,
"children_count": 0,
"ctime": "Sun Oct 16 10:24:04 UTC 2016 (1476613444895)",
"cversion": 0,
"czxid": 479,
"ephemeralOwner": 0,
"mtime": "Fri Mar 24 09:48:50 UTC 2017 (1490348930211)",
"mzxid": 31451,
"pzxid": 479,
"dataLength": 111
},
"data": "#Fri Mar 24 09:48:50 UTC 2017\nname.last_index_time=2017-03-24 09\\:48\\:49\nlast_index_time=2017-03-24 09\\:48\\:49\n"
},
"tree": [
{
"data": {
"title": "dataimport.properties",
"attr": {
"href": "admin/zookeeper?detail=true&path=%2Fconfigs%2Fmy-search%2Fdataimport.properties"
}
}
}
]
}
btw, my Solr is configured in cloud mode.
Given that your Solr is running in Cloud mode the file /configs/my-search/dataimport.properties is into Zookeeper.
SolrJ does not have native API to easily read from Zookeeper.
To read into the Zookeeper I suggest to use use Apache Curator Framework
String dataPath = "/configs/my-search/dataimport.properties";
String connectionString = "zookeeper-ensemble:2181";
CuratorFramework client = CuratorFrameworkFactory.newClient(connectionString, new ExponentialBackoffRetry(1000, 3));
client.start();
byte[] barray = client.getData().forPath(dataPath);
if (barray != null) {
String data = new String(barray);
System.out.println(data);
}
I have some problems concerning charater encoding with gson.
I have a webservice which sends following JSON response:
{
"version": "v2",
"result": {
"class": "SearchSizeBean",
"result_list": [
{
"id": 0,
"name": "Bitte w\u00c3\u00a4hlen"
},
{
"id": "21",
"name": "176"
}
]
}
}
I request the webservice with following code:
ClientResponse response = webResource.path(version)
.path(path)
.accept(MediaType.APPLICATION_JSON)
.method(method.name(), ClientResponse.class, paramMap);
if (response.getStatus() == 200)
{
responseBody = response.getEntity(String.class);
...
}
When checking responseBody the resulting string shows exactly the same result as the JSON mentioned above. But after using GSon to convert the JSON string into an Object, the characters are converted wrong:
Parsing with GSON:
Gson gson = new GsonBuilder().disableHtmlEscaping()
.setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES)
.registerTypeAdapter(Transferable.class, new TransferableDeserializer())
.setDateFormat("yyyy-MM-dd HH:mm:ss")
.create();
RestResponse responseDTO = gson.fromJson(responseBody, RestResponse.class);
Resulting content:
Response: SearchSizeBean[
resultList={
PosSize [id=0, name=Bitte wählen],
PosSize [id=21, name=176]
}
]
UPDATE:
I had copied the wrong JSON content. I have edited the JSON to show the correct JSON string. The data is sent in unicode w\u00c3\u00a4. When I check the response after conversion by GSon it becomes hexadecimal: C3 82 C2 A4
The problem was on the server side php code. I have a script, which was already encoded in UTF-8. I have encoded the string "Bitte wählen" with the php function utf8_encode() which led to my problem.
Thank you McDowell, your hints brought me on the right track!
The solution was not to use the function and just send the string "Bitte wählen".