I'm trying to retrieve this page using Apache HttpClient: http://quick-dish.tablespoon.com/
Unfortunately, when I try to do this, it just returns the following (as returned by JSoup, so probably it's really just returning the HTTP... string itself):
<html>
<head></head>
<body>
HTTP/1.1 200 OK [Server: nginx/1.0.11, Content-Type: text/html;charset=UTF-8, Last-Modified: Mon, 02 Jul 2012 15:30:40 GMT, Vary: Accept-Encoding, Cookie,Accept-Encoding, X-Powered-By: PHP/5.3.6, X-Pingback: http://quick-dish.tablespoon.com/xmlrpc.php, X-Powered-By: ASP.NET, Content-Encoding: gzip, X-Blz: lb1.blaze.io, Date: Mon, 02 Jul 2012 16:06:21 GMT, Content-Length: 11723, Connection: keep-alive]
</body>
</html>
Here is my code (note that I'm emulating the Google Bot as I've found that web servers tend to be better behaved that way):
URL sourceURL = new URL("http://quick-dish.tablespoon.com/");
HttpClient httpClient = new ContentEncodingHttpClient();
httpClient.getParams().setBooleanParameter("http.protocol.handle-redirects", true);
final HttpGet httpget = new HttpGet(sourceURL.toURI());
httpget.setHeader("User-Agent", "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)");
httpget.setHeader("Accept", "text/html");
httpget.setHeader("Accept-Charset", "utf-8");
final HttpResponse response = httpClient.execute(httpget);
return Jsoup.parse(response.toString());
Needless to say, the page returns fine in my web browser. Any ideas?
Instead of toString you need to get the response entity
// Get hold of the response entity
HttpEntity entity = response.getEntity();
Then you can get the contents of that
HttpEntity entity = response.getEntity();
String pageHTML = EntityUtils.toString(entity);
Jsoup.parse(response.toString());
Related
I am working with Microsoft Cognitive Services and submitting some TTS over the Rest API. If I submit the xml listed below directly to the service via postman, it works fine, if I submit the xml below via my java code I get:
HttpResponseProxy{HTTP/1.1 400 Synthesis failed. StatusCode: FailedPrecondition, Details: SSML parsing error: 8004507A. [Server: openresty/1.15.8.2, Date: Mon, 07 Jun 2021 20:48:40 GMT, Content-Type: text/xml, Transfer-Encoding: chunked, Connection: keep-alive, Strict-Transport-Security: max-age=15724800; includeSubDomains] ResponseEntityProxy{[Content-Type: text/xml,Chunked: true]}}
The java code works flawlessly if I do not inject the SSML
<phoneme alphabet="ipa" ph="təˈmeɪtoʊ"> tomato </phoneme>
The only thing I can think of going wrong is the Entity for the body. With postman this is a raw body. Is there something else I should be doing for the body in java?
HttpPost httpPost = new HttpPost("https://eastus.tts.speech.microsoft.com/cognitiveservices/v1");
httpPost.setEntity(new StringEntity(xml));
httpPost.addHeader("Content-Type", "application/ssml+xml");
httpPost.addHeader("Ocp-Apim-Subscription-Key", key);
httpPost.addHeader("X-Microsoft-OutputFormat", "audio-48khz-192kbitrate-mono-mp3");
org.apache.http.HttpResponse resp = httpclient.execute(httpPost);
<speak xmlns="http://www.w3.org/2001/10/synthesis" xmlns:mstts="http://www.w3.org/2001/mstts" xmlns:emo="http://www.w3.org/2009/10/emotionml" version="1.0" xml:lang="en-US"><voice name="en-US-JennyNeural"><mstts:express-as style="assistant"><prosody rate="5%" pitch="13%">
we can totally get a <phoneme alphabet="ipa" ph="təˈmeɪtoʊ"> tomato </phoneme> made for you right now!</prosody></mstts:express-as></voice></speak>
i have a implementation for oauth 2.0 in Java, i wanted to try out some things with the Shopware 6 API. I can get the access token without problems and, as far as i see, i'm doing everything right to request a ressource with this access_token. In the header for the GET Request i put the 'Authorization Bearer' + access_token header and also the "Content-Type", "application/json" header.
HttpGet get = new HttpGet(resourceURL);
get.addHeader("Content-Type", "application/json");
and later
if (isValid(accessToken)) {
// update the access token
// System.out.println("New access token: " + accessToken);
oauthDetails.setAccessToken(accessToken);
// remove the old auth header
get.removeHeaders(OAuthConstants.AUTHORIZATION);
// add the new auth header
get.addHeader(OAuthConstants.AUTHORIZATION,
getAuthorizationHeaderForAccessToken(oauthDetails.getAccessToken()));
get.releaseConnection();
response = client.execute(get);
code = response.getStatusLine().getStatusCode();
The Error Code i always get is 415.
This is the complete response:
HttpResponseProxy{HTTP/1.1 415 Unsupported Media Type [Date: Thu, 04 Jul 2019 08:45:38 GMT, Server: Apache/2.4.25 (Debian), Cache-Control: no-cache, private, Access-Control-Allow-Origin: *, Access-Control-Allow-Methods: GET,POST,PUT,PATCH,DELETE, Access-Control-Allow-Headers: Content-Type,Authorization,sw-context-token,sw-access-key,sw-language-id,sw-version-id, sw-version-id: , sw-language-id: , sw-context-token: , x-frame-options: deny, X-Debug-Token: c1766c, X-Debug-Token-Link: http://localhost:8000/_profiler/c1766c, X-Robots-Tag: noindex, Vary: Authorization, Keep-Alive: timeout=5, max=100, Connection: Keep-Alive, Transfer-Encoding: chunked, Content-Type: application/json] ResponseEntityProxy{[Content-Type: application/json,Chunked: true]}}
The endpoint im trying to get is the "http://localhost:8000/api/v1/category/" endpoint. If im doing this whole thing with Insomnia/Postman i get the expected category information.
Does anyone have any idea what could be wrong? What am i missing here?
Please add following header
'Accept': 'application/json'
As nuriselcuk pointed out in the comment, the missing thing was the Accept header.
I added
post.addHeader("Accept", "application/json");
and now its working fine.
I'm trying http requests and create http get request to server. Server must return .zip file. here is the code:
url = new URL(urlToRead);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
is = conn.getInputStream();
then I want to write it to file like this:
r = 1;
while(r > 0){
r = is.read(buf);
if(r > 0)
fos.write(buf, 0, r);
}
but to create fileoutputstream i want to use the file name provided by server. I've found that server answer has the file name and all the structure looks like this:
HTTP/1.1 200 OK
Date: Mon, 07 Apr 2003 14:51:19 GMT
Server: Apache/1.3.20 (Win32) PHP/4.3.0
Last-Modified: Mon, 07 Apr 2003 14:51:00 GMT
Accept-Ranges: bytes
Content-Length: 673
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: application/zip
Content-Disposition: attachment; filename=test.zip
Pragma: no-cache
....(zip content)
how to get filename?
You can use the HttpUrlConnection object's getHeaderField() method to read the Content-Disposition header. Also make sure you handle the case when it is not present.
Have you had a look at the methods that HttpURLConnection offers?
Especially the ones that involve headers?
if u have servletRequest object(lets say obj) then
i think this will help
obj.getRequestURL() ;
I'm trying to login to a grade database with HttpClient. I send it valid LogOnDetails.Username and LogOnDetails.Password information, but whenever I submit it consistently sends back errors. I'm not sure where it's happening, it may log in successfully and then hang up, or it may not even be making it in. here is the html it outputs after the request:
output
and this is my code:
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("https://home-access.cfisd.net/HomeAccess/Account/LogOn");
List<NameValuePair> list = new ArrayList<NameValuePair>();
list.add(new BasicNameValuePair("LogOnDetails.Username", "s491670"));
list.add(new BasicNameValuePair("LogOnDetails.Password","qrrp4ji6t"));
post.setEntity(new UrlEncodedFormEntity(list));
HttpResponse response = client.execute(post);
BufferedReader file = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
PrintWriter pw = new PrintWriter(new FileWriter(new File("output.txt")));
String line = null;
while((line = file.readLine())!=null)
pw.println(line);
pw.close();
if anyone could shed some light on this, I'd love them forever. if HTML for the login form is needed let me know. thanks!
When the form submits the following form data is sent:
Form Data
Database:10
LogOnDetails.UserName:sadf
LogOnDetails.Password:sdf
Add the following:
list.add(new BasicNameValuePair("Database", "10")); // or 20
As you are well aware I can't test this.
If you add #robbmj's patch, you do not get the 500 error page, but a 302 (moved), which is a good start:
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
You can advise the http-client to handle redirects by itself, but depending on the version of http-client, handling redirects is done differently (they're always refactoring this code). Which version are you using?
Maybe the 302 means that it was all successful and I'm logged in now. ;)
Headers returned:
Cache-Control : private
Content-Type : text/html; charset=utf-8
Location : /HomeAccess/
Server : Microsoft-IIS/7.5
X-AspNetMvc-Version : 4.0
X-AspNet-Version : 4.0.30319
Set-Cookie : ASP.NET_SessionId=hvjw3jqjoaa5ohofaaxu4od1; path=/; HttpOnly
Set-Cookie : .AuthCookie=; expires=Tue, 12-Oct-1999 05:00:00 GMT; path=/; HttpOnly
Set-Cookie : .AuthCookie=0863B972684CC784E4D9D5594354B6F08FF6FF7225836F01A9715D0ABA633042946B032987F7926588610F5FB7C18757CE759338B75E341DF56DB3FB71BC326B3D6E49EA94EEE43B39FCC84BB98F236CA0D63CE668E14434169C6B835FA671DD; path=/; HttpOnly
X-Powered-By : ASP.NET
Date : Thu, 12 Mar 2015 23:37:12 GMT
Content-Length : 129
I want to know how can you know if you are sending correctly a Json message by reading the response.
I'm actually receiving this message:
HttpResponseProxy{HTTP/1.0 201 Created [Server: nginx, Date: Thu, 11
Sep 2014 21:01:52 GMT, Content-Type: application/json; charset=utf-8,
X-Frame-Options: SAMEORIGIN, X-XSS-Protection: 1; mode=block,
X-Content-Type-Options: nosniff, Location:
http://newduoc.efizity.com/api/telemetry_consumptions/16, ETag:
"cf80e1320a6e11352acb0a9dc17d9f75", Cache-Control: max-age=0, private,
must-revalidate, X-Request-Id: 341e5904-bb0b-4451-b0fb-f45690c0a43c,
X-Runtime: 0.010819, X-Cache: MISS from router.dhemax.cl,
X-Cache-Lookup: MISS from router.dhemax.cl:3128, Via: 1.0
router.dhemax.cl (squid/3.1.10), Connection: close]}
The method that I'm using to send the post json message is this one:
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
HttpPost request = new HttpPost(Config.urlJSON);
StringEntity params = new StringEntity(json.toString());
request.addHeader("content-type", "application/json");
request.setEntity(params);
Log.debug(httpClient.execute(request).toString());
httpClient.close();
Basically I'm just doing a .toString() to the CloseableHttpResponse object returned by the CloseableHttpClient object.
I'm using java-json to send the Json message and it looks like the server is ngix.
Thank you.
I have already found the way of getting the server response and understand it.
I don't know if this is a standard but when you use the method getStatusCode you get the code:
httpClient.execute(request).getStatusLine().getStatusCode();
At least in this case the server send this codes. I repeat, I'm not sure if this is for all the servers but at least this one goes for me I think it is a nginx server:
422: Incorrect data.
400: Json structure invalid.
401: Access not allowed.
201: Correct.
So basically the code goes like this:
HttpPost request = new HttpPost(Config.urlJSON);
StringEntity params = new StringEntity(json.toString());
request.addHeader("content-type", "application/json");
request.setEntity(params);
int statusCode = httpClient.execute(request).getStatusLine().getStatusCode();
Log.debug("[STATUS:[" + String.valueOf(statusCode) + "]");
All the magic goes by the CloseableHttpClient obj.getStatusLine().getStatusCode();