How to get mobile response of webpage using jsoup - java

I am trying to get response of website using JSoup.
I am able to get the response of website using JSoup as follows, it returns the desktop website's response
String str = "http://hplus.com.vn/xem-tivi-htv9-full-hd-1080-52834.html";
doc = Jsoup.connect(str)
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36")
.get();
Same way, I am trying to get the response for mobile version to this same site as follows,
doc = Jsoup.connect("http://hplus.com.vn/xem-tivi-htv9-full-hd-1080-52834.html")
.userAgent("Mozilla/5.0(Linux; U; Android 4.2; en-gb; LG-P500 Build/FRF91) AppleWebKit/533.0 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1")
.get();
But this gives only desktop/laptop version response and not the mobile response.
Picture Format web :
Picture Format mobile :
How to get the mobile response from jsoup.
Thanks in advance.

Related

Java, Get response via socket

I know that I had to search on Google, Youtube and Stackoverflow before I ask my question on Stackoverflow .
But I swear I watched many videos on Youtube, And I searched for many ways on google and Stackoverflow ! to solve my problem. ,
I can't get the full response,
And now i'm stuck with this,

Facebook checks for browser:
Add agent string to request:
String content1 = "GET /zuck HTTP/1.1\r\nHost: www.facebook.com\r\nuser-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36\r\n\r\n";

How do I access WhatsAppWeb in an Android WebView?

I want to use WhatsAppWeb (web.whatsapp.com) in an Android app. I've tried using a WebView and load the URL but I'm redirected to the regular WhatsApp page (www.whatsapp.com). I tried changing the User Agent
mWebView.getSettings().SetUserAgentString("Mozilla / 5.0 (Windows NT 10.0; Win64; x64) AppleWebKit / 537.36 (KHTML, like Gecko) Chrome / 78.0.3945.79 Safari / 537.36");
but this only works if the app is started the first time. Each further start it redirects. Deleting Cache works (no redirect) but then I have to "authenticate" the device again.
Any ideas?

Did you try using custom webview client? Please check this http://www.technotalkative.com/android-webviewclient-example/

Perform a search using JSoup

Since the Soundcloud Java API is discontinued, I want to perform a search on their site using JSoup. I am currently using this code:
Document doc = Jsoup
.connect("https://soundcloud.com/search?q=deep%20house")
.userAgent("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36")
.timeout(5000).get();
But the webpage is giving me a message that I should be using a newer browser:
<p class="messageText sc-text-light">Your current browser isn't compatible with SoundCloud.<br>Please download one of our supported browsers. Need help?</p>
I have tried using other user agents which I found here but none seems to work so far. What can I do to prevent this message from popping up?

You can try the below snippet. User agent string taken from this thread.
Document doc = Jsoup
.connect("https://soundcloud.com/search?q=deep%20house")
.userAgent("Mozilla/5.0 (Linux; Android 4.1.1; Nexus 7 Build/JRO03D) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Safari/535.19")
.timeout(5000).get();
System.out.println(doc);

Jsoup exception: HTTP error fetching URL. Status=-1

I'm recently struggling with this error:
org.jsoup.HttpStatusException: HTTP error fetching URL. Status=-1, URL=http://ekstraklasa.net/wisla-krakow/
Everything works just fine when running from Windows 8 machine, but when uploaded on Ubuntu server it just throws this exception.
I've tried to escape dashes in URL, print response body (there's none, only java.io.IOException: Invalid Http response) and so on.
It's not a problem with timeout setting.
I'm using Jsoup and simply ran out of ideas.
My code:
org.jsoup.Connection con = Jsoup.connect("http://ekstraklasa.net/wisla-krakow/")
.userAgent("Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36")
.timeout(1 * 60 * 1000);
Document doc = con.get();

Guessing what is the client browser

I got the following information using request.getHeader("User-Agent") method inside a Servlet:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10
Actually what is the client browser?

It's Chrome 8.0.552.
This website may be useful for future consultations: http://user-agent-string.info. Paste the UA string there and click Analyze. They have even a XML-RPC webservice.

Categories

Resources