I'm using the UserAgentUtils Java library to extract user agent details from the user agent string of browsers during a PDI transform, but no matter what I do I always get back a null version from the library after parsing the user agent string, even when I can clearly see the version in the string. For example:
String userAgentString = "Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US) AppleWebKit/533.3 (KHTML, like Gecko) capybara-webkit Safari/533.3"
UserAgent userAgent = new UserAgent(userAgentString)
userAgent.getBrowserVersion() //always comes back null
Two questions. What am I not doing right to get back the data from UserAgentUtils (it doesn't seem to be a bug because there's no history of issues related to this in their bug tracking system)?
Alternatively, is there another Java or JavaScript library I could use to extract the component information from user agent strings? Either one is okay, since I can equally easily use either in the PDI job where this code lives.
are you trying to set the http agent value for jetty http client requests?
i do this on my user defined java class:
import java.lang.System.*;
...
System.setProperty("http.agent", "my cool crawler, mycoolcrawler#example.com");
now all your http requests from kettle will send user agent header with this info
Related
I am trying to download an image from Java code. My code is already working fine for tons of other images, but this one refuses to download.
I'm sure the image exists and I am able to view it inside the browser: http://lemonde-emploi.blog.lemonde.fr/files/2017/02/La-Ru%C3%A9e-des-licornes-Hazard.jpg
I'm using Play framework WS Scala client to download the image. It's just a wrapper around Java famous AsyncHttpClient with a Netty implementation.
I'm running the following code, which work fine for many other images, but fails just for this one:
WS
.url(url)
.withQueryString(queryString: _*)
.withHeaders("User-Agent" -> "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36")
.get
I've set exactly the same User-Agent as my local browser which succeed to access the image.
Here's the server response in debug: it returns a 400 status code.
Any idea why it happens?
The legacy "com.ning" package in your image shows that you're still running AHC1, that has reached end-of-life and I no longer maintain.
You're most likely hitting an old AHC bug related to URL encoding that's been long fixed (I checked that modern releases of AHC2 work fine with your URL).
Basically, it's time to consider upgrading Play/AHC versions.
I am setting user-agent to test a iOS app from my Java client this way -
urlc.setRequestProperty("user-agent", "Mozilla/5.0 (Windows NT 5.1; rv:19.0) Gecko/20100101 Firefox/19.0");
However, in the JSON response I am getting an error that this app can only be tested on an iOS device (which is a custom response I have when the app is tested from a non-IOS device. So what is the correct way to set user-agent in Java?
A browser sends a special string, called a user agent, to websites to identify itself. The web server, or JavaScript in the downloaded webpage, detects the client’s identity and can modify its behavior accordingly. In the simplest case, the user agent string includes an application name—for example, Navigator as the application name and 6.0 as the version. Safari on the desktop and Safari on iOS have their own user agent strings, too.
https://developer.apple.com/library/ios/documentation/AppleApplications/Reference/SafariWebContent/OptimizingforSafarioniPhone/OptimizingforSafarioniPhone.html
UserAgent complete set
UserAgent complete set ssfari
I'm aware I can call out to Active Directory and do queries provided I have a cleartext username and password. (I don't want to do that)
In VB, I can set authorisation levels by NT group - and the user doesn't have to enter their password nor, store it in a text file. (My understanding is that this has access to the Windows AD ticket).
I also know I can shell out to the command line and parse the output - to get the users groups - this is problematic.
How can I replicate getting the executing user's NT groups without a password in Java?
(It is beginning to sound like I'll have to call the Win32 API with JNA to get the kerberos ticket - I'm hoping there is a simpler way.)
You should split up your question in two because you're mixing authentication with authorization. Kerberos works very pleasently with Java on Windows with some caveats due to MS. Having said that use the Kerberos ticket with the provided principal to authenticate against AD and retrieve the user's memberOf values to see in which groups the user is in.
I'm writing an HTTP client, which needs to parse the response from a webserver, and I have run into (another) problem.
I found that for one page I was redirected to their mobile content portal:
example: www.example.com/m/public. This is not what I want.
When using a "normal" browser, this redirect did not take place.
After looking into the capture I made, I found that this could be because my user-agent is interpreted as that of a mobile handset browser (user agent was "Java/1.6.0_22").
So I changed the user agent, using this:
URL url = new URL(endpoint);
URLConnection conn = url.openConnection();
conn.setRequestProperty ( "User-agent", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 3.5.30729; InfoPath.1; .NET CLR 3.0.30618)");
To my surpise it still did not work, and I found that I was still sending user-agent "Java/1.6.0_22".
Then I looked a bit closer at my capture, and I saw that after a couple of GET requests (after the first GET I send GETs to the sources on the main page) the user-agent magically changed from java to "Mozilla...".
It seems my setRequestProperty does not become active until after a while...
Has anyone seen this? Any way to get around it?
Thanks!
This SO answer suggests setting the system property before-hand.
I had the same problem. I wrote a web crawler and web pages grabbed were mobile versions. Now I used both
System.setProperty("http.agent", "");
urlconn.setRequestProperty("User-Agent", "IE/9.0");
and it worked.
I'm following Scott Davis' tutorials on developing grails apps, but whenever i try to run my app (or indeed his source code) i get "Firefox has detected that the server is redirecting the request for this address in a way that will never complete." Safari gives a similar error message as does Opera.
As i've tested the original authors source code which gives the same error i'm fairly confident it's nothing to do with the code.
Is this a problem with the web server on my machine? I use Mac OS Snow Leopard so i'm assuming it's apache that's generating this error.
Edit: Seems Grails as standard uses Jetty, so probably not Apache that is causing the problem. However also tested the app on Glassfish and i get the same error.
Anyone know what i can do to fix this?
Cheers
It depends on the code and Apache configuration you are using. I assume that the web server sends cyclic HTTP redirections, eg. from /root/ to /root (without the slash) and vice versa. This causes a redirection infinite loop.
Check your configuration on conditions that cause a HTTP redirect. For example, Apache automatically adds slashes to directory URLs in standard configuration (like the /root/ example above). I don't know Grails, so I cannot give you a hint on how URLs are processed within the app.
You can also use manual HTTP requests for debugging to see whats going on behind the scenes, using telnet on a terminal:
$ telnet localhost 80
GET / HTTP/1.0
I guess the response will be something like that:
HTTP/1.0 302 Found
Location: XXX
...
Now do a second request on the URL passed in the Location header and so on.
I was getting the same error a little while ago, heres how I fixed:
Try the same page on a different internet setup (it could be your ISP)
Open up Safari, Firefox or whatever your using and empty the cache and delete ALL your cookies
Reboot your computer and try again
It may work now, but if it doesn't:
open up Firefox and type 'about:config' (without the quotes) into the URL bar
You will get some little warning, just press OK
Type 'redirect' into the Filter box
You should see a listing for 'network.http.redirection-limit'
Double click the listing and type a large number (anything above 50 and lower than 200)
Press OK, quit and re-open FireFox
Basically all that does is make FireFox's tolerance for redirect loops higher which should fix your problem - but usually, just borrowing someone else's internet connection fixes it
Hope that all helps =)
Just carefully check your URLMappings configuration:
YOUR_APP/grails-app/conf/UrlMappings.groovy
Common case:
You configured request to be handled like this:
"/anything" (controller:"someController")
So without action, request will be handled by default one, "index". "index" action usually redirects to "list", and "list", in some cases redirect back to "index"
There is your loop.
Good luck