Need timeout setting for remote data request made using java.net.URL class. After some googling found out that there are two system properties which can be used to set timeout for URL class as follows.
sun.net.client.defaultConnectTimeout
sun.net.client.defaultReadTimeout
I don't have control over all the systems and don't want everybody to keep setting the system properties. Is there any other alternative for making remote request which will allow me to set timeouts.
Without any library, If available in java itself is preferable.
If you're opening a URLConnection from URL you can set the timeouts this way:
URL url = new URL(urlPath);
URLConnection con = url.openConnection();
con.setConnectTimeout(connectTimeout);
con.setReadTimeout(readTimeout);
InputStream in = con.getInputStream();
How are you using the URL or what are you passing it to?
A common replacement is the Apache Commons HttpClient, it gives much more control over the entire process of fetching HTTP URLs.
Related
I have HttpURLConnection instance created from URL and also I set query parameters and called some setters on this HttpURLConnection instance. I use this instance to get response from web service.
Is there some way to get the HTTP request string that will be sent over the network when using the given HttpURLConnection instance ? (just for debugging purposes). Can we do this programatically using HttpURLConnection or if it's not possible how can I monitor the outgoing HTTP traffic ?
The reason I need this that in some cases it can be easier to detect what is wrong with your configuration of HttpURLConnection by looking directly at the request that is defined by this instance than trying to figure out what is wrong with particular configuration of HttpURLConnection by checking what setters was called.
Thank you for any suggestion.
You can monitor your http Traffic by using Fiddler
Here is the download link
I want to use server cache for 15 minutess so what i have to use in setRequestProperty() ?
Please Help me..
Here is my code which i used..
private HttpURLConnection httpCon = null;
httpCon = (HttpURLConnection) httpUrl.openConnection();
httpCon.setRequestMethod("GET");
httpCon.setRequestProperty("Connection", "Keep-Alive");
httpCon.setRequestProperty("Pragma","public");
httpCon.setRequestProperty("Cache-Control","maxage=900");
httpCon.setUseCaches(true);
You are telling the server you are willing for it to cache responses, but there's no guarantee that the server will do that or is enabled to do that (unless you control the server also and implement that).
You can also try setting up an intermediate HTTP cache the client and server, such as a proxy cache such as Varnish, Pound, or Squid.
Lastly, you can do browser caching on your own, which is supported the the Android java.net package but doesn't have a default implementation. To do this:
-Check out HttpURLConnection which details (in the "Response Caching" section) that you must implement ResponseCache and call setDefault.
-Also check out ResponseCache Example which has examples of this, and something quirky to watch out for at the end (which may or may not still be true).
Good luck!
Instead of using the HttpConnection, use DefaultHttpClient and CachingHttpClient (part of Apache Http Client, bundled by default with Android).
Have a look at http://hc.apache.org/httpcomponents-client-ga/tutorial/html/caching.html to get more details on how to use caching.
I have an authentication protected url : www.domain.com/alias
that when requested will return another url: www.another.com/resource.mp4 (not protected)
I would like to know if exists a method in Java that will return the real url from a given one.
Something like: second = resolve(first)
I'm thinking of loading the first and try to read into the response maybe the location attribute, but since I'm not a java guru I would like to know if Java already faces this.
This is a problem i used to have concerning URL redirects. Try the following code:
URL url = new URL(url);
HttpURLConnection ucon = (HttpURLConnection) url.openConnection();
ucon.setInstanceFollowRedirects(false);
URL secondURL = new URL(ucon.getHeaderField("Location"));
URLConnection conn = secondURL.openConnection();
The "magic" here happens in these 2 steps:
ucon.setInstanceFollowRedirects(false);
URL secondURL = new URL(ucon.getHeaderField("Location"));
By default InstanceFollowRedirects are set to true, but you want to set it to false to capture the second URL. To be able to get that second URL from the first URL, you need to get the header field called "Location".
I have eliminated this issue on sites where we have a MikroTik router by using a Layer 7 protocol filter as shown below. This doesn't help the devices off the WiFi network (obviously) but at least gives them some reprieve when they are connected to home and/or work WiFi networks.
Firstly, create the protocol definition:
/ip firewall layer7-protocol
add comment="Frigging javascript redirects on chrome browsers" \
name=Javascript_Redirect \
regexp="^.+(spaces.slimspot.com|mostawesomeoffers.com).*\$"
Now, to actually filter this traffic out
/ip firewall filter
add action=drop chain=forward comment=\
"Block and log Javascript_Redirect L7 Protocol" layer7-protocol=\
Javascript_Redirect log=yes log-prefix=JSredirect_
Other firewalls that have Layer 7 filtering capacity could also block these redirects in a similar way.
If you are using Ktor:
import io.ktor.client.statement.*
val resp = HttpClient.get<HttpResponse>(urlString = yourUrl)
val redirectedUrl = resp.request.url
I am trying to create a HttpServlet that forwards all incoming requests as is, to another serlvet running on a different domain.
How can this be accomplished? The RequestDispatcher's forward() only operates on the same server.
Edit: I can't introduce any dependencies.
You can't when it doesn't run in the same ServletContext or same/clustered webserver wherein the webapps are configured to share the ServletContext (in case of Tomcat, check crossContext option).
You have to send a redirect by HttpServletResponse.sendRedirect(). If your actual concern is reusing the query parameters on the new URL, just resend them along.
response.sendRedirect(newURL + "?" + request.getQueryString());
Or when it's a POST, send a HTTP 307 redirect, the client will reapply the same POST query parameters on the new URL.
response.setStatus(HttpServletResponse.SC_TEMPORARY_REDIRECT);
response.setHeader("Location", newURL);
Update as per the comments, that's apparently not an option as well since you want to hide the URL. In that case, you have to let the servlet play for proxy. You can do this with a HTTP client, e.g. the Java SE provided java.net.URLConnection (mini tutorial here) or the more convenienced Apache Commons HttpClient.
If it's GET, just do:
InputStream input = new URL(newURL + "?" + request.getQueryString()).openStream();
OutputStream output = response.getOutputStream();
// Copy.
Or if it's POST:
URLConnection connection = new URL(newURL).openConnection();
connection.setDoOutput(true);
// Set and/or copy request headers here based on current request?
InputStream input1 = request.getInputStream();
OutputStream output1 = connection.getOutputStream();
// Copy.
InputStream input2 = connection.getInputStream();
OutputStream output2 = response.getOutputStream();
// Copy.
Note that you possibly need to capture/replace/update the relative links in the HTML response, if any. Jsoup may be extremely helpful in this.
As others have pointed out, what you want is a proxy. Your options:
Find an open-source Java library that does this. There are a few out there, but I haven't used any of them, so I can't recommend any.
Write it yourself. Shouldn't be too hard, just remember to deal with stuff like passing along all headers and response codes.
Use the proxy module in Apache 2.2. This is the one I'd pick, because I already know that it works reliably.
Jetty has a sample ProxyServlet implementation that uses URL.openConnection() under the hood. Feel free to use as-is or to use as inspiration for your own implementation. ;-)
Or you can use Apache HttpClient, see the tutorial.
I want to use HTTP GET and POST commands to retrieve URLs from a website and parse the HTML. How do I do this?
You can use HttpURLConnection in combination with URL.
URL url = new URL("http://example.com");
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("GET");
connection.connect();
InputStream stream = connection.getInputStream();
// read the contents using an InputStreamReader
The easiest way to do a GET is to use the built in java.net.URL. However, as mentioned, httpclient is the proper way to go, as it will allow you among others to handle redirects.
For parsing the html, you can use html parser.