I have HttpURLConnection instance created from URL and also I set query parameters and called some setters on this HttpURLConnection instance. I use this instance to get response from web service.
Is there some way to get the HTTP request string that will be sent over the network when using the given HttpURLConnection instance ? (just for debugging purposes). Can we do this programatically using HttpURLConnection or if it's not possible how can I monitor the outgoing HTTP traffic ?
The reason I need this that in some cases it can be easier to detect what is wrong with your configuration of HttpURLConnection by looking directly at the request that is defined by this instance than trying to figure out what is wrong with particular configuration of HttpURLConnection by checking what setters was called.
Thank you for any suggestion.
You can monitor your http Traffic by using Fiddler
Here is the download link
Related
I need to send data to another system in a Java aplication via HTTP POST method. Using the Apache HttpClient library is not an option.
I create a URL, httpconection without problems. But when sending special character like Spanish Ñ, the system complains it is receiving
Ñ instead of Ñ.
I've read many post, but I don't understand some things:
When doing a POST connection, and writing to the connection object, is it mandatory to do the URLEncode.encode(data,encoding) to the data being sent?
When sending the data, in some examples I have seen they use the
conn.writeBytes(strData), and in other I have seen conn.write(strData.getBytes(encoding)). Which one is it better? Is it related of using the encode?
Update:
The current code:
URL url = new URL(URLstr);
conn1 = (HttpsURLConnection) url.openConnection();
conn1.setRequestMethod("POST");
conn1.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(conn1.getOutputStream());
wr.writeBytes(strToSend);//data sent
wr.flush();
wr.close();
(later I get the response)
strToSend has been previously URLENCODE.encode(,"UTF-8")
I still don't know if I must use urlencode in my code and/or setRequestProperty("Contentype","application/x-www-formurlencode");
Or if I must use .write(strToSend.getByte(??)
Any ideas are welcome. I am testing also the real server (I dont know very much about it)
I need to find the HTTP response code of URLs in java. I know this can be done using URL & HTTPURLConnection API and have gone through previous questions like this
and this.
I need to do this on around 2000 links so speed is the most required attribute and among those I already have crawled 150-250 pages using crawler4j and don't know a way to get code from this library (due to which I will have to make connection on those links again with another library to find the response code).
In Crawler4J, the class WebCrawler has a method handlePageStatusCode, which is exactly what you are looking for and what you would also have found if you had looked for it. Override it and be happy.
The answer behind your first link contains everything you need:
How to get HTTP response code for a URL in Java?
URL url = new URL("http://google.com");
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("GET");
connection.connect();
int code = connection.getResponseCode();
The response code is the HTTP code returned by the server.
I have page like localhost:7001/MyServlet. I am making a http connection request from like below
String url = "http://localhost:7001/MyServlet"
PostMethod method = new PostMethod(url);
HttpClient client = new HttpClient();
However "MyServlet" is protected by j_security_check. So when I am making my connection , getting redirected to login page.
How to get authenticated and access my url , in one HttpConnection
Note: I use apache common httpclient
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.PostMethod;
I doubt you can log in and call the server in a single request, unless HTTP BASIC authentication is enabled. While I do not know the details of HTTPClient's API yet, basically you will need to track a session using cookies; POST your login to /j_security_check; then access the servlet. (The same basic process works for /j_acegi_security_check if using ACEGI Security.)
A nasty wrinkle in Tomcat is that just posting right away to /j_security_check gives a 400 "bad request"; its authenticator is rather finicky about state transitions and was clearly not designed with programmatic clients in mind. You need to first access /loginEntry (you can throw away the response other than session cookies); then post your login information to /j_security_check; then follow the resulting redirect (back to /loginEntry I think) which will actually store your new login information; finally post to the desired servlet! NetBeans #5c3cb7fb60fe shows this in action logging in to a Hudson server using Tomcat's container authentication.
Need timeout setting for remote data request made using java.net.URL class. After some googling found out that there are two system properties which can be used to set timeout for URL class as follows.
sun.net.client.defaultConnectTimeout
sun.net.client.defaultReadTimeout
I don't have control over all the systems and don't want everybody to keep setting the system properties. Is there any other alternative for making remote request which will allow me to set timeouts.
Without any library, If available in java itself is preferable.
If you're opening a URLConnection from URL you can set the timeouts this way:
URL url = new URL(urlPath);
URLConnection con = url.openConnection();
con.setConnectTimeout(connectTimeout);
con.setReadTimeout(readTimeout);
InputStream in = con.getInputStream();
How are you using the URL or what are you passing it to?
A common replacement is the Apache Commons HttpClient, it gives much more control over the entire process of fetching HTTP URLs.
I'm having a very strange issue.
My company uses a centralized user registration web-service for our various properties. We generally send a request to the web service via HttpURLConnection with request-method GET, setting parameters via qs. This has worked fine in the past.
For another property with which we've recently acquired and plugged into our registration web service, HttpURLConnection seems to be duplicating parameters when sent along. The expected value of a parameter is paramName=value, but we're receiving paramName=value, value instead. Here's a representation of what it looks like in our logs:
Note: Removing information specific to my employer and our systems.
01-26 15:21:54 [TP-Processor17] INFO com].[/] - parameter=userName=nameValue65, nameValue65
01-26 15:21:54 [TP-Processor17] INFO com].[/] - parameter=policyAccepted=true, true
This, of course, caused the end-point validation to error and disable user-registration.
Here's a representation of the code used to create the connection:
URL url = new URL("http://account-ws.domain.tld/register.action?responseType=json&userName=nameValue65&age=24&country=US&password1=Passw3rt&emailAddress=name#domain.tld&tosAccepted=true&policyAccepted=true");
HttpURLConnection urlc = (HttpURLConnection) url.openConnection();
urlc.setRequestMethod("GET");
urlc.setDoOutput(true);
urlc.setAllowUserInteraction(false);
PrintStream ps = new PrintStream(urlc.getOutputStream());
ps.print(restEndPoint);
ps.close();
Perhaps(?) useful info:
The registration form submits to itself using POST, at which point we validate using Struts forms, and send the request to the web service using the values returned by the Struts form validation class. (These values are checked for accuracy once more before sending.)
Wireshark and log4j debug messages indicate that the URL sent to the web service is correct / what we would expect, with single values for each parameter.
The initiating form's post fields are named identically to the query keys sent with the web service request.
Please ask for more info if you find what's here to be insufficient.
Thank you in advance! :)
When you use GET method, the query string is added to URL string. The GET method is a default Http request method for HttpURLConnection. You do not need to explicitly set the request method to GET.
A GET method is used to obtain the content of the requested URL. You should not write to output stream of the GET connection.
If you want to use POST method, you can set it via setRequestMethod("POST") but I am not sure if you need to have setDoOutput(true) as well. However, the setDoOutput(true) will, by default, set request method to POST so you might as well ignore the setRequestMethod("POST"). If you want to write to output stream using POST, here is my previous answer of how to do it using HttpURLConnection.
It should be noted that when you do a POST (or PUT), the URL should not contain query part. Since you have a mixture of GET and POST, this might be the cause of your problem but I am not certain.
One possible case where you have to use both setRequestMethod and setDoOutput(true) is when you want to do a Http PUT.