I want to make some API calls to a server using HttpURLConnection. But the requests are not successful, returning:
<error>
<http_status>400 Bad Request</http_status>
<message>Unexpected request Content-Type header ''. Expecting 'application/x-www-form-urlencoded'.</message>
</error>
So I want to check what the "real" content is that is being sent to the server. By real content I mean the exact HTTP request.
Any ideas how I can see this?
Edit:
Based on the first answers here I should clarify my problem: I want to avoid using an external program like HTTP sniffer or anything and I was hoping that there is a function or a property or whatever that holds the information I am looking for.
If that is not the case, does someone know if this information can be manually rebuilt (for example by calling several functions like getRequestMethod(), etc.)
I am facing this problem kinda often so that it's worth the effort to build such functionality myself. Just need to know how :)
You can put the HttpURLConnection in debug mode by enabling java.logging with
-Djava.util.logging.config.file=logging.properties
and put in logging.properties (by default in JRE_HOME\lib) the following property
sun.net.www.protocol.http.HttpURLConnection.level = ALL
tcpdump will work, but it can be hard to make it do what you want. NetCat is more user-friendly (here's the project page: http://netcat.sourceforge.net/ - most Unix platforms already include it).
nc -l 9999
This will listen on TCP port 9999, and when an HTTP client connects, it'll print out the full text of the request.
The accepted solution did not work for me. But what did was
static {
ConsoleHandler handler = new ConsoleHandler();
handler.setLevel(Level.ALL);
Logger log = LogManager.getLogManager().getLogger("");
log.addHandler(handler);
log.setLevel(Level.ALL);
System.setProperty("javax.net.debug","all");
}
Use something like tcpdump, which can dump the actual network packets that are emitted or received by your computer.
On JDK 11 I was able to log all the http connections, setting java.util.logging.ConsoleHandler.level to FINEST and adding the following lines in the file logging.properties which is by default in %JAVA_HOME%/conf:
sun.net.www.protocol.http.HttpURLConnection.level = FINEST
sun.net.www.protocol.http.HttpURLConnection.handlers = java.util.logging.ConsoleHandler
Related
I want to ask you how can I trace all the 'http' traffic triggered by a simple Java standalone application.
I run under openjdk version "1.8.0_212".
This doesn't works for me:
-Djava.util.logging.config.file=<path_to_file>/logging.properties
In the logging.properties:
sun.net.www.protocol.http.HttpURLConnection.level = FINEST
You say it doesn't work, but you don't specify what's the behaviour you are getting. You get no logs at all? You get some log traces but not all you would expect?
Nevertheless, here are some shots in the dark:
Try this in the logging.properties file:
handlers = java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level = ALL
sun.net.www.protocol.http.HttpURLConnection.level = ALL
In the java.util.logging.config.file property, make sure to specify the full path (and not a relative one) to the logging.properties file
I assume when you say http traffic you really mean http and not https; in case you want to debug SSL connections (https), try passing the following property to the JVM:
-Djavax.net.debug=all
If none of the above works, you may want to post a minimal, self contained example along with the full command line used to run it.
I have added below new code in protobuf file and compiled to get the generated grpc_pb files.
service EchoService {
rpc Echo(EchoMessage) returns (EchoMessage) {
#-----New code start-----
option (google.api.http) = {
post: "/v1/echo"
body: "*"
};
#-----New code end------
}
}
From cURL command executed below command
curl -X POST -k https://localhost:10000/v1/echo -d '{"Key": "Value"}'
After making above request, not able to get the proper response.
My doubt is, any server side code changes needed to prepare the response to send back to caller? If so, Please suggest me with the code (Java) & also how to make request. If not, how we need to make http request to grpc?
Working example is much appreciated.
In order to test gRPC server without client, we have to use grpcurl not curl. Please take a look at https://github.com/fullstorydev/grpcurl
However, based on my experience there is a requirement to make it works. First, please ensure that your service support Reflection, you can read about it from https://github.com/sourcegraph/gophercon-2018-liveblog/issues/27. There are different ways of doing Reflection across programming languages. My advice is, just do this for the development phase, otherwise, people may querying your gRPC endpoint. Maybe you can use if() to make a conditional block for it. For Golang, i did this
import "google.golang.org/grpc/reflection"
if os.Getenv("GO_ENV") == "development" {
reflection.Register(s)
}
then, you need to know available services in your gRPC server. There is two way you can know the structure. First You can read them from your proto file, second by executing command grpcurl localhost:10000 list
If you like to read from your proto file, the path should be packageName.Service/rpcMethodName. So, based on your proto, it should be something like EchoService/Echo or if you have package name it will be packageName.EchoService/Echo
If you prefer to do grpcurl localhost:10000 list, just type those command and it will output service path.
The last thing to note when you test it locally and you don't setup SSL/TLS, please use -plaintext option, otherwise, it will tell you that TLS handshake failed.
Example command, based on your proto, a call in local will looks like:
grpcurl -plaintext -d '{"Key": "Value"}' 127.0.0.1:10000 EchoService/Echo
Hope it helps.
UPDATE 30 June 2020:
After a few months working with gRPC I found another interesting gRPC tool:
Interactive tool: https://github.com/ktr0731/evans
even easier GUI tool: https://github.com/uw-labs/bloomrpc
I created a soap client with wsimport and a given wsdl. I also used SoapUI to test the service. Using SoapUI I had no problem but when using my Java client I get
java.net.ConnectException: Connection timed out: connect
The default values I have in the requestContext are as follows
com.sun.xml.internal.ws.connect.timeout=100000
javax.xml.ws.service.endpoint.address=[fully qualified domain name endpoint]
com.sun.xml.internal.ws.request.timeout=100000
javax.xml.ws.soap.http.soapaction.use=null
com.sun.xml.internal.ws.client.ContentNegotiation=none
javax.xml.ws.soap.http.soapaction.uri=null
I've tried increasing the timeout but it still doesn't connect.
Has anyone else had a similar problem?
As you mentioned the problem is of proxy, it has been answered in below links.
How to use an HTTP proxy in java
Proxy settings in a java program
If you are using proxy with authentication then you have set authenticator along with the proxy. This is answered here.
Authenticated HTTP proxy with Java
EDIT:
As correctly mentioned by William Burnham, you have set to set the properties before calling them.
Morever, I recommend you to clear the property soon after getting response using System.clearProperty(key) as the property is set for complete instance of jvm till it is restarted and hence can cause problems for other outgoing connections.
The problem was I was behind a proxy. I did different tests and found that using a web browser (or SoapUI) I was able to access the resource but from the command line it wasn't working.
After much searching, it was a simple fix: either passing the property as a jvm argument or manually setting it in the code with System.setProperty("java.net.useSystemProxies", "true"). The JVM wasn't using the proxy on its own.
EDIT
As I used wsimport I have a jax-ws client. It's important that proxy settings be configured prior to instantiantion.
ANOTHER EDIT
If by chance you're having problems and you're using an application server to make the soap request through the proxy, you may have to specify java.net.useSystemProxies=true (or similar) in the server's configuration--for example catalina.properties if using tomcat.
I was expecting this code to return a 404, however it produces the output :
"Response code is 200"
Would it be possible to learn how to differentiate between existent and non-existent web pages . . . thanks so much,
try
{
// create the HttpURLConnection
URL url = new URL("http://www.thisurldoesnotexist");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
System.out.println("Response code is " + connection.getResponseCode());
}
EDIT: I see you've call openConnection() but not connect() - could that be the problem? I would expect getResponseCode() to actually make the request if it hasn't already, but it's worth just trying that...
That suggests you've possible got some DNS resolver which redirects to a "helper" (spam) page, or something like that.
The easiest way to see exactly what's going on here is to use Wireshark - have that up and capturing traffic (HTTP-only, to make life easier) and then run your code. You should be able to see what's going on that way.
Note that I wouldn't have expected a 404 - because that would involve being able to find a web server to talk to to start with. If you're trying to go to a host which doesn't involve, there shouldn't be an HTTP response at all. I'd expect connect() to throw an exception.
try adding a "connection.connect();" or look at the contents returned...
it could be a dns issue, ie: your dns is being sent to a parking place... for example: freedns does this.
You could:
Resolve the IP from the host of the page
Try to connect to port 80 on the resolved IP using plain sockets
This is a bit low level however and will add complexity since you will need to make a simple GET request through the socket. Then validate the response so you're sure that its actually a HTTP server running on port 80.
NMap might be able to help you here.
Ideally you should be getting this error:
java.net.UnknownHostException: www.thisurldoesnotexist
But it looks like your URL is resolved by you DNS provider.
For instance on my company's network running your code with URI "http://profile/" displays
the employee profile.
Please also check etc.home file if you are on windows to check if any settings have been changed.
Like #spgennard - I think this is most likely a DNS issue.
The URL you have chosen is owned by a DNS speculator.
The URL you have chosen is "parked" by a DNS provider.
Your ISP is messing with your DNS results to send your browser to some search page.
It is also possible that you are accessing the web via a proxy, and the proxy is doing something strange.
The way to diagnose this is to look at the other information in the HTTP responses you are getting, particularly the response body.
I'm following Scott Davis' tutorials on developing grails apps, but whenever i try to run my app (or indeed his source code) i get "Firefox has detected that the server is redirecting the request for this address in a way that will never complete." Safari gives a similar error message as does Opera.
As i've tested the original authors source code which gives the same error i'm fairly confident it's nothing to do with the code.
Is this a problem with the web server on my machine? I use Mac OS Snow Leopard so i'm assuming it's apache that's generating this error.
Edit: Seems Grails as standard uses Jetty, so probably not Apache that is causing the problem. However also tested the app on Glassfish and i get the same error.
Anyone know what i can do to fix this?
Cheers
It depends on the code and Apache configuration you are using. I assume that the web server sends cyclic HTTP redirections, eg. from /root/ to /root (without the slash) and vice versa. This causes a redirection infinite loop.
Check your configuration on conditions that cause a HTTP redirect. For example, Apache automatically adds slashes to directory URLs in standard configuration (like the /root/ example above). I don't know Grails, so I cannot give you a hint on how URLs are processed within the app.
You can also use manual HTTP requests for debugging to see whats going on behind the scenes, using telnet on a terminal:
$ telnet localhost 80
GET / HTTP/1.0
I guess the response will be something like that:
HTTP/1.0 302 Found
Location: XXX
...
Now do a second request on the URL passed in the Location header and so on.
I was getting the same error a little while ago, heres how I fixed:
Try the same page on a different internet setup (it could be your ISP)
Open up Safari, Firefox or whatever your using and empty the cache and delete ALL your cookies
Reboot your computer and try again
It may work now, but if it doesn't:
open up Firefox and type 'about:config' (without the quotes) into the URL bar
You will get some little warning, just press OK
Type 'redirect' into the Filter box
You should see a listing for 'network.http.redirection-limit'
Double click the listing and type a large number (anything above 50 and lower than 200)
Press OK, quit and re-open FireFox
Basically all that does is make FireFox's tolerance for redirect loops higher which should fix your problem - but usually, just borrowing someone else's internet connection fixes it
Hope that all helps =)
Just carefully check your URLMappings configuration:
YOUR_APP/grails-app/conf/UrlMappings.groovy
Common case:
You configured request to be handled like this:
"/anything" (controller:"someController")
So without action, request will be handled by default one, "index". "index" action usually redirects to "list", and "list", in some cases redirect back to "index"
There is your loop.
Good luck