I have to send some command using HTTP request to some applications that have embedded http server.I'm using sockets,so far I did this but I'm a little lost:
URI uri = URI.create(rawData);
try {
String host = uri.getHost();
String path = uri.getRawPath( );
if (path == null || path.length( ) == 0) {
path = "/";
}
String protocol = uri.getScheme( );
int port = uri.getPort( );
if (port == -1) {
if (protocol.equals("http")) {
port = 80; // http port
}
else if (protocol.equals("https")) {
port = 443; // https port
}
}
Socket socket = new Socket( host, port );
PrintWriter request = new PrintWriter( socket.getOutputStream() );
request.print( "GET " + path + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Connection: close\r\n\r\n");
request.flush( );
Is this all I need to do?
An example of rawData is "http://somemessage".Is the protocol written correctly?
Thanks
Is this all I need to do?
Nope.
You need to process the response, and there is complexity in dealing with the various kinds of responses you could receive.
The way you are handling HTTPS is all wrong. You need to deal with SSL connection negotiation ... and if you try to do that using plain sockets you have a huge amount of coding to do.
An example of rawData is "http://somemessage"
What is a strange URL. The stuff after the "//" should be (or include) a resolvable hostname or IP address. If you tried to fetch a URL like that using a web browser, it would not work.
Is the protocol written correctly?
Certainly the URL is not written correctly. If you try to use that so-called URL with HttpURLConnection, or any the other client-side HTTP API (see below), IT WILL NOT WORK!!!
A typical well-formed URL (in this case, with an explicit port number) looks like this:
http://example.com:8080/path/to/resource
You can also express URIs in relative form; e.g.
http:/path/to/resource
or
http:relpath/to/resource
or even
resource
but those two forms need to be turned into absolute (URL) form before they can be used by a client library.
But frankly, this is the wrong way to go about things. There are existing implementations of the client-side HTTP / HTTPS stack in the Java SE libraries, and also in the Apache HTTP libraries. Attempting to reimplement them is a waste of time.
You can use higher-level JDK APIs instead of the lower-level Socket API for your requirement. Please refer to the following links -
http://docs.oracle.com/javase/7/docs/api/java/net/HttpURLConnection.html
http://docs.oracle.com/javase/7/docs/api/javax/net/ssl/HttpsURLConnection.html
Below is a sample snippet with HttpURLConnection API -
String urlString = "http://myhost/mywebapp/myresource";
URL url= new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(timeout);
conn.setRequestMethod("GET");
conn.getResponseCode();
Related
I'm working under a AS400 system using Java 1.4.2, to get an XML but when I do the request using getOutputStream() I got an IOException and the message only returns the domain of the provider.
Here a part of my code:
try {
url = new URL("https://test.example.it/27/xml/"); //Example URL...
} catch(MalformedURLException exMAL) {
return exMAL.getMessage();
}
//Set parameters
LinkedHashMap params = new LinkedHashMap();
params.put("id", id);
params.put("password", password);
...
params.put("description", description);
String data = "";
Set set = params.entrySet();
Iterator i = set.iterator();
//Create URL of parameters
while(i.hasNext()){
if(data != ""){
data += "&";
}
Map.Entry me = (Map.Entry)i.next();
data += me.getKey() + "=" + me.getValue();
}
try {
//Create connection
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setUseCaches(false);
//Do request
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream()); //Error here
wr.write(data);
wr.flush();
//Get resource
InputStream is = conn.getInputStream();
outputData = readAll(is);
} catch (IOException ioEx) {
return "ERROR IOException: " + ioEx.getMessage();
}
And the message error is "ERROR IOException: test.example.it".
I tested my code in a computer with Windows XP, Netbeans 4.1, Java 1.4.2 Build 19 (the same in the AS400) and I get the XML without problems.
I can compile my code into the AS400 but when I run the class in QSHELL or using a RPG Program i get the error.
Somebody knows what else I have to do or why I get this error?
Thanks!
EDIT:
Its a POST method, and actually I included setDoInput and setDoOutput in the code.
Now printing the exception I get this:
java.net.UnknownHostException: test.example.it
ERROR IOException: test.example.it
EDIT:
Looks like nslookup doesn't resolve the domain but after that I retry using the IP and now I have this error...
java.net.SocketException: The value specified for the argument is not correct.
ERROR IOException: The value specified for the argument is not correct.
I think it would be a problem with AS400 security, probably should I add this domain in a list or edit a system value?
EDIT:
Using TCPCFG and then option 10, in this table of hosts I don't have the domain test.example.it, I suppose that I need to add this domain and IP, isn't it?
EDIT:
The version is V6R1M0. I added the IPs (Option 10. Work with TCP/IP host table entries) and now the AS400 resolve the domain but now I have a new problem.
java.net.ConnectException: A remote host refused an attempted connect operation.
In the option 12 I have *SAME in all the options. For option 1 the following:
Internet Subnet Line Line Opt Address Mask Description Type
127.0.0.1 255.0.0.0 *LOOPBACK *NONE
151.208.xxx.xx 255.255.255.0 ETHLINE *ELAN
I found in a group this: "Is the 8080 port on the 400. Is the 400 setup to accept 8080 traffic via SSL".
How can I able the 400 to accept traffic?
Finally I resolve my problem, adding the IP and Domain in the host table (as I mentioned in a edit) but here is another problem.
The provider gave me one IP for requests and two more IPs for responses, an I added the three IPs with the same domain in the host table. I deleted two of three IPs and then I tried using only the IP for request and it worked!
Now the problem is that the request via Java is too slow! but that's another topic. Thank you very much for all.
I am trying to find a way to get the protocol from a URL that the user types in. I have an EditText set as uri in an android layout file. The user types in his web address as www.thiersite.com or theirsite.com.
Now how can I get the correct protocol from what they have typed in? It seems everywhere I look that you need to have either https:// or http:// as the protocol in the http request. I get a malformed exception when I don't have a protocol for their web address.
Is there a way to check the URL without having the need to have the protocol when they typed their address? So in essence, do I need to ask the User to type in the protocol as part of the URL? I would prefer to do it programmatically.
/**
* Protocol value could be http:// or https://
*/
boolean usesProtocol(String url,String protocol){
boolean uses = false;
try{
URL u = new URL( protocol.concat(url) );
URLConnection con = u.openConnection();
con.connect();
// the following line will be hit only if the
// supplied protocol is supported
uses = true;
}catch(MalformedURLException e){
// new URL() failed
// user has made a typing error
}catch(IOException e){
// openConnection() failed
// the supplied protocol is not supported
}finally{
return uses;
}
}
I believe that the code is self-explaining. The above code uses no external dependencies. If you do not mind using JSoup, there is another answer on SO that deals with the same: Java how to find out if a URL is http or https?
My Source: http://docs.oracle.com/javase/tutorial/networking/urls/connecting.html
I want to read the second line of the text at this URL: "http://vuln2014.picoctf.com:51818/" (this is a capture-the-flag competition but only asking for flags or direction to flags breaks the competition rules). I am attempting to open an input stream from the URL but I get an Invalid HTTP Response exception. Any help is appreciated, and I recognize that my error is likely quite foolish.
Code:
URL url = new URL("http://vuln2014.picoctf.com:51818");
URLConnection con = url.openConnection();
InputStream is = con.getInputStream()
The error occurs at the third line.
java.io.IOException: Invalid Http response at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1342) at name.main(name.java:41)
curl happily gets the text from the page, and it is perfectly accessible from a web browser.
When you do this:
URL url = new URL("http://vuln2014.picoctf.com:51818");
URLConnection con = url.openConnection();
You are entering into a contract that says that this URL uses the http protocol. When you call openConnection it expects to get http responses because you used http:// in the URL as the protocol. The Java Documentation says:
If for the URL's protocol (such as HTTP or JAR), there exists a public, specialized URLConnection subclass belonging to one of the following packages or one of their subpackages: java.lang, java.io, java.util, java.net, the connection returned will be of that subclass. For example, for HTTP an HttpURLConnection will be returned, and for JAR a JarURLConnection will be returned.
The server you are connecting to just returns a couple lines of data. I retrieved them with the command nc vuln2014.picoctf.com 51818. There is no http response code like HTTP/1.1 200 OK:
Welcome to the Daedalus Corp Spies RSA Key Generation Service. The public modulus you should use to send your updates is below. Remember to use exponent 65537.
b4ab920c4772c5247e7d89ec7570af7295f92e3b584fc1a1a5624d19ca07cd72ab4ab9c8ec58a63c09f382aa319fa5a714a46ffafcb6529026bbc058fc49fb1c29ae9f414db4aa609a5cab6ff5c7b4c4cfc7c18844f048e3899934999510b2fe25fcf8c572514dd2e14c6e19c4668d9ad82fe647cf9e700dcf6dc23496be30bb
In this case I would use java.net.Socket to establish a connection and then read the lines. This is a simplistic approach that assumes there are 2 lines of data:
Socket theSocket;
try {
theSocket = new Socket("vuln2014.picoctf.com", 51818);
BufferedReader inFile = new BufferedReader(new InputStreamReader(theSocket.getInputStream()));
String strGreet = inFile.readLine();
String strData = inFile.readLine();
} catch (IOException e) {
e.printStackTrace();
}
As for why curl and browsers may render it properly? They are likely more lenient about the data they read and will just dump what is read from the port even if it doesn't conform to the specified protocol (like http)
I have a Java application which opens an existing company's website using the Socket class:
Socket sockSite;
InputStream inFile = null;
BufferedWriter out = null;
try
{
sockSite = new Socket( presetSite, 80 );
inFile = sockSite.getInputStream();
out = new BufferedWriter( new OutputStreamWriter(sockSite.getOutputStream()) );
}
catch ( IOException e )
{
...
}
out.write( "GET " + presetPath + " HTTP/1.1\r\n\r\n" );
out.flush();
I would read the website with the stream inFile and life is good.
Recently this started to fail. I was getting an HTTP 301 "site has moved" error but no moved-to link. The site still exists and responds using the same original HTTP reference and any web browser. But the above code comes back with the HTTP 301.
I changed the code to this:
URL url;
InputStream inFile = null;
try
{
url = new URL( presetSite + presetPath );
inFile = url.openStream();
}
catch ( IOException e )
{
...
}
And read the site with the original code from inFile stream and it now works again.
This difference doesn't just occur in Java but it also occurs if I use Perl (using IO::Socket::INET approach opening the website port 80, then issuing a GET fails, but using LWP::Simple method get just works). In other words, I get a failure if I open the web page first with port 80, then do a GET, but it works fine if I use a class which does it "all at once" (that just says, "get me web page with such-and-such an HTTP address").
I thought I'd try the different approaches on http://www.microsoft.com and got an interesting result. In the case of opening port 80, followed by issuing the GET /..., I received an HTTP 200 response with a page that said, "Your current user agent
In one case, I tried the "port 80" open followed by GET / on www.microsoft.com and I received an HTTP 200 response page that said, "Your current user agent appears to be from an automated process...". But if I use the second method (URL class in Java, or LWP in Perl) I simply get their web page.
So my question is: how does the URL class (in Java) or the LWP module (in Perl) do its thing under the hood that makes it different from opening the website on port 80 and issuing a GET?
Most servers require the Host: header, to allow virtual hosting (multiple domains on one IP)
If you use a packet capturing software to see what's being sent when URL is used, you'll realize that there's a lot more than just "GET /" being sent. All sorts of additional header information are included. If a server gets just a simple "GET /", it's easy to deduct that it can't be a very sophisticated client on the other end.
Also, HTTP 1.0 is "outdated", the current version is 1.1.
Java URL implementation delegates to HttpURLConnection if it starts with "http:"
I want to connect to as site through proxy in java. This is the code which I have written:
public class ConnectThroughProxy
{
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy ip", 8080));
public static void main(String[] args)
{
try
{
URL url = new URL("http://www.rgagnon.com/javadetails/java-0085.html");
URLConnection connection=url.openConnection();
String encoded = new String(Base64.encode(new String("user_name:pass_word").getBytes()));
connection.setDoOutput(true);
connection.setRequestProperty("Proxy-Authorization","Basic "+encoded);
String page="";
String line;
StringBuffer tmp = new StringBuffer();
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while ((line=in.readLine()) != null)
{
page.concat(line + "\n");
}
System.out.println(page);
}
catch(Exception ex)
{
ex.printStackTrace();
}
}
While trying to run this code it throws the following error:
java.lang.IllegalArgumentException: Illegal character(s) in message header value: Basic dXNlcl9uYW1lOnBhc3Nfd29yZA==
at sun.net.www.protocol.http.HttpURLConnection.checkMessageHeader(HttpURLConnection.java:323)
at sun.net.www.protocol.http.HttpURLConnection.setRequestProperty(HttpURLConnection.java:2054)
at test.ConnectThroughProxy.main(ConnectThroughProxy.java:30)
Any Idea how to do it?
If you're just trying to make HTTP requests through an HTTP proxy server, you shouldn't need to go to this much effort. There's a writeup here: http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html
But it basically boils down to just setting the http.proxyHost and http.proxyPort environment properties, either on the command line, or in code:
// Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setProperty("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.clearProperty("http.proxyHost");
// From now on HTTP connections will be done directly.
It seems to me, that you are not using your Proxy instance at all. I think you should pass it when you are creating URLConnection instance:
URLConnection connection=url.openConnection(proxy);
Setting of environment properties http.proxy is easier and when using some 3rd party libraries without Proxy instance passing support only possible solution, but its drawback is that it is set globally for the whole process.
I was using the Google Data APIs and the only way I got the proxy settings to work was to provide ALL the parameters related to proxy, even thought they are set to be empty:
/usr/java/jdk1.7.0_04/bin/java -Dhttp.proxyHost=10.128.128.13
-Dhttp.proxyPassword -Dhttp.proxyPort=80 -Dhttp.proxyUserName
-Dhttps.proxyHost=10.128.128.13 -Dhttps.proxyPassword -Dhttps.proxyPort=80
-Dhttps.proxyUserName com.stackoverflow.Runner
Where, username and password are NOT required, and the same http and https servers are set to be the same, as well as the port number (if that's your case as well). Note that the same HTTP proxy is also provided as the HTTPS server, as well as its port number (reference from https://code.google.com/p/syncnotes2google/issues/detail?id=2#c16).
If your Java class has an instance of the class "URL", it should pick those configurations up...