Develop Reference

Java HttpUrlConnection throws Connection Refused

I know there are several question regarding this topic But I did't find an answer in any of them.
I'm trying to open a connection to my local server but I keep getting connection refused.
I have the server running and I tested the connection with the Browser and with a Google App called Postman and it works.
It's failing when opening the connection as if there where nothing to connect to. or maybe something is blocking the connection? I tested with firewall and antivirus down, no luck.
testing in Postman the URL returns a User as it should...
If I replace the url with "http://www.google.com" It Works fine.
here is my code:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
/**
*
* #author Gabriel
*/
public class HttpConnection {
public HttpConnection() {
}
public void makeRequest() throws MalformedURLException, IOException {
String url = "http://localhost:8000/users/1";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
//add request header
con.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36");
con.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
con.setRequestProperty("Accept-Encoding", "gzip,deflate,sdch");
con.setRequestProperty("Accept-Language", "en-US,en;q=0.8,es;q=0.6");
con.setRequestProperty("Connection", "keep-alive");
con.setRequestProperty("Host", "localhost:8000");
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//print result
System.out.println(response.toString());
}
}

I faced exactly the same problem. Use this instead of localhost:
http://[::1]:8000/index.php

I have similar code that is working, but my request header is a lot simpler. Basically just:
con.setRequestProperty("User-Agent", "Mozilla/5.0");
If simplifying the header does not help, I would capture the traffic when using your browser with something like fiddler and then making the request look exactly like that.

I will make a wild guess what can be the problem. It is possible a IPv4/IPv6 problem.
If so, here is two possible solutions
If the server is only listening on an ipv6 address, change it to listening to ipv4.
If the server is listening to ipv4, then force Java to use ipv4 with
java.net.preferIPv4Stack=true

You can try implementing CORS at the API you are trying to connect by setting access-control-allow-origin:* property in response header.

The code is good and works great. Now the problem must be on the transportation or network part. What I want to mean is you don't request the right server. If you use 127.0.0.1 instead of localhost I think you won't get a problem. So, my guest will be that you have a problem in /etc/hosts or C:\Windows\System32\drivers\etc\hosts.
I advice you to try a simple test: ping the hostname and check in the output if the ip address is good.

Well, put http://localhost:8000/users/1 in your web browser and what do you get? A simple Connection Refused error. It's not you, it's the website. Also, Url returns websites using Protocol Identifiers(http://, https://), Ending Domains(.com, .edu, .gov) that's also another reason why you get an error.

You mentioned that you were opening a connection to your "local server"
I am assuming that you are doing this on the same computer that you're hosting the server on?
Try to open the connection to your local server using a different computer.

Can't read in HTML content from valid URL

I am trying out a simple program for reading the HTML content from a given URL. The URL I am trying in this case doesn't require any cookie/username/password, but still I am getting a io.IOException: Server returned HTTP response code: 403 error. Can anyone tell me what am I doing wrong here? (I know there are similar question in SO, but they didn't help):
import java.net.*;
import java.io.*;
import java.net.MalformedURLException;
import java.io.IOException;
public class urlcont {
public static void main(String[] args) {
try {
URL u = new URL("http://www.amnesty.org/");
URLConnection uc = u.openConnection();
uc.addRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)");
uc.connect();
InputStream in = uc.getInputStream();
int b;
File f = new File("C:\\Users\\kausta\\Desktop\\urlcont.txt");
f.createNewFile();
OutputStream s = new FileOutputStream(f);
while ((b = in.read()) != -1) {
s.write(b);
}
}
catch (MalformedURLException e) {System.err.println(e);}
catch (IOException e) {System.err.println(e);}
}
}

If you can fetch the URL in a browser, but not via Java, that indicates, to me, that they are blocking programmatic access to the page via user-agent filtering. Try setting the user-agent on your connection so that your code appears, to the webserver, to be a web-browser.
See this thread for help on that: What is the proper way of setting headers in a URLConnection?

There is a permission problem:
A web server may return a 403 Forbidden HTTP status code in response to a request from a client for a web page or resource to indicate that the server refuses to allow the requested action

you are not doing anything "wrong", the server you are trying to access is blocking your request, as you are not allowed to access the file
Http-Error 403 means Forbidden --> the remote server blocks the request.
check if you need to give authentification to access the document you want and in that case provide it with the request ;)

How can I prevent a 403 HTTP error code in Java?

I use simple code to get html for http://www.ip-adress.com, but it shows error http code 403.
I try it in other website like google.com in program, it can work. i can also open www.ip-adress.com in browse, why i can't use it in java program.
public class urlconnection
{
public static void main(String[] args)
{
StringBuffer document = new StringBuffer();
try
{
URL url = new URL("http://www.ip-adress.com");
URLConnection conn = url.openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null)
document.append(line + " ");
reader.close();
}
catch (MalformedURLException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
System.out.println(document.toString());
}
}
java.io.IOException: Server returned HTTP response code: 403 for URL: http://www.ip-adress.com/
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at urlconnection.main(urlconnection.java:14)

This is the line you required
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2");
refer this

The web-server can detect that you are not actually trying to access it via HTTP, so it rejects your request. There are ways to fake that to trick the server into thinking that you are a browser.

I suppose the site checks user agent header and blocks what it seems to be "a robot". You need to mimic normal browser. Check this solution Setting user agent of a java URLConnection or try to use commons http client AND set user agent.

I don't believe that this is fundamentally a Java problem. You're doing the right thing to make an HTTP connection, and the server is doing "the right thing" from its perspective by responding to your request with a 403 response.
Let's be clear about this - the response you're getting is due to whatever logic is being employed by the target webserver.
So if you were to ask "how can I modify my request so that http://www.ip-address.com returns a 200 response", then people may be able to come up with workarounds that keep that server happy. But this is a host-specific process; your Java code is arguably correct, though it should have better error handling because you can always get non-2xx responses.

Try to change Connection User-Agent to something like Browsers, most of times I use Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1

java.io.IOException: Server returned HTTP response code: 403 for URL

I want to download the mp3 file from url : "http://upload13.music.qzone.soso.com/30671794.mp3", i always got java.io.IOException: Server returned HTTP response code: 403 for URL. But it's ok when open the url using browser. Below is part of my code:
BufferedInputStream bis = null;
BufferedOutputStream bos = null;
try {
URL url = new URL(link);
URLConnection urlConn = url.openConnection();
urlConn.addRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
String contentType = urlConn.getContentType();
System.out.println("contentType:" + contentType);
InputStream is = urlConn.getInputStream();
bis = new BufferedInputStream(is, 4 * 1024);
bos = new BufferedOutputStream(new FileOutputStream(
fileName.toString()));​
Anyone could help me? Thanks in advance!

You can also use
System.setProperty("http.agent", "Chrome");
it worked for me.
//Update
Explanation
Because HttpURLConnection reads the property "http.agent" if set.
You can read it here: https://www.innovation.ch/java/HTTPClient/advanced_info.html
Or you can look it up in the source code of the HttpURLConnection Class:
String agent = java.security.AccessController.doPrivileged(new sun.security.action.GetPropertyAction("http.agent"));

Instead of using URLConnection in java, if you use HttpURLConnection you should beable to access the requested web page from java. Try the following code:
HttpURLConnection httpcon = (HttpURLConnection) url.openConnection();
httpcon.addRequestProperty("User-Agent", "Mozilla/4.76");
Normal java using urlConnection wont be accepted to access the internet. To access the browser it will need to perform a search without theexception HTTP response code : 403 for URL
EDIT (#Mordechai): No need to do the casting, just add the user agent.

When I access the URL with my browser I also get 403. Perhaps you're logged in to the site with your browser?
If that's the case you need to duplicate the cookie from your browser and send it along, perhaps even do more to replicate your browser's signature if the site does any extra checks.
You can set the cookie by adding:
urlConn.setRequestProperty("Cookie", "foo=bar");
Where foo=bar is the key-value pair you'll find when you locate the site's cookie in your browser.

The problem is given by the Status code. 403 means actually "Forbidden" and implies The request was denied for a reason the server does not want to (or has no means to) indicate to the client.
the problem lies at the server-side.

I would also check if the server were the resource is located has an ACL or similar in place, we just resolved a "java.io.IOException: 403" issue this way.
It happens that 403 errors are very generic and you cannot really be sure of the source as it can be just anything.

Connect to a site using proxy code in java

I want to connect to as site through proxy in java. This is the code which I have written:
public class ConnectThroughProxy
{
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy ip", 8080));
public static void main(String[] args)
{
try
{
URL url = new URL("http://www.rgagnon.com/javadetails/java-0085.html");
URLConnection connection=url.openConnection();
String encoded = new String(Base64.encode(new String("user_name:pass_word").getBytes()));
connection.setDoOutput(true);
connection.setRequestProperty("Proxy-Authorization","Basic "+encoded);
String page="";
String line;
StringBuffer tmp = new StringBuffer();
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while ((line=in.readLine()) != null)
{
page.concat(line + "\n");
}
System.out.println(page);
}
catch(Exception ex)
{
ex.printStackTrace();
}
}
While trying to run this code it throws the following error:
java.lang.IllegalArgumentException: Illegal character(s) in message header value: Basic dXNlcl9uYW1lOnBhc3Nfd29yZA==
at sun.net.www.protocol.http.HttpURLConnection.checkMessageHeader(HttpURLConnection.java:323)
at sun.net.www.protocol.http.HttpURLConnection.setRequestProperty(HttpURLConnection.java:2054)
at test.ConnectThroughProxy.main(ConnectThroughProxy.java:30)
Any Idea how to do it?

If you're just trying to make HTTP requests through an HTTP proxy server, you shouldn't need to go to this much effort. There's a writeup here: http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html
But it basically boils down to just setting the http.proxyHost and http.proxyPort environment properties, either on the command line, or in code:
// Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setProperty("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.clearProperty("http.proxyHost");
// From now on HTTP connections will be done directly.

It seems to me, that you are not using your Proxy instance at all. I think you should pass it when you are creating URLConnection instance:
URLConnection connection=url.openConnection(proxy);
Setting of environment properties http.proxy is easier and when using some 3rd party libraries without Proxy instance passing support only possible solution, but its drawback is that it is set globally for the whole process.

I was using the Google Data APIs and the only way I got the proxy settings to work was to provide ALL the parameters related to proxy, even thought they are set to be empty:
/usr/java/jdk1.7.0_04/bin/java -Dhttp.proxyHost=10.128.128.13
-Dhttp.proxyPassword -Dhttp.proxyPort=80 -Dhttp.proxyUserName
-Dhttps.proxyHost=10.128.128.13 -Dhttps.proxyPassword -Dhttps.proxyPort=80
-Dhttps.proxyUserName com.stackoverflow.Runner
Where, username and password are NOT required, and the same http and https servers are set to be the same, as well as the port number (if that's your case as well). Note that the same HTTP proxy is also provided as the HTTPS server, as well as its port number (reference from https://code.google.com/p/syncnotes2google/issues/detail?id=2#c16).
If your Java class has an instance of the class "URL", it should pick those configurations up...