I am writing a code that connects to websites and checks some code, like a crawler. But I need to connect trough a proxy and change the IP address (so it doesn't show the client's IP in the server logs).
How can this be done through java?
You can use the java system properties to set up a proxy or pass it as command line options.
You can find some details and samples here.
Ex: Before opening the connection
System.setProperty("http.proxyHost", "myProxyServer.com");
System.setProperty("http.proxyPort", "80");
Or you can use the default network proxies configured in the sytem
System.setProperty("java.net.useSystemProxies", "true");
Since Java 1.5 you can create a instance of proxy and pass it to the openConnection() method.
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("123.0.0.1", 8080));
URL url = new URL("http://www.yahoo.com");
HttpURLConnection uc = (HttpURLConnection)url.openConnection(proxy);
uc.connect();
Or as lisak suggested, you can use some 3rd party libraries which supports your need better.
Or you can also use HttpClient which would suit your needs better. Check out the documentation, it's brief and very informative.
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpHost proxy = new HttpHost("someproxy", 8080);
httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("123.0.0.1", 8080));
URL url = new URL("http://www.yahoo.com");
HttpURLConnection uc = (HttpURLConnection)url.openConnection(proxy);
uc.connect();
This worked for me. I was able to use the proxy for the specific connection or transfer. Earlier we were using System.setProperty which used to set it at system level and all the requests internal/external started using the same proxy.
Also Proxy.Type.HTTP works for both http and https
Related
I used Jodd Http library to connect with the proxy:
ProxyInfo proxyInfoObj = new ProxyInfo(ProxyType.HTTP, "10.30.56.70", 8080, "", "");
SocketHttpConnectionProvider provider = new SocketHttpConnectionProvider();
provider.useProxy(proxyInfoObj);
HttpRequest request = HttpRequest.get(url);
request.method("GET");
request.charset("UTF-8");
HttpResponse response = request.open(provider).send();
result = response.bodyText();
But i got this error:
jodd.http.HttpException: HTTP: Invalid code
at jodd.http.net.HTTPProxySocketFactory.createHttpProxySocket(HTTPProxySocketFactory.java:113)
at jodd.http.net.HTTPProxySocketFactory.createSocket(HTTPProxySocketFactory.java:32)
If I use SOCKS4 type, the program hang and don't return anything. Can anyone help me?
But I can connect via proxy using following code:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("10.30.56.70", 8080));
HttpURLConnection connection =(HttpURLConnection)new URL("http://tvl.csmtalk.vn/api/sms/receive").openConnection(proxy);
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestProperty("Content-type", "text/xml");
connection.setRequestProperty("Accept", "text/xml, application/xml");
connection.setRequestMethod("GET");
connection.connect();
For me both codes hangs. When I try Jodd, it hangs because it can not open proxy socket to 10.30.56.70:8080. When I try to
telnet 10.30.56.70 8080
from command line it hangs as well. It looks like proxy is not responding. (You can contact Jodd support if you need more details, or if you want to send some private data regarding the connectivity.)
btw, you don't need to:
request.method("GET");
request.charset("UTF-8");
as method is already set to GET by method get() and charset is not used for requests, but response (to set one if not set by server).
In Java 1.4, I am trying to make an connection to a URL, but since I want to configure a timeout, I am using Apache's httpclient instead of the old school URLConnection.openConnection();
So I have the following:
HttpConnection conn = null;
SimpleHttpConnectionManager httpMgr = new SimpleHttpConnectionManager(true);
HostConfiguration hostConf = new HostConfiguration();
hostConf.setHost(new HttpHost("http://www.google.com"));
conn = httpMgr.getConnectionWithTimeout(hostConf, 30);
conn.open();
But seems like everytime I try to open a connection, I would get a java.net.UnknownHostException: http://www.google.com, even to google.com.
Did I do something wrong..?
I think protocol (http://) is what breaks it. Try using "www.google.com" as a host name.
HTTP CLient tutorial is here.
I have implemented a WS client in Java from an existing WSDL I got.
I can connect, invoke WS methods and everything works great. But....
I am supposed to use a Proxy for communication!!!
Already tried many ways to do that, but nothing works:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("host",Integer.parseInt("8008")));
URL url = new URL(theUrl);
HttpURLConnection uc = (HttpURLConnection)url.openConnection(proxy);
uc.connect();
Or i tried
System.setPropertry(String, String)
So can set host and proxy and proxySet= true.
But according to colleagues this is not what I should do as it sets global properties and it will not work with many individual runs within out System.
Any ideas how I am supposed to do this?
Any help appreciated, thanks
Eventually I used a class to extend java.net.ProxySelector let's call it MyProxySelector.java
And in the implemented select() method, I check the URI, if it is a URI I which to connect through a proxy, that is the Web service's URI then I create a proxy:
SocketAddress addr = new InetSocketAddress(theHost, thePort);
proxy = new Proxy(Proxy.Type.HTTP, addr);
add it to my proxy list:
List<Proxy> proxyList = new ArrayList<Proxy>();
proxyList.add(proxy);
and return the proxy list.
I am currently working at a web requests project and I am using Apache Http Client library. I try to connect to a server (E.g. http://www.google.com) with an working Socks v4/5 tested with mozilla firefox but the problem is that I never get a response. Only different errors...
Here is a code snippet:
//HttpClient
DefaultHttpClient http = new DefaultHttpClient();
//A class defined by me
Proxy proxy = bla bla;
HttpHost host = new HttpHost(proxy.getIP(), proxy.getPort());
if (proxy.getUsername() != null) {
http.getCredentialsProvider().setCredentials(
new AuthScope(proxy.getIP(), proxy.getPort()),
new UsernamePasswordCredentials(proxy.getUsername(), proxy.getPassword()));
}
http.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, host);
Can anyone tell a proper way to initiate comunnication through SOCKS proxies? Thanks!
Note: The code above works perfect with HTTP proxies.
Http proxy and socks proxy has very different protocols ( http://en.wikipedia.org/wiki/SOCKS#Comparison ).
For your question:
You can do thTis by native java socket ( How can I configure HTTPClient to authenticate against a SOCKS proxy? ) or create your own implementation over DefaultClientConnectionOperator, here is good guide ( http://4devops.blogspot.com/2011/10/httphttps-over-socks-proxy-with-java.html )
I want to connect to as site through proxy in java. This is the code which I have written:
public class ConnectThroughProxy
{
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy ip", 8080));
public static void main(String[] args)
{
try
{
URL url = new URL("http://www.rgagnon.com/javadetails/java-0085.html");
URLConnection connection=url.openConnection();
String encoded = new String(Base64.encode(new String("user_name:pass_word").getBytes()));
connection.setDoOutput(true);
connection.setRequestProperty("Proxy-Authorization","Basic "+encoded);
String page="";
String line;
StringBuffer tmp = new StringBuffer();
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while ((line=in.readLine()) != null)
{
page.concat(line + "\n");
}
System.out.println(page);
}
catch(Exception ex)
{
ex.printStackTrace();
}
}
While trying to run this code it throws the following error:
java.lang.IllegalArgumentException: Illegal character(s) in message header value: Basic dXNlcl9uYW1lOnBhc3Nfd29yZA==
at sun.net.www.protocol.http.HttpURLConnection.checkMessageHeader(HttpURLConnection.java:323)
at sun.net.www.protocol.http.HttpURLConnection.setRequestProperty(HttpURLConnection.java:2054)
at test.ConnectThroughProxy.main(ConnectThroughProxy.java:30)
Any Idea how to do it?
If you're just trying to make HTTP requests through an HTTP proxy server, you shouldn't need to go to this much effort. There's a writeup here: http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html
But it basically boils down to just setting the http.proxyHost and http.proxyPort environment properties, either on the command line, or in code:
// Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setProperty("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.clearProperty("http.proxyHost");
// From now on HTTP connections will be done directly.
It seems to me, that you are not using your Proxy instance at all. I think you should pass it when you are creating URLConnection instance:
URLConnection connection=url.openConnection(proxy);
Setting of environment properties http.proxy is easier and when using some 3rd party libraries without Proxy instance passing support only possible solution, but its drawback is that it is set globally for the whole process.
I was using the Google Data APIs and the only way I got the proxy settings to work was to provide ALL the parameters related to proxy, even thought they are set to be empty:
/usr/java/jdk1.7.0_04/bin/java -Dhttp.proxyHost=10.128.128.13
-Dhttp.proxyPassword -Dhttp.proxyPort=80 -Dhttp.proxyUserName
-Dhttps.proxyHost=10.128.128.13 -Dhttps.proxyPassword -Dhttps.proxyPort=80
-Dhttps.proxyUserName com.stackoverflow.Runner
Where, username and password are NOT required, and the same http and https servers are set to be the same, as well as the port number (if that's your case as well). Note that the same HTTP proxy is also provided as the HTTPS server, as well as its port number (reference from https://code.google.com/p/syncnotes2google/issues/detail?id=2#c16).
If your Java class has an instance of the class "URL", it should pick those configurations up...