I am using apache commons 3.1 to implement httpClient with proxy support.
I am trying to connect to a remote host through proxy. The proxy server is configured without any authentication, however the the remote host is configured with authentication.
When I am passing the proxy parameters through properties file, it gives warning while execution:
WARN - Required proxy credentials not available for BASIC #xx.xx.xx.xx
WARN - Preemptive authentication requested but no default proxy credentials availble
But the execution goes ahead.
On the other hand when I am passing the proxy parameters through the JVM arguments then the again the same warning is given and the execution is stopped.
Is there any specific reason for this behavior? Is there any difference in passing the proxy parameters through properties file and through JVM arg?
Here is the code:
if(System.getProperty("http.proxyHost") != null && System.getProperty("http.proxyPort") != null) {
httpClient.getHostConfiguration().setProxy(System.getProperty("http.proxyHost"), Integer.parseInt(System.getProperty("http.proxyPort")));
}
else if(AMXAdminTask.props.getProperty("http.proxyHost") != null && AMXAdminTask.props.getProperty("http.proxyPort") != null) {
httpClient.getHostConfiguration().setProxy(Propfile.props.getProperty("http.proxyHost"), Integer.parseInt(Propfile.props.getProperty("http.proxyPort")));
}
Looks like you are trying to combine two very different things. The code you posted above properly gets you through your proxy, but the remote host requires BASIC authentication. The below example uses the Jersey client (used in an existing project for making RESTful calls), but you should get the idea of what you need to do. If you're stuck on using Apache HttpComponents, take a look at this:
http://hc.apache.org/httpcomponents-client-ga/tutorial/html/authentication.html
import org.apache.commons.lang.StringUtils;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.filter.HTTPBasicAuthFilter;
import com.sun.jersey.client.apache.ApacheHttpClient;
import com.sun.jersey.client.apache.config.ApacheHttpClientConfig;
import com.sun.jersey.client.apache.config.DefaultApacheHttpClientConfig;
public abstract class BaseProxyProvider {
protected Client getHttpClient() {
final DefaultApacheHttpClientConfig cc = new DefaultApacheHttpClientConfig();
if (StringUtils.isNotEmpty(System.getProperty("http.proxyHost"))) {
cc.getProperties()
.put(ApacheHttpClientConfig.PROPERTY_PROXY_URI,
"http://" + System.getProperty("http.proxyHost") + ":"
+ System.getProperty("http.proxyPort") + "/");
}
Client c = ApacheHttpClient.create(cc);
c.addFilter(new HTTPBasicAuthFilter(WebAppPropertyReader.getProperties().getProperty(
WebAppPropertyReader.SERVICE_USER), WebAppPropertyReader.getProperties().getProperty(
WebAppPropertyReader.SERVICE_PASSWORD)));
return c;
}
}
Related
I have developed a web site locally on my mac using tomcat 8.0.027 and java jdk 7. The site relies on websockets and all works well locally. When I deploy the site to a virtual private java server I get an error 404. The url for the websocket is ws://www.modelstudio3d.com/handler, my site is www.modelstudio3d.com. I've tried appending :8080 to the host name, that changes the error to connection refused. Changing the url to ../handlerx also generates a connection refused. I've checked that I'm not packaging additional websocket jars in my war file (I'm not as far as I can tell). My server is running tomcat 8.0.22 and JDK 7. My ISP is unable to provide any guidance.
My client code is
this.getWebSocketURI = function () {
var loc = window.location, wsUri;
if (loc.protocol === "https:") {
wsUri = "wss:";
} else {
wsUri = "ws:";
console.log("Websocket is not secure.");
}
wsUri += "//" + loc.host;
wsUri += loc.pathname + "handler";
console.log("wsUri" + wsUri);
return wsUri;
};
this.init = function () {
var self = this;
self.websocket = new WebSocket(self.getWebSocketURI());
...
The interesting part of the server code is
#ServerEndpoint(value = "/handler",
configurator = GetHttpSessionConfigurator.class)
public class WebSocket {
I've read something here about an Apache location filter that might be interfering by redirecting ws: to http: but have no idea where that resides or how to change it.
My hosting service finally agreed to investigate the problem. They say mod_proxy_wstunnel is not enabled and cannot be enabled. Don't use Mocha Host if you need to use websockets.
I have problem with vertx HttpClient.
Here's code which shows that tests GET using vertx and plain java.
Vertx vertx = Vertx.vertx();
HttpClientOptions options = new HttpClientOptions()
.setTrustAll(true)
.setSsl(false)
.setDefaultPort(80)
.setProtocolVersion(HttpVersion.HTTP_1_1)
.setLogActivity(true);
HttpClient client = vertx.createHttpClient(options);
client.getNow("google.com", "/", response -> {
System.out.println("Received response with status code " + response.statusCode());
});
System.out.println(getHTML("http://google.com"));
Where getHTML() is from here: How do I do a HTTP GET in Java?
This is my output:
<!doctype html><html... etc <- correct output from plain java
Feb 08, 2017 11:31:21 AM io.vertx.core.http.impl.HttpClientRequestImpl
SEVERE: java.net.UnknownHostException: failed to resolve 'google.com'. Exceeded max queries per resolve 3
But vertx can't connect. What's wrong here? I'm not using any proxy.
For reference: a solution, as described in this question and in tsegismont's comment here, is to set the flag vertx.disableDnsResolver to true:
-Dvertx.disableDnsResolver=true
in order to fall back to the JVM DNS resolver as explained here:
sometimes it can be desirable to use the JVM built-in resolver, the JVM system property -Dvertx.disableDnsResolver=true activates this behavior
I observed this DNS resolution issue with a redis client in a kubernetes environment.
I had this issue, what caused it for me was stale DNS servers being picked up by the Java runtime, i.e. servers registered for a network the machine was no longer connected to. The issue is first in the Sun JNDI implementation, it also exists in Netty which uses JNDI to bootstrap its list of name servers on most platforms, then finally shows up in VertX.
I think a good place to fix this would be in the Netty layer where the set of default DNS servers is bootstrapped. I have raised a ticket with the Netty project so we'll see if they agree with me! Here is the Netty ticket
In the mean time a fairly basic workaround is to filter the default DNS servers detected by Netty, based on whether they are reachable or not. Here is a code Sample in Kotlin to apply before constructing the main VertX instance.
// The default set of name servers provided by JNDI can contain stale entries
// This default set is picked up by Netty and in turn by VertX
// To work around this, we filter for only reachable name servers on startup
val nameServers = DefaultDnsServerAddressStreamProvider.defaultAddressList()
val reachableNameServers = nameServers.stream()
.filter {ns -> ns.address.isReachable(NS_REACHABLE_TIMEOUT)}
.map {ns -> ns.address.hostAddress}
.collect(Collectors.toList())
if (reachableNameServers.size == 0)
throw StartupException("There are no reachable name servers available")
val opts = VertxOptions()
opts.addressResolverOptions.servers = reachableNameServers
// The primary Vertx instance
val vertx = Vertx.vertx(opts)
A little more detail in case it is helpful. I have a company machine, which at some point was connected to the company network by a physical cable. Details of the company's internal name servers were set up by DHCP on the physical interface. Using the wireless interface at home, DNS for the wireless interface gets set to my home DNS while the config for the physical interface is not updated. This is fine since that device is not active, ipconfig /all does not show the internal company DNS servers. However, looking in the registry they are still there:
Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces
They get picked up by the JNDI mechanism, which feeds Netty and in turn VertX. Since they are not reachable from my home location, DNS resolution fails. I can imagine this home/office situation is not unique to me! I don't know whether something similar could occur with multiple virtual interfaces on containers or VMs, it could be worth looking at if you are having problems.
Here is the sample code which works for me.
public class TemplVerticle extends HttpVerticle {
public static void main(String[] args) {
Vertx vertx = Vertx.vertx();
// Create the web client and enable SSL/TLS with a trust store
WebClient client = WebClient.create(vertx,
new WebClientOptions()
.setSsl(true)
.setTrustAll(true)
.setDefaultPort(443)
.setKeepAlive(true)
.setDefaultHost("www.w3schools.com")
);
client.get("www.w3schools.com")
.as(BodyCodec.string())
.send(ar -> {
if (ar.succeeded()) {
HttpResponse<String> response = ar.result();
System.out.println("Got HTTP response body");
System.out.println(response.body().toString());
} else {
ar.cause().printStackTrace();
}
});
}
}
Try using web client instead of httpclient, here you have an example (with rx):
private val client: WebClient = WebClient.create(vertx, WebClientOptions()
.setSsl(true)
.setTrustAll(true)
.setDefaultPort(443)
.setKeepAlive(true)
)
open fun <T> get(uri: String, marshaller: Class<T>): Single<T> {
return client.getAbs(host + uri).rxSend()
.map { extractJson(it, uri, marshaller) }
}
Another option is to use getAbs.
I have a load server application which switches between two applications, App1 & App2. Each App has its own IFrame which talks to a bank, but when the bank returns back data, the load server cannot distinguish which app was that. I investigated on that, then I found out that it is the proxy issue.
As a result, I tried to force the application to use proxyHost & proxyport but it did n't work, then I forced the Jboss to use the specific proxyHost & proxyport but it did n't work either. It use a default proxy on the box and it does n't care about my configuration.
Setting the proxyHost & proxyPort at the Jboss startup script (bin\run.bat):
the Jboss logs showed that it had accepted the java VM arguments (e.g. -Dhttp.proxyHost=XX.XX.XX.XX -Dhttp.proxyPort=8080), when we ran
the application, it would still go through the default proxy settings on the box.
I searched in google & stackoverflow and tried different ideas but didn't work.
Could someone please shed some light on it?
Could you try running your apps with the following JVM args: -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0?
The DNS settings get cached after a successful request, these parameters will disable the caching.
You can force an application to ignore any VM arguments for proxyHost and
just use the default proxy of the box it’s running on. You can write a small java program which just prints the default proxy and ran it on the specific box (like this one):
import java.net.InetSocketAddress;
import java.net.ProxySelector;
import java.net.URISyntaxException;
import java.util.Iterator;
import java.util.List;
import java.net.URI;
public class PrintDefaultProxy {
public static void main(String[] args) {
// If you clear these 2 properties then set java.net.useSystemProxies to true, it
// will use the default System Proxy and ignore any settings given to the VM
// e.g. http.proxyHost & http.proxyPort
System.setProperty("http.proxyHost", "");
System.setProperty("http.proxyPort", "");
System.setProperty("java.net.useSystemProxies", "true");
System.out.println("detecting proxies");
List l = null;
try {
String url = "http://google.com.au/";
l = ProxySelector.getDefault().select(new URI(url));
}
catch (URISyntaxException e) {
e.printStackTrace();
}
if (l != null) {
for (Iterator iter = l.iterator(); iter.hasNext();) {
java.net.Proxy proxy = (java.net.Proxy) iter.next();
System.out.println("proxy Type : " + proxy.type());
InetSocketAddress addr = (InetSocketAddress) proxy.address();
if (addr == null) {
System.out.println("No Proxy");
} else {
System.out.println("proxy hostname : " + addr.getHostName());
System.setProperty("http.proxyHost", addr.getHostName());
System.out.println("proxy port : " + addr.getPort());
System.setProperty("http.proxyPort", Integer.toString(addr.getPort()));
}
}
}
}
}
Basically in the code, if you clear the http.proxyHost and http.proxyPort and then set java.net.useSystemProxies to true, it will use
the system default proxy (if any) and ignore any VM arguments.
System.setProperty("http.proxyHost", "");
System.setProperty("http.proxyPort", "");
System.setProperty("java.net.useSystemProxies", "true");
System.out.println("detecting proxies");
Then you run this on your box and passed it some bogus proxy addresss:
C:\t>"C:\Program Files\Java\jdk1.6.0_19\bin\java.exe" -Dhttp.proxyHost=99.0.0.99.9 -Dhttp.proxyPort=8080 PrintDefaultProxy
detecting proxies
proxy Type : DIRECT
No Proxy
Note, if you do not clear those 2 properties, it will use the arguments you pass to the JVM – but from my understanding it’s not what your application seems to be doing. This is a way that the application should ‘just work’ without having to specifically set the proxyHost – and this is most likely why it ignores whatever setting you provide it at the JVM/Jboss level.
When the behaviour is consistent with what you’ve experienced trying to change those settings in Jboss, it means you can’t configure it in an application or Jboss level to use a proxyHost and most likely needs to be done at a Network level.
I have Gui Application written which running on windows,and i want to connect to remote unix machine and perform actions there such like API's ,go over the log file in the machines and send back to the application the last log file or others API that i want to perform on the remote machine.
In the remote machine i don;t have application server i just have Java which installed there.
I want to use Java in order to perform remote API over the remote machine;
what is the advice ,can i use web services ,can any one please advise.
Thanks in advance.
If Java can perform the actions you're talking about, I would use Sockets to communicate with the UNIX-Machine (over TCP/IP).
Your Windows-PC would be the client sending commands to the Unix-PC.
Web services would be a bit heavy handed option, esp if you opt for the SOAP ones. If you don't have a problem with the client and server always being Java, RMI seems to be the simplest solution to this problem since it's communication between two different JVM's using the normal method calling mechanism (with some additional interfaces and rules to be followed to please the RMI specification).
The Spring Framework ships with a number of remoting options that are all very easy to setup. You can use their classes for simpler configuration of something standard like RMI or JMS, or use a lightweight web services protocol such as Spring's HTTP invoker or Hessian.
For analyzing log files of remote machines you can always use Apache Commons sftp programmatically to FTP a copy of the remote log file to your PC.
If you configure the log files to be rotatable or to rotate each time they reach a specific size, you can avoid reloading the same information over and over.
You can use Ganymed SSH-2 for Java to ssh to the remote host from Client Java App and run the commands. No need to run any additional components on remote server. You can do password based authentication or key based authentication to login to remote host. We had successfully used it to administer (start/stop/grep log files, etc.) applications running on remote UNIX hosts. You can capture output of the remote command using the StreamGobbler class provided in the package. You can pass multiple commands separated by semi-colon in one remote call.
Basic Example included in the package:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import ch.ethz.ssh2.Connection;
import ch.ethz.ssh2.Session;
import ch.ethz.ssh2.StreamGobbler;
public class Basic
{
public static void main(String[] args)
{
String hostname = "127.0.0.1";
String username = "joe";
String password = "joespass";
try
{
/* Create a connection instance */
Connection conn = new Connection(hostname);
/* Now connect */
conn.connect();
/* Authenticate.
* If you get an IOException saying something like
* "Authentication method password not supported by the server at this stage."
* then please check the FAQ.
*/
boolean isAuthenticated = conn.authenticateWithPassword(username, password);
if (isAuthenticated == false)
throw new IOException("Authentication failed.");
/* Create a session */
Session sess = conn.openSession();
sess.execCommand("uname -a && date && uptime && who");
System.out.println("Here is some information about the remote host:");
/*
* This basic example does not handle stderr, which is sometimes dangerous
* (please read the FAQ).
*/
InputStream stdout = new StreamGobbler(sess.getStdout());
BufferedReader br = new BufferedReader(new InputStreamReader(stdout));
while (true)
{
String line = br.readLine();
if (line == null)
break;
System.out.println(line);
}
/* Show exit status, if available (otherwise "null") */
System.out.println("ExitCode: " + sess.getExitStatus());
/* Close this session */
sess.close();
/* Close the connection */
conn.close();
}
catch (IOException e)
{
e.printStackTrace(System.err);
System.exit(2);
}
}
}
I am following the Jetty HttpClient Example, but I am unable to get an SSL connection working. When I connect using a proxy, it throws a "Not Implemented" exception. When I don't use a proxy, it doesn't return anything.
HttpClient client = new HttpClient();
client.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL);
client.setProxy(new Address("www.example.com", 80));
client.start();
// create the exchange object, which lets you define where you want to go
// and what you want to do once you get a response
ContentExchange exchange = new ContentExchange()
{
// define the callback method to process the response when you get it
// back
protected void onResponseComplete() throws IOException
{
super.onResponseComplete();
String responseContent = this.getResponseContent();
// do something with the response content
System.out.println(responseContent);
}
};
exchange.setMethod("GET");
exchange.setURL("https://www.example.com");
exchange.setScheme(HttpSchemes.HTTPS_BUFFER);
// start the exchange
client.send(exchange);
exchange.waitForDone();
System.err.println("Response status: " + exchange.getResponseStatus());
Jetty v7.4.1:
if (dest.isSecure()) {
if (dest.isProxied()) {
SSLEngine engine=newSslEngine(channel); ep = new ProxySelectChannelEndPoint(channel, selectSet, key, _sslBuffers, engine, (int)_httpClient.getIdleTimeout());
} else { ...
Yeah weird, the source code for the Jetty-Client's SelectConnector looks like the following:
if (dest.isProxied()) {
String connect = HttpMethods.CONNECT+" "+dest.getAddress()+HttpVersions.HTTP_1_0+"\r\n\r\n";
// TODO need to send this over channel unencrypted and setup endpoint to ignore the 200 OK response.
throw new IllegalStateException("Not Implemented");
}
so the functionality doesn't exist at present - at least in the version I'm using (6.1.16) for using a proxy in this kind of way. It's also the same in the milestone Jetty 7 version (I found after downloading the source code).
I suggest your try a different client - check out Apache HttpClient:
http://hc.apache.org/httpclient-3.x/
The Jetty developers should really have marked this clearly in the Javadocs. another alternative is to implementinghave a go at implementing the feature for them and submitting it back as a patch.
try ProxyHandler (jetty 7) which handle connect-command for tunneling https-connection (via proxy)