Develop Reference

Connect to remote instance of Elasticsearch through spring data elasticsearch using an URL

I'm very newbie to ES and I'm having a lot of troubles to connect to an external instance od ES through my java web app, let's say that I have the following client config
#Configuration
#ComponentScan(basePackages = {"com.elasticsearchlogs.elasticsearchlogsbackend"})
public class Config extends AbstractElasticsearchConfiguration {
#Value("${elasticsearch.url}")
public String elasticsearchUrl;
#Bean
#Override
public RestHighLevelClient elasticsearchClient() {
final ClientConfiguration config = ClientConfiguration.builder()
.connectedTo(elasticsearchUrl)
.build();
return RestClients.create(config).rest();
}
}
When I was making some tests with elastichsearchUrl = "localhost:9200" all was fine, but know i should connect to an ES instance that is in a external server, that is accesible, let suppose by the following url https://elasticinstance.com (yes it uses Https).
The problem is that if a put the url like in the example, java complains because it asks for a port number. I don't really know what port number have I to give it, I have tried with 9300 and 9200 because they are the default ports in ES, also with 443 because is the default port in Https.
What should I do? Shall I ask my admin to give me a port number to connect? Or shall I configure the client in any special way?

I think as you are trying to connect to external Elasticsearch server with Https, first
Go through, https://www.elastic.co/blog/configuring-ssl-tls-and-https-to-secure-elasticsearch-kibana-beats-and-logstash, this tells you how to Create SSL certificates and enable TLS for Elasticsearch.
Then you can try and access Elasticsearch with https://[ip/domainname:port].com
Or to validate first if everything is OK with current setup
Just access the server only with HTTP instead of HTTPS.

Upload files to AWS S3 using Spring Boot works fine without Proxy but fails with Proxy

Uploading files to AWS S3 using spring boot works great when it executed without proxy and when I add proxy in the VM args it fails with following error,
Internal Server Error (Service: Amazon S3; Status Code: 500; Error Code: 500 Internal Server Error; Request ID: null; S3 Extended Request ID: null; Proxy: 192.168.1.171)
Below are the vm arguments that I have provided,
-Dhttp.proxyHost=192.168.1.171 -Dhttp.proxyPort=9999 -Dhttps.proxyHost=192.168.1.171 -Dhttps.proxyPort=9999
When I started to execute the package the AWS SDK auto initialize the proxy as it finds in the args list
and it prints in the console
com.amazonaws.http.AmazonHttpClient - Configuring Proxy. Proxy Host: 192.168.1.171 Proxy Port: 9999
I can not remove the proxy because I am using Oauth2 authentication in spring security.
Is there any way that I can disable auto initializing the proxy in AWS SDK?

In effect to communicate over a network "endpoints" are an address, but links require "connections" so it may be more of a connection/connected "host conflict" by configuration.
For example i could have both a webserver and a DB that takes requests via http direct but it depends if they are configured for each other and/or to be connected through an irq to network to anything else.

I was able to resolve the issue after contacting AWS, I had to configure the method “setNoneProxyHosts (String nonProxyHosts)” method in the ClientConfiguration class. This sets the optional hosts the client will access without going through the proxy.
The parameter – “nonProxyHosts” takes the hosts the client will access without going through the proxy. Please follow this link here for more information about the class and the method: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/ClientConfiguration.html#setNonProxyHosts-java.lang.String-
Based on the instructions provided by AWS Support I had amended the config and added following;
public ClientConfiguration clientConfiguration() {
final ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setNonProxyHosts("*.s3.<ZONE>.amazonaws.com|*.s3-<ZONE>.amazonaws.com");
return clientConfiguration;
}

Using Proxy server for Java application

So i have a java app that uses Google Analytics API to gather some info. I am putting this application to run in my oracle cloud managed server which has a firewall and blocks any web calls to work. So, they setup a proxy for me to use....I've never set up a proxy to work with a java application before, I've been reading at tutorials like this: http://docs.oracle.com/javase/1.5.0/docs/guide/net/proxies.html
And i have no idea how to set this up...anyone want to point me in the right direction?

You must tell your application that there's a proxy somewhere.
As the documentation says, you must set some properties in your virtual machine. You can do it programatically:
//Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setPropery("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.clearProperty("http.proxyHost");
// From now on http connections will be done directly.
Or use https.proxy... if the connection is HTTPS.
Besides, if you have access to the application server start script, you could add those properties as VM properties with -Dhttp.proxyHost....

The solution in my case was to configure the JVM with a HTTPS proxy:
System.setProperty("https.proxyHost", "proxy");
System.setProperty("https.proxyPort", "3128");

How to get a response from localhost server in GWT client side?

I have a WCF based web service and hosted on local machine written in c#. I want to get a response from this hosted service in my GWT client side.
Please check the following code:
url = "localhost:8089/request"
RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, url);
Request response = builder.sendRequest(null, new RequestCallback() {
#override
public void onResponseReceived(Request request, Response response) {
Window.alert(response + " ");
}
});
Everytime when i am trying to run the code it shows an "http://localhost:8089 is not allowed by Access-Control-Allow-Origin."
PS: i searched for this error but i don't want to disable the web security of all browsers.
Is there any alternative solution to do get response of localhost server running on different porn on same machine. That is why i want to call a url in client side.
Please suggest a solution.
Edit
Let me explain you full scenario in points:
GWT application is hosted on some server (www.abc.com)
WCF is a web service installed at each client.
A client open his/her browser and put the url (www.abc.com/page)
This page want to access the web services hosted on a client machine.

Can't you configure your WCF service to send an Access-Control-Allow-Origin: www.abc.com response header?
Otherwise, I guess you could use a proxy servlet in www.abc.com that proxies the call to the getRemoteAddr, assuming there's no proxy in-between.

SOAP connections through a proxy using URLEndpoint

I've had to update a previous java application that requests a SOAP response from an external web service. This service is outside our firewall which now requires us to go through a proxy instead of hitting the URL directly.
Currently the Java App uses URLEndpoint that takes a string for the URL. Usually when I am getting to a URL through a proxy I create a URL like so:
URL url = new URL("http", "theproxy.com", 5555, finalUrl);
The problem is URLEndpoint only takes a string for the url, I tried to convert URL to string using toExternalForm() but it malformed the URL.
Any ideas as to a way around this?
EDIT: I can't use System.setProperty as this runs with a whole heap of other Java applications in tomcat.
second edit: I can't set a system properties as it will override all other applications running on the server, I can't use jsocks as the proxy we run through squid proxy which does not support socks4/5
Any help appreciated.

That's not how proxy's work. The way a proxy works is that you take your normal URL:
http://example.com/service
and instead of looking up "example.com" and port 80, the message is sent to your proxy host instead (http://theproxy.com:5555).
Java has built in proxy handling using http.proxyHost and http.proxyPort System properties.
So in your case you would need to do:
System.setProperty("http.proxyHost", "theproxy.com");
System.setProperty("http.proxyPort", "5555");
Then your code should, ideally, "Just Work".
Here is a page documenting the proxy properties.

Use Apache HttpClient and do as show in this example.

About the URL constructor with individual proxy setting:
http://edn.embarcadero.com/article/29783
(sorry don't have privileges to comment)