I am creating a Elasticsearch Connection with Java API. I am using TransportConnection and I need to set the timeout for the connection.
I haven't configured any property and the connect takes three minutes to give me a timeout.
Anybody know if any property exists to set the value of timeout?
Thaks.
Settings settings = ImmutableSettings.settingsBuilder()
.put("cluster.name", cluster_name).put("client.transport.ping_timeout", "30s").build();
TransportClient transport = new TransportClient(settings);
See also the following from ES documentation page:
Fault Detection
Use the code below to update the TransportClient's connection time out value:
Settings.builder().put("transport.tcp.connect_timeout", "240s")
The Complete TransportClient code:
Settings settings = Settings.builder()
.put("cluster.name", "elasticsearch")
.put("client.transport.sniff", true)
.put("transport.tcp.connect_timeout", "240s")
.build();
Client transportClient = new PreBuiltTransportClient(settings)
.addTransportAddresses(
new TransportAddress("127.0.0.1"), "9300"));
Important: Each Elasticsearch version has different config key. You can read this document to learn about other settings you can change:
www.elastic.co/guide/en/elasticsearch/reference/6.4/modules-transport.html
If you are using AWS Elasticsearch, check the load balancer time out settings.
docs.aws.amazon.com/elasticloadbalancing/latest/classic/config-idle-timeout.html
Related
I am maintaining a JSP/Servlet application that uses the MongoDB 3.8 Java driver. At this point, I cannot change versions to a newer one.
I have occasionally experienced some timeouts when connecting from the application to the database. Based on what I read in https://mongodb.github.io/mongo-java-driver/3.8/driver/tutorials/connect-to-mongodb/ I wrote the following code:
CodecRegistry pojoCodecRegistry = fromRegistries(com.mongodb.MongoClient.getDefaultCodecRegistry(),
fromProviders(PojoCodecProvider.builder().automatic(true).build()));
MongoCredential credential
= MongoCredential.createCredential(theuser, database, thepassword.toCharArray());
MongoClientSettings settings = MongoClientSettings.builder()
.credential(credential)
.codecRegistry(pojoCodecRegistry)
.applyToClusterSettings(builder
-> builder.hosts(Arrays.asList(new ServerAddress(addr, 27017))))
.build();
MongoClient client = MongoClients.create(settings);
This works, but with the eventual timeout (usually when reloading a JSP page).
I figured out I could create a SocketSettings instance with:
SocketSettings socketOptions = SocketSettings.builder().connectTimeout(60,TimeUnit.SECONDS).build();
But I cannot figure out how to apply these settings to the creation of the instance of MongoClient. Any hints?
thanks
I try to connect to Elasticsearch with Searchguard from Spring Boot app.
I create a bean for TransportClient.
It looks like that:
Settings settings = Settings.builder()
.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_TYPE, "PKCS12")
.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_FILEPATH, keyStore)
.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_KEYSTORE_PASSWORD, keyPassword)
.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_TRUSTSTORE_FILEPATH, trustStore)
.put("cluster.name", clusterName)
.build();
TransportClient client = new PreBuiltTransportClient(settings, SearchGuardPlugin.class);
TransportAddress[] addresses = clusterNodes.stream()
.map(node -> {
String[] url = StringUtils.deleteWhitespace(node).split(":");
return new TransportAddress(new InetSocketAddress(url[0], Integer.parseInt(url[1])));
}).toArray(TransportAddress[]::new);
client.addTransportAddresses(addresses);
I have my repository extended ElasticsearchRepository.
But I receive a strange exception when my app starts:
ERROR com.floragunn.searchguard.ssl.transport.SearchGuardSSLNettyTransport - SSL Problem error:10000410:SSL routines:OPENSSL_internal:SSLV3_ALERT_HANDSHAKE_FAILURE
javax.net.ssl.SSLHandshakeException: error:10000410:SSL routines:OPENSSL_internal:SSLV3_ALERT_HANDSHAKE_FAILURE
What can be a reason? What part of code should I check?
I have another app which uses ElasticsearchTemplate directly (only SearchQuery). And there I don't have any problems.
Elasticsearch version: 6.4.3
Fixed, when I set searchguard.ssl.transport.enable_openssl_if_available to false:
...
.put(SSLConfigConstants.SEARCHGUARD_SSL_TRANSPORT_ENABLE_OPENSSL, false)
...
I have some troubles with the usage of java async driver (3.8.1).
I'll describe my environment:
I have a replica set (rs0) with 3 istances: let me call them A,B,C.
In my application I use Mongo and two different java driver, sync and async.
At the beginning I reached no problems but when the primary went down (and come up after some minutes changing its behavior as secondary) the part of code when I use async driver was not able to use transactions and session.
The error is the following:
com.mongodb.MongoClientException: Sessions are not supported by the MongoDB cluster to which this client is connected
at com.mongodb.async.client.MongoClientImpl$1.onResult(MongoClientImpl.java:90)
at com.mongodb.async.client.MongoClientImpl$1.onResult(MongoClientImpl.java:83)
at com.mongodb.async.client.ClientSessionHelper$2.onResult(ClientSessionHelper.java:77)
at com.mongodb.async.client.ClientSessionHelper$2.onResult(ClientSessionHelper.java:73)
at com.mongodb.internal.connection.BaseCluster$ServerSelectionRequest.onResult(BaseCluster.java:433)
at com.mongodb.internal.connection.BaseCluster.handleServerSelectionRequest(BaseCluster.java:309)
at com.mongodb.internal.connection.BaseCluster.access$800(BaseCluster.java:65)
at com.mongodb.internal.connection.BaseCluster$WaitQueueHandler.run(BaseCluster.java:482)
at java.lang.Thread.run(Unknown Source)
2019-01-21 17:02:01.906 ERROR 17560 --- [271de4498944329] org.mongodb.driver.client : Callback onResult call produced an error
java.lang.NullPointerException: null
at it.mypackage.mongo.service.ProcessoDocumentService$1.onResult(ProcessoDocumentService.java:124)
at it.mypackage.mongo.service.ProcessoDocumentService$1.onResult(ProcessoDocumentService.java:1)
at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:49)
at com.mongodb.async.client.MongoClientImpl$1.onResult(MongoClientImpl.java:90)
at com.mongodb.async.client.MongoClientImpl$1.onResult(MongoClientImpl.java:83)
at com.mongodb.async.client.ClientSessionHelper$2.onResult(ClientSessionHelper.java:77)
at com.mongodb.async.client.ClientSessionHelper$2.onResult(ClientSessionHelper.java:73)
at com.mongodb.internal.connection.BaseCluster$ServerSelectionRequest.onResult(BaseCluster.java:433)
at com.mongodb.internal.connection.BaseCluster.handleServerSelectionRequest(BaseCluster.java:309)
at com.mongodb.internal.connection.BaseCluster.access$800(BaseCluster.java:65)
at com.mongodb.internal.connection.BaseCluster$WaitQueueHandler.run(BaseCluster.java:482)
at java.lang.Thread.run(Unknown Source)
Just FYI, if I comment the part of code when I use session and transactions, the error is a classic timeout, as the driver was not longer able to find replica set anymore.
Someone could help me? What I'm missing?
This is how I create my MongoClient:
connectionString = new ConnectionString("mongodb://address1:27017,address2:27018,address3:27019/?replicaSet=rs0");
MongoClientSettings settings = MongoClientSettings.builder().applyConnectionString(connectionString)
.build();
settings = settings.builder().credential(credential).build();
asyncMongoClientInstance = MongoClients.create(settings);
I found the solution by myself, as the wise man once said: "If you want an help, find it at the end of your arm".
Let's us focus on this part of code:
connectionString = new connectionString("mongodb://address1:27017,address2:27018,address3:27019/?replicaSet=rs0");
MongoClientSettings settings = MongoClientSettings.builder().applyConnectionString(connectionString)
.build();
settings = settings.builder().credential(credential).build();
asyncMongoClientInstance = MongoClients.create(settings);
I'm reallocating the settings object to another object without the connection string.
So, async library doesn't know anymore where to address the connection.
Why I did that? I wanted to dinamically add credentials to the settings. But is not possible in this way. So I created two different settings object, one with credentials and one without.
MongoClientSettings settings = MongoClientSettings.builder().applyConnectionString(connectionString).credential(credential).build();
It definitely works with this object now.
With the deprecation of SearchTye.SCAN and the newly Reindex API we want to migrate our elasticsearch cluster and clients from 2.1.1 to 2.3.3.
We use java and the appropiate libraries to access elasticsearch. To access the cluster we use the TransportClient, for embedded Unittests we use the NodeClient.
Unfortunatly the Reindex API is provided as plugin, which the NodeClient seems to be unable to deal with.
So the question is how to use the NodeClient with the Reindex-Plugin?
I already tried exposing the protected NodeClient constructor to pass the ReindexPlugin class as argument without success.
Using the NodeClient to start an embedded ElasticSearch and using the TransportClient with the added ReindexPlugin didn't worked either. All I get here is an Exception: ActionNotFoundTransportException[No handler for action [indices:data/write/reindex]]
Dependencies of interest:
org.elasticsearch:elasticsearch:2.3.3
org.elasticsearch.module:reindex:2.3.3
org.apache.lucene:lucene-expressions:5.5.1
org.codehaus.groovy:groovy:2.4.6
Starting the NodeClient:
Settings.Builder settings = Settings.settingsBuilder();
settings.put("path.data", "/some/path/data");
settings.put("path.home", "/some/path/home");
//settings.put("plugin.types", ReindexPlugin.class.getName()); > No effect
settings.put("http.port", 9299);
settings.put("transport.tcp.port", 9399);
node = NodeBuilder.nodeBuilder()
.clusterName("testcluster")
.settings(settings)
.local(true)
.node();
// also tested with local(false), then no transport port is available, resulting in NoNodeAvailableException
Using TransportClient to access the Node:
Settings settings = Settings.settingsBuilder()
.put("cluster.name", "testcluster")
.put("discovery.zen.ping.multicast.enabled", false)
.build();
InetSocketTransportAddress[] addresses = new InetSocketTransportAddress[]
{new InetSocketTransportAddress(new InetSocketAddress("localhost", 9399))};
client = TransportClient.builder()
.settings(settings)
.addPlugin(ReindexPlugin.class)
.build()
.addTransportAddresses(addresses);
Main part of triggering reindex:
ReindexRequestBuilder builder = ReindexAction.INSTANCE.newRequestBuilder(getClient())
.source(indexFrom)
.destination(indexTo)
.refresh(true);
I was able to solve this, by combining both approaches described above.
So creating the NodeClient involves overriding the Node:
class ExposedNode extends Node {
public ExposedNode(Environment tmpEnv, Version version, Collection<Class<? extends Plugin>> classpathPlugins) {
super(tmpEnv, version, classpathPlugins);
}
}
And using it when starting the NodeClient:
Settings.Builder settings = Settings.settingsBuilder();
settings.put("path.data", "/some/path/data");
settings.put("path.home", "/some/path/home");
settings.put("http.port", 9299);
settings.put("transport.tcp.port", 9399);
// Construct Node without NodeBuilder
List<Class<? extends Plugin>> classpathPlugins = ImmutableList.of(ReindexPlugin.class);
settings.put("node.local", false);
settings.put("cluster.name", "testcluster");
Settings preparedSettings = settings.build();
node = new ExposedNode(InternalSettingsPreparer.prepareEnvironment(preparedSettings, null), Version.CURRENT, classpathPlugins);
node.start();
After that you can use the TransportClient which adds the ReindexPlugin, as described in the question.
Nevertheless this is a dirty hack, which may break in a future release, and shows how poorly Elasticsearch supports plugin development imo.
We've got a Rails app using Resque to push jobs on the queue. The consumer of the jobs is a Java app using the Jesque client. Both apps run on Heroku. What I can't figure out is how to use Jesque's ConfigBuilder class to populate the redis connection parameters from Heroku's REDISTOGO_URL config var. The source documentation is pretty thin. Examples other than the default final Config config = new ConfigBuilder().build(); would be great.
I'm not sure how to do it with Jesque's ConfigBuilder but here is how you do it with a JedisPool:
URI redisURI = new URI(System.getenv("REDISTOGO_URL"));
JedisPool pool = new JedisPool(new JedisPoolConfig(),
redisURI.getHost(),
redisURI.getPort(),
Protocol.DEFAULT_TIMEOUT,
redisURI.getUserInfo().split(":",2)[1]);