Develop Reference

mTLS/TLS Redis 6 Issues Java

Currently I am running Redis 6 with ACL and mTLS with a C# client just fine. I am trying to update our Java side to also use ACL and mTLS but have been running into issues. I am primarily focused on mTLS at the moment and have not been getting anywhere with it. This could be user fault in these that I have not used Java for 5-6 years before attempting to do this, so please advise. Not sure what or how to really progress from this error and I have done google searches with not success really. Any help greatly appreciated, again I have not done Java in a long time so that most likely might be the issue.
Trace:
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to localhost:6379
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78)
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56)
at io.lettuce.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:295)
at io.lettuce.core.RedisClient.connect(RedisClient.java:214)
at io.lettuce.core.RedisClient.connect(RedisClient.java:199)
at blah blah blah my code....
... 48 more
Caused by: javax.net.ssl.SSLException: SSLEngine closed already
at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:834)
at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:797)
at io.netty.handler.ssl.SslHandler.handleUnwrapThrowable(SslHandler.java:1254)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1230)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
... 2 more
Redis Server Debug logs:
1:M 30 Jul 2020 15:23:10.837 - Accepted 10.0.2.2:62023
1:M 30 Jul 2020 15:23:11.024 # Error accepting a client connection: (null)
Java code:
final RedisClient client = RedisClient.create(RedisURI.Builder.redis(hostConfig,portConfig)
.withSsl(true).withVerifyPeer(false).build().toURI().toString());
if (redisTruststorePath != null && !redisTruststorePath.isEmpty()) {
SslOptions sslOptions;
if (redisKeystorePath != null && !redisKeystorePath.isEmpty()) {
sslOptions = SslOptions.builder()
.jdkSslProvider()
.keystore(new File(redisKeystorePath), redisKeystorePass)
.truststore(new File(redisTruststorePath), redisTruststorePass)
.build();
}
else {
sslOptions = SslOptions.builder()
.jdkSslProvider()
.truststore(new File(redisTruststorePath), redisTruststorePass)
.build();
}
client.setOptions(ClientOptions.builder().sslOptions(sslOptions).build());
}
client.connect();
Versions:
Lettuce version(s): 6.0.0.M1 (Running on windows locally)
Redis version: 6.0.5 (Running on linux VM locally)
Notes:
C# client is working fine so doubt its a Redis Server issue.
Redis URI (printed in my real code before set): rediss://localhost:6379

Please check your client-side logs.
16797:M 03 Aug 2020 09:11:11.246 # Error accepting a client connection: (null)
This message above happens when Redis wasn't able to continue with the connection phase. Such a message occurs in SSL arrangements when the SSL handshake wasn't completed successfully, e.g. caused by a failed certificate validation.
Looking at the code above, the client gets created with:
RedisClient.create(RedisURI.Builder.redis(hostConfig,portConfig) .withSsl(true).withVerifyPeer(false).build().toURI().toString());
The RedisURI object gets converted into a string which causes a loss of the verifyPeer flag.
Please change your code to:
RedisClient.create(RedisURI.Builder.redis(hostConfig,portConfig) .withSsl(true).withVerifyPeer(false).build());
by removing .toURI().toString().

As #mp911de mentioned I removed .toURI().toString(); as well as, updated to lettuce-core 6.0.0.RC and started using RESP2 (as suggested here). This resolved my problem. I think the main solution here was switching to RESP2, which again was a suggestion from #mp911de. Thank you for the assistance #mp911de!!

java.net.UnknownHostException: <<hostname>> : unknown error

I'm trying to establish Oracle DB connection from the application which runs on an EC2 instance of AWS. Oracle DB is in on-prem server. Firewall has been opened and I'm able to telnet to the SCAN and VIP hosts of that DB from my EC2 instance. But, still I'm getting the below exception:
Caused by: oracle.net.ns.NetException: The Network Adapter could not establish the connection
at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:392) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:434) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:687) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:343) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1102) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:320) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
... 239 common frames omitted
Caused by: java.net.UnknownHostException: <<hostname>>: unknown error
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) ~[na:1.8.0_40-internal]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:907) ~[na:1.8.0_40-internal]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1302) ~[na:1.8.0_40-internal]
at java.net.InetAddress.getAllByName0(InetAddress.java:1255) ~[na:1.8.0_40-internal]
at java.net.InetAddress.getAllByName(InetAddress.java:1171) ~[na:1.8.0_40-internal]
at java.net.InetAddress.getAllByName(InetAddress.java:1105) ~[na:1.8.0_40-internal]
at oracle.net.nt.TcpNTAdapter.connect(TcpNTAdapter.java:117) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
at oracle.net.nt.ConnOption.connect(ConnOption.java:133) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:370) ~[ojdbc6-11.2.0.3.jar:11.2.0.3.0]
JDBC URL:
jdbc:oracle:thin:#(DESCRIPTION =(ADDRESS_LIST =(ADDRESS = (PROTOCOL = TCP)
(HOST = <<IP address of the host>>)(PORT = 1590)))(LOAD_BALANCE = yes)
(CONNECT_DATA =(SERVICE_NAME = <<example.service.com>>)(FAILOVER_MODE =(TYPE = SELECT)
(METHOD = BASIC))))`

Are you using java 8 ? If so then probably it's related to this bug
Java 7 or 9 would probably give you a more useful error message instead of "unknown error" (eg possibly "Name or service not known")
Apart from that, did you try a tnsping from the host you're trying to connect from ?
Also as per the below in the Oracle driver documentation when using a tnsnames entry in the jdbc URL it should be as below, using OCI driver :
Note that you can also specify the database by a TNSNAMES entry. You can find the available TNSNAMES entries listed in the file tnsnames.ora on the client computer from which you are connecting. For example, if you want to connect to the database on host myhost as user scott with password tiger that has a TNSNAMES entry of MyHostString, enter:
Connection conn = DriverManager.getConnection
("jdbc:oracle:oci8:#MyHostString","scott","tiger");

GemFire with GFSH - Region creation with server start throws BindException

I am trying to create a GemFire cache and region at the time of starting a cache server using GFSH command.
GFSH:
gfsh start server --name=server1 --server-port=40405 --classpath=$CLASSPATH --cache-xml=/tmp/gemfire/8.2.7/config/cache.xml --locators=hostA[10334],hostB[10334] --mcast-port=0
Cache.xml:
<?xml version="1.0" encoding="UTF-8"?><cache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schema.pivotal.io/gemfire/cache" xsi:schemaLocation="http://schema.pivotal.io/gemfire/cache http://schema.pivotal.io/gemfire/cache/cache-8.1.xsd" version="8.1" lock-lease="120" lock-timeout="60" search-timeout="300" is-server="false" copy-on-read="false">
<pdx>
<pdx-serializer>
<class-name>
com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer
</class-name>
<parameter name="classes">
<string>com.gemfire.DomainObjects</string>
</parameter>
</pdx-serializer>
</pdx>
<region name="Customer" refid="REPLICATE"></region>
</cache>
When I run the gfsh command, I am getting below exception.
Caused by: java.net.BindException: Failed to create server socket on null[40,405]
at com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:828)
at com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:758)
at com.gemstone.gemfire.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:466)
at com.gemstone.gemfire.internal.cache.BridgeServerImpl.start(BridgeServerImpl.java:342)
at com.gemstone.gemfire.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:607)
... 11 more
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
at java.net.ServerSocket.bind(ServerSocket.java:375)
at com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:825)
... 15 more
Any help?
I used netstat to find if the port is occupied. I couldnt find any and i executed the script still i am getting same exception.
bash-4.1$ netstat | grep 40405
bash-4.1$ ./startServer.sh
.......................................................................................................................................................................................................................................................The Cache Server process terminated unexpectedly with exit status 1. Please refer to the log file in /var/tmp/sn17180/gemfire/8.2.7/config/server1 for full details.
[severe 2018/05/06 04:37:53.156 IST libgemfire.so nid=0x17921700] SIGQUIT received, dumping threads
java.io.EOFException: Locator at hostB(server1:18877)<v73>:10334 did not respond. This is normal if the locator was shutdown. If it wasn't check its log for exceptions.
at com.gemstone.org.jgroups.stack.tcpserver.TcpClient.requestToServer(TcpClient.java:125)
at com.gemstone.org.jgroups.stack.tcpserver.TcpClient.requestToServer(TcpClient.java:78)
at com.gemstone.gemfire.internal.cache.ClusterConfigurationLoader.requestConfigurationFromLocators(ClusterConfigurationLoader.java:171)
at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.requestAndApplySharedConfiguration(GemFireCacheImpl.java:874)
at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.init(GemFireCacheImpl.java:1025)
at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:688)
at com.gemstone.gemfire.cache.CacheFactory.create(CacheFactory.java:182)
at com.gemstone.gemfire.cache.CacheFactory.create(CacheFactory.java:229)
at com.gemstone.gemfire.distributed.ServerLauncher.startWithGemFireApi(ServerLauncher.java:793)
at com.gemstone.gemfire.distributed.ServerLauncher.start(ServerLauncher.java:695)
at com.gemstone.gemfire.distributed.ServerLauncher.run(ServerLauncher.java:625)
at com.gemstone.gemfire.distributed.ServerLauncher.main(ServerLauncher.java:195)
Exception in thread "main" com.gemstone.gemfire.GemFireIOException: While starting bridge server CacheServer on port=40405 client subscription config policy=none client subscription config capacity=1 client subscription config overflow directory=.
at com.gemstone.gemfire.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:611)
at com.gemstone.gemfire.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:340)
at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4269)
at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.initializeDeclarativeCache(GemFireCacheImpl.java:1184)
at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.init(GemFireCacheImpl.java:1026)
at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:688)
at com.gemstone.gemfire.cache.CacheFactory.create(CacheFactory.java:182)
at com.gemstone.gemfire.cache.CacheFactory.create(CacheFactory.java:229)
at com.gemstone.gemfire.distributed.ServerLauncher.startWithGemFireApi(ServerLauncher.java:793)
at com.gemstone.gemfire.distributed.ServerLauncher.start(ServerLauncher.java:695)
at com.gemstone.gemfire.distributed.ServerLauncher.run(ServerLauncher.java:625)
at com.gemstone.gemfire.distributed.ServerLauncher.main(ServerLauncher.java:195)
Caused by: java.net.BindException: Failed to create server socket on null[40,405]

This means some other process running on localhost is already bound and listening to client socket connections on port 40405. Perhaps you have another GemFire Server already running on that port?
In which case, you need to vary the port numbers for as many servers as you start, for example...
gfsh> start server --name=One --server-port=40405 ...
...
gfsh> start server --name=Two --server-port=40406 ...
...
gfsh> start server --name=Three --server-port=40407 ...
...
... and so and so forth.
If a non-GemFire process is already running using that port number than either considering killing that process or using another port for your GemFire CacheServers.
You can use the Linux netstat command to find which process is using the port and act appropriately.

Partner not reached error in JCO.destination

I'm Having a problem establishing a connection to SAP in my java program.
I'm following the examples that come in the JCO download but i always get this error:
com.sap.conn.jco.JCoException: (102) RFC_ERROR_COMMUNICATION: Connect to SAP gateway failed
Connection parameters: TYPE=A DEST=ABAP_AS_WITHOUT_POOL ASHOST=xx.xx.x.xx SYSNR=00 PCS=1
LOCATION CPIC (TCP/IP) on local host with Unicode
ERROR partner 'xx.xx.x.xx:3300' not reached
TIME Wed Jul 08 08:18:28 2015
RELEASE 711
COMPONENT NI (network interface)
VERSION 39
RC -10
MODULE nixxi.cpp
LINE 3147
DETAIL NiPConnect2: xx.xx.x.xx:3300
SYSTEM CALL connect
ERRNO 10060
ERRNO TEXT WSAETIMEDOUT: Connection timed out
COUNTER 2
I don't know what it can be, i'm writing down the correct connection properties (ashost,user,passwd,sysnr,etc).
Has anybody else has had a problem like this?
This is my connection code:
Properties connectProperties = new Properties();
connectProperties.setProperty(DestinationDataProvider.JCO_ASHOST, "xx.xx.x.xx");
connectProperties.setProperty(DestinationDataProvider.JCO_SYSNR, "00");
connectProperties.setProperty(DestinationDataProvider.JCO_CLIENT, "020");
connectProperties.setProperty(DestinationDataProvider.JCO_USER, "xxxxxx");
connectProperties.setProperty(DestinationDataProvider.JCO_PASSWD, "xxxxxxx");
connectProperties.setProperty(DestinationDataProvider.JCO_LANG, "en");
createDataFile(ABAP_AS, "jcoDestination", connectProperties);
After that i just instantiate the object with those properties and call the method connect that is written like this:
JCoDestination destination = JCoDestinationManager.getDestination(ABAP_AS);
System.out.println("Attributes:");
System.out.println(destination.getAttributes());
System.out.println();
I'm working on Java, using netbeans, the sapjco3.jar is added to my libraries.
Do i have to do anything with the dll file that comes?

Cannot programmatically submit Spark application (with Cassandra connector) to cluster from remote client

I'm running a standalone Spark cluster on EC2, and I'm writing a application using Spark-Cassandra connector driver and try to submit job to Spark cluster programmatically.
The job itself is simple:
public static void main(String[] args) {
SparkConf conf;
JavaSparkContext sc;
conf = new SparkConf()
.set("spark.cassandra.connection.host", host);
conf.set("spark.driver.host", "[my_public_ip]");
conf.set("spark.driver.port", "15000");
sc = new JavaSparkContext("spark://[spark_master_host]","test",conf);
CassandraJavaRDD<CassandraRow> rdd = javaFunctions(sc).cassandraTable(
"keyspace", "table");
System.out.println(rdd.first().toString());
sc.stop();
}
Which runs fine when I run that in the Spark Master node of my EC2 cluster.
I'm trying to running this in a remote Windows client.
The problem was from these two lines:
conf.set("spark.driver.host", "[my_public_ip]");
conf.set("spark.driver.port", "15000");
First, if i comment out these 2 lines, application would not throw a exception, but the Executor is not running, with following log:
14/12/06 22:40:03 INFO client.AppClient$ClientActor: Executor updated: app-20141207033931-0021/3 is now LOADING
14/12/06 22:40:03 INFO client.AppClient$ClientActor: Executor updated: app-20141207033931-0021/0 is now EXITED (Command exited with code 1)
14/12/06 22:40:03 INFO cluster.SparkDeploySchedulerBackend: Executor app-20141207033931-0021/0 removed: Command exited with code 1
Which never ends, when I check the worker node log, I found:
14/12/06 22:40:21 ERROR security.UserGroupInformation: PriviledgedActionException as:[username] cause:java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException: Unknown exception in doAs
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1134)
at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:52)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:113)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:156)
at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
Caused by: java.security.PrivilegedActionException: java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
... 4 more
Caused by: java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:107)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:125)
at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:53)
at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:52)
... 7 more
I've no idea what that's about, my guess is that probably worker node could not connect to driver, which probably initially set as:
14/12/06 22:39:30 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver#[some_host_name]:52660]
14/12/06 22:39:30 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkDriver#[some_host_name]:52660]
Obviously, no DNS is going to resolve my host name...
Since I can't set deploy mode to "client" or "cluster", if not via ./spark-submit script.(Which I think that's absurd...). I try to add a host resolution "XX.XXX.XXX.XX [host-name]" in /etc/hosts of all Spark Master Worker nodes.
No luck of course...
That leads me to the second, un-comment that two line;
Which gives me:
14/12/06 22:59:41 INFO Remoting: Starting remoting
14/12/06 22:59:41 ERROR Remoting: Remoting error: [Startup failed] [
akka.remote.RemoteTransportException: Startup failed
at akka.remote.Remoting.akka$remote$Remoting$$notifyError(Remoting.scala:129)
at akka.remote.Remoting.start(Remoting.scala:194)
...
Cause:
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /[my_public_ip]:15000
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:391)
at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:388)
I double checked my firewall setting and router setting, confirm that my firewall is diabled; and netstat -an to confirm port 15000 is not in use (in fact I tried to change to several available port, no luck); and I ping my public ip from both other machine and machine from my cluster, no problem.
Now I'm utterly screw up, I just run out of idea try to fix this. Any suggestions? Any help is appreciated!

Please check if 15000 is in your security group.