Develop Reference

Calendar API calls begin to produce "SSL peer shut down"

I have a middleware (developped with Tibco ActiveMatrix BusinessWorks, but I don't think it's very important) that have been working for months, and suddently on 2020-10-17 started to fail on a high number of requests (around 8000) with the error:
javax.ws.rs.ProcessingException: javax.net.ssl.SSLException: SSL peer shut down incorrectly
Any idea why this error would suddently start occuring, and how to solve it ? Thanks in advance.
Full stacktrace:
Job-3158 Error in [BusinessProcesses/AgendasSecurises/Helpers/Gmail/Get Event.process/Get Event] Activity invocation failed at com.tibco.pe.plugin.Activity.postEval(Unknown Source) at com.tibco.pe.plugin.Activity.postEval(Unknown Source) at com.tibco.pe.core.TaskImpl.eval(Unknown Source) at com.tibco.pe.core.Job.a(Unknown Source) at com.tibco.pe.core.Job.k(Unknown Source) at com.tibco.pe.core.JobDispatcher$JobCourier.a(Unknown Source) at com.tibco.pe.core.JobDispatcher$JobCourier.run(Unknown Source) caused by: javax.ws.rs.ProcessingException: javax.net.ssl.SSLException: SSL peer shut down incorrectly at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287) at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252) at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:701) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:228) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444) at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:697) at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:420) at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:316) at com.tibco.plugin.json.activities.RestActivity.doGet(RestActivity.java:485) at com.tibco.plugin.json.activities.RestActivity.doService(RestActivity.java:473) at com.tibco.plugin.json.activities.RestActivity$1.run(RestActivity.java:418) at com.tibco.pe.util.ThreadPool$ThreadPoolThread.run(Unknown Source) Caused by: javax.net.ssl.SSLException: SSL peer shut down incorrectly at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:596) at sun.security.ssl.InputRecord.read(InputRecord.java:532) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:990) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1388) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1416) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1400) at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352) at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:399) at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285) ... 13 more

This is very likely that something changed on the server side on the mentioned date.
It could be a change of certificate or some changes that require use of a robust encryption library (for example TLS 1.2 might now be mandatory).
If you are using a recent version of BusinessWorks TLS 1.2 should be enabled by default. If you are using older versions (6.3.x) you may have to edit the bwappnode.tra file with the following :
java.extended.properties=-Xmx1024m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Dhttps.protocols=TLSv1.1,TLSv1.2
You can use the elements from the following TIBCO KB to get more trace and debug your issue:
https://support.tibco.com/s/article/Tibco-KnowledgeArticle-Article-44615

Cannot connect to Lotus Notes DIIOP on SSL port 63149

The company I work for creates software that crawls and indexes various sources for use in search, in this case we're completely stuck on attempting to connect to Lotus Notes over SSL. This is a classic case of "it worked in dev, but failed in prod".
The problem is further compounded by the fact that we're devs and techies, not Lotus Notes experts.
Basically we have a piece of Java software that connects to Lotus to grab the DIIOP IOR text file using this command:
String ior = NotesFactory.getIOR(host, userIOR, pwdIOR);
Now, this is a non-SSL connection and requires username and password to connect. This works perfectly every time.
Then we attempt to create a session using this:
session = NotesFactory.createSessionWithIOR(ior, args, this.user, this.pwd);
where:
args[0] = "-ORBEnableSSLSecurity";
When connecting we consistently receive this stack trace:
NotesException: Session closed due to communications failure
at lotus.domino.cso.ORBCallback.make_error(Unknown Source)
at lotus.priv.CORBA.iiop.Generic.make_error(Unknown Source)
at lotus.priv.CORBA.portable.ObjectImpl._invoke(Unknown Source)
at lotus.domino.corba._IObjectServerStub.createSession(Unknown Source)
at lotus.domino.cso.Session.initSession(Unknown Source)
at lotus.domino.cso.Session.<init>(Unknown Source)
at lotus.domino.cso.Session.createSession(Unknown Source)
at lotus.domino.NotesFactory.createSessionUP(Unknown Source)
at lotus.domino.NotesFactory.createSessionWithIOR(Unknown Source)
[ ... ]
Caused by: org.omg.CORBA.COMM_FAILURE: java.net.SocketException: Connection reset: Connection closed: Host: <redacted> Port: 63149 vmcid: 0x0 minor code: 1 completed: Maybe
at lotus.priv.CORBA.iiop.IIOPConnection.purge_calls(Unknown Source)
at lotus.priv.CORBA.iiop.ReaderThread.run(Unknown Source)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:154)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at lotus.priv.CORBA.iiop.Message.readFully(Unknown Source)
at lotus.priv.CORBA.iiop.Message.createFromStream(Unknown Source)
at lotus.priv.CORBA.iiop.IIOPInputStream.prefill(Unknown Source)
at lotus.priv.CORBA.iiop.IIOPConnection.createInputStream(Unknown Source)
... 1 more
I understand the most common reason for this is because we're not correctly importing the TrustedCerts.class file, however after double and triple checking it's in the class path just fine (and the fact it works in dev environment) we're confident that it's being loaded in.
We know if we disable SSL (aka remove the aforementioned arg) our code will connect to the non-SSL DIIOP port 63148 and work just fine however it is a project requirement that all data must be passed through SSL.
Which finally brings me to my actual question: How do we get Lotus to report why this connection is failing?
I've tried upping the DIIOP log level to 4 which apparently should show all transmissions but the logs show nothing when I attempt to connect. Also it's worth noting the DIIOP service is definitely running and on the correct ports as well.
I've also checked that we're making it through the firewall to the remote box and TCPDUMP on our RedHat machine clearly shows that we establish a connection with the remote Lotus Notes host before it responds closing the connection.
[Pertinent Information]
Application: Java
Library: NCSO.jar
Lotus Version: Domino 8.5.3 FP6
Lotus Server: Windows Server 2008 Standard
Ports: 63148 (non-SSL), and 63149 (SSL)

Alright, the answer was clear as crystal but not something I was actively looking for.
The reason this doesn't work is that the third party certificate is not a SHA-1 certificate and Lotus Notes 8.5.x only supports SHA-1 (as per this documentation).
So basically that certificate of theirs can never be used for SSL authentication programatically or otherwise.

You mentioned DIIOP logging, but if the connection is failing in the SSL handshake, then the guts of the diiop task probably don't know what's wrong since that's occurring at a lower level in the stack. There are, however, additional debug variables that can be set on the server.
DEBUG_SSL_HANDSHAKE=1
DEBUG_SSL_CIPHERS=1
DEBUG_SSL_ALL=1
The last setting can be pushed up to 2 or 3 for more detailed information.
Restart of the server (or possibly of the server's diiop task) may be required to get the settings to take effect.

JMeter: Remote host closed connection during handshake

I am running a JMeter test with "Java Request" sampler which performs POST request to HTTPS service which uses TLS v1.2. The request fails with
2015/12/01 19:11:50 ERROR - jmeter.protocol.java.sampler.AbstractJavaSamplerClient: 2015-12-01T16:11:50.189Z message:**** Exception in JMeter node for Login java.lang.RuntimeException: Failed to getServerToken
at com.chrysler.loadtest.GUIMimicLogin.getServerToken(GUIMimicLogin.java:227)
at com.chrysler.loadtest.GUIMimicLogin.runTest(GUIMimicLogin.java:97)
at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:191)
at org.apache.jmeter.threads.JMeterThread.process_sampler(JMeterThread.java:434)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:261)
at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:436)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at com.chrysler.loadtest.GUIMimicLogin.getServerToken(GUIMimicLogin.java:215)
... 5 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read(Unknown Source)
... 18 more
JMeter used to work with this server when it used TLS v1.1. Is there a setting in JMeter properties which can set the applicable protocol.
The JMeter version is 2.13, Java version is 1.7.

This is an SSL error and can mean several things. Make sure you have TLS 1.2 enabled in your application (JVM argument?)(-Dhttps.protocols=TLSv1.2). You can use openssl to check the connection and see if you are getting back a certificate. If you are then check your keystore and make sure you have the correct certificate. You may not be authorized to access that server.

Axis2 client is unable to initiate a second https call on Tomcat 7.0.42

I am trying to migrate an Java Axis2 client from Websphere6/JRE1.5.x to Tomcat 7.0.42/JRE1.6.x on Solaris. In the process, I have come across a issue where the client can communicate with a webservice through https the first time, but would fail from the second attempt onwards due to ssl errors.
Extracted from the log of the failed attempts showed -
%% Client cached [Session-1, SSL_RSA_WITH_RC4_128_MD5]
%% Try resuming [Session-1, SSL_RSA_WITH_RC4_128_MD5] from port 34092
** ClientHello, TLSv1
...
Thread-606, WRITE: TLSv1 Handshake, length = 179
Thread-606, READ: TLSv1 Alert, > length = 2
Thread-606, RECV TLSv1 ALERT: fatal, unexpected_message
and the actual exception was (I suspect it was due to no open socket)-
Caused by: javax.net.ssl.SSLException: Received fatal alert:
unexpected_message at
com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown
Source) at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown
Source) at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown
Source) at
com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(Unknown Source)
at java.io.BufferedOutputStream.flushBuffer(Unknown Source) at
java.io.BufferedOutputStream.write(Unknown Source) at
java.io.FilterOutputStream.write(Unknown Source) at
org.apache.commons.httpclient.WireLogOutputStream.write(WireLogOutputStream.java:86)
at
org.apache.axis2.transport.http.AxisRequestEntity.writeRequest(AxisRequestEntity.java:89)
Comparing this to the logs on the Websphere environment, I have found a couple of differences and hence my questions -
Is SSL configuration on the Tomcat going to affect the behaviour of a Axis2 client calling a different server? Or is my issue more at the OS level?
on the old environment, I have noticed more than one ssl sessions were cached. When resume failed on one session, it simply tried using a different session and continued with the call. On the new server, however, I have ever only seen one session created and was cached. When resuming, it kept trying to resume the same sessionn a different port but kept failing. As it was the only session, if couldn't try a different session nor creating a new session. My question is why is there only one session created and is it a configuration issue to enable caching of more than one sessions?
Any help/pointers are greatly appreciated.

Finally tracked down the fault. Thought I would share it here for anyone that may come across a similar issue.
As it turned out, it has to do with the Security Provider on the JRE itself. Here is a link on how to configure it -
http://pic.dhe.ibm.com/infocenter/ssp/v3r4/index.jsp?topic=%2Fcom.ibm.help.sspreleasenotes.doc%2FSSP_RN_RemovePKCS11SecurityProvider.html
It seemed the default security provider on Solaris was causing the weird behaviour I saw. Following the instruction in the link, and by using the Java default provider, resolved the ssl problem I saw.

Tomcat start failure probably due to Linux network setup: java.net.BindException: Cannot assign requested address [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
I can't start tomcat due to the pasted stack trace, and I don't believe the answer is in Java.
I've tried changing ports and confirmed that another port (8005) also fails with the same error message.
How do I verify that my linux network setup allows Java to use ports?
EDIT2: It appears this box has been the victim of an intrusion. I believe it makes sense to flag this post for moderator to close since a compromised box can't be expected to behave normally. Thank you everyone for your time and effort.
EDIT:
there is a possibility that corruption of packages in /bin is responsible. I had to update the a few with up2date.
EXTRA INFO:
Re: similar questions, I've eliminated the JVM_BIND possibility by checking netstat.
ping localhost simply hangs:
$ ping localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
-- no responses ---
--- localhost ping statistics ---
9 packets transmitted, 0 received, 100% packet loss, time 8000ms
ping google works:
$ ping google.com
PING google.com (74.125.225.228) 56(84) bytes of data.
-- responses removed ---
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 1.389/1.417/1.443/0.048 ms, pipe 2
the exception:
SEVERE: StandardServer.await: create[8036]:
java.net.BindException: Cannot assign requested address
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(Unknown Source)
at java.net.ServerSocket.bind(Unknown Source)
at java.net.ServerSocket.(Unknown Source)
at org.apache.catalina.core.StandardServer.await(StandardServer.java:373)
at org.apache.catalina.startup.Catalina.await(Catalina.java:662)
at org.apache.catalina.startup.Catalina.start(Catalina.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Jun 5, 2013 12:49:05 PM org.apache.coyote.http11.Http11Protocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8035
Jun 5, 2013 12:49:05 PM org.apache.catalina.connector.Connector pause
SEVERE: Protocol handler pause failed
java.net.SocketException: Invalid argument or cannot assign requested address
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.(Unknown Source)
at java.net.Socket.(Unknown Source)
at org.apache.jk.common.ChannelSocket.unLockSocket(ChannelSocket.java:492)
at org.apache.jk.common.ChannelSocket.pause(ChannelSocket.java:289)
at org.apache.jk.server.JkMain.pause(JkMain.java:725)
at org.apache.jk.server.JkCoyoteHandler.pause(JkCoyoteHandler.java:153)
at org.apache.catalina.connector.Connector.pause(Connector.java:1029)
at org.apache.catalina.core.StandardService.stop(StandardService.java:566)
at org.apache.catalina.core.StandardServer.stop(StandardServer.java:744)
at org.apache.catalina.startup.Catalina.stop(Catalina.java:648)
at org.apache.catalina.startup.Catalina$CatalinaShutdownHook.run(Catalina.java:692)

Seems to me that your /etc/host configuration is not correct. Are you able to ping 127.0.0.1? Also make sure you have following line in your host file
/etc/host
127.0.0.1 localhost
.
Please check if your "The Loopback Interface" is setup correctly.

"java.net.BindException: Cannot assign requested address" suggests another instance of Tomcat didn't shutdown properly and is still using the port you assigned to tomcat.

Answer from user1628502 is correct.Another instance is running and which is using this address, Kill that process and try to run the server again. Are you running as root? then all the port and resources used by the server will be unbinded and released automatically upon server close/shutdown. And in case of user , when you are not shutting down the server properly, the address remains to be binded, that might be your pblm.
Restart system & Try to run as root user

If you've already verified that no other process is using the same port as configured for Tomcat, you need to verify your network configuration now.
Check that your /etc/host file does not contain multiple entries for localhost. If it does, verify that the IP addresses listed are actually assigned to your machine. If not, either remove or update those entries with your current IP address.