java.net.SocketTimeoutException at peak traffic on Production - java

We have been facing this issue on Prodcution. Now this issue occurs whenever a notification is rolled out to all the Mobile Apps and Desktop Apps. In the notification one image is there which is cached at CDN(Akamai). So a lot of users are getting the following exception. I tried to figure out the issue but a lot of people say it might be due to
network issue on the device but my question is DNS of XXXX.XXXX.com has been resolved to akamai ip (117.239.141.27). So, that should not be a problem. Moreover I had a work with Akamai guys they couldn't find any issue at their end. So, I have following questions:
1) How do i figure out what is causing the issue? Secondly, does anyone has faced the similar issue on production at peak traffic?
2) As mentioned in the below stack track which i got from crashlytics. In few cases we are getting private client ip (192.168.1.33) why is it so?
Shouln't public ip of client should be logged in the crashlytics.
Caused by java.net.SocketTimeoutException: failed to connect to XXXX.XXXX.com/117.239.141.27 (port 80) from /192.168.1.33 (port 50974) after 15000ms
at libcore.io.IoBridge.connectErrno(IoBridge.java:185)
at libcore.io.IoBridge.connect(IoBridge.java:129)
at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:137)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:390)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:230)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:212)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:436)
at java.net.Socket.connect(Socket.java:621)
at com.android.okhttp.internal.Platform.connectSocket(Platform.java:145)
at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:1416)
at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:1368)
at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:219)
at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:142)
at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:104)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:392)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:325)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:488)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:434)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:565)
at com.squareup.picasso.UrlConnectionDownloader.load(UrlConnectionDownloader.java:125)
at com.squareup.picasso.NetworkRequestHandler.load$71fa0c91(NetworkRequestHandler.java:6)
at com.squareup.picasso.BitmapHunter.hunt(BitmapHunter.java:72)
at com.XXXXX.XXXXX.utils.AppNotificationHandler.getTabCategoryIdForScreen(AppNotificationHandler.java:82)
at com.XXXXX.XXXXX.utils.AppNotificationHandler.handleExtras(AppNotificationHandler.java:806)
at com.XXXXX.XXXXX.fcm.FCMIntentService.onMessageReceived(FCMIntentService.java:112)
at com.google.firebase.messaging.FirebaseMessagingService.zzd(FirebaseMessagingService.java:394)
at com.google.firebase.iid.zzg.run(zzg.java:26)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at com.google.android.gms.common.util.concurrent.zza.run(zza.java:7)
at java.lang.Thread.run(Thread.java:764)

Related

Pubnub Socket Timeout exception

This is the error I get in PubNub, when I publish. I am publishing from a normal Java file on eclipse.
[Error: 103-2] : HTTP Error. Please check network connectivity. Please contact support with error details if issue persists. : java.net.SocketTimeoutException: connect timed out
My network is connected and is working. Ping requests to google & other servers are successful. Yet, the error keeps occurring randomly.

Unknown SMTP host

I am running a Java web application using tomcat to send generated reports via emails to the users.
I am able to send the emails but after few hours the server stops sending emails, with the following error.
javax.mail.MessagingException: Unknown SMTP host: mail.mydomain.co.uk;
nested exception is:
java.net.UnknownHostException: mail.mydomain.co.uk
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1970)
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:642)
at javax.mail.Service.connect(Service.java:317)
at javax.mail.Service.connect(Service.java:176)
at javax.mail.Service.connect(Service.java:125)
at javax.mail.Transport.send0(Transport.java:194)
at javax.mail.Transport.send(Transport.java:124)
at com.turnkey.email.SendEmail.sendMail(SendEmail.java:119)
at com.turnkey.thread.CommunicationThread.run(CommunicationThread.java:399)
Caused by: java.net.UnknownHostException: mail.mydomain.co.uk
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:195)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at com.sun.mail.util.SocketFetcher.createSocket(SocketFetcher.java:319)
at com.sun.mail.util.SocketFetcher.getSocket(SocketFetcher.java:233)
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1938)
... 8 more
After some time the server starts sending emails again.
Can anyone tell me what could be the problem.
And how do I solve this problem?
Thanks
This looks like a failure in your name service. The JDK isn't able to look up the host name to find its internet address. Since this works sometimes and not others, it looks like an intermittent failure of the name service. The name service failure could be due to some failure in your local operating system, or it could be due to some network failure communicating with your DNS server or other name service server, or it could be a failure in that DNS server or name service server itself. Determining the exact cause of the failure will require some debugging. Note that the JDK caches the results of name server lookups for some time so you'll need to factor that into your debugging.
Also make sure there is no space at the end of smtp hostname eg. mail.google.comSPACEHERE . Surprisingly this happened to me and finally after removing this space there was no complain about smtp host . Email was successfully sent
Set for host the ip address of the domain name instead of the domain name.
use nslookup mail.mydomain.co.uk on cmd to find the ip address.
It worked for me.
Specially for AIX or Linux OS environment,
We need to add the hostname in the etc/hosts file.. to sole this.
Windows operating system , this may work on Windows system as there is no strict security check however AIX or Linux must add host name to etc/hosts file in order for ping the SMTP server.
avoid doing this may lead Unknown host issue

Network is unreachable?

I want to run a cyc (opencyc) Java class, but when I run this class I got an error mentioning that the network is unreachable. I am working on NetBeans 7.0 and opencyc 2.0 win 32 version. I am using localhost (127.0.0.1) to test. The error is below.
Network is unreachable: connect
java.net.SocketException: Network is unreachable: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at java.net.Socket.connect(Socket.java:469)
at java.net.Socket.<init>(Socket.java:366)
at java.net.Socket.<init>(Socket.java:180)
at org.opencyc.api.CycConnection.initializeApiConnections(CycConnection.java:223)
at org.opencyc.api.CycConnection.<init>(CycConnection.java:199)
at org.opencyc.api.CycAccess.<init>(CycAccess.java:121)
at org.opencyc.api.CycAccess.<init>(CycAccess.java:61)
at org.opencyc.api.ApiDemo.<init>(ApiDemo.java:50)
at org.opencyc.api.ApiDemo.main(ApiDemo.java:147)
Exception in thread "main" java.lang.NullPointerException
at org.opencyc.api.ApiDemo.<init>(ApiDemo.java:56)
at org.opencyc.api.ApiDemo.main(ApiDemo.java:147)
Interruption while waiting Cyc connection establishment, closing sockets
Exception in thread "Thread-1" java.lang.NullPointerException
at org.opencyc.api.CycConnection$TaskProcessorBinaryResponseHandler.access$002(CycConnection.java:988)
at org.opencyc.api.CycConnection.close(CycConnection.java:268)
at org.opencyc.api.CycConnection$ConnectionTimer.run(CycConnection.java:1351)
Why is this happening? How can I connect to the cyc?
The problem is that your network settings do not allow you to connect to this machine. This is not a Java problem as such.
try
ping {hostname}
until this works, there is nothing you can do in java to work around this.
If you have the same version I have, it's trying to establish a connection to a Cyc image with the hostname and port defined by
CycConnection.DEFAULT_HOSTNAME
CycConnection.DEFAULT_BASE_PORT
You can find the proper values to use by evaluating the following in your Cyc image:
(get-machine-name)
*BASE-TCP-PORT*
Hope this works for you.
A dev on my team encountered this today, only their machine is seeing this issue out of many on the team.
Through much debugging we eventually discovered that localhost would not resolve for some reason. It would work in some JVM contexts and fail in others so it was completely inconsistent.
Replacing localhost with explicitly 127.0.0.1 fixed the issue. My hunch is that something was different with a local network and IPv6 perhaps, but we still don't really understand why this is the fix.

I need some help with Sakai 2.7.1 and Tomcat 5.5.33, in regards to SQL issues

Today I managed to recreate the farms with Scalr.net and apparently after a few times restarting tomcat and fixing issues, I get this error once again. The thing is I was using MySQL with a clean install on the entire server, that includes Java 6.1_24, Tomcat 5.5.33, Sakai 2.7.1. The issue I keep running into is user denied when the fact that I have this user in the MySQL Instance, as well giving it complete remote access with sakai#% and even this is not working when it was working about an hour ago since this post was made.
... Continued from above log, everything before logs just fine
2011-03-31 18:31:14,120 WARN main org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy - Could not retrieve default auto-commit and transaction isolation settings
org.apache.commons.dbcp.SQLNestedException: Error preloading the connection pool
... continued over 400+ lines...
Here is another error in regards to the access denied error...
2011-03-31 18:31:16,854 WARN main org.hibernate.cfg.SettingsFactory - Could not obtain connection metadata
java.sql.SQLException: Access denied for user 'sakai'#'ec2-50-17-184-70.compute-1.amazonaws.com' (using password: YES)
.... continued....
I now get this error whenever I startup, this is with a fresh install of tomcat/sakai
SEVERE: Unable to set localhost. This prevents creation of a GUID. Cause was: ec2-72-44-56-167.compute-1.amazonaws.com: ec2-72-44-56-167.compute-1.amazonaws.com
java.net.UnknownHostException: ec2-72-44-56-167.compute-1.amazonaws.com: ec2-72-44-56-167.compute-1.amazonaws.com
(This most recent error (Localhost) was simply fixed by restarting the amazon aws instance. Thankfully) Although I keep getting the same errors even with a fresh install... Almost as if the information is being refreshed from a cache... Or something
As with the last question you posted on this topic, the error message seems very clear: the user 'sakai'#... does not have access to login to the database you have set it up to. I recommend taking a look at the Mysql documentation to understand how to administer the user accounts to find out if you've missed a setting somewhere to allow this account to have access.
I believe I may have figured out how to fix this problem. It has nothing to do with mysql, or the apache server itself. It has to do with the failure of Scalr.net not Initializing the IP or something of that sort. After doing some research I found some issues with the HostInit issues such as....
Cannot deliver message 'HostInit' (message_id: af9dcfdb-a09e-4971-bdb7-7871b3f7e21c) via REST to server '50.17.135.98' (server_id: e49cfec9-5bcb-44d1-bbc5-fde32450fc89). Error: 0 Timeout was reached; connect() timed out! (http://50.17.135.98:8013/control)
Cannot deliver message 'BlockDeviceAttached' (message_id: a153d83f-3d96-4d53-920a-ccb80701675a) via REST to server '50.17.135.98' (server_id: e49cfec9-5bcb-44d1-bbc5-fde32450fc89). Error: 0 Timeout was reached; connect() timed out! (http://50.17.135.98:8013/control)
Cannot deliver message 'HostUp' (message_id: 1adde27c-9982-4551-b266-c3c432d1dd44) via REST to server '50.17.135.98' (server_id: e49cfec9-5bcb-44d1-bbc5-fde32450fc89). Error: 0 Timeout was reached; connect() timed out! (http://50.17.135.98:8013/control)
Cannot deliver message 'HostInit' (message_id: f1aa4b14-ef57-4361-ae56-87702d674b11) via REST to server '50.17.135.98' (server_id: e49cfec9-5bcb-44d1-bbc5-fde32450fc89). Error: 0 Timeout was reached; connect() timed out! (http://50.17.135.98:8013/control)
So what I did was I made a snapshot image of the apache server/mysql etc. and terminated them allowing the recreation of the instance and this managed to solve the problem in one manner.

IOException : The network adapter could not establish the connection

I am trying to connect to an Oracle database from my Java application. I am using oracle.jdbc.driver.OracleDriver, Version : 0/1 to connect to the database. But DriverManager.getConnection() is giving me the "The network adapter could not establish the connection". This is not happening evertime. Its happening sometimes only, may be once in 8-10 times. The stack trace am getting is :
Driver Class : oracle.jdbc.driver.OracleDriver, version 0/1
java.sql.SQLException: Io exception: The Network Adapter could not establish the connection
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:134)
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:179)
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:334)
at oracle.jdbc.ttc7.TTC7Protocol.handleIOException(TTC7Protocol.java:3678)
at oracle.jdbc.ttc7.TTC7Protocol.logon(TTC7Protocol.java:352)
at oracle.jdbc.driver.OracleConnection.<init>(OracleConnection.java:365)
at oracle.jdbc.driver.OracleDriver.getConnectionInstance(OracleDriver.java:547)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:347)
at java.sql.DriverManager.getConnection(DriverManager.java:316)
at java.sql.DriverManager.getConnection(DriverManager.java:297)
Since I am not getting this everytime, I am not able to guess that there is any connectivity issue. I can see lots of developers have the same problem, but for them it happens everytime. Please help me, since i am literally stuck in solving this problem.
I was having a similar problem, I tried using the IP address instead of the host name in the database URL, and it worked for me.
Example jdbc:
...(DESCRIPTION=(ADDRESS.....(HOST=1.1.1.1)..)
This is probably a network issue:
Is there a firewall that's giving you trouble? Try to telnet to the port. Is the firewall stateful?
Is the DNS too slow (use IP address instead of hostname, try DNS lookup)
Do you close the connections? Are you overloading the Listener?
I would have a closer look and perhaps set a break point in
at oracle.jdbc.ttc7.TTC7Protocol.handleIOException(TTC7Protocol.java:3678)
to see the precise IOException handled. This will tell you the underlying cause, which can help diagnose the problem.

Categories

Resources