Problem statement:
I have two Amazon EC2 instances running on same VPC. One ec2 is running web application (a spring mvc java website), while second instance is running mongodb.
I can do ssh from my local machine to both instances as well as to two instances from each other. Also telnet works perfectly fine from each other using private ip.
I also can connect to ec2 mongodb instance from my personal (local) workspace, as necessary rules are created to accept connections from local and web application's ec2 instance.
However, my web application instance cannot connect to this mongodb instance and throwing following exception whenever trying to load index page.
Exception:
com.mongodb.diagnostics.logging.JULLogger log
INFO: Cluster created with settings {hosts=[xxx.xx.xx.xxx:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
com.mongodb.diagnostics.logging.JULLogger log
INFO: Exception in monitor thread while connecting to server xxx.xx.xx.xxx:27017
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.SocketStream.open(SocketStream.java:63)
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115)
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Permission denied (connect failed)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57)
at com.mongodb.connection.SocketStream.open(SocketStream.java:58)
... 3 more
com.mongodb.diagnostics.logging.JULLogger log
INFO: No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=xxx.xx.xx.xxx:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Permission denied (connect failed)}}]}. Waiting for 30000 ms before timing out
Other details:
- Two security group have been created. 1 for Web application 2 Mongodb
- EC2 Web application is running on RHEL + Java 1.8 + Tomcat
- EC2 Mongo instance is running on RHEL (followed instructions from https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat/)
- Java Driver is used to connect to MongoDB is { driver: { name: "mongo-java-driver", version: "3.4.1" }
- bind_ip is set to 0.0.0.0 and no administrative security at this moment.
Assumption:
I am certain that mongodb ec2 instance is accepting connection as i can connect from local workspace.
Any clue or direction in this regards will be really helpful. Thanks.
bind_ip set to 0.0.0.0
Security groups are accepting connections from pretty much everywhere and inbound and outbound 0.0.0.0/0
iptables rules set for INPUT and OUTPUT on both instances.
Related
please help me with starting WAR Spring MVC Web app on AWS (Amazon Web service)
it fails with connection timeout to database ( to RDS - also Amazon).
I works localy absolutely fine. via tomcat it connects to my RDS and works,
but when I deploy this war on AWS - timeout.
I tried to add configs to increase timeout, but seems it is ignored by AWS.
Could you please help ??
structure of WAR file:
-META-INF
-WEB-INF
|- classes
|- lib
|- view (*.jsp here)
-.ebextensions
|- timeout.config
timeout.config content:
option_settings:
- namespace: aws:elasticbeanstalk:command
option_name: Timeout
value: 1800
Logs :
Where com.books.dao.impl.GenreDAOImpl.findAll(GenreDAOImpl.java:49) my first request to DB.
16-Jun-2018 20:24:46.050 SEVERE [http-nio-8080-exec-8] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [dispatcher] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection] with root cause
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at oracle.net.nt.TcpNTAdapter.connect(TcpNTAdapter.java:161)
at oracle.net.nt.ConnOption.connect(ConnOption.java:159)
at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:428)
at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:506)
at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:595)
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:230)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1452)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:496)
at oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:666)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:566)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:153)
at org.springframework.jdbc.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:144)
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnectionFromDriver(AbstractDriverBasedDataSource.java:196)
at org.springframework.jdbc.datasource.AbstractDriverBasedDataSource.getConnection(AbstractDriverBasedDataSource.java:159)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:394)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:474)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:484)
at com.books.dao.impl.GenreDAOImpl.findAll(GenreDAOImpl.java:49)
Security for elasticbeanstalk
Security Group for elastic bean
Security for ec2
Security Group for EC2 instance where public access failed
Actually, it's not an issue of config file not working, its due to security group is blocking for port or not allowing your application to get connected with your Database.
So if you are using RDS Instance then check the Security Group of your RDS, and also in which VPC it is. Best is just try to ping your RDS Instance and see whether you are getting the reply from it or not.
If you are using Mysql, Running inside your Instance, then just Allow the port (3306 for MySQL) in your EC2 Security Group. As your Port is not open, so AWS won't able to make the connection
In this case, you have to Install Mysql in your Instance if not installed before the deployment.
What happens it when you try with your Local development (Tomcat), So what happens is, in our code, we have mention localhost:3306 as the connection URL and being on the same machine (your development and MySQL) it works fine.
I want to built the zookeeper cluster in three cloud servers (centos 7) but it is desperate that it hasn't started successful yet....
Here is the conf:
zoo1:
clientPort=2181
dataDir=/tmp/zookeeper/data
syncLimit=5
initLimit=10
tickTime=2000
server.1=0.0.0.0:2888:3888
server.2=x.x.x.x:2888:3888
server.3=x.x.x.x:2888:3888
The other two machines had setup the corresponding localhost to 0.0.0.0 respectively, and confirmed that the iptables and firewalld is closed.
Everytime I run 'zkServer.sh start', it prints STARTED, like this:
[root#VM_144_102_centos zookeeper-3.5.2-alpha]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/iuShu/components/zookeeper/zookeeper-3.5.2-alpha/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
but their state is...
[root#VM_144_102_centos zookeeper-3.5.2-alpha]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/iuShu/components/zookeeper/zookeeper-3.5.2-alpha/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Error contacting service. It is probably not running.
I can see the election port 3888 is listening by using 'netstat -tunlp' and also telnet is OK, here is the primary logs:
2017-12-12 22:06:58,914 [myid:3] - WARN [WorkerSender[myid=3]:QuorumCnxManager#459] - Cannot open channel to 2 at election address /x.x.x.x:3888
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
I'm new to zookeeper. I had tried any ways i could find but it doesn't did the trick, maybe the reason is the cloud servers ? Hope someone can help me, please~~
I'm having an issue with the MongoDB Java driver (Using driver version 3.0.4 and MongoDB version 3.2.3). I'm trying to check whether a connection opens to the MongoDB server and throw an exception if not. I should be able to catch it with:
servName = "Username";
servPW = "Password";
try {
MongoCredential cred = MongoCredential.createCredential(servName, "DatabaseName", servPW.toCharArray());
MongoClient mongo = new MongoClient(new ServerAddress("localhost", 27017), Arrays.asList(cred));
} catch (MongoException e){
System.out.println("ERROR");
}
System.out.println("WE ARE HERE");
The problem I'm having is that the exception that is thrown isn't caught, it seems to be coming from a thread that I can't access maybe? For testing I don't have a Mongo server running just to see the exception that gets thrown. Similar to the issue in this question (MongoDB java driver 3.0 can't catch exception when authenticate). This is the output:
Mar 02, 2016 12:24:17 PM com.mongodb.diagnostics.logging.JULLogger
log
INFO: Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN,
serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
WE ARE HERE
Mar 02, 2016 12:24:18 PM com.mongodb.diagnostics.logging.JULLogger log
INFO: Exception in monitor thread while connecting to server localhost:27017
com.mongodb.MongoSocketOpenException: Exception opening socket at com.mongodb.connection.SocketStream.open(SocketStream.java:63) at
com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:114)
at
com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:127)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused: connect at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at
java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) at
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
java.net.Socket.connect(Socket.java:589) at
com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:50)
at com.mongodb.connection.SocketStream.open(SocketStream.java:58)
... 3 more
I've tried to catch "com.mongodb.MongoSocketOpenException", "Exception" and a number of others but nothing seems to work! Is anybody able to help me with where I'm going wrong?
I was unable to find an answer to this question and couldn't work it out myself. In order to work around this issue I downgraded to driver version 2.11.1 and used that!
From what I can gather, the move to Mongo 3 drivers alters the behaviour of creating a Mongo Client - it becomes a non-blocking operation and you only find out whether you actually have a working connection when you try to use it.
The video I watched about this (https://www.mongodb.com/presentations/mongodb-drivers-and-high-availability-deep-dive ) suggests using the client immediately after creating it to check whether it has been successful - it recommends using the command that asks whether or not the connected server is the "master" ( not sure what this is in Java ) .
While adding a document using solr cloud server I'm getting following exception
60 [main] INFO org.apache.solr.common.cloud.ConnectionManager - Waiting for client to connect to ZooKeeper
65 [main-SendThread(jmajeed.ibsorb.com:8982)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server jmajeed.ibsorb.com/192.168.70.91:8982. Will not attempt to authenticate using SASL (unknown error)
69 [main-SendThread(jmajeed.ibsorb.com:8982)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to jmajeed.ibsorb.com/192.168.70.91:8982, initiating session
Exception in thread "main" java.lang.RuntimeException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 192.168.70.91:8982/#/hotelcontent within 10000 ms
Does anybody has any idea why this is happening??
Thanks.
Have the disturbed the default configuration of solr nodes because by default if you do not specify the port the first node in the cluster will start in 8983 port so check this first. If this is not the problem then check whether the cluster is up or not by accessing admin UI of solr cloud. Then see whether all the shards in the cluster are alive by clicking on the cloud tab.
If everything is fine and still you are facing the above problem then are you trying to access a remote solr cloud server and it is firewall issue.
I am trying to connect to cassandra. I installed the latest stable version that is apache-cassandra-1.2.4 and extracted it on my desktop. As I run cassandra it sets up nicely listening for thrift client and displaying following :
sudo cassandra -f
log :
INFO 15:30:34,646 Cassandra version: 1.0.12
INFO 15:30:34,646 Thrift API version: 19.20.0
INFO 15:30:34,646 Loading persisted ring state
INFO 15:30:34,650 Starting up server gossip
INFO 15:30:34,661 Enqueuing flush of Memtable-LocationInfo#1117603949(29/36 serialized/live bytes, 1 ops)
INFO 15:30:34,661 Writing Memtable-LocationInfo#1117603949(29/36 serialized/live bytes, 1 ops)
INFO 15:30:34,877 Completed flushing /var/lib/cassandra/data/system/LocationInfo-hd-54-Data.db (80 bytes)
INFO 15:30:34,892 Starting Messaging Service on port 7000
INFO 15:30:34,901 Using saved token 143186062733850112297005303551620336860
INFO 15:30:34,903 Enqueuing flush of Memtable-LocationInfo#1282534304(53/66 serialized/live bytes, 2 ops)
INFO 15:30:34,904 Writing Memtable-LocationInfo#1282534304(53/66 serialized/live bytes, 2 ops)
INFO 15:30:35,102 Completed flushing /var/lib/cassandra/data/system/LocationInfo-hd-55-Data.db (163 bytes)
INFO 15:30:35,106 Node localhost/127.0.0.1 state jump to normal
INFO 15:30:35,107 Bootstrap/Replace/Move completed! Now serving reads.
INFO 15:30:35,108 Will not load MX4J, mx4j-tools.jar is not in the classpath
INFO 15:30:35,150 Binding thrift service to localhost/127.0.0.1:9160
INFO 15:30:35,155 Using TFastFramedTransport with a max frame size of 15728640 bytes.
INFO 15:30:35,160 Using synchronous/threadpool thrift server on localhost/127.0.0.1 : 9160
INFO 15:30:35,168 Listening for thrift clients...
Now as I run : cassandra-cli -h localhost -p 9160, it throws up the error. I have checked for the port to be free and cassandra is listening at the port. :
**
org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:183)
at org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
at org.apache.cassandra.cli.CliMain.connect(CliMain.java:80)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:256)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:178)
... 3 more
Exception connecting to localhost/9160. Reason: Connection refused.
**
I had the same error.Now, it is OK.
The main problem is that the configuration is wrong.
My configuration is as following:
My visual machine ip is 192.168.11.11.My cassandra was installed into the machine.So, I configurate thar
listen_address: 192.168.11.11
rpc_address: 0.0.0.0
broadcast_rpc_address: 192.168.11.11
That is OK。
The documentation of cassandra-stress seems to be sketchy. Maybe in due course that would be corrected. As of now, this command worked for me
./cassandra-stress write -node <IP_OF_NODE1>
Once this works, we could try putting in the other optional parameters to tweak our command.
Option 1:
Run jps command under root user and kill CassandraDaemon if you will see it. After this you will start Cassandra again.
Option2:
Try to connect Cassandra with CQL
./cqlsh 10.234.31.232 9042
Final Check:
An intermediate firewall is blocking the JVM from making the connection.
An operating system firewall, or antivirus that is causing the problems as well.
I think you installed in windows and looks like firewall is blocking your connection.