My Spring boot application is using Spring Data JPA to interact with a AWS MySQL RDS. The application is deployed on 2 ec2 instances on top of ELB.
MySQL DB has a maximum connection limit of 147 connections, so I have added following entry in my application-prod.properties file:
spring.datasource.hikari.connection-timeout=30000
spring.datasource.hikari.maximum-pool-size=70
After this my application runs fine for few days but then I again start getting JDBC connection not found error.
On further debugging running following command on my MySQl gives me only 73 active connections whereas for the time when application was running properly, it returned 143.
Could it be possible that 1 of the ec2 instance is stopped or terminated?
If so what can I do to debug further?
SHOW STATUS WHERE `variable_name` = 'Threads_connected';
I have had a hard time to debug this issue, need some help.
Related
I have an application runing in App engine Standar Environment done in java and accessing to the datastore with datanucleus. It is working fine and, as it is running inside the Standard Environment, it connects to datastore perfecly.
Now, I am deploying a new service for the application done in Spring Boot with the spring-cloud-gcp-data-datastore library. I have the .json file for the connection for the service account and it is working perfectly on my local environment with the gcloud beta emulator.
But now, in the production environment, when the app try to connect to the datastore, I can see the error in the logs:
com.google.cloud.datastore.DatastoreException: I/O error
...
Caused by: com.google.datastore.v1.client.DatastoreException: I/O error
...
Caused by: java.net.ConnectException: Connection refused (Connection refused)
Seems that the service can't connect to the datastore.
How can I see what is wrong in the connection?
Is there anyway to list the connection process for the Spring boot application and datastore?
Resolved
Thanks to Mike for all his support. He gave me the key with the Spring Cloud GCP documentation.
The problem was that I must disable the configuration for the local emulator when I deploy to Production. If you don't do that, the application doesn't find the Datastore service.
Once I commented the line for the spring.cloud.gcp.datastore.host configuration (Datastore Emulator Autoconfiguration), the application worked fine in production.
I have a spring boot application running on cloud run, so far I only had to add the spring cloud gcp mysql
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-sql-mysql</artifactId>
<version>1.2.8.RELEASE</version>
</dependency>
dependency in my POM, and configure my application.yml file to set database name, connection name etc, and it runs fine locally and on cloud run.
My application.yml:
spring:
cloud:
gcp:
sql:
enabled: true
database-name: pos_database
instance-connection-name: pos-sys:asia-southeast2:pos-server-database
datasource:
driver-class-name: com.mysql.cj.jdbc.Driver
username: ***
password: ***
hikari:
maximum-pool-size: 20
However I realized cold start performance has taken a hit, because on startup the socket factory connects to the database instance via SSL socket:
2021-05-31 13:10:07.152 INFO 1539 --- [onnection adder] c.g.cloud.sql.core.CoreSocketFactory :
Connecting to Cloud SQL instance [pos-sys:asia-southeast2:owl-server-database] via SSL socket.
and i get a bunch of lines just repeating
2021-05-31 13:10:09.461 INFO 1539 --- [connection adder] c.g.cloud.sql.core.CoreSocketFactory :
Connecting to Cloud SQL instance [pos-sys:asia-southeast2:pos-server-database] via SSL socket.
I know there is a faster way to connect then the application is running on the cloud, I have been following this tutorial so far:
https://cloud.google.com/sql/docs/mysql/connect-run
But i'm very confused on the last part where it says I have to connect with unix socket, is this a docker thing or within my application? where does the ConnectionPoolContextListener.java
file have to go?
It also says in a comment within the file itself not to use this for java users, and to instead use
Cloud SQL JDBC Socket Factory
But when I go to that link it says to add a dependency to for mysql-connector, but isnt that already included in spring-gcp-starter-mysql? It also says make a connection string in this format:
jdbc:mysql:///<DATABASE_NAME>?cloudSqlInstance=<INSTANCE_CONNECTION_NAME>&socketFactory=com.google.cloud.sql.mysql.SocketFactory&user=<MYSQL_USER_NAME>&password=<MYSQL_USER_PASSWORD>
But doesnt mention where do I put this?
So to summarise:
I have a cloud mysql instance, with the admin api enabled.
I did the Enable connecting to a Cloud SQL in my cloud run by selecting my db instance.
I am very confused by the documentation on what the next step is and what to do next.
Cloud Run provide a Unix domain socket when configured with a Cloud SQL instance - it's a file that can be used to connect to a database. You are using the Cloud SQL Java connector, which allows you to bypass using the Unix socket (which is usually preferred on Java, since Unix sockets aren't natively supported).
Instead to improve your cold start time, I recommend doing two things:
Reduce the number of connections in your pool. While the optimal number varies greatly between applications, 20 is almost certainly way more than you need. As a rule of thumb, try 2 * the number of cores used as your starting value, and increase/decrease as needed. Hikari uses maximumPoolSize to do this.
Adjust the number of starting connections in your pool. Hikari offers minimumIdle, which sets the minimum number of idle connections in the pool, and up to maximumPoolSize. While Hikari recommends not setting this value (so you have a fixed pool), setting it to 0 means your pool won't establish connections on startup. This means your application will start faster, but will take longer to get a connection from the pool on average.
I'm new to h2, but I read a lot of documentation,
I successfully created embeded H2 mem or file database.
But I'm still not able to use server mode or mixed mode :
I launch my Web server and tcp server like this :
java -jar h2-1.4.196.jar -web -webAllowOthers -tcp -tcpAllowOthers -browser -tcpPort 9092
Trying server mode :
I create file database like this :
I open H2 console and enter following :
jdbc:h2:~/maBaseH2FichierTCP
username: sa
password:
Database is created, and I see a lock file
In my java application
url: jdbc:h2:tcp://localhost:9092/~/maBaseH2FichierTCP
same username and pwd
--> When I launch my java application properties file, it stays sticked at the phase of datasource creation.
I tried to change url: jdbc:h2:tcp://localhost:9092/~/maBaseH2FichierTCP
to jdbc:h2:tcp://localhost/~/maBaseH2FichierTCP, but no effect.
Trying mixed mode :
In H2 console, I created jdbc:h2:C:/data/maBaseFichierAutoServeur;AUTO_SERVER=TRUE
In java, url: jdbc:h2:tcp://192.168.1.152:55000/file:c:/data/maBaseFichierAutoServeur;AUTO_SERVER=TRUE
--> Behaviour is the same, application start, but when at datasource creation, it waits, and nothing more happen ...
Does someone take has a idea to make one of orther server mode work ?
I solved this first problem : It was caused by a incompatibility between h2 version of h2 console, and my h2 client in application. 1.4.196 vs 1.4.192; when debuging, we can see an exception is launched, but nothing is writen on debug or even trace I think, and programme try to execute a rollback on a connection. It ended in a infinite loop here, even if I don't understand why. When changing version, I can connect my remote base.
Howerver now, when application update the database, the updated data cannot been seen in h2 console. Does someone know why ?
I'm a newbie to create project by spring + hibernate.
I'm using Oracle SQL Developer for DB and eclipse for development.
In Spring boot application.properties file we have the following options:
spring.datasource.tomcat.max-active = 1
When i start project, there are 2 sessions in DB (1 of DB tool).
During application run, error occur
Occured:: MSG = org.apache.tomcat.jdbc.pool.PoolExhaustedException:
[https-jsse-nio-8888-exec-2] Timeout: Pool empty. Unable to fetch a
connection in 10 seconds, none available[size:1; busy:1; idle:0;
lastwait:10000].
Please give me the way to explain or resolve this error.
Thank you very much.
In one of my project, we are trying to connect to Postgres server via Java application. It gives an error when try to open connection. Stack trace is below -
{"timestamp":"2016-03-17T06:57:18.853Z",
"level":"ERROR",
"thread":"main",
"logger":"com.abc.messaging.status.configuration.ValidateDatabaseConfiguration",
"message":"Failed to initialize data source: \n
### Error opening session. Cause: java.lang.NullPointerException\n
### Cause: java.lang.NullPointerException","context":"default"}
While we have tried following -
Ping db server from application server - it's successful
Telnet db server and port (from app server)- it's also successful
We also tried a simple python script that connects to postgre server. It also connects to db successfully.
We then moved back to our staging server and tried to connect production db server from there. This is also done successfully.
We tried connecting staging db server from production app server, which also we COULDN'T.
We are assuming that we don't have any issue in our application. This is because, from staging server, application can connect to both staging db and production db. There by, it must be some application server issue or something.
PostgreSQL version - Production - 9.4.5 Staging - 9.4.4
Java version - 1.8.31
Any kind of help would be much appreciated.
Thanks in advance.