Am using Spring Boot 1.5.4 with Spring JDBC.
Have a Spring Boot Microservice which uses Spring JDBC has the following issue when trying to conduct an HTTP PUT (after a bunch of users try conducting an HTTP PUT) which trickles to this Spring JDBC call:
2018-10-10 19:40:02 [http-nio-8081-exec-4] ERROR c.v.r.RepositoryImpl - Problem in updateData() method:
"org.springframework.dao.DataAccessResourceFailureException: PreparedStatementCallback; SQL [select a.user_id,b.user_id, from user a join user_profile b where a.user_id=b.user_id and a.date=?;]; No operations allowed after connection closed.; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:79)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
at com.mysql.jdbc.Util.getInstance(Util.java:360)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 28,915,589 milliseconds ago. The last packet sent successfully to the server was 9 milliseconds ago.
at com.myapp.repository.RepositoryImpl.updateData(RepositoryImpl.java:74)
at com.myapp.repository.RepositoryImpl$$FastClassBySpringCGLIB$$1be9dd8e.invoke(<generated>)
... 52 common frames omitted
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2914)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3337)
... 83 common frames omitted
pom.xml:
<artifactId>MyService</artifactId>
<packaging>jar</packaging>
<version>1.0</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.4.RELEASE</version>
</parent>
<properties>
<java.version>1.7</java.version>
</properties>
<dependencies>
<!-- Spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.33</version>
</dependency>
</dependencies>
Am guessing that I need to setup a JDBC connection pool...
Inside my application.properties settings, (I have two different databases - one local and one remote, its losing connection with the remote database; database2):
# Local
spring.datasource.database1.url=jdbc:mysql://localhost/database1?zeroDateTimeBehavior=convertToNull
spring.datasource.database1.username=root
spring.datasource.database1.password=ret2my
spring.datasource.database1.driverClassName=com.mysql.jdbc.Driver
# Remote
spring.datasource.database2.url=jdbc:mysql://read-replica-database-production.cranmichpmc.us-west-2.rds.amazonaws.com/database2?zeroDateTimeBehavior=convertToNull
spring.datasource.database2.username=root
spring.datasource.database2.password=ret2a$$
spring.datasource.database2.driverClassName=com.mysql.jdbc.Driver
Should I add this for the second database:
spring.datasource.database2.hikari.maximum-pool-size=10
spring.datasource.database2.hikari.connection-timeout=60000
Are there other useful params that I should consider?
spring.datasource.url=jdbc:mysql://10.168.143.140:3306/database_name
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.hikari.maximum-pool-size=4
more detailed information can be found at :
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html
Related
Im using the Keycloak admin client (version 4.5.0.Final) and am trying to do some simple queries such as looking up a user. The client code is running in a plugin module in another java server, not standalone. The code looks like this:
...
try {
Keycloak kc = Keycloak.getInstance(URL, REALM, USER, PWD, CLIENT_ID);
UserRepresentation kcuser = kc.realm(REALM).users().get(USER).toRepresentation();
trace(String.format("Got user: %s", kcuser.toString()));
} catch (Exception e) {
trace("Error authenticating: " + e);
}
...
It creates the kc instance successfully, but barfs when trying to look up the user.
This is the error:
javax.ws.rs.ProcessingException: RESTEASY003215: could not find writer for content-type application/x-www-form-urlencoded type: javax.ws.rs.core.Form$1
at org.jboss.resteasy.core.interception.ClientWriterInterceptorContext.throwWriterNotFoundException(ClientWriterInterceptorContext.java:40)
at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.getWriter(AbstractWriterInterceptorContext.java:146)
at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:121)
at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.writeRequestBody(ClientInvocation.java:394)
at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.writeRequestBodyToOutputStream(ApacheHttpClient4Engine.java:666)
at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.buildEntity(ApacheHttpClient4Engine.java:631)
at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.loadHttpMethod(ApacheHttpClient4Engine.java:509)
at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:310)
at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:439)
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invokeSync(ClientInvoker.java:148)
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invoke(ClientInvoker.java:112)
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientProxy.invoke(ClientProxy.java:76)
at com.sun.proxy.$Proxy362.grantToken(Unknown Source)
at org.keycloak.admin.client.token.TokenManager.grantToken(TokenManager.java:89)
at org.keycloak.admin.client.token.TokenManager.getAccessToken(TokenManager.java:69)
at org.keycloak.admin.client.token.TokenManager.getAccessTokenString(TokenManager.java:64)
at org.keycloak.admin.client.resource.BearerAuthFilter.filter(BearerAuthFilter.java:52)
at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.filterRequest(ClientInvocation.java:587)
at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:436)
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invokeSync(ClientInvoker.java:148)
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invoke(ClientInvoker.java:112)
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientProxy.invoke(ClientProxy.java:76)
at com.sun.proxy.$Proxy372.toRepresentation(Unknown Source)
...
My pom has the latest dependencies and classpath seems ok, any ideas why this is not working?
<properties>
<keycloak.version>4.5.0.Final</keycloak.version>
<resteasy.version>3.6.1.Final</resteasy.version>
</properties>
<dependencies>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-admin-client</artifactId>
<version>${keycloak.version}</version>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-client</artifactId>
<version>${resteasy.version}</version>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-multipart-provider</artifactId>
<version>${resteasy.version}</version>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-jackson2-provider</artifactId>
<version>${resteasy.version}</version>
</dependency>
</dependencies>
I noticed that during the instantiation of a new Keycloak instance, resteasy is checking here for the available providers with the help of the current thread. In version 3.9.1.Final which is currently used by the last keycloak-admin-client so far (version 11.0.0).
In my specific case we are using keycloak-admin-client in combination with graphql-java and CompletableFuture.supplyAsync for our data loaders. Which implies that in some cases, without further configuration, the current thread is not an instance of Thread but actually ForkJoinWorkerThread. Which apparently breaks the retrieval of the providers.
I am still a beginner to java so I would be glad if someone could explain why the registerProviders method does not work with a ForkJoinWorkerThread.
I learned on DZone is that JVM sizes the commonPool to two threads when you have more than 2 CPUs available. So I tried and noticed that my app works with 2 CPUs, but I have the same error (RESTEASY003215) with 3 CPUs.
My current "workaround" is to to use CompletableFuture.completedStage when loading data using the keycloak-admin-client.
In my spring boot project I am losing sometimes the connection to the teradata database. I would like to know how I can tell spring to close and open the connection again.
I see after several days of runtime exceptions like
Caused by: java.sql.SQLException: [Teradata JDBC Driver] [TeraJDBC 16.10.00.03] [Error 1095] [SQLState HY000] Cannot call a method on closed connection
at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDriverJDBCException(ErrorFactory.java:95) ~[terajdbc4-16.10.00.03.jar!/:16.10.00.03]
at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDriverJDBCException(ErrorFactory.java:65) ~[terajdbc4-16.10.00.03.jar!/:16.10.00.03]
at com.teradata.jdbc.jdbc_4.TDSession.prepareStatement(TDSession.java:1323) ~[terajdbc4-16.10.00.03.jar!/:16.10.00.03]
at com.teradata.jdbc.jdbc_4.TDSession.prepareStatement(TDSession.java:1375) ~[terajdbc4-16.10.00.03.jar!/:16.10.00.03]
at com.teradata.jdbc.jdbc_4.TDSession.prepareStatement(TDSession.java:1361) ~[terajdbc4-16.10.00.03.jar!/:16.10.00.03]
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162]
at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126) ~[tomcat-jdbc-8.5.20.jar!/:na]
at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108) ~[tomcat-jdbc-8.5.20.jar!/:na]
at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:75) ~[tomcat-jdbc-8.5.20.jar!/:na]
at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108) ~[tomcat-jdbc-8.5.20.jar!/:na]
at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:81) ~[tomcat-jdbc-8.5.20.jar!/:na]
at com.sun.proxy.$Proxy73.prepareStatement(Unknown Source) ~[na:na]
at org.springframework.jdbc.core.JdbcTemplate$SimplePreparedStatementCreator.createPreparedStatement(JdbcTemplate.java:1521) ~[spring-jdbc-4.3.11.RELEASE.jar!/:4.3.11.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:627) ~[spring-jdbc-4.3.11.RELEASE.jar!/:4.3.11.RELEASE]
... 25 common frames omitted
When trying to access the database.
I am using an autowired JdbcTemplate when accessing the database. I can catch the exception, but I do not know how to tell spring to kill and reconnect to the database.
I am using the following dependencies.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>com.teradata.jdbc</groupId>
<artifactId>terajdbc4</artifactId>
<version>16.10.00.03</version>
</dependency>
<dependency>
<groupId>com.teradata.jdbc</groupId>
<artifactId>tdgssconfig</artifactId>
<version>16.10.00.03</version>
</dependency>
After searching stackoverflow, I found examples how to add reconnection configs for different databases, but not for teradata. Can I use similar configs?
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.7.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
java.sql.Connection conn;
java.sql.Statement stmt;
try {
stmt = conn.createStatement();
stmt.executeQuery("SELECT USER");
} catch (SQLException e) {
// ... open connection here
}
i can't mvn clean install my project.
I need to use a remote JNDI in WAR(not jar), but having the name in application.properties throws error during mvn clean install(due to "datasource jndi name not found")
if I have to use jndi name from a remote server, that I have no control over, how do i compile my package in WAR without lookup fail, but let it look up during the deployment/installation of the app on the server? any hint, or direction is very much appreicated...
in app prop, i have
spring.datasource.jndi-name=jdbc/ff-app
I have tried: Spring boot JNDI datasource lookup failure - Name comp/env/jdbc not found in context "java:"
tried, http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-connecting-to-a-jndi-datasource
and pretty much every questions asked in stackoverflow about a remote jndi..
Some of the errors:
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is org.springframework.jdbc.datasource.lookup.DataSourceLookupFailureException: Failed to look up JNDI DataSource with name 'jdbc/ff-app'; nested exception is javax.naming.NoInitialContextException: Need to specify class name in environment or system property, or as an applet parameter, or in an application resource file: java.naming.factory.initial
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189) ~[spring-beans-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588) ~[spring-beans-4.3.6.RELEASE.jar:4.3.6.RELEASE]
... 69 common frames omitted
Caused by: org.springframework.jdbc.datasource.lookup.DataSourceLookupFailureException: Failed to look up JNDI DataSource with name 'jdbc/ff-app'; nested exception is javax.naming.NoInitialContextException: Need to specify class name in environment or system property, or as an applet parameter, or in an application resource file: java.naming.factory.initial
at org.springframework.jdbc.datasource.lookup.JndiDataSourceLookup.getDataSource(JndiDataSourceLookup.java:48) ~[spring-jdbc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
at org.springframework.boot.autoconfigure.jdbc.JndiDataSourceAutoConfiguration.dataSource(JndiDataSourceAutoConfiguration.java:62) ~[spring-boot-autoconfigure-1.5.1.RELEASE.jar:1.5.1.RELEASE]
at org.springframework.boot.autoconfigure.jdbc.JndiDataSourceAutoConfiguration$$EnhancerBySpringCGLIB$$9509801a.CGLIB$dataSource$0() ~ ........
My POM.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.1.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
</dependency>
</dependencies>
The point is that when you try to compile by using maven, maven tries to execute also Junit tests; in your case you have in src/resources/java some unit tests and so maven tries to execute them. Since you are using a JNDI datasource, it means that also the servlet container must be active for executing tests otherwise you can't get connection.
In this case when maven tries to execute tests maybe no servlet container (and, so, no JNDI datasource) is active and your test throws error.
To solve this (and to compile the maven artifact) you must:
or exclude tests execution: this is done by adding <maven.test.skip>true</maven.test.skip> in pom.xml. No test will be executed
or ignore tests error this is done by adding <maven.test.failure.ignore>true</maven.test.failure.ignore> in pom.xml. In this case tests will be executed but any test error will not stop maven compilation
The configuration I gave not only tells maven to not execute tests but it tells maven to not stop on any test error
I hope i have been clearer... my english is not the best
I was referring to a post here:
Connecting to Zookeeper in a Apache Kafka Multi Node cluster
Its mentioned here that from kafka V9 version, Producer and Consumer does not have to use the zookeeper.connect property and just the bootstrap.servers is enough to producer/consume data.
My POM.xml looks like this in the consumer side:
<properties>
<java.version>1.7</java.version>
<kafka.version>0.9.0.1-cp1</kafka.version>
<kafka.scala.version>2.11</kafka.scala.version>
<confluent.version>2.0.1</confluent.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.10.0.0</version>
</dependency>
I run into the following issue in the consumer side, without zookeeper.connect property. Does anyone has the consumer part working without the zookeeper connect property ?
[WARNING]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: requirement failed: Missing required property 'zookeeper.connect'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getString(VerifiableProperties.scala:177)
at kafka.utils.ZKConfig.<init>(ZkUtils.scala:902)
at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:101)
at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:105)
at io.confluent.examples.consumer.ConsumerGroup.<init>(ConsumerGroup.java:30)
at io.confluent.examples.consumer.ConsumerGroup.main(ConsumerGroup.java:113)
... 6 more
Only the new consumer works without connecting to Zookeeper and that is available in the kafka-clients artifact. You have to add the dependency:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.0.0</version>
</dependency>
and use the implementation from the org.apache.kafka.clients.consumer. package.
in java i want to use oracle XMLType.
I add
XMLType type = new XMLType(conn, "<shipTo></shipTo>");
but i always get
Caused by: java.lang.ClassNotFoundException: oracle.xml.parser.v2.XMLParseException
why? i am using xdb6.jar for oracle 11 database. For connection i am using JDBC.
Here's the excerpt from my POM for doing the same (in JBoss):
<!-- required for Oracle XML support -->
<dependency>
<groupId>com.oracle</groupId>
<artifactId>xmlparserv2</artifactId>
<version>11.2.0.1.0</version>
</dependency>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>xdb</artifactId>
<version>11.2.0.1.0</version>
</dependency>
i.e. you need the XDB and XMLParserv libraries.