How do I get Spring Boot, Java, MyBatis, Jackson, and MySQL to return a UTC time for a date without converting it? The query returns a date like
+------------+
| hitDate |
+------------+
| 2018-04-24 |
+------------+
The API class has a field like
#JsonFormat(pattern="yyyy-MM-dd'T'HH:mm:ss'Z'", timezone="UTC")
Date hitDate;
API calls return values like
"hitDate":"2018-08-01T04:00:00Z"
Obviously it cannot have a time component. So it thinks the date with 00:00 time is in EST (the system time zone -4:00) and tries to convert it to UTC by adding 4 hours. It works in production where everything is set to UTC. My local system time zone is set to Eastern Standard Time.
I tried
&useLegacyDatetimeCode=false&serverTimezone=UTC in the MySQL connection for spring.datasource.url in config/application.properties
user.timezone=UTC in config/application.properties
SET ##global.time_zone = '+00:00'; in MySQL
mvn spring-boot:run -Dexec.args="-Duser.timezone=UTC"
My datasource upon initialization:
DATASOURCE = org.apache.tomcat.jdbc.pool.DataSource#67c7bbdf{ConnectionPool[defaultAutoCommit=null; defaultReadOnly=null; defaultTransactionIsolation=-1; defaultCatalog=null; driverClassName=com.mysql.jdbc.Driver; maxActive=100; maxIdle=100; minIdle=10; initialSize=10; maxWait=30000; testOnBorrow=true; testOnReturn=false; timeBetweenEvictionRunsMillis=5000; numTestsPerEvictionRun=0; minEvictableIdleTimeMillis=60000; testWhileIdle=false; testOnConnect=false; password=********; url=jdbc:mysql://localhost:3306/appdb?useSSL=false&useUnicode=yes&characterEncoding=UTF-8&useLegacyDatetimeCode=false&serverTimezone=UTC; username=root; validationQuery=/* ping */ SELECT 1; validationQueryTimeout=-1; validatorClassName=null; validationInterval=3000; accessToUnderlyingConnectionAllowed=true; removeAbandoned=false; removeAbandonedTimeout=60; logAbandoned=false; connectionProperties=null; initSQL=null; jdbcInterceptors=null; jmxEnabled=true; fairQueue=true; useEquals=true; abandonWhenPercentageFull=0; maxAge=0; useLock=false; dataSource=null; dataSourceJNDI=null; suspectTimeout=0; alternateUsernameAllowed=false; commitOnReturn=false; rollbackOnReturn=false; useDisposableConnectionFacade=true; logValidationErrors=false; propagateInterruptState=false; ignoreExceptionOnPreLoad=false; useStatementFacade=true; }
My MySQL time zone settings:
mysql> SHOW VARIABLES LIKE '%zone%';
+------------------+-----------------------+
| Variable_name | Value |
+------------------+-----------------------+
| system_time_zone | Eastern Standard Time |
| time_zone | +00:00 |
+------------------+-----------------------+
Setting the JVM time zone to UTC was sufficient
mvn spring-boot:run -Drun.jvmArguments="-Duser.timezone=UTC"
This works for Spring Boot v1. It is a different argument for v2. Using java.time.*, MySQL connection parameters, or setting MySQL variables was unnecessary. Using LocalDate will give an error com.fasterxml.jackson.databind.JsonMappingException: Unsupported field: HourOfDay because of the JSON Jackson annotation using timestamps. Instant does work though.
I added
#SpringBootApplication
public class App extends SpringBootServletInitializer {
#PostConstruct
public void init(){
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
}
For closer DEV/PROD parity, and in case production accidentally gets a time zone set.
For a Spring project, the mysql-connector-java has been migrated from 6.0.6 to 8.0.11.
Thus with 8.0.11 the problem is the following:
Caused by: com.mysql.cj.exceptions.InvalidConnectionAttributeException:
The server time zone value 'PET' is unrecognized or represents more than one time zone.
You must configure either the server or JDBC driver (via the serverTimezone configuration property)
to use a more specifc time zone value if you want to utilize time zone support.
After to do a research
The server time zone value 'AEST' is unrecognized or represents more than one time zone
the solution is change the URL (I don't want return to a previous release)
from: mysql.jdbcUrl = jdbc:mysql://localhost:3306/web_v01?useSSL=false
to: mysql.jdbcUrl = jdbc:mysql://localhost:3306/web_v01?useSSL=false&serverTimezone=UTC
Observe the addition of &serverTimezone=UTC
In my DB I have the following:
mysql> select * from persona;
+-----+--------------+-------------+------------+
| id | nombre | apellido | fecha |
+-----+--------------+-------------+------------+
...
| 088 | Something | Something | 1981-07-06 |
...
+-----+--------------+-------------+------------+
When the Spring application does a retrieve from the db through the RowMapper<Persona> I can confirm that rs.getDate("fecha") returns 1981-07-05 (observe the day has been decreased by one, it is not correct)
If the mysql-connector-java returns to 6.0.6 and thus mysql.jdbcUrl = jdbc:mysql://localhost:3306/web_v01?useSSL=false (no serverTimezone=UTC) rs.getDate("fecha") returns 1981-07-06 (how is expected)
Thus how fix this working with 8.0.11?.
I want have the same behaviour when serverTimezone never was declared from the beginning, of course avoiding the exception.
Therefore the solution would be better if is take it in consideration that does not matter what value for serverTimezone was declared.
There are several attributes related to timezone:
useTimezone:
Convert time/date types between client and server time zones (true/false, defaults to 'false')? This is part of the legacy date-time code, thus the property has an effect only when "useLegacyDatetimeCode=true."
Default: false
useLegacyDatetimeCode:
Use code for DATE/TIME/DATETIME/TIMESTAMP handling in result sets and statements that consistently handles time zone conversions from client to server and back again, or use the legacy code for these datatypes that has been in the driver for backwards-compatibility? Setting this property to 'false' voids the effects of "useTimezone," "useJDBCCompliantTimezoneShift," "useGmtMillisForDatetimes," and "useFastDateParsing."
Default: true
serverTimezone:
Override detection/mapping of time zone. Used when time zone from server doesn't map to Java time zone
If mysql-connector-java is 5.1, you should specify three attributes, like this:
jdbc:mysql://host:port/dbname?useTimezone=true&useLegacyDatetimeCode=true&serverTimezone=GMT%2B08:00
If mysql-connector-java is 8.0, you should specify one attribute, like this:
jdbc:mysql://host:port/dbname?serverTimezone=GMT%2B08:00
Try to use
jdbc:mysql://localhost:3306/test?useSSL=false&serverTimezone=CST
For my case the most unlikely suspect was jackson.
If your using Spring Boot add the following property to application.properties
spring.jackson.time-zone=Asia/Colombo
I've answered this with detail here -> https://stackoverflow.com/a/68016006/9183199
I am developing a Spring JPA web app on OSX which is currently using MariaDB hosted on my local machine. A query of my timezone settings on MariaDB shows the following:
MariaDB [testdb]> show variables like '%time_zone%';
+------------------+--------+
| Variable_name | Value |
+------------------+--------+
| system_time_zone | EST |
| time_zone | SYSTEM |
+------------------+--------+
2 rows in set (0.00 sec)
MariaDB [testdb]>
I have a #Query which uses a date. The table being queried has a field defined as date. If I turn on trace-level logging I see that the date is properly passed but has been adjusted down by 1 day by the time the select statement is assembled. This obviously causes the query to return incorrect results. If I run the query manually, it obviously fails to return the desired row. If I run the query manually with the unadjusted date, the proper row is returned.
2016-11-25 12:11:46 TRACE org.hibernate.type.descriptor.sql.BasicBinder:81 - binding parameter [3] as [DATE] - [2016-11-04]
2016-11-25 12:29:00 TRACE org.hibernate.loader.Loader:1934 - Bound [4] parameters total
2016-11-25 12:29:00 TRACE org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl:437 - Registering result set [com.mysql.cj.jdbc.result.ResultSetImpl#d38f349]
2016-11-25 12:29:00 TRACE org.hibernate.loader.Loader:946 - Processing result set
2016-11-25 12:29:00 TRACE org.hibernate.loader.Loader:970 - Done processing result set (0 rows)
2016-11-25 12:29:00 TRACE org.hibernate.loader.Loader:1112 - Total objects hydrated: 0
2016-11-25 12:29:00 TRACE org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl:412 - Releasing statement [com.mysql.cj.jdbc.PreparedStatement#bf12e91: select vacationda0_.EMPLOYEE_ID as EMPLOYEE1_3_, vacationda0_.TYPE as TYPE2_3_, vacationda0_.VACATION_DAY as VACATION3_3_ from VACATION_DAY vacationda0_ where (vacationda0_.EMPLOYEE_ID, vacationda0_.TYPE)=('000448719', 'personal') and vacationda0_.VACATION_DAY='2016-11-03']
With tracing, I can see that the problem occurs when the the date is formatted for the EST timezone. I would think that if the database is running the same TimeZone as the app, then the date shouldn't be adjusted. Obviously this isn't the case.
Can someone suggest a way to fix this issue?
The issue is because when you call it by application, it uses the default of system (sync time if it is AWS or other cloud using NTP time server). So from application specify the time zone then your problem will be resolved.
Also, in java 8, timezone is
LocalDateTime date = Instant
.ofEpochMilli(new Date().getTime()).atZone(ZoneId.systemDefault())
.toLocalDateTime();
I am new with Cassandra CQL, I want to get the Cassandra query execution time. Can i do it in CQL shell by storing the current time in the variable, execute the query and then store the current time in another variable and calculate the actual execution time by taking the difference of both variables. Can anyone guide me.
From within cqlsh, your best option is probably to use tracing (output shortened for brevity):
aploetz#cqlsh:stackoverflow> tracing on;
Now Tracing is enabled
aploetz#cqlsh:stackoverflow> SELECT * FROM sujata WHERE id=2;
id | roll_number | age
----+-------------+-----
2 | 10 | 26
2 | 20 | 26
(2 rows)
Tracing session: 35072590-99fb-11e5-beaa-8b496c707234
activity | timestamp | source | source_elapsed
-------------------------------------------------------------------------------------------------+----------------------------+-----------+----------------
Execute CQL3 query | 2015-12-03 14:19:51.027000 | 127.0.0.1 | 0
Parsing SELECT * FROM sujata WHERE id=2; [SharedPool-Worker-1] | 2015-12-03 14:19:51.034000 | 127.0.0.1 | 12378
Preparing statement [SharedPool-Worker-1] | 2015-12-03 14:19:51.035000 | 127.0.0.1 | 13415
Executing single-partition query on roles [SharedPool-Worker-2] | 2015-12-03 14:19:51.036000 | 127.0.0.1 | 14052
.................................................
Read 2 live and 0 tombstone cells [SharedPool-Worker-2] | 2015-12-03 14:19:51.054001 | 127.0.0.1 | 32768
Request complete | 2015-12-03 14:19:51.063069 | 127.0.0.1 | 36069
Edit:
can I store this tracing log report to some file...?
Yes, you can. If I were to run the above trace from the Linux command line, and output that to a file, I would start by creating a file to hold my cqlsh commands:
aploetz#dockingBay94:~/cql$ cat traceSujata.cql
use stackoverflow;
tracing on;
SELECT * FROM sujata WHERE id=2;
Then, I'd use the -f flag on cqlsh to run commands from that file, and then redirect the output to another text file.
aploetz#dockingBay94:~/cql$ cqlsh -f traceSujata.cql > queryTrace_20151204.txt
Now you can peruse the query trace file at your leisure!
Option A
With datastax devcenter you directly have access to the request used time.
Go in the "query_trace" tab, just next to "Results".
More info : http://docs.datastax.com/en/developer/devcenter/doc/devcenter/dcQueryTrace.html
Option B
tracing on
More info : http://www.datastax.com/dev/blog/tracing-in-cassandra-1-2
Nb : Option A uses Option B
(1) If query is a small, just use like:
use nishi;
tracing on;
select * from family where name='nkantkumar';
(2) If query statement is very big say 1k, 5k select statement at a time.
cd <CASSANDRA_HOME>/bin
cqlsh -f '/apps/nkantkumar/query.txt' > '/apps/nkantkumar/traceQuery.cql'
here your query file will be like:-
use nishi;
tracing on;
select * from family where name='nkantkumar';
select * from family where name='nkantkumar1';
select * from family where name='nkantkumar2';
I attempted to use the tracing approach suggested by other answers, but something about the nature of my query meant that tracing was taking a very, very long time to return.
Since it was just a one-off timing for me to compare query performance between two options, I instead wrote a file with SQL commands to execute the command, getting timestamp before & after executing my query. I then copied the timestamps into Microsoft Excel and used it to calculate the difference in seconds (after stripping off the +0000 time zone information so Excel could understand it).
timing.sql
SELECT toTimestamp(now()) FROM system.local;
SELECT * from TABLE_1;
SELECT toTimestamp(now()) FROM system.local;
SELECT * from TABLE_2;
SELECT toTimestamp(now()) FROM system.local;
Execute timing.sql
cqlsh example.com -u my_user --file=timing.sql
Current time output
SELECT toTimestamp(now()) FROM system.local;
system.totimestamp(system.now())
----------------------------------
2020-11-18 16:10:35.745000+0000
Excel date difference
=(C1-B1)*60*60*24
I get an exception (see below) if I try to do
resultset.getString("add_date");
for a JDBC connection to a MySQL database containing a DATETIME value of 0000-00-00 00:00:00 (the quasi-null value for DATETIME), even though I'm just trying to get the value as string, not as an object.
I got around this by doing
SELECT CAST(add_date AS CHAR) as add_date
which works, but seems silly... is there a better way to do this?
My point is that I just want the raw DATETIME string, so I can parse it myself as is.
note: here's where the 0000 comes in: (from http://dev.mysql.com/doc/refman/5.0/en/datetime.html)
Illegal DATETIME, DATE, or TIMESTAMP
values are converted to the “zero”
value of the appropriate type
('0000-00-00 00:00:00' or
'0000-00-00').
The specific exception is this one:
SQLException: Cannot convert value '0000-00-00 00:00:00' from column 5 to TIMESTAMP.
SQLState: S1009
VendorError: 0
java.sql.SQLException: Cannot convert value '0000-00-00 00:00:00' from column 5 to TIMESTAMP.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
at com.mysql.jdbc.ResultSetImpl.getTimestampFromString(ResultSetImpl.java:6343)
at com.mysql.jdbc.ResultSetImpl.getStringInternal(ResultSetImpl.java:5670)
at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5491)
at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5531)
Alternative answer, you can use this JDBC URL directly in your datasource configuration:
jdbc:mysql://yourserver:3306/yourdatabase?zeroDateTimeBehavior=convertToNull
Edit:
Source: MySQL Manual
Datetimes with all-zero components (0000-00-00 ...) — These values can not be represented reliably in Java. Connector/J 3.0.x always converted them to NULL when being read from a ResultSet.
Connector/J 3.1 throws an exception by default when these values are encountered as this is the most correct behavior according to the JDBC and SQL standards. This behavior can be modified using the zeroDateTimeBehavior configuration property. The allowable values are:
exception (the default), which throws an SQLException with an SQLState of S1009.
convertToNull, which returns NULL instead of the date.
round, which rounds the date to the nearest closest value which is 0001-01-01.
Update: Alexander reported a bug affecting mysql-connector-5.1.15 on that feature. See CHANGELOGS on the official website.
I stumbled across this attempting to solve the same issue. The installation I am working with uses JBOSS and Hibernate, so I had to do this a different way. For the basic case, you should be able to add zeroDateTimeBehavior=convertToNull to your connection URI as per this configuration properties page.
I found other suggestions across the land referring to putting that parameter in your hibernate config:
In hibernate.cfg.xml:
<property name="hibernate.connection.zeroDateTimeBehavior">convertToNull</property>
In hibernate.properties:
hibernate.connection.zeroDateTimeBehavior=convertToNull
But I had to put it in my mysql-ds.xml file for JBOSS as:
<connection-property name="zeroDateTimeBehavior">convertToNull</connection-property>
Hope this helps someone. :)
My point is that I just want the raw DATETIME string, so I can parse it myself as is.
That makes me think that your "workaround" is not a workaround, but in fact the only way to get the value from the database into your code:
SELECT CAST(add_date AS CHAR) as add_date
By the way, some more notes from the MySQL documentation:
MySQL Constraints on Invalid Data:
Before MySQL 5.0.2, MySQL is forgiving of illegal or improper data values and coerces them to legal values for data entry. In MySQL 5.0.2 and up, that remains the default behavior, but you can change the server SQL mode to select more traditional treatment of bad values such that the server rejects them and aborts the statement in which they occur.
[..]
If you try to store NULL into a column that doesn't take NULL values, an error occurs for single-row INSERT statements. For multiple-row INSERT statements or for INSERT INTO ... SELECT statements, MySQL Server stores the implicit default value for the column data type.
MySQL 5.x Date and Time Types:
MySQL also allows you to store '0000-00-00' as a “dummy date” (if you are not using the NO_ZERO_DATE SQL mode). This is in some cases more convenient (and uses less data and index space) than using NULL values.
[..]
By default, when MySQL encounters a value for a date or time type that is out of range or otherwise illegal for the type (as described at the beginning of this section), it converts the value to the “zero” value for that type.
DATE_FORMAT(column name, '%Y-%m-%d %T') as dtime
Use this to avoid the error. It return the date in string format and then you can get it as a string.
resultset.getString("dtime");
This actually does NOT work. Even though you call getString. Internally mysql still tries to convert it to date first.
at com.mysql.jdbc.ResultSetImpl.getDateFromString(ResultSetImpl.java:2270)
~[mysql-connector-java-5.1.15.jar:na] at com.mysql.jdbc.ResultSetImpl.getStringInternal(ResultSetImpl.java:5743)
~[mysql-connector-java-5.1.15.jar:na] at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5576)
~[mysql-connector-java-5.1.15.jar:na]
If, after adding lines:
<property
name="hibernate.connection.zeroDateTimeBehavior">convertToNull</property>
hibernate.connection.zeroDateTimeBehavior=convertToNull
<connection-property
name="zeroDateTimeBehavior">convertToNull</connection-property>
continues to be an error:
Illegal DATETIME, DATE, or TIMESTAMP values are converted to the “zero” value of the appropriate type ('0000-00-00 00:00:00' or '0000-00-00').
find lines:
1) resultSet.getTime("time"); // time = 00:00:00
2) resultSet.getTimestamp("timestamp"); // timestamp = 00000000000000
3) resultSet.getDate("date"); // date = 0000-00-00 00:00:00
replace with the following lines, respectively:
1) Time.valueOf(resultSet.getString("time"));
2) Timestamp.valueOf(resultSet.getString("timestamp"));
3) Date.valueOf(resultSet.getString("date"));
I wrestled with this problem and implemented the 'convertToNull' solutions discussed above. It worked in my local MySql instance. But when I deployed my Play/Scala app to Heroku it no longer would work. Heroku also concatenates several args to the DB URL that they provide users, and this solution, because of Heroku's use concatenation of "?" before their own set of args, will not work. However I found a different solution which seems to work equally well.
SET sql_mode = 'NO_ZERO_DATE';
I put this in my table descriptions and it solved the problem of '0000-00-00 00:00:00' can not be represented as java.sql.Timestamp
I suggest you use null to represent a null value.
What is the exception you get?
BTW:
There is no year called 0 or 0000. (Though some dates allow this year)
And there is no 0 month of the year or 0 day of the month. (Which may be the cause of your problem)
I solved the problem considerating '00-00-....' isn't a valid date, then, I changed my SQL column definition adding "NULL" expresion to permit null values:
SELECT "-- Tabla item_pedido";
CREATE TABLE item_pedido (
id INTEGER AUTO_INCREMENT PRIMARY KEY,
id_pedido INTEGER,
id_item_carta INTEGER,
observacion VARCHAR(64),
fecha_estimada TIMESTAMP,
fecha_entrega TIMESTAMP NULL, // HERE IS!!.. NULL = DELIVERY DATE NOT SET YET
CONSTRAINT fk_item_pedido_id_pedido FOREIGN KEY (id_pedido)
REFERENCES pedido(id),...
Then, I've to be able to insert NULL values, that means "I didnt register that timestamp yet"...
SELECT "++ INSERT item_pedido";
INSERT INTO item_pedido VALUES
(01, 01, 01, 'Ninguna', ADDDATE(#HOY, INTERVAL 5 MINUTE), NULL),
(02, 01, 02, 'Ninguna', ADDDATE(#HOY, INTERVAL 3 MINUTE), NULL),...
The table look that:
mysql> select * from item_pedido;
+----+-----------+---------------+-------------+---------------------+---------------------+
| id | id_pedido | id_item_carta | observacion | fecha_estimada | fecha_entrega |
+----+-----------+---------------+-------------+---------------------+---------------------+
| 1 | 1 | 1 | Ninguna | 2013-05-19 15:09:48 | NULL |
| 2 | 1 | 2 | Ninguna | 2013-05-19 15:07:48 | NULL |
| 3 | 1 | 3 | Ninguna | 2013-05-19 15:24:48 | NULL |
| 4 | 1 | 6 | Ninguna | 2013-05-19 15:06:48 | NULL |
| 5 | 2 | 4 | Suave | 2013-05-19 15:07:48 | 2013-05-19 15:09:48 |
| 6 | 2 | 5 | Seco | 2013-05-19 15:07:48 | 2013-05-19 15:12:48 |
| 7 | 3 | 5 | Con Mayo | 2013-05-19 14:54:48 | NULL |
| 8 | 3 | 6 | Bilz | 2013-05-19 14:57:48 | NULL |
+----+-----------+---------------+-------------+---------------------+---------------------+
8 rows in set (0.00 sec)
Finally: JPA in action:
#Stateless
#LocalBean
public class PedidosServices {
#PersistenceContext(unitName="vagonpubPU")
private EntityManager em;
private Logger log = Logger.getLogger(PedidosServices.class.getName());
#SuppressWarnings("unchecked")
public List<ItemPedido> obtenerPedidosRetrasados() {
log.info("Obteniendo listado de pedidos retrasados");
Query qry = em.createQuery("SELECT ip FROM ItemPedido ip, Pedido p WHERE" +
" ip.fechaEntrega=NULL" +
" AND ip.idPedido=p.id" +
" AND ip.fechaEstimada < :arg3" +
" AND (p.idTipoEstado=:arg0 OR p.idTipoEstado=:arg1 OR p.idTipoEstado=:arg2)");
qry.setParameter("arg0", Tipo.ESTADO_BOUCHER_ESPERA_PAGO);
qry.setParameter("arg1", Tipo.ESTADO_BOUCHER_EN_SERVICIO);
qry.setParameter("arg2", Tipo.ESTADO_BOUCHER_RECIBIDO);
qry.setParameter("arg3", new Date());
return qry.getResultList();
}
At last all its work. I hope that help you.
To add to the other answers: If yout want the 0000-00-00 string, you can use noDatetimeStringSync=true (with the caveat of sacrificing timezone conversion).
The official MySQL bug: https://bugs.mysql.com/bug.php?id=47108.
Also, for history, JDBC used to return NULL for 0000-00-00 dates but now return an exception by default.
Source
you can append the jdbc url with
?zeroDateTimeBehavior=convertToNull&autoReconnect=true&characterEncoding=UTF-8&characterSetResults=UTF-8
With the help of this, sql convert '0000-00-00 00:00:00' as null value.
eg:
jdbc:mysql:<host-name>/<db-name>?zeroDateTimeBehavior=convertToNull&autoReconnect=true&characterEncoding=UTF-8&characterSetResults=UTF-8