My code is like
String path = "/home/user/tmp/file1";
Path p = FileSystems.getDefault().getPath(path);
PosixFileAttributes attrs = Files.readAttributes(p, PosixFileAttributes.class);
System.out.println("Last Modified Time: "+attrs.lastModifiedTime());
System.out.println("Last Access Time: "+attrs.lastAccessTime());
The time returned by lastModifiedTime() and lastAccessTime() are 4 hours difference with the correct one.
The output is
Last Modified Time: 2014-06-25T12:50:31Z
Last Access Time: 2014-06-25T18:26:07Z
stat file1 produce:
Access: 2014-06-25 14:26:07.870281008 -0400
Modify: 2014-06-25 08:50:31.922861913 -0400
Change: 2014-06-25 08:50:31.922861913 -0400
Any one can help me?
A time like
2014-06-25T12:50:31Z
is in UTC (that's the Z at the end), so it may be off according to your time zone.
Related
Environment: Java 1.8, VM Cloudera Quickstart.
I have data into Hadoop hdfs from a csv file. Each row represents a bus route.
id vendor start_datetime end_datetime trip_duration_in_sec
17534 A 1/1/2013 12:00 1/1/2013 12:14 840
68346 A 1/1/2013 12:13 1/1/2013 12:18 300
09967 B 1/1/2013 12:34 1/1/2013 12:39 300
09967 B 1/1/2013 12:44 1/1/2013 12:51 420
09967 A 1/1/2013 12:54 1/1/2013 12:56 120
.........
.........
So, i want for every day, to find the hour that each vendor (A and B) has the most bus routes. With java and spark.
A result could be:
1/1/2013 (Day 1) - Vendor A has 3 bus routes at 12:00-13:00 hour. (That time 12:00-13:00, vendor A had the most bus routes..)
1/1/2013 (Day 1) - Vendor B has 2 bus routes at 12:00-13:00 hour. (That time 12:00-13:00, vendor B had the most bus routes..)
....
Mu java code is:
import static org.apache.spark.sql.functions;
import static org.apache.spark.sql.Row;
Dataset<Row> ds;
ds.groupBy(functions.window(col("start_datetime"), "1 hour").count().show();
But i cant find in which hour are the max routes per day.
I'm not so familiar in Java so I tried to explain it in Scala.
The key to find out the hour of max routes per day per vendor, is to count by (vendor, day, hour), then aggregate by (vendor, day) to calculate the hour corresponding to maximum cnt of each group. The day and the hour of each record could be parsed by start_datetime.
val df = spark.createDataset(Seq(
("17534","A","1/1/2013 12:00","1/1/2013 12:14",840),
("68346","A","1/1/2013 12:13","1/1/2013 12:18",300),
("09967","B","1/1/2013 12:34","1/1/2013 12:39",300),
("09967","B","1/1/2013 12:44","1/1/2013 12:51",420),
("09967","A","1/1/2013 12:54","1/1/2013 12:56",120)
)).toDF("id","vendor","start_datetime","end_datetime","trip_duration_in_sec")
df.rdd.map(t => {
val vendor = t(1)
val day = t(2).toString.split(" ")(0)
val hour = t(2).toString.split(" ")(1).split(":")(0)
((vendor, day, hour), 1)
})
// count by key
.aggregateByKey(0)((x: Int, y: Int) =>x+y, (x: Int, y: Int) =>x+y)
.map(t => {
val ((vendor, day, hour), cnt) = t;
((vendor, day), (hour, cnt))
})
// solve the max cnt by key (vendor, day)
.foldByKey(("", 0))((z: (String, Int), i: (String, Int)) => if (i._2 > z._2) i else z)
.foreach(t => println(s"${t._1._2} - Vendor ${t._1._1} has ${t._2._2} bus routes from ${t._2._1}:00 hour."))
I am trying to get page load time in automation testing project.
pageLoad3.start();
WebDriverWait wait3 = new WebDriverWait(driver, 30);
wait3.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(hotelNameDetailsXpath)));
pageLoad3.stop();
long pageLoadTime_ms5 = pageLoad3.getTime();
long pageLoadTime_Seconds5 = pageLoadTime_ms5 / 1000;
//System.out.println("Time taken to get load Total price element ::");
System.out.println("pageLoadTime_ms5 ::"+pageLoadTime_ms5);
System.out.println("Time taken to load DB Response :: " + pageLoadTime_Seconds5 + " seconds");
Output:
pageLoadTime_ms5 ::11479
Time taken to load DB Response :: 11 seconds
I am getting 11 seconds but not able to get for 479, how can I get it in below format?
Actual Requirement: I want time to get in below format like
11 seconds 47 milliseconds i.e. (00:11:47)
To get the output as 11 seconds 47 milliseconds you can use the following solution:
Code Block:
public class division_by_1000 {
public static void main(String[] args) {
//long pageLoadTime_ms5 = pageLoad3.getTime();
// assuming pageLoadTime_ms5 = 11479
String pageLoadTime_ms5 = "11479";
System.out.println("pageLoadTime_ms5 ::"+pageLoadTime_ms5);
System.out.println("Time taken to load DB Response :: " + (Integer.parseInt(pageLoadTime_ms5)/1000) + " seconds " + (Integer.parseInt(pageLoadTime_ms5)%1000) + " millisseconds ");
}
}
Console Output:
pageLoadTime_ms5 ::11479
Time taken to load DB Response :: 11 seconds 479 millisseconds
Having been burned by mysql timezone and Daylight Savings "hour from hell" issues in the past, I decided my next application would store everything in UTC timezone and only interact with the database using UTC times (not even the closely-related GMT).
I soon ran into some mysterious bugs. After pulling my hair out for a while, I came up with this test code:
try(Connection conn = dao.getDataSource().getConnection();
Statement stmt = conn.createStatement()) {
Instant now = Instant.now();
stmt.execute("set time_zone = '+00:00'");
stmt.execute("create temporary table some_times("
+ " dt datetime,"
+ " ts timestamp,"
+ " dt_string datetime,"
+ " ts_string timestamp,"
+ " dt_epoch datetime,"
+ " ts_epoch timestamp,"
+ " dt_auto datetime default current_timestamp(),"
+ " ts_auto timestamp default current_timestamp(),"
+ " dtc char(19) generated always as (cast(dt as character)),"
+ " tsc char(19) generated always as (cast(ts as character)),"
+ " dt_autoc char(19) generated always as (cast(dt_auto as character)),"
+ " ts_autoc char(19) generated always as (cast(ts_auto as character))"
+ ")");
PreparedStatement ps = conn.prepareStatement("insert into some_times "
+ "(dt, ts, dt_string, ts_string, dt_epoch, ts_epoch) values (?,?,?,?,from_unixtime(?),from_unixtime(?))");
DateTimeFormatter dbFormat = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss").withZone(ZoneId.of("UTC"));
ps.setTimestamp(1, new Timestamp(now.toEpochMilli()));
ps.setTimestamp(2, new Timestamp(now.toEpochMilli()));
ps.setString(3, dbFormat.format(now));
ps.setString(4, dbFormat.format(now));
ps.setLong(5, now.getEpochSecond());
ps.setLong(6, now.getEpochSecond());
ps.executeUpdate();
ResultSet rs = stmt.executeQuery("select * from some_times");
ResultSetMetaData md = rs.getMetaData();
while(rs.next()) {
for(int c=1; c <= md.getColumnCount(); ++c) {
Instant inst1 = Instant.ofEpochMilli(rs.getTimestamp(c).getTime());
Instant inst2 = Instant.from(dbFormat.parse(rs.getString(c).replaceAll("\\.0$", "")));
System.out.println(inst1.getEpochSecond() - now.getEpochSecond());
System.out.println(inst2.getEpochSecond() - now.getEpochSecond());
}
}
}
Note how the session timezone is set to UTC, and everything in the Java code is very timezone-aware and forced to UTC. The only thing in this entire environment which is not UTC is the JVM's default timezone.
I expected the output to be a bunch of 0s but instead I get this
0
-28800
0
-28800
28800
0
28800
0
28800
0
28800
0
28800
0
28800
0
0
-28800
0
-28800
28800
0
28800
0
Each line of output is just subtracting the time stored from the time retrieved. The result in each row should be 0.
It seems the JDBC driver is performing inappropriate timezone conversions. For an application which interacts fully in UTC although it runs on a VM that's not in UTC, is there any way to completely disable the TZ conversions?
i.e. Can this test be made to output all-zero rows?
UPDATE
Using useLegacyDatetimeCode=false (cacheDefaultTimezone=false makes no difference) changes the output but still not a fix:
0
-28800
0
-28800
0
-28800
0
-28800
0
-28800
0
-28800
0
-28800
0
-28800
0
0
0
0
0
0
0
0
UPDATE2
Checking the console (after changing the test to create a permanent table), I see all the values are STORED correctly:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 27148
Server version: 5.7.12-log MySQL Community Server (GPL)
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> set time_zone = '-00:00';
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT * FROM some_times \G
*************************** 1. row ***************************
dt: 2016-11-18 15:39:51
ts: 2016-11-18 15:39:51
dt_string: 2016-11-18 15:39:51
ts_string: 2016-11-18 15:39:51
dt_epoch: 2016-11-18 15:39:51
ts_epoch: 2016-11-18 15:39:51
dt_auto: 2016-11-18 15:39:51
ts_auto: 2016-11-18 15:39:51
dtc: 2016-11-18 15:39:51
tsc: 2016-11-18 15:39:51
dt_autoc: 2016-11-18 15:39:51
ts_autoc: 2016-11-18 15:39:51
1 row in set (0.00 sec)
mysql>
The solution is to set JDBC connection parameter noDatetimeStringSync=true with useLegacyDatetimeCode=false. As a bonus I also found sessionVariables=time_zone='-00:00' alleviates the need to set time_zone explicitly on every new connection.
There is some "intelligent" timezone conversion code that gets activated deep inside the ResultSet.getString() method when it detects that the column is a TIMESTAMP column.
Alas, this intelligent code has a bug: TimeUtil.fastTimestampCreate(TimeZone tz, int year, int month, int day, int hour, int minute, int seconds, int secondsPart) returns a Timestamp wrongly tagged to the JVM's default timezone, even when the tz parameter is set to something else:
final static Timestamp fastTimestampCreate(TimeZone tz, int year, int month, int day, int hour, int minute, int seconds, int secondsPart) {
Calendar cal = (tz == null) ? new GregorianCalendar() : new GregorianCalendar(tz);
cal.clear();
// why-oh-why is this different than java.util.date, in the year part, but it still keeps the silly '0' for the start month????
cal.set(year, month - 1, day, hour, minute, seconds);
long tsAsMillis = cal.getTimeInMillis();
Timestamp ts = new Timestamp(tsAsMillis);
ts.setNanos(secondsPart);
return ts;
}
The return ts would be perfectly valid except when further up in the call chain, it is converted back to a string using the bare toString() method, which renders the ts as a String representing what a clock would display in the JVM-default timezone, instead of a String representation of the time in UTC. In ResultSetImpl.getStringInternal(int columnIndex, boolean checkDateTypes):
case Types.TIMESTAMP:
Timestamp ts = getTimestampFromString(columnIndex, null, stringVal, this.getDefaultTimeZone(), false);
if (ts == null) {
this.wasNullFlag = true;
return null;
}
this.wasNullFlag = false;
return ts.toString();
Setting noDatetimeStringSync=true disables the entire parse/unparse mess and just returns the string value as-is received from the database.
Test output:
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
The useLegacyDatetimeCode=false is still important because it changes the behaviour of getDefaultTimeZone() to use the database server's TZ.
While chasing this down I also found the documentation for useJDBCCompliantTimezoneShift is incorrect, although it makes no difference: documentation says [This is part of the legacy date-time code, thus the property has an effect only when "useLegacyDatetimeCode=true."], but that's wrong, see ResultSetImpl.getNativeTimestampViaParseConversion(int, Calendar, TimeZone, boolean).
I was trying a example code with spring. And a part of code is like below;
private List<Point> points;
long timeTakeninMilis = System.currentTimeMillis();
public List<Point> getPoints() {
return points;
}
public void setPoints(List<Point> points) {
this.points = points;
}
public void drawJava8() {
points.stream().forEachOrdered(
point -> System.out.println("Point : (" + point.getX() + ", "
+ point.getY() + ")"));
System.out.println("Total Time Taken drawJava8(): "
+ (System.currentTimeMillis() - timeTakeninMilis)
+ " miliseconds");
}
public void draw() {
for (Point point : points) {
System.out.println("Point = (" + point.getX() + ", " + point.getY()
+ " )");
}
System.out.println("Total Time Taken draw(): "
+ (System.currentTimeMillis() - timeTakeninMilis)
+ " miliseconds");
}
The OUTPUT,
Jun 30, 2015 11:30:53 AM org.springframework.context.support.ClassPathXmlApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext#7daf6ecc: startup date [Tue Jun 30 11:30:53 IST 2015]; root of context hierarchy
Jun 30, 2015 11:30:53 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [spring.xml]
Point = (0, 0 )
Point = (-50, 0 )
Point = (0, 50 )
Total Time Taken draw(): 70 miliseconds
Point : (0, 0)
Point : (-50, 0)
Point : (0, 50)
Total Time Taken drawJava8(): 124 miliseconds
Jun 30, 2015 11:30:54 AM org.springframework.context.support.ClassPathXmlApplicationContext doClose
INFO: Closing org.springframework.context.support.ClassPathXmlApplicationContext#7daf6ecc: startup date [Tue Jun 30 11:30:53 IST 2015]; root of context hierarchy
Why it is taking more time?
Or i am doing something wrong?
I was expecting it to be faster or of similar speed...
Please help me understand what is the benefit of the Lambda Expressions?
INFO: I did it in two different programs. Times are taken from those. I merged them here to make it short.
Adding this as an analysis per original poster's request.
We can not really predict the sophisticated analysis and transformation that the modern JIT compiler performs on running code. Hence while benchmarking items such as these, you should not conclude it just by running two method calls.
Instead, create various sample input sets (boundary cases) and check the perofmance by repeatedly calling your test cases without shutting down JVM. In this case for example :
for (int i=0;i<100;i++){draw(); drawJava8();}
Once you have the results, find out average execution and you can safely ignore first execution result as it might not have had optimizations.
So the conclusion you have drawn from your tests is not completely correct.
Please suggest, I am retrieving data using ibatis ORM from postgres database.
When I fire the below query using ibatis the timestamp "PVS.SCHED_DTE" get reduced to "2013-06-07 23:30:00.0" where as in db timestamp is "2013-06-08 05:00:00.0".
Though when the same query is executed using postgresAdminII, its gives proper PVS.SCHED_DTE.
SELECT PVS.PTNT_VISIT_SCHED_NUM, PVS.PTNT_ID,P.FIRST_NAM AS PTNT_FIRST_NAM,P.PRIM_EMAIL_TXT,P.SCNDRY_EMAIL_TXT, PVS.PTNT_NOTIFIED_FLG,PVS.SCHED_DTE,PVS.VISIT_TYPE_NUM,PVS.DURTN_TYPE_NUM,PVS.ROUTE_TYPE_NUM, C.FIRST_NAM AS CARE_FIRST_NAM,C.MIDDLE_INIT_NAM AS CARE_MIDDLE_NAM,C.LAST_NAM AS CARE_LAST_NAM, C.PHONE_NUM_1,ORGNIZATN.EMAIL_HOST_SERVER_TXT,ORGNIZATN.PORT_NUM, ORGNIZATN.MAIL_SENDER_USER_ID,ORGNIZATN.MAIL_SENDER_PSWD_DESC,ORGNIZATN.SOCKET_FCTRY_PORT_DESC, ORGNIZATN.TIMEZONE_ID FROM IHCAS.PTNT_VISIT_SCHED PVS , IHCAS.PTNT P, IHCAS.ROUTE_LEG RL, IHCAS.ROUTE R, IHCAS.CAREGIVER C,IHCAS.ORG ORGNIZATN WHERE ORGNIZATN.ORG_NUM = ? AND PVS.SCHED_STAT_CDE = '2002' AND PVS.PTNT_NOTIFIED_FLG = FALSE AND **PVS.SCHED_DTE >= (CURRENT_DATE + interval '1 day') AT TIME ZONE ? at TIME ZONE 'UTC'** AND **PVS.SCHED_DTE < (CURRENT_DATE + interval '2 day' - interval '1 sec') AT TIME ZONE ? at TIME ZONE 'UTC'** AND PVS.PTNT_ID = P.PTNT_ID AND PVS.PTNT_VISIT_SCHED_NUM = RL.PTNT_VISIT_SCHED_NUM AND RL.ROUTE_NUM = R.ROUTE_NUM AND C.CAREGIVER_NUM=R.ASGN_TO_CAREGIVER_NUM AND PVS.ORG_NUM= ORGNIZATN.ORG_NUM ORDER BY PVS.SCHED_DTE ASC,PVS.ROUTE_TYPE_NUM ASC,PVS.ORG_NUM ASC
Parameters: **[1, EST, EST]**
Header: [PTNT_VISIT_SCHED_NUM, PTNT_NOTIFIED_FLG, **SCHED_DTE**, VISIT_TYPE_NUM, DURTN_TYPE_NUM, ROUTE_TYPE_NUM, PTNT_ID, PTNT_FIRST_NAM, PRIM_EMAIL_TXT, SCNDRY_EMAIL_TXT, CARE_FIRST_NAM, CARE_MIDDLE_NAM, CARE_LAST_NAM, PHONE_NUM_1, EMAIL_HOST_SERVER_TXT, PORT_NUM, MAIL_SENDER_USER_ID, MAIL_SENDER_PSWD_DESC, SOCKET_FCTRY_PORT_DESC, TIMEZONE_ID]
Result: [1129, false, **2013-06-07 23:30:00.0**, 1002, 1551, 1702, PID146, Ricky, Receiver.test#******.com, , Vikas, S, Jain, 956-**-5**8, 172.17.*.***, 25, Sender.test#******.com, ********123, 465, EST]
For me.. it seems to be some timezone issue but what and how to resolve.
N.B : Local Time zone : IST
Application Time zone : EST
In database, datatype of column is timestamp without timezone.
Any suggestion.. ??
It may be best to set the database timezone to UTC and implement an UTC-preserving MyBatis type handler as shown in the accepted answer to this:
What's the right way to handle UTC date-times using Java, iBatis, and Oracle?.