Problem. I need a way to find Starteam server time through Starteam Java SDK 8.0. Version of server is 8.0.172 so method Server.getCurrentTime() is not available since it was added only in server version 9.0.
Motivation. My application needs to use views at specific dates. So if there's some difference in system time between client (where the app is running) and server then obtained views are not accurate. In the worst case the client's requested date is in the future for server so the operation results in exception.
After some investigation I haven't found any cleaner solution than using a temporary item. My app requests the item's time of creation and compares it with local time. Here's the method I use to get server time:
public Date getCurrentServerTime() {
Folder rootFolder = project.getDefaultView().getRootFolder();
Topic newItem = (Topic) Item.createItem(project.getTypeNames().TOPIC, rootFolder);
newItem.update();
newItem.remove();
newItem.update();
return newItem.getCreatedTime().createDate();
}
If your StarTeam server is on a Windows box and your code will be executing on a Windows box, you could shell out and execute the NET time command to fetch the time on that machine and then compare it to the local time.
net time \\my_starteam_server_machine_name
which should return:
"Current time at \\my_starteam_server_machine_name is 10/28/2008 2:19 PM"
"The command completed successfully."
We needed to come up with a way of finding the server time for use with CodeCollab. Here is a (longish) C# code sample of how to do it without creating a temporary file. Resolution is 1 second.
static void Main(string[] args)
{
// ServerTime replacement for pre-2006 StarTeam servers.
// Picks a date in the future.
// Gets a view, sets the configuration to the date, and tries to get a property from the root folder.
// If it cannot retrieve the property, the date is too far in the future. Roll back the date to an earlier time.
DateTime StartTime = DateTime.Now;
Server s = new Server("serverAddress", 49201);
s.LogOn("User", "Password");
// Getting a view - doesn't matter which, as long as it is not deleted.
Project p = s.Projects[0];
View v = p.AccessibleViews[0]; // AccessibleViews saves checking permissions.
// Timestep to use when searching. One hour is fairly quick for resolution.
TimeSpan deltaTime = new TimeSpan(1, 0, 0);
deltaTime = new TimeSpan(24 * 365, 0, 0);
// Invalid calls return faster - start a ways in the future.
TimeSpan offset = new TimeSpan(24, 0, 0);
// Times before the view was created are invalid.
DateTime minTime = v.CreatedTime;
DateTime localTime = DateTime.Now;
if (localTime < minTime)
{
System.Console.WriteLine("Current time is older than view creation time: " + minTime);
// If the dates are so dissimilar that the current date is before the creation date,
// it is probably a good idea to use a bigger delta.
deltaTime = new TimeSpan(24 * 365, 0, 0);
// Set the offset to the minimum time and work up from there.
offset = minTime - localTime;
}
// Storage for calculated date.
DateTime testTime;
// Larger divisors converge quicker, but might take longer depending on offset.
const float stepDivisor = 10.0f;
bool foundValid = false;
while (true)
{
localTime = DateTime.Now;
testTime = localTime.Add(offset);
ViewConfiguration vc = ViewConfiguration.CreateFromTime(testTime);
View tempView = new View(v, vc);
System.Console.Write("Testing " + testTime + " (Offset " + (int)offset.TotalSeconds + ") (Delta " + deltaTime.TotalSeconds + "): ");
// Unfortunately, there is no isValid operation. Attempting to
// read a property from an invalid date configuration will
// throw an exception.
// An alternate to this would be proferred.
bool valid = true;
try
{
string testname = tempView.RootFolder.Name;
}
catch (ServerException)
{
System.Console.WriteLine(" InValid");
valid = false;
}
if (valid)
{
System.Console.WriteLine(" Valid");
// If the last check was invalid, the current check is valid, and
// If the change is this small, the time is very close to the server time.
if (foundValid == false && deltaTime.TotalSeconds <= 1)
{
break;
}
foundValid = true;
offset = offset.Add(deltaTime);
}
else
{
offset = offset.Subtract(deltaTime);
// Once a valid time is found, start reducing the timestep.
if (foundValid)
{
foundValid = false;
deltaTime = new TimeSpan(0,0,Math.Max((int)(deltaTime.TotalSeconds / stepDivisor), 1));
}
}
}
System.Console.WriteLine("Run time: " + (DateTime.Now - StartTime).TotalSeconds + " seconds.");
System.Console.WriteLine("The local time is " + localTime);
System.Console.WriteLine("The server time is " + testTime);
System.Console.WriteLine("The server time is offset from the local time by " + offset.TotalSeconds + " seconds.");
}
Output:
Testing 4/9/2009 3:05:40 PM (Offset 86400) (Delta 31536000): InValid
Testing 4/9/2008 3:05:40 PM (Offset -31449600) (Delta 31536000): Valid
...
Testing 4/8/2009 10:05:41 PM (Offset 25200) (Delta 3): InValid
Testing 4/8/2009 10:05:38 PM (Offset 25197) (Delta 1): Valid
Run time: 9.0933426 seconds.
The local time is 4/8/2009 3:05:41 PM
The server time is 4/8/2009 10:05:38 PM
The server time is offset from the local time by 25197 seconds.
<stab_in_the_dark>
I'm not familiar with that SDK but from looking at the API if the server is in a known timezone why not create and an OLEDate object whose date is going to be the client's time rolled appropriately according to the server's timezone?
</stab_in_the_dark>
Related
In my application I read file using following method,
public void readFIleData(String path) {
BufferedReader br = null;
try {
String sCurrentLine;
br = new BufferedReader(new FileReader(path));
while ((sCurrentLine = br.readLine()) != null) {
System.out.println("Data : "+sCurrentLine);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (br != null)br.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
Also I get last access time and last modified time of the file using following method,
public void getFIleInfo(String path) {
Path file = Paths.get(path);
try {
BasicFileAttributes attrs = Files.readAttributes(file, BasicFileAttributes.class);
FileTime accessTime = attrs.lastAccessTime();
System.out.println("accessTime : "+accessTime.toMillis());
FileTime modifiedTime = attrs.lastModifiedTime();
System.out.println("modifiedTime : "+modifiedTime.toMillis());
} catch (IOException e) {
e.printStackTrace();
}
}
I ran above methods in following order,
1-> getFIleInfo()
2-> readFIleData()
3-> getFIleInfo()
I got following as output,
accessTime : 1462943491685
modifiedTime : 1462943925846
Data : erteuyuittdgfdfghjkhw5643rtrr66664fdghf
accessTime : 1462943491685
modifiedTime : 1462943925846
Here is output times in string format,
accessTime : 2016-05-11T05:11:31.685881Z
modifiedTime : 2016-05-11T07:39:28.237884Z
Data : erteuyuittdgfdfghjkhw5643rtrr66LE229F1HBQ664fdghf
accessTime : 2016-05-11T05:11:31.685881Z
modifiedTime : 2016-05-11T07:39:28.237884Z
I have a doubt about this output because access time remains the same as before reading the data of the file. Can somebody please explain to me what is actually mean by last access time and last modified time in java?
First, let's focus on what these things mean.
Access - the last time the file was read, i.e., the last time the file data was accessed.
Modify - the last time the file was modified (content has been modified), i.e., time when file data last modified.
Change - the last time meta data of the file was changed (e.g. permissions), i.e., time when file status was last changed.
Edit.
The access time IS changing. I suggest you use Thread.sleep(100) or something and then see if this problem persists.
If it does, the culprit would have to the be the OS you are running since Java simply reads from the filesystem. #Serge Ballesta's comments should give an understanding about the Windows NTFS having an option to disable writing every change made to the file attributes back to the hard drive for performance reasons. There is actually more to this.
From [docs],
NTFS delays updates to the last access time for a file by up to one hour after the last access. NTFS also permits last access time updates to be disabled. Last access time is not updated on NTFS volumes by default.
Following is some data from running the script on mac os x.
calling getFileInfo() at: 11.4.2016 3:13:08:738
accessTime : 11.4.2016 3:12:53:0
modifiedTime : 29.10.2015 1:49:14:0
--------------------
sleeping for 100ms
--------------------
calling readFIleData() at: 11.4.2016 3:13:08:873
--------------------
sleeping for 100ms
--------------------
re-calling getFileInfo() at: 11.4.2016 3:13:08:977
accessTime : 11.4.2016 3:13:08:0 <---- READING FILE CHANGES ACCESS TIME
modifiedTime : 29.10.2015 1:49:14:0
--------------------
sleeping for 100ms
--------------------
re-calling getFileInfo() at: 11.4.2016 3:13:09:81
accessTime : 11.4.2016 3:13:08:0 <---- READING FILE ATTRIBUTES DOES NOT CHANGE ACCESS TIME
modifiedTime : 29.10.2015 1:49:14:0
To enhance clarity, you can convert the milliseconds you have, to something more readable. The following code snippet will elaborate on that.
long accessTimeSinceEpoch = Files.readAttributes(file, BasicFileAttributes.class).lastAccessTime().toMillis();
Calendar calendar = Calendar.getInstance();
calendar.setTimeInMillis(accessTimeSinceEpoch);
int mYear = calendar.get(Calendar.YEAR);
int mMonth = calendar.get(Calendar.MONTH);
int mDay = calendar.get(Calendar.DAY_OF_MONTH);
int mHour = calendar.get(Calendar.HOUR);
int mMin = calendar.get(Calendar.MINUTE);
int mSec = calendar.get(Calendar.SECOND);
int mMilisec = calendar.get(Calendar.MILLISECOND);
String st = mDay + "." + mMonth + "." + mYear + " " + mHour + ":" + mMin + ":" + mSec + ":" + mMilisec;
If you look into the api you have this.
If the file system implementation does not support a time stamp
to indicate the time of last access then this method returns
an implementation specific default value, typically the last-modified-time or a FileTime
representing the epoch (1970-01-01T00:00:00Z).
It looks pretty much like the "problem" is related to your file system and your operating system. I don't think your code has anything wrong in it.
For example, for a windows operating system, the NtfsDisableLastAccessUpdate option was enabled by default in Vista and Windows 7, but you can disable it by using the following command line:
fsutil behavior set disablelastaccess 0
As I said in the comment to your question I was able to solve this problem in Windows in a real machine, but not in a virtual one. If you are still struggling with this issue then issue this command prior to anything to see whats going on with the registry:
fsutil behavior query disablelastaccess
On a last note, I did not had to restart windows or Intellij (where I ran my tests). The results were immediate and I could see that for value 1 the timestamp for the last access does not change and for value 0 it does.
I have an android service that runs on background, sample sensors and write them to a file with their time-stamp.
I have also a java application that read from this file, and want to compare the time now with the time-stamp from the file.
My android device time set to my PC time, but:
The time-stamp (on milli seconds) are different from my java time.
Any one have an idea how to compare the different times?
Java:
System.currentTimeMillis()
1460272414374
Android:
/**
* The time in nanosecond at which the event happened
*/
public long timestamp;
event.timestamp: 190174864000
Thanks
long timeInMillis = (new Date()).getTime()
+ (sensorEvent.timestamp - System.nanoTime()) / 1000000L;
SystemClock.elapsedRealtimeNanos() is the API . sensorTimeStamp in device's local time reference = System.currentTimeMillis() + ((event.timestamp-SystemClock.elapsedRealtimeNanos())/1000000L)
private long sensorTimeReference = 0l;
private long myTimeReference = 0l;
public void onSensorChanged(SensorEvent event) {
// set reference times
if(sensorTimeReference == 0l && myTimeReference == 0l) {
sensorTimeReference = event.timestamp;
myTimeReference = System.currentTimeMillis();
}
// set event timestamp to current time in milliseconds
event.timestamp = myTimeReference +
Math.round((event.timestamp - sensorTimeReference) / 1000000.0);
// some code...
}
I have 2 classes doing a similar task in Apache Spark but the one using data frame is many times slower than the "regular" one using RDD. (30x)
I would like to use data frame since it will eliminate a lot of code and classes we have but obviously I can't have it be that much slower.
The data set is nothing big. We have 30 some files with json data in each about events triggered from activities in another piece of software. There are between 0 to 100 events in each file.
A data set with 82 events will take about 5 minutes to be processed with data frames.
Sample code:
public static void main(String[] args) throws ParseException, IOException {
SparkConf sc = new SparkConf().setAppName("POC");
JavaSparkContext jsc = new JavaSparkContext(sc);
SQLContext sqlContext = new SQLContext(jsc);
conf = new ConfImpl();
HashSet<String> siteSet = new HashSet<>();
// last month
Date yesterday = monthDate(DateUtils.addDays(new Date(), -1)); // method that returns the date on the first of the month
Date startTime = startofYear(new Date(yesterday.getTime())); // method that returns the date on the first of the year
// list all the sites with a metric file
JavaPairRDD<String, String> allMetricFiles = jsc.wholeTextFiles("hdfs:///somePath/*/poc.json");
for ( Tuple2<String, String> each : allMetricFiles.toArray() ) {
logger.info("Reading from " + each._1);
DataFrame metric = sqlContext.read().format("json").load(each._1).cache();
metric.count();
boolean siteNameDisplayed = false;
boolean dateDisplayed = false;
do {
Date endTime = DateUtils.addMonths(startTime, 1);
HashSet<Row> totalUsersForThisMonth = new HashSet<>();
for (String dataPoint : Conf.DataPoints) { // This is a String[] with 4 elements for this specific case
try {
if (siteNameDisplayed == false) {
String siteName = parseSiteFromPath(each._1); // method returning a parsed String
logger.info("Data for site: " + siteName);
siteSet.add(siteName);
siteNameDisplayed = true;
}
if ( dateDisplayed == false ) {
logger.info("Month: " + formatDate(startTime)); // SimpleFormatDate("yyyy-MM-dd")
dateDisplayed = true;
}
DataFrame lastMonth = metric.filter("event.eventId=\"" + dataPoint + "\"").filter("creationDate >= " + startTime.getTime()).filter("creationDate < " + endTime.getTime()).select("event.data.UserId").distinct();
logger.info("Distinct for last month for " + dataPoint + ": " + lastMonth.count());
totalUsersForThisMonth.addAll(lastMonth.collectAsList());
} catch (Exception e) {
// data does not fit the expected model so there is nothing to print
}
}
logger.info("Total Unique for the month: " + totalStudentForThisMonth.size());
startTime = DateUtils.addMonths(startTime, 1);
dateDisplayed = false;
} while ( startTime.getTime() < commonTmsMetric.monthDate(yesterday).getTime());
// reset startTime for the next site
startTime = commonTmsMetric.StartofYear(new Date(yesterday.getTime()));
}
}
There are a few things that are not efficient in this code but when I look at the logs it only adds a few seconds to the whole processing.
I must be missing something big.
I have ran this with 2 executors and 1 executor and the difference is 20 seconds on 5 minutes.
This is running with Java 1.7 and Spark 1.4.1 on Hadoop 2.5.0.
Thank you!
So there a few things, but its hard to say without seeing the breakdown of the different tasks & their time. The short version is you are doing way to much work in the driver and not taking advantage of Spark's distributed capabilities.
For example, you are collecting all of the data back to the driver program (toArray() and your for loop). Instead you should just point Spark SQL at the files in needs to load.
For the operators, it seems like your doing many aggregations in the driver, instead you could use the driver to generate the aggregations and have Spark SQL execute them.
Another big difference between your in-house code and the DataFrame code is going to be Schema inference. Since you've already created classes to represent your data, it seems likely that you know the schema of your JSON data. You can likely speed up your code by adding the schema information at read time so Spark SQL can skip inference.
I'd suggest re-visiting this approach and trying to build something using Spark SQL's distributed operators.
I have sqlite db(around 10k entries) with time stored in following format: hh:mmam/pm for example 12:40pm, 6:50am and I need it in milliseconds so they can be compared. Is there a way to make it happen? I am working with Java.
EDIT: Sorry, my question is ambiguous. I want to take value, transform it to milliseconds and overwrite it back, so all values will be stored in milliseconds rather than current format.
Problem was solved with the following python code, post it just in case anyone else will need to do something similar. After prog is done had to manually change type of the column from TEXT to NUMERIC
import sqlite3
from datetime import datetime
def unix_time(dt):
"""Takes datetime object and returns its unix time since epoch"""
epoch = datetime.utcfromtimestamp(0) #January 1st 1970
delta = dt - epoch
return delta.total_seconds()
def unix_time_millis(dt):
return unix_time(dt) * 1000 #milliseconds
db = sqlite3.connect("your_db.sqlite")#connect to initial database
cursor = db.cursor()
cursor.execute("select * from fancy_table")
all_entries = cursor.fetchall() #get our stuff
#new database. Make a copy of initial to prevent serious damage
db_new = sqlite3.connect("your_db_new.sqlite")
for entry in all_entries:
entry = str(entry[0].strip())#cursor returns tuple
#since it is time not a date, get milliseconds of the epoch
date_object = datetime.strptime("Jan 1 1970 " + entry, '%b %d %Y %I:%M%p')
new_time = unix_time_millis(date_object)
#print(entry + " to " + str(new_time))
cursor_update = db_new.cursor()#new cursor
try:
#updating
cursor_update.execute("UPDATE fancy_table SET time = '" + str(new_time) + "' WHERE arr_time = '" + entry + "'")
except Exception as error:
print(error)
db_new.commit()#needs to be commited to take affect
print("done")
I'm a Stata user trying to make use of Stata's Java API. I want to use Java to return the time of last modification of a file to Stata. However, I think I'm running into issues with time zones.
Quick background on times in Stata for non-Stata users: Times are represented by the number of milliseconds since January 1, 1960 00:00:00.000. Stata does not have time zones, so I want the return value from the Java function to be the number of milliseconds since midnight on January 1, 1960 in my time zone (America/New York).
Here is my attempt at the Java function (I can post it in its entirety if that would be helpful):
import com.stata.sfi.*;
// Other imports
public class SFileAttribs
{
// Syntax: lastModified(String filename, String scalar_name)
// Stores in the Stata scalar scalar_name the time of last
// modification of filename.
public static int lastModified(String[] args)
{
// Code here...
// Get the time.
FileTime time;
try {
time = (FileTime) Files.getAttribute(path, "lastModifiedTime");
}
catch (IOException e) {
SFIToolkit.errorln("file I/O error on read");
return(692);
/*NOTREACHED*/
}
// Store the time in the scalar.
Calendar cal1960 = Calendar.getInstance();
cal1960.set(1960, 0, 1, 0, 0, 0);
cal1960.set(Calendar.MILLISECOND, 0);
Scalar.setValue(args[1], time.toMillis() - cal1960.getTimeInMillis());
return(0);
}
}
Back in Stata, I run this command:
// Calls SFileAttribs.lastModified("Test.txt", "filetime").
javacall SFileAttribs lastModified, args("Test.txt" "filetime")
However, the result is an hour earlier than it should be:
. display %tc filetime
24oct2013 12:54:36
The time of last modification in my time zone is 13:54, not 12:54.
I am really struggling on this one and would appreciate any suggestions.
EDIT:
Perhaps predictably given that I'm coming from a language with no time zones, I think I messed up daylight savings. In my time zone, it is now DST, but in the same time zone in January 1, 1960, it was not DST. I need to account for that:
// Store the time in the scalar.
Calendar now = Calendar.getInstance();
Calendar cal1960 = Calendar.getInstance();
cal1960.set(1960, 0, 1, 0, 0, 0);
cal1960.set(Calendar.MILLISECOND, 0);
int dstHour = (cal1960.getTimeZone().inDaylightTime(cal1960.getTime()) ? 1 : 0) -
(now.getTimeZone().inDaylightTime(now.getTime()) ? 1 : 0);
cal1960.set(Calendar.HOUR, dstHour);
Scalar.setValue(args[1], time.toMillis() - cal1960.getTimeInMillis());
Is there an easier way to do this?
The community-contributed command filesys is the easiest way to do exactly what you want:
. findfile auto.dta
. filesys `r(fn)', attributes
. return list
macros:
(some output omitted)
r(accessednum) : "1871843910000"
r(modifiednum) : "1745362526000"
r(creatednum) : "1745362526000"
r(accessed) : "25apr2019 20:38:30"
r(modified) : "22apr2015 22:55:26"
r(created) : "22apr2015 22:55:26"
. display %tcHH:MM `r(modifiednum)'
22:55
You can install the filesys command by using the
community-contributed command github:
. net install github, from("https://haghish.github.io/github/")
. github install wbuchanan/StataFileSystem