I am reading a text file which has a field in Timestamp in this format "yyyy-MM-dd HH:mm:ss"
I want to be able to convert it to a field in Impala as BigInt and should like yyyMMddHHmmss in Java.
I am using Talend for the ETL but I get this error "schema's dbType not correct for this component"
and so I want to have the right transformation in my tImpalaOutput component
One obvious option is to read the date in as a string, format it to the output you want and then convert it to a long before sending it to Impala.
To do this you would start by using Talend's parseDate function with something like:
TalendDate.parseDate("yyyy-MM-dd HH:mm:ss",row1.date)
This parses the date string into a Date type object. From here you can convert this into your desired string format with:
TalendDate.formatDate("yyyMMddHHmmss",row2.date)
Alternatively this can be done in one go with:
TalendDate.formatDate("yyyMMddHHmmss",TalendDate.parseDate("yyyy-MM-dd HH:mm:ss",row1.date))
After this you should have a date string in your desired format. You can then cast this to a Long using a tConvertType component or the following Java code:
Long.valueOf(row3.date)
Or, once again we can do the whole thing in a one liner:
Long.valueOf(TalendDate.formatDate("yyyMMddHHmmss",TalendDate.parseDate("yyyy-MM-dd HH:mm:ss",row1.date)))
From here you should be able to send this to Impala as a Java Long to an Impala BIGINT field.
Related
I have a database that has the date as String and I have to delete the rows that have the same data in the csv file as in the database. More exactly, my dates look like this 2018-03-31T23:30:24+00:00. I want that when it gets a date like this, to delete from database where data LIKE %2018-03-31%, so it will delete all the records from that day, even if the time is not the same.
I have a job where tFileInputDelimited is connected with a tSortRow and then to tFlowToIterate. After that, I have a tJava where I extract the date and then a tMysqlInput where the query has the where clause like this: WHERE purchase_date LIKE '%"+context.date+"%' . Then, with a run if connection, I have tMysqlRow in which I have the delete statement with the same where clause. After that, of course, I have the tMysqlCommit.
The context.date is made like this:
context.dataaa=(String)globalMap.get("row6.purchase_date");
context.month=context.dataaa.substring(5,7);
context.year=context.dataaa.substring(0,4);
context.day=context.dataaa.substring(8,10);
context.date=context.year+"-"+context.month+"-"+context.day;
The problem is that, it doesn't delete from database. I want it to go row by row and delete all my records that have the same day in the csv and database.
The problem was from an if statement that compared the full date separately, so I had to compare only the day not day,year and month.
In Talend, you can define the pattern of a Date column to some usage.
Here, I have a tFileInputDelimited with a column date :
Column: date
Type : Date
pattern : "yyyy-MM-dd'T'hh:mm:ss"
That let me read a file with a value like 2018-03-31T23:30:24 because here, the pattern is used to "parse" the String from the file into a Date. Now if we add a LogRow and update the schema of this component, we can define a different pattern to format the Date
Column: date
Type : Date
pattern : "yyyy-MM-dd"
Input : 2018-03-31T23:30:24
Output : 2018-03-31
Know, if you don't want to play with the Schema, you can convert/format a date into a String with TalendDate.formatDate("yyyy-MM-dd", row1.date)
I am trying to convert the DB2 data base Date field format into H2 in memory database date format as shown below. It did not work for me.
Please help.
select CREATE_DATE from PX.MY_DB2TABLE ;
Here DB2 DATE format in database table is 'MM/DD/YYYY'. CREATE_DATE is DATE data type in db2.
H2 database is accepting only 'yyyy-MM-dd' format.
Tried the following way in H2:
select PARSEDATETIME(CREATE_DATE,'MM/dd/yyyy') FROM PX.MY_DB2TABLE;
select PARSEDATETIME(CREATE_DATE,'yyyy-MM-dd') FROM PX.MY_DB2TABLE;
Error:
Cannot parse DATE constant
To obtain a character-string representation from a DATE data-type value with the format 'YYYY-MM-DD', the DATE field selected in the original query [from the OP] can be wrapped in a CHAR casting scalar with an additional argument requesting that particular date-formatting for the Datetime to character casting; no mention of which variant of DB2, but here is some doc: DB2 for Linux UNIX and Windows 9.7.0->Database fundamentals->SQL->Functions->Scalar functions->CHAR:
select CHAR(CREATE_DATE, ISO) from PX.MY_DB2TABLE ;
For the second argument, see Date Strings Table 1. Formats for String Representations of Dates in [imagining LOCAL as the valid argument, in place of the LOC as seen in the table]:
DB2 for Linux UNIX and Windows 9.7.0->Database fundamentals->SQL->Language elements->Data types->Data type list->Datetime values
I am using MySQL for my database and working with Spring.
For export functionality, I am generating a report(excel sheet). In that I have a field called "Created On". I am putting the data i get from database into that field which is in DATETIME format.
like this:
'2016-05-22 17:06:55'
But what I would like to have in my report need to be like this: '05-22-2016 05:06:55 PM'
Thanks in advance.
Use date_format;
select date_format('2016-05-22 17:06:55', '%m-%d-%Y %h:%i:%s %p')
SQLFiddle DEMO HERE
I have to use a web service, where one of the parameters is date represented as a 5-digit number like this:
DATETIME INT
--------- ----
1/1/2009 39814
2/1/2009 39845
3/1/2009 39873
4/1/2009 39904
After googeling this I found out that this is MS-SQL representation. Do you guys have any idea how to make this integer in Java?
Thanks
That integer is the number of days since 1899-12-30. This is how Microsoft Excel stores dates, not SQL Server.
--integer to date
SELECT DATEADD(day,#integer,'1999-12-30')
--date to integer
SELECT DATEDIFF(day,'1999-12-30',#date)
I am using Compass to make queries on data inside in memory data structure. It works fine for searching String and enum values, now I want to search dates.
Search criteria are annotated by #SearchRestriction annotation. Example about someDate:
#SearchRestriction(path="fooBar.someDate" type = SearchRestrictionType.EQUAL)
String someDate;
At searchable data SomeDate is annotated like the following:
#SearchableProperty
Date someDate;
SomeDate inside the searchable data is generated with new Date();) and query String is given as 20120802.
Situation on debugger:
This code generates queries like this:
someDate:20120802
Here someDate is the name of the field I am looking for and 20120802 is a date in order yyyyMMdd.
Problem:
No results is returned, when this query is run. I get an empty list. The Date in query is the same as in the Date object.
What is wrong??
Is this wrong way to search Dates with Compass? I can find only range queries about Date, but a search with exact Date or part of exact Date I cannot find.
You need to specify the format for Searchable property [Date]
#SearchableProperty(format = "yyyyMMdd")
To some extent, it relates to Grails: Lucene, Compass Query Builder and date ranges