Error "org.apache.hadoop.hive.ql.exec.vector.TimestampColumnVector.changeCalendar(ZZ)V" while using Timestamp filed in SQL query

Hi Team,
We are using Hopsworks 2.4 community edition and trying to execute a sql query having timestamp attribute to create a feature and it gives below error.

An error was encountered:
An error occurred while calling o180.showString.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 765) ( executor 1): java.lang.NoSuchMethodError: org.apache.hadoop.hive.ql.exec.vector.TimestampColumnVector.changeCalendar(ZZ)V

df=spark.sql(""“select order_id,
row_number()over(partition by order_id,order_year,order_week,order_amount order by Source_txn_commit desc,source_txn_csn desc,source_txn_rsn desc) as Rid
from order
where order_year >= ‘2018’
and COALESCE(Comments,‘Empty’) NOT LIKE (’%sample request%’)
group by 1,2,3,4,5,6,7
order by Source_txn_commit, source_txn_csn, source_txn_rsn desc”"")

HI @Rajendra_Tamboli,

In which format is the data stored and could you send the schema of the table you are trying to query?


Hi Fabio,

The data is stored in orc in adls and fetched via storage connector. Please find below schema of the order table used in SQL -

|-- order_id: integer (nullable = true)
|-- order_year: integer (nullable = true)
|-- order_week: integer (nullable = true)
|-- order_amount: decimal(18,4) (nullable = true)
|-- last_updated: timestamp (nullable = true)
|-- updated_by: string (nullable = true)
|-- status: string (nullable = true)
|-- data_source: string (nullable = true)
|-- comments: string (nullable = true)
|-- source_txn_commit: timestamp (nullable = true)
|-- source_txn_type: string (nullable = true)
|-- source_file_name: string (nullable = true)
|-- source_txn_csn: string (nullable = true)
|-- source_txn_rsn: long (nullable = true)
|-- load_txn_tm: string (nullable = true)
|-- dl_load_flag: string (nullable = true)
|-- etl_load_date: timestamp (nullable = true)
|-- end_date: timestamp (nullable = true)
|-- batchid: long (nullable = true)
|-- errormapcol: map (nullable = true)
| |-- key: string
| |-- value: string (valueContainsNull = true)
|-- source_file_name_dl: string (nullable = true)
|-- date_part: date (nullable = true)