I am trying to get last access time on a hive table in pyspark function and have no success. I need this to determine which tables I can safely remove because they were not accessed in last 2 years.
I am able to get create date with this
f.regexp_extract('information', r'transient_lastDdlTime=(\d+)', 1).cast("bigint").alias('ddl_tm'),
I tried similar for AccessTime but had no success:
f.regexp_extract('information', r'transient_lastAccessTime=(\d+)', 1).cast("bigint").alias('lastAccessTime')