I have a dataset registered in Glue / Athena, call it my_db.table
. I'm able to query it via Athena and everything generally seems to be in order.
I'm trying to use this table in a Glue job, but am getting the following fairly opaque error message:
py4j.protocol.Py4JJavaError: An error occurred while calling o54.getCatalogSource.
: java.lang.Error: No classification or connection in my_db.table
This would appear to indicate that Glue can't see the catalog entry for my table, or can't use the information in that entry, but I don't have any further visibility than that.
Has anyone experience with this error and what might be causing it?
The error message actually describes the problem well - there was no classification for the table being queried.
Tables created via Glue are registered with a Classification -
csv
,parquet
,orc
,avro
,json
. See Creating Tables Using Athena for AWS Glue Jobs.The table I created 'manually' via Athena did not have a classifcation. See the below screenshot from the Glue 'tables' page.
The solution is easy: at the end of the
CREATE TABLE
script user must append aclassification
property like soNow the table has a classification within the Glue interface and is accessible via a Glue job.