ERROR - Exception rendering Jinja template for task from BigQueryExecuteQueryOperator in Airflow

618 views Asked by At

I've been getting this following issue, for the last couple of weeks out of no where.

ERROR - Exception rendering Jinja template for task 'sometask', field 'sql'

jinja2.exceptions.TemplateNotFound: sql/test.sql

I have a task to execute a piece of sql through BigQueryExecuteQueryOperator. This is how my task looks like

tt = BigQueryExecuteQueryOperator(
        task_id="test1",
        sql=const.sqlfolder + "test.sql",
        destination_dataset_table="target_table",
    )

I also do have a user_defined_macros which i'm referring to in the SQL file.

user_defined_macros={
        "test_function": const.TEST_FUNCTION
} 

and this is how i'm referring to it in the SQL file:

SELECT
    {{ test_function }} as test,
    *
FROM
    sometable

Airflow version that I'm using is 2.4.3. I did go through some of the similar questions, but those seem not related to this at all. One was for BashOperator where we have to leave a space at the end of the path of the file or try using the variable through Params rather than macros.

This issue happens once in a while during daily runs.

Any help is really appreciated, if anyone has gone through the same.

0

There are 0 answers