DBT 404 Not found: Dataset hello-data-pipeline:staging_benjamin was not found in location EU

Question

When doing "DBT run" I get the following error

{{ config(materialized='table') }}

SELECT customer_id FROM `hello-data-pipeline.adwords.google_ads_campaign_stats` 

I am making sure that my FROM location contains 3 parts

• A project (hello-data-pipeline)
• A database (adwords)
• A table (google_ads_campaign_stats)

But I get the following error

15:41:51 | 2 of 3 START table model staging_benjamin.yo......................... [RUN]
15:41:51 | 2 of 3 ERROR creating table model staging_benjamin.yo................ [ERROR in 
0.32s]
Runtime Error in model yo (models/yo.sql)
404 Not found: Dataset hello-data-pipeline:staging_benjamin was not found in location EU

NB. Bigquery does not show any error when doing this query in Bigquery Editor. NB 2 DBT does not show any error when "running sql" command directly in the script editor

What I am doing wrong ?

Answer 1

You may need to specify a location where your query will run. Queries that run in a specific location may only reference data in that location. You may choose auto-select to run the query in the location where the data resides.

Read more about Dataset locations

Answer 2

OK I found. I needed to specify the location in the profile.yml file. => https://docs.getdbt.com/reference/warehouse-profiles/bigquery-profile/#dataset-locations

In DBT cloud you will find it when setting up your project

Answer 3

I had a similar error to your 'hello-data-pipeline:staging_benjamin was not found in location EU'

However, my issue was not that the dataset was not in the incorrect location. If was that DBT was not targeting the schema I wanted.

e.g. for your example it would be that hello-data-pipeline:staging_benjamin would actually not be the target schema you initially wanted.

Adding this bit of code on top of my query solved the issue.

{{ config(schema='marketing') }}

select ...

cf DBT's schemas: https://docs.getdbt.com/docs/building-a-dbt-project/building-models/using-custom-schemas

here is another doc that helped me understand why this was happening:

"dbt Cloud IDE: The values are defined by your connection and credentials. To check any of these values, head to your account (via your profile image in the top right hand corner), and select the project under "Credentials". https://docs.getdbt.com/reference/dbt-jinja-functions/target