GCP ssh via identity access proxy fails locally but not via cloud shell

Question

I am trying to SSH into a Compute Engine VM that has only a private IP.

I am trying to use IAP to access it. I can SSH using cloud shell just fine, but I'd like to be able to use gcloud to do the same, but I get the following error when trying to connect.

[21:53] atte@x1:terraform $ gcloud compute ssh bastion --project=my-project
External IP address was not found; defaulting to using IAP tunneling.
kex_exchange_identification: banner line contains invalid characters
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].

Here is the relevant firewall configuration enabling IAP access

resource "google_compute_firewall" "my_vpc_fw_allow_iap" {
    project = google_project.project.name
    name = "${google_compute_network.my_vpc.name}-allow-iap"
    network = google_compute_network.my_vpc.name

    allow {
        protocol = "tcp"
    }

    source_ranges = ["35.235.240.0/20"]
}

(I also have a rule enabling SSH ingress from everywhere, although the above should be enough?) and here is my Compute Engine instance

resource "google_compute_instance" "bastion" {
    name = "bastion"
    project = google_project.project.project_id
    machine_type = "f1-micro"
    zone = var.zone

    boot_disk {
        initialize_params {
          image = "debian-cloud/debian-9"
        }
    }

    network_interface {
      subnetwork = google_compute_subnetwork.my_vpc_subnet1.self_link
    }

    metadata = {
        enable-oslogin = "TRUE"
    }

    # tags = ["iap-access"]
    deletion_protection = false
}

I am using a project owner account to try to SSH, so I don't see how it would be a credential issue.