Kafka advertised listener over AWS privatelink

Question

I have two VPCs in AWS:

VPC-A has an ec2 instance in it.
VPC-B has an ec2 instance in it running kafka and zookeeper via docker-compose

The VPCs are connected via AWS Privatelink (endpoint --> endpoint service --> nlb (in VPC-B) --> kafka) I have given the privatelink endpoint a DNS name: broker.confluent-playground

I can telnet fine to both port 9092 and 2181 from VPC A to Kafka and Zookeeper in VPC-B. No problems

[ec2-user@ip-10-1-0-90 etc]$ telnet broker.confluent-playground 9092
Trying 10.1.1.200...
Connected to broker.confluent-playground.

My problem is that while the network connectivity is there between the VPCs, I seem to be having problems with the kafka listener configuration. When I set the advertised.listeners to broker.confluent-playground:9092, my producer cannot seem to connect, and when running kafkacat -b broker.confluent-playground -L I only list 7 of the 40 topics. (they seem like internal system topics eg."_confluent_balancer_partition_samples").

Here is an excerpt from the docker-compose file:

broker:
    image: confluentinc/cp-server:6.1.1
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "9101:9101"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://broker:29092,EXTERNAL://broker.confluent-playground:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL

Now if I change the advertised listener to the private IP of the ec2 holding kafka ie:
FROM

EXTERNAL://broker.confluent-playground:9092

to

EXTERNAL://192.168.35.65:9092

then kafkacat in VPC-A can see all the 40 topics correctly! I still cannot produce (because i assume i have been given an advertised private ip address in a different network).

I seem to have these listener configurations messed up somewhere and I am confused how by changing the advertised listener I can get a subset of topics with one setting and then all the topics with another.

Another interesting thing
My kafkacat scanning using broker.confluent-playground advertised listener returns this

[ec2-user@ip-10-1-0-90 ~]$ sudo docker run --rm --network=host edenhill/kafkacat:1.5.0 kafkacat -b broker.confluent-playground:9092 -L

Metadata for all topics (from broker 1: broker.confluent-playground:9092/1):
 1 brokers:
  broker 1 at broker.confluent-playground:9092 (controller)
 7 topics:

And when I use the private IP address as (that I cannot reach from VPC-A) as the advertised listener I get the /bootstrap with all the topics:

[ec2-user@ip-10-1-0-90 ~]$ sudo docker run --rm --network=host edenhill/kafkacat:1.5.0 kafkacat -b broker.confluent-playground:9092 -L
Metadata for all topics (from broker -1: broker.confluent-playground:9092/bootstrap):
 1 brokers:
  broker 1 at 192.168.54.226:9092 (controller)
 40 topics: