I haven't been able to find a clear answer to my question anywhere so hoping people can help me out or point me to some documentation and/or articles that discuss this topic.
Let's say I have 3 services, each deployed as their own CDK app. Lets call them Service1, Service2 and MessageBus.
The MessageBus service is, as you may have guessed, a message bus which has some sort of endpoint to publish messages into the bus and an endpoint for other services to register subscriptions so that they will be delivered the messages they want. Lets say the publish endpoint is an SNS topic and the subscribe is an HTTP endpoint (API Gateway).
Service1 will publish into the message bus messages of type 1 and Service2 wants to subscribe to those same messages of type 1, so each of those CDK apps will need to reference something that doesn't exist until the MessageBus service is deployed.
Unlike multiple stacks within the same CDK app, apps cannot directly reference resources of other apps.
The naive way to handle this would be to deploy the MessageBus service then simply read the ARNs of the endpoints and hard code them into other services. This obviously has multiple downsides, one of the biggest being that not everything can be easily deployed at once (which is one of the things we are trying to do, switch to an Nx monorepo for our backend)
The best idea I can come up with is for the MessageBus to register its endpoints in something like AWS Cloud Map after it's deployment. It could also then expose the names used to save those resources in an interface shared by the MessageBus package. This means that Service1 and Service2 would have a clear dependency on MessageBus as they would need that name to properly lookup the resources in Cloud Map. This dependency could then be leveraged by the build system to say that Service1 and Service2 cannot de deployed until after their dependencies (MessageBus) have been deployed, meaning the values they get from Cloud Map would be up to date.
Is there anything wrong with this approach? Is there a better way to accomplish what I am trying to do?
What you're looking for is called "service discovery" and yes I think that CloudMap is a fine place to put that in.
Generally you can use some kind of DNS for this as well with stable domain names or fancier solutions depending on your requirements.
Realistically your services still depend very hard on MessageBus being there, they just are not coupled to it at deploy time. This means your contract guarantees that MessageBus will be up and you need to figure out what the other services will do when MessageBus happens to be down either permanently or transiently (ie. will they start throwing errors, will they crash out, will they recover when MessageBus comes up again, etc.).
Other than that, there's no real way around it.