Been trying to diagnose an issue with a couple ambassador mappings which appear to result in a 301 Permanent Redirect when from what I understand it shouldn't.
Here is the behavior:
- Go to https://[api domain]/pubsub/[random string]
- Redirected to https://[api domain]/[same random string] with 404 response (expected from main api service)
- Go to https://[api domain]/pubsub/[same random string] again
- Redirected with 301 to https://[api domain]/[same random string]
API Mapping
apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
name: api
spec:
prefix: /
service: api
regex_rewrite:
pattern: ^/docs/$
substitution: /docs/index.html
host: [api domain]
timeout_ms: 60000
PubSub Mapping
apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
name: pubsub-api
spec:
prefix: /pubsub
service: pubsub
host: [api domain]
timeout_ms: 60000
allow_upgrade:
- websocket
---
apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
name: pubsub-edge
spec:
prefix: /
service: pubsub
host: [deprecated websocket domain]
timeout_ms: 60000
allow_upgrade:
- websocket
This does NOT happen when using the deprecated websocket domain but appears to be some sort of conflict with the two mappings on the api domain.
I should note that there are zero 301 redirects in my code so this appears to be coming from emissary itself and that the issue happens with any path so long as it starts with /pubsub
My theory was that the issue was caused by the ordering of my deployments in Kustomize but changing the order to add the api mapping after the pubsub mapping seemed to only make the issue worse, resulting in blank responses for valid requests to the api service.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- api
- graphql
- pubsub
prefix: /pubsub
? Just to be safe, check that theprefix
for thepubsub-api
mapping ends with a/
to avoid conflicts:Mappings in Ambassador are evaluated in order, and the first match will be used. You should make sure more specific mappings are placed before more general ones.
See more at Advanced Mapping configuration / Mapping evaluation order.
You might want to make sure the mappings are applied in the correct order, which could be influenced by how resources are organized in your Kustomize setup.
You could try and define each mapping in its own file. That makes managing their order easier.
For example, have
pubsub-api-mapping.yaml
andapi-mapping.yaml
.In your Kustomize configuration, explicitly list resources in the order you want them to be applied. Kustomize applies resources in the order they are listed in the
resources
block of thekustomization.yaml
file.kustomization.yaml
:Use
kubectl apply -k path/to/kustomize/directory
to apply the Kustomization configuration. That will apply the resources in the order they are listed.After applying, you can check the order in which the mappings were applied using
kubectl get mappings -o yaml
.Given that changing the order in your Kustomize configuration did not resolve the issue and even worsened it, check if applying resources in different namespaces can implicitly manage the order. If applicable, you could try deploying the
pubsub
andapi
services in separate namespaces. (mentioned in this answer and in the documentation)Also, an init container in Kubernetes runs to completion before the main container starts. That can be used to enforce ordering. For instance, an init container could run a script that checks for the existence of the
pubsub-api
mapping and only allows the main application to start once it is confirmed.As a temporary measure, for debugging, you might manually apply the configurations in steps, first deploying the
pubsub-api
mapping and verifying its proper function before deploying theapi
mapping.Check the: