I'm trying to setup kubernetes for the first time, probably it's something stupid. So I have an Ubuntu server with microk8s recently installed, and I'm trying to setup a deployment for an app with nginx. I've managed to successfully start said deployment, I can connect to it directly from server's ssh. There's also an ingress controller in different from app's namespace. When I'm trying to connect via browser I get 503. There are logs of my attempts in ingress, but not in webapp.
My configs:
---
# Source: yii/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: myapp-yii
labels:
helm.sh/chart: yii-0.1.0
app.kubernetes.io/name: yii
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.4.0"
app.kubernetes.io/managed-by: Helm
---
# Source: yii/templates/host-config.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: backend-host-config
data:
default.conf: |
server {
client_max_body_size 10m;
root /var/www/web;
location /health {
return 200;
}
location / {
# try to serve file directly, fallback to app.php
try_files $uri /index.php$is_args$args;
}
location ~* ^.+\\.(css|js)$ {
access_log off;
log_not_found off;
expires max;
add_header Access-Control-Allow-Origin \"*\";
}
location ~* ^.+\\.(svg|svgz|eot|otf|woff|woff2|ttf|ttc|rss|atom|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|wav|bmp|rtf)$ {
access_log off;
expires max;
add_header Access-Control-Allow-Origin \"*\";
}
location ~ ^/index\\.php(/|$) {
fastcgi_pass localhost:9000;
fastcgi_split_path_info ^(.+\\.php)(/.*)$;
include fastcgi_params;
fastcgi_send_timeout 300s;
fastcgi_read_timeout 60s;
fastcgi_param HTTPS on;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $realpath_root;
http2_push_preload on;
proxy_buffer_size 2048k;
proxy_buffers 4 2048k;
proxy_busy_buffers_size 4096k;
fastcgi_buffers 16 256k;
fastcgi_buffer_size 512k;
internal;
}
location ~ \\.php$ {
return 404;
}
}
---
# Source: yii/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp-yii
labels:
helm.sh/chart: yii-0.1.0
app.kubernetes.io/name: yii
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.4.0"
app.kubernetes.io/managed-by: Helm
spec:
type: ClusterIP
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
- port: 9000
name: fastcgi
selector:
app.kubernetes.io/name: yii
app.kubernetes.io/instance: myapp
---
# Source: yii/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-yii
labels:
helm.sh/chart: yii-0.1.0
app.kubernetes.io/name: yii
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.4.0"
app.kubernetes.io/managed-by: Helm
spec:
revisionHistoryLimit: 3
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: yii
app.kubernetes.io/instance: myapp
template:
metadata:
labels:
app.kubernetes.io/name: yii
app.kubernetes.io/instance: myapp
spec:
imagePullSecrets:
- name: docker-token
serviceAccountName: myapp-yii
securityContext:
{}
volumes:
- name: vendor-dir
emptyDir: { }
- name: upload-dir
emptyDir: { }
- name: runtime-dir
emptyDir: { }
- name: assets-dir
emptyDir: { }
- name: host-config
configMap:
name: backend-host-config
initContainers:
- name: app-dependencies
image: "[app-image]"
imagePullPolicy: IfNotPresent
command: ['sh', '-c', 'composer install --no-interaction --classmap-authoritative']
envFrom: &envFrom
- secretRef:
name: backend-env
volumeMounts:
- mountPath: /var/www/vendor
name: vendor-dir
- mountPath: /var/www/web/upload
name: upload-dir
- mountPath: /var/www/runtime
name: runtime-dir
- name: app-migrations
image: "[app-image]"
imagePullPolicy: IfNotPresent
envFrom: *envFrom
volumeMounts:
- mountPath: /var/www/vendor
name: vendor-dir
- mountPath: /var/www/web/upload
name: upload-dir
- mountPath: /var/www/runtime
name: runtime-dir
command: [ 'sh', '-c', './yii migrate --interactive=0' ]
- name: app-cache-clear
image: "[app-image]"
imagePullPolicy: IfNotPresent
envFrom: *envFrom
volumeMounts:
- mountPath: /var/www/vendor
name: vendor-dir
- mountPath: /var/www/web/upload
name: upload-dir
- mountPath: /var/www/runtime
name: runtime-dir
command: [ 'sh', '-c', './yii cache/flush-all --interactive=0' ]
containers:
- name: app
securityContext:
{}
image: "[app-image]"
envFrom: *envFrom
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /var/www/vendor
name: vendor-dir
- mountPath: /var/www/web/upload
name: upload-dir
- mountPath: /var/www/runtime
name: runtime-dir
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /health
port: http
resources:
{}
- name: webapp
image: "[webapp-image]"
volumeMounts:
- mountPath: /etc/nginx/conf.d
name: host-config
---
# Source: yii/templates/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-yii
labels:
helm.sh/chart: yii-0.1.0
app.kubernetes.io/name: yii
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.4.0"
app.kubernetes.io/managed-by: Helm
annotations:
cert-manager.io/issuer: cert-issuer-staging
certmanager.k8s.io/issuer: cert-issuer-staging
ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/backend-protocol: http
nginx.ingress.kubernetes.io/client-body-buffer-size: 1M
nginx.ingress.kubernetes.io/fastcgi-index: index.php
nginx.ingress.kubernetes.io/fastcgi-params-configmap: default/fastcgi-config
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/http2-push-preload: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- "myapp.site"
secretName: myapp-tls
rules:
- host: "myapp.site"
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: myapp-yii
port:
name: http
---
# Source: yii/templates/issuer.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: cert-issuer-staging
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: "[email]"
privateKeySecretRef:
name: myapp-tls
solvers:
- http01:
ingress:
class: nginx
---
# Source: yii/templates/tests/test-connection.yaml
apiVersion: v1
kind: Pod
metadata:
name: "myapp-yii-test-connection"
labels:
helm.sh/chart: yii-0.1.0
app.kubernetes.io/name: yii
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.4.0"
app.kubernetes.io/managed-by: Helm
annotations:
"helm.sh/hook": test
spec:
containers:
- name: wget
image: busybox
command: ['wget']
args: ['myapp-yii:80/health']
restartPolicy: Never
When trying to access via browser this appears in ingress logs:
<client-ip> - - [28/Oct/2023:11:52:01 +0000] "GET / HTTP/1.1" 503 592 "-" <client-useragent> [myapp-myapp-yii-http] [] - - - - cbf11499bb04aa4c4cc2fdff838a6ff4
kubectl get service myapp-yii -n myapp
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
myapp-yii ClusterIP 10.152.183.239 <none> 80/TCP,9000/TCP 22h
kubectl get ingress -n myapp
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
myapp myapp-yii nginx myapp.site 127.0.0.1 80, 443 13h
kubectl get deployment -n myapp
NAME READY UP-TO-DATE AVAILABLE AGE
myapp-yii 1/1 1 1 22h
kubectl get pod -n myapp
NAME READY STATUS RESTARTS AGE
myapp-yii-775597f9d8-p76rf 2/2 Running 0 13h
I don't know if external-ip being <none> and/or ingress address 127.0.0.1 affects the issue, but I could not resolve both problems.
Edit: to minimize the human factor, we switched to helm. Also tried exposing pod outside by adding
spec:
externalIPs:
- [server-ip]
this way we were able to access webapp via browser
Turns out Ingress' annotations ruins our case, we only left two: issuer and ingress.class, and now it all works fine