How to seldon-core quick-start on kind with port-forward?

Question

Following the documentation I try to setup the Seldon-Core quick-start https://docs.seldon.io/projects/seldon-core/en/v1.11.1/workflow/github-readme.html

I don't have LoadBalancer so I would like to use port-fowarding for accessing to the service.

I run the following script for setup the system:

#!/bin/bash -ev
kind create cluster --name seldon

kubectl cluster-info --context kind-seldon
sleep 10
kubectl get pods -A

istioctl install -y
sleep 10
kubectl get pods -A

kubectl create namespace seldon-system
kubens seldon-system
helm install seldon-core seldon-core-operator \
     --repo https://storage.googleapis.com/seldon-charts \
     --set usageMetrics.enabled=true \
     --namespace seldon-system \
     --set istio.enabled=true
sleep 100
kubectl get validatingwebhookconfigurations
kubectl create namespace modelns
kubens modelns
kubectl apply -f - << END           
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: iris-model
  namespace: modelns
spec:
  name: iris
  predictors:
  - graph:
      implementation: SKLEARN_SERVER
      modelUri: gs://seldon-models/v1.12.0-dev/sklearn/iris
      name: classifier
    name: default
    replicas: 1
END
sleep 100
kubectl get pods -A
kubectl get svc -A
INGRESS_GATEWAY_SERVICE=$(kubectl get svc --namespace istio-system --selector="app=istio-ingressgateway" --output jsonpath='{.items[0].metadata.name}')
kubectl port-forward --namespace istio-system svc/${INGRESS_GATEWAY_SERVICE} 8080:80 &

I gess the port-forwarding argument 8080:80 is probably wrong.

I'm using the following script for testing:

#!/bin/bash -ev

export INGRESS_HOST=localhost
export INGRESS_PORT=8080
SERVICE_HOSTNAME=$(kubectl get inferenceservice sklearn-iris -n kserve-test -o jsonpath='{.status.url}' | cut -d "/" -f 3)

curl -X POST http://$INGRESS_HOST:$INGRESS_PORT/seldon/modelns/iris-model/api/v1.0/predictions \
    -H 'Content-Type: application/json' \
    -d '{ "data": { "ndarray": [1,2,3,4] } }'

But I got the following error:

Handling connection for 8080
E1012 10:52:32.074812   
52896 portforward.go:400] an error occurred forwarding 8080 -> 8080: 
error forwarding port 8080 to pod b9bd4ff03c6334f4af632044fe54e1c2531e95976a5fe074e30b4258d145508a, 
uid : failed to execute portforward in network namespace "/var/run/netns/cni-2b4d8573-3cfe-c70e-1c36-e0dc53cbd936": failed to connect to localhost:8080 inside namespace "b9bd4ff03c6334f4af632044fe54e1c2531e95976a5fe074e30b4258d145508a",
 IPv4: dial tcp4 127.0.0.1:8080: connect: connection refused IPv6 dial tcp6 [::1]:8080: connect: connection refused

Please can somebody known how to fix this? What is the right port forwarding argument?

Answer 1

If you install with istio enabled you also need to install the istio gateway.

I've tested your flow and it didn't work, and then did work after installing the following istio gateway.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: seldon-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"

You can read more about istio configuration on Seldon Core here: https://docs.seldon.io/projects/seldon-core/en/latest/ingress/istio.html