Isito Ingress Controller VirtualService returning 404 for kserve service

Question

I have installed kserve 0.9 and then deployed a tutorial kserve service using-

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: sklearn-iris
  namespace: kserve-test
  labels:
    app: sklearn-iris
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: gs://kfserving-examples/models/sklearn/1.0/model

Now, I wanted to add an Istio Gateway and VirtualService because I want to change some properties of these resources without disturbing the resources created by the above yaml. I want to do this because I want to apply rate limiting and other things like accepting another host.

I tried creating Gateway-

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: gateway-iris
  namespace: kserve-test
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        name: http
        number: 8190
        protocol: HTTP
      hosts: [abc.com]

and VirtualService-

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: iris-vs
  namespace: kserve-test
spec:
  gateways: [kserve-test/gateway-iris]
  hosts: ['*']
  # hosts:
  #   - abc.com
  http:
    # - match:
    #     - uri:
    #         prefix: /v2/models/sklearn-iris/infer
    - route:
        - destination:
            host: sklearn-iris-predictor-default.kserve-test.svc.cluster.local
            port:
              number: 80

But this is returning 404 when I hit the ingress gateway URL. I have checked that the response comes whenever I send the request with the host as defined by the resources created by kserve.