Can I deploy kserve inference service using XGBoost model on kserve-tritonserver?

202 views Asked by At

I want to deploy XGBoost model on kserve. I deployed it on default serving runtime. But I want to try it on kserve-tritonserver.

I know kserve told me kserve-tritonserver supports Tensorflow, ONNX, PyTorch, TensorRT. And NVIDIA said triton inference server supported XGBoost model.

so.. is there a way to deploy kserve inference service using XGBoost model on kserve-tritonserver?

k apply -n kserve-test -f - <<EOF
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "digits-classification-xgboost"
spec:
  predictor:
    model:
      modelFormat:
        name: xgboost
      protocolVersion: v2
      storageUri: "s3://.../digits_classification_model"
      runtime: kserve-tritonserver
EOF

I tried it. But I got this description

Status:
  Model Status:
    Last Failure Info:
      Message:  Specified runtime does not support specified framework/version
      Reason:   NoSupportingRuntime
    States:
      Active Model State:
      Target Model State:  FailedToLoad
    Transition Status:     InvalidSpec
Events:
  Type     Reason         Age                  From                Message
  ----     ------         ----                 ----                -------
  Warning  InternalError  65s (x19 over 9m9s)  v1beta1Controllers  specified runtime kserve-tritonserver does not support specified framework/version
0

There are 0 answers