odh-model-controller failed to support old scheme of inferenceservice

[Jooho]

New scheme:

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "sklearn-iris"
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"

Old scheme:

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "sklearn-iris"
spec:
  predictor:
    sklearn:
      storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"

current implementation occurs this error

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x149a051]

goroutine 552 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:118 +0x1f4
panic({0x15fa4a0, 0x2698f00})
        /usr/lib/golang/src/runtime/panic.go:1038 +0x215
github.com/opendatahub-io/odh-model-controller/controllers.(*OpenshiftInferenceServiceReconciler).findSupportingRuntimeForISvc(0xc0006cf180, {0x1a115d8, 0xc0024af740}, {{0x1a2e048, 0xc0024af800}, 0x0}, 0xc000092200)
        /workspace/controllers/inferenceservice_route.go:102 +0x91
github.com/opendatahub-io/odh-model-controller/controllers.(*OpenshiftInferenceServiceReconciler).reconcileRoute(0xc0006cf180, 0xc000092200, {0x1a115d8, 0xc0024af740}, 0x18a7cd8)
        /workspace/controllers/inferenceservice_route.go:164 +0x1c7
github.com/opendatahub-io/odh-model-controller/controllers.(*OpenshiftInferenceServiceReconciler).ReconcileRoute(...)
        /workspace/controllers/inferenceservice_route.go:247
github.com/opendatahub-io/odh-model-controller/controllers.(*OpenshiftInferenceServiceReconciler).Reconcile(0xc0006cf180, {0x1a115d8, 0xc0024af740}, {{{0xc000322510, 0x18}, {0xc0003ae660, 0x40d3a7}}})
        /workspace/controllers/inferenceservice_controller.go:77 +0x256
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1a11530, {0x1a115d8, 0xc0024af740}, {{{0xc000322510, 0x17190c0}, {0xc0003ae660, 0x40ee1d}}})
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121 +0xd1
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0001aec80, {0x1a11530, 0xc0006cf100}, {0x16530e0, 0xc000738060})
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320 +0x33c
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0001aec80, {0x1a11530, 0xc0006cf100})
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273 +0x205
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:230 +0x36f
        /workspace/controllers/inferenceservice_route.go:247
github.com/opendatahub-io/odh-model-controller/controllers.(*OpenshiftInferenceServiceReconciler).Reconcile(0xc0006cf180, {0x1a115d8, 0xc0024af740}, {{{0xc000322510, 0x18}, {0xc0003ae660, 0x40d3a7}}})
        /workspace/controllers/inferenceservice_controller.go:77 +0x256
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1a11530, {0x1a115d8, 0xc0024af740}, {{{0xc000322510, 0x17190c0}, {0xc0003ae660, 0x40ee1d}}})
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121 +0xd1
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0001aec80, {0x1a11530, 0xc0006cf100}, {0x16530e0, 0xc000738060})
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320 +0x33c
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0001aec80, {0x1a11530, 0xc0006cf100})
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273 +0x205
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
        /opt/app-root/src/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:230 +0x36f

[heyselbi]

should be resolved by #100 but needs to be checked/tested.

[davidesalerno]

@vaibhavjainwiz could you confirm that the #100 is fixing also this?

[spolti]

Validated, it is working with the new and old isvc:

Tested with:

• New

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  annotations:
    serving.knative.openshift.io/enablePassthrough: "true"
    sidecar.istio.io/inject: "true"
    sidecar.istio.io/rewriteAppHTTPProbers: "true"
  generation: 1
  name: sklearn-iris-v2-rest
  namespace: issue-mm-100
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      name: ""
      resources: {}
      runtime: kserve-mlserver
      storageUri: gs://seldon-models/sklearn/mms/lr_model

Old:

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  annotations:
    serving.knative.openshift.io/enablePassthrough: "true"
    sidecar.istio.io/inject: "true"
    sidecar.istio.io/rewriteAppHTTPProbers: "true"
  name: "sklearn-iris-old"
spec:
  predictor:
    sklearn:
      runtime: kserve-mlserver
      storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"

both as worked as expected.

Closing this issue.