Serving

A trained model is published as an inference service: an HTTP endpoint inside the lab that receives data and returns predictions. Classic models (scikit-learn, XGBoost, PyTorch) are published this way, from the KServe Endpoints section of the lab panel; for language models, the path is the Model catalog and InferNode.

Publish a scikit-learn model

1. Save the model on a lab volume, for example model.joblib on the modelo-fugas volume. Train with the same library version the server will use, to avoid incompatibilities when loading.
2. Create the service:

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: detector-fugas
spec:
  predictor:
    model:
      modelFormat: { name: sklearn }
      storageUri: pvc://modelo-fugas/

3. In the lab panel, KServe Endpoints shows the service with its predictor, runtime, protocol and Storage URI; wait until the status is green. New Endpoint lets you create it from a form instead of the file above, and View Graphs shows the request metrics.

   

4. Test from a notebook in the same lab:

import requests
r = requests.post("http://detector-fugas.<namespace>/v1/models/detector-fugas:predict",
                  json={"instances": [[2.1, 9.8], [5.2, 10.1]]})
print(r.json())

Resources

The service consumes CPU and memory from the lab budget while it is published. Delete it when you are not using it.

Next step

• Catalog apps