Hi,
I have installed single hopswork host on GCP ( redhat 7 ) without TLS. I have setup a external Kubernetes cluster on GCP/GKE.
I want to use kubernetes for model serving. Do you have documents/blog that shows how to do it ?
regards,
Vivek
Hi Vivek,
Hopsworks EE comes with it’s own Kubernetes cluster which is also used for model serving, for example with TensorFlow.
User guide: https://hopsworks.readthedocs.io/en/latest/hopsml/tf_model_serving.html
Notebook example: https://github.com/logicalclocks/hops-examples/blob/master/notebooks/ml/Inference/Inference_Hello_World.ipynb
Currently it’s on-going work to enable Hopsworks to start model serving on a user-provided Kubernetes cluster, it is not available yet.
If you have developed your model in Hopsworks and you want to serve it from another Kubernetes cluster, another option is to download the model from Hopsworks and transfer it to that Kubernetes cluster.
Hi Vivek. If you just want to try out model-serving on Hospworks without installing the Enterprise version, you can just install Hopsworks Community on an Ubuntu server. Currently, Hopsworks community for Redhat/Centos only supports scikit-learn model serving. Hopsworks community for Ubuntu supports both scikit-learn and tf-model-serving. It is not using Kubernetes at the backend, but the client experience is still the same - you can write a model serving REST client that uses an API key (generated from menu settings (top-right hand corner of screen)) and the REST endpoint to the model that you can get by clicking on ‘info’ for a running model server.