Hi @sarahbhord ,

thanks for the feedback. So far I was able to test a few of your suggestions. Unfortunately, no success yet.
1. Thanks for the hints. I essentially used the default conda.yaml file generated by mlflow. Unfortunately, using your suggested version did not work either.
2. Did not have the time to test that version yet. But will definitely try it next week or so 🙂
3. Tried that, using the databricks-sdk for python. Unfortunately, I got an error: databricks.sdk.errors.platform.InvalidParameterValue: Workload type 'GPU_LARGE' with size 'Large' is not supported. Please choose a node type from

4. I Am aware of the T4's memory size. To clarify, I wanted to make the following point: In a notebook, I could use 4xT4 GPUs to run the model, thus my assumption was, that it should work for the 4xMedium Size GPUs Instances provided for model serving endpoints.

Currently, I am trying 8 bit quantization, so far with little success. (Running in Timeouts on Endpoint Creation, instead of the OOM though).

Anyway, thank's for the effort.
Greetings, Daniel