topic No Spark Session Available Within Model Serving Environment in Machine Learning

No Spark Session Available Within Model Serving Environment

mharrison — Wed, 18 Dec 2024 15:50:25 GMT

Hi,

Is it possible to have a Spark session, that can be to used query the Unity Catalog etc, available within a Model Serving?

I have an MLFlow Pyfunc model that needs to get data from a Feature Table as part of its `.predict()` method. See my earlier question for more context, and why I can't use the Feature Lookups via the Feature Engineering client.

My solution following that was to instead just query for the historical data I need within the `.predict()` method via `spark.read.table()`.

This works fine within a notebook environment, that already has a Spark session created (with access to the Unity Catalog).

However, when I deploy a model serving for the model, and try to use it for inference -- I ultimately get the following exception: `Exception: No SparkSession Available!`. Presumably because the serving's environment does not have any Spark session created.

(I get the same error if I try the MLFlow validation `mlflow.models.predict()` function).

I suppose one alternative solution could be to create a Feature Serving endpoint for the Unity Catalog table I need to query, then query that from within my model's `predict()` method? Is there a convenient way of handling this, or will I simply need to send a POST request to the appropriate URL?

Re: No Spark Session Available Within Model Serving Environment

Alberto_Umana — Wed, 18 Dec 2024 15:58:11 GMT

Hi @mharrison

Creating a Spark session within a Model Serving environment is not directly supported, which is why you are encountering the Exception: No SparkSession Available! error. This limitation arises because the serving environment does not automatically create a Spark session.

Here are a few potential solutions to address this issue:

Feature Serving Endpoint: As you suggested, creating a Feature Serving endpoint for the Unity Catalog table you need to query is a viable solution. You can then query this endpoint from within your model's predict() method. This approach involves sending a POST request to the appropriate URL to retrieve the necessary data.
Alternative Data Retrieval Methods: If creating a Feature Serving endpoint is not feasible, consider other methods to retrieve the data required for your model's predictions. This could involve pre-fetching the data and storing it in a format that your model can access without needing a Spark session.
Batch Inference: If real-time inference is not a strict requirement, you might consider performing batch inference using a Spark cluster. This way, you can leverage the Spark session to read from the Unity Catalog and perform the necessary computations