topic Re: Error: "[REQUIRES_SINGLE_PART_NAMESPACE] sparkcatalog requires a single-part namespace" in Machine Learning

Error: "[REQUIRES_SINGLE_PART_NAMESPACE] sparkcatalog requires a single-part namespace"

Nawneet — Mon, 27 Jan 2025 17:35:28 GMT

Hi,

In my code I am creating a Spark session, that can be to used query the Unity Catalog delta tables.I have an MLFlow Pyfunc model that uses these table to retrieve information. Things work well from my cluster but getting below error from model serving:

"Error retrieving information for table catlogue_name.schema_name.table_name: [REQUIRES_SINGLE_PART_NAMESPACE] spark_catalog requires a single-part namespace, but got `catlogue_name`.`schema_name`."

Code Snippet:

schema="catlogue_name.schema_name"

table=table_name

full_table_name = f"{schema}.{table_name}"

try:

df = spark.read.table(full_table_name)

except Exception as e:

logger.error(f"Error retrieving information for table {full_table_name}: {e}")

What am I missing here? Can spark sessions be used from Model Serving? Do I need some authentication to retrieve information from delta tables? Is any specific model packaging step while model registration?

Re: Error: "[REQUIRES_SINGLE_PART_NAMESPACE] sparkcatalog requires a single-part namespace"

Alberto_Umana — Mon, 27 Jan 2025 17:45:18 GMT

Hi @Nawneet,

Please refer to this post: https://community.databricks.com/t5/machine-learning/no-spark-session-available-within-model-serving-environment/m-p/102515

Spark session are not directly supported in Model serving.

Re: Error: "[REQUIRES_SINGLE_PART_NAMESPACE] sparkcatalog requires a single-part namespace"

Nawneet — Tue, 28 Jan 2025 02:44:12 GMT

Hi @Alberto_Umana ,

But I am not getting exception as "Exception: No SparkSession Available!" ,probably because I am installing that as part of conda environment creation. Spark session is available. Exception is "[REQUIRES_SINGLE_PART_NAMESPACE] spark_catalog requires a single-part namespace" and my complete table name is "catlogue_name.schema_name.table_name" that includes catlogue name schema name and table name.

Kindly suggest.