Issue with Delta Live tables access from workflow ( dlt pipelines )
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-08-2022 07:44 AM
Hi all,
I am trying to write data from external s3 bucket to delta live tables at unity catalog( not dbfs ) from a workflow. I am seeing the following error ( saying catalog namespace is not supported. Please check full error below )
Error Detailspy4j.Py4JException: An exception was raised by the Python Proxy. Return Message: Traceback (most recent call last):
File "/databricks/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py", line 617, in _call_proxy
return_value = getattr(self.pool[obj_id], method)(*params)
File "/databricks/spark/python/dlt/helpers.py", line 26, in call
res = self.func()
File "<command--1>", line 4, in demo_purpose
return base64.b64decode("L1VzZXJzL3NyaWthbnRoQG1vdmlzdGEuY29tL3Rlc3R2ZXJzaW9uX2RsdF8y").decode("utf-8")
File "<command--1>", line 30, in dlt_table_fn
File "/databricks/spark/python/pyspark/sql/session.py", line 1120, in table
return DataFrame(self._jsparkSession.table(tableName), self)
File "/databricks/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", line 1321, in __call__
return_value = get_return_value(
File "/databricks/spark/python/pyspark/sql/utils.py", line 202, in deco
raise converted from None
pyspark.sql.utils.AnalysisException: Catalog namespace is not supported.
- Labels:
-
Delta Live
-
Python Proxy
-
Unity Catalog