Error with mosaic.enable_mosaic() when created DLT Pipeline with Mosaic lib

slothPetete — Fri, 08 Mar 2024 10:24:27 GMT

The error was raised when I tried to start a DLT pipeline with simple code, just to start experimenting the DLT.

The primary library was Mosaic, which is already instructed to installed first before importing.

The code is roughly as follow

$ %pip install databricks-mosaic import mosaic as mos mos.enable_mosaic(spark, dbutils) # Error line import dlt from pyspark.sql.functions import * from pyspark.sql.types import * @dlt.table(comment="Testing a DLT table for geospatial coverage") def geo_area(): return spark.read.table("geo_coverage_area")

Below is the DLT Pipeline setting JSON

{ "id": "<>", "pipeline_type": "WORKSPACE", "clusters": [ { "label": "default", "node_type_id": "m5d.large", "driver_node_type_id": "m5d.large", "custom_tags": { "type": "test" }, "num_workers": 1 }, { "label": "maintenance", "custom_tags": { "type": "test" } } ], "development": true, "continuous": false, "channel": "CURRENT", "photon": false, "libraries": [ { "notebook": { "path": "/<>/test DLT" } } ], "name": "helloCellCoverage", "edition": "CORE", "catalog": "sigint_workspace", "target": "default", "data_sampling": false }

And here is the error raised

java.lang.RuntimeException: Failed to execute python command for notebook "<Removed>" and error AnsiResult(--------------------------------------------------------------------------- Py4JError Traceback (most recent call last) File <command--1>:3 1 import mosaic as mos ----> 3 mos.enable_mosaic(spark, dbutils) 5 import dlt 6 # import pyspark.sql.functions as

Re: Error with mosaic.enable_mosaic() when created DLT Pipeline with Mosaic lib

slothPetete — Mon, 11 Mar 2024 04:50:49 GMT

Hi, thank you for the reply. However, I couldn't related the OAuth issue to the Java RunTime error from my question. Additionally, I tried checked the Photon feature when started the workflow, and the error still persists.

Thank you for your response anyway.

topic Error with mosaic.enable_mosaic() when created DLT Pipeline with Mosaic lib in Data Engineering

Error with mosaic.enable_mosaic() when created DLT Pipeline with Mosaic lib

Re: Error with mosaic.enable_mosaic() when created DLT Pipeline with Mosaic lib