Can't run Delta Live Tables pipeline while using Mosaic

[Gabrosciurti]

I'm just learning to use mosaic and delta live table in Databricks. I was following this example https://github.com/databrickslabs/mosaic/tree/main/notebooks/examples/python/OpenStreetMaps and after configuring everything I'm not able to run the pipeline.

The error is in the line where i enable mosaic

mos.enable_mosaic(spark, dbutils)

I also tried to install a specific version of mosaic to downgrade at the mosaic v3 and use the preview version of DLT.

This the configuration file of my DLT
```
{
"id": "<id>",
"pipeline_type": "WORKSPACE",
"clusters": [
{
"label": "default",
"node_type_id": "Standard_DS3_v2",
"num_workers": 1
}
],
"development": true,
"continuous": false,
"channel": "CURRENT",
"photon": true,
"libraries": [
{
"notebook": {
"path": "/Users/<user>/Mosaic-Example/02_Process"
}
}
],
"name": "open_street_map",
"edition": "ADVANCED",
"storage": "/temp/mosaic/open_street_maps",
"target": "open_street_maps",
"data_sampling": false
}
```

What am I missing?

and here the complete Trace Error

[Kaniz_Fatma]

Hi @Gabrosciurti, It seems you’re encountering an issue while working with Mosaic and Delta Live Tables (DLT) in Databricks.

Let’s troubleshoot this together!

1. Enabling Mosaic Error: The error occurs when you try to enable Mosaic using the line:

   mos.enable_mosaic(spark, dbutils)
   

   To address this, consider the following steps:

   • Check Dependencies: Ensure that you have installed the necessary dependencies for Mosaic. You can verify this by running:

     %pip show mosaic
     

     If not installed, you can install Mosaic using:

     %pip install mosaic
     

   • Cluster Configuration: Confirm that your Databricks cluster is configured correctly. Make sure it has the required libraries and configurations for Mosaic.

   • Cluster Restart: Sometimes, restarting the cluster can resolve issues related to library initialization.

2. Downgrading Mosaic: If you want to downgrade to Mosaic v3, you can specify the version during installation:

   %pip install mosaic==3.0.0
   

3. DLT Configuration: Your DLT configuration file appears to be correctly structured. However, ensure the following:

   • The notebook path (/Users/<user>/Mosaic-Example/02_Process) exists and contains the necessary code.
   • The target and storage paths are valid and accessible.

4. Trace Error: Please share the complete trace error message so that we can pinpoint the issue more accurately.

If you encounter any specific error messages, feel free to share them, and we’ll continue troubleshooting! 🚀

¹: GitHub - databrickslabs/mosaic ²: Tutorial: Declare a data pipeline with SQL in Delta Live Tables ³: High Scale Geospatial Processing With Mosaic - Databricks