Hi,

I am trying to install mosaic on my cluster, but get the error once I use 'enable_mosaic': 

ImportError: cannot import name '_to_java_column' from 'pyspark.sql.functions' (/databricks/spark/python/pyspark/sql/functions/__init__.py)
File <command-1412689069782833>, line 2
      1 from pyspark.sql.functions import *
----> 2 from mosaic import enable_mosaic
      3 enable_mosaic(spark, dbutils)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-d2578267-976a-48ca-ae64-6686226a93e0/lib/python3.10/site-packages/mosaic/__init__.py:1
----> 1 from .api import *
      2 from .core import MosaicLibraryHandler, MosaicContext
      3 from .utils.display_handler import displayMosaic
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-d2578267-976a-48ca-ae64-6686226a93e0/lib/python3.10/site-packages/mosaic/api/__init__.py:1
----> 1 from .accessors import *
      2 from .aggregators import *
      3 from .constructors import *
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-d2578267-976a-48ca-ae64-6686226a93e0/lib/python3.10/site-packages/mosaic/api/accessors.py:2
      1 from pyspark.sql import Column
----> 2 from pyspark.sql.functions import _to_java_column as pyspark_to_java_column
      4 from mosaic.config import config
      5 from mosaic.utils.types import ColumnOrName

The commands I used to import mosaic are:

%pip install databricks-mosaic

%python
from pyspark.sql.functions import *
from mosaic import enable_mosaic
enable_mosaic(spark, dbutils)

 I have already verified that I have the dependencies required using 

%python pip show databricks-mosaic

which outputs the following:

Name: databricks-mosaic
Version: 0.4.2
Summary: Mosaic: geospatial analytics in python, on Spark
Home-page: https://github.com/databrickslabs/mosaic
Author: Databricks
Author-email: 
License: Databricks License
Location: /local_disk0/.ephemeral_nfs/envs/pythonEnv-d2578267-976a-48ca-ae64-6686226a93e0/lib/python3.10/site-packages
Requires: geopandas, h3, ipython, keplergl, pyspark
Required-by: 

My cluster configuration is as follows, with Photon enabled: