Issue: Lakeflow Connect Microsoft Teams Community Connector - No module named 'databricks.labs'

ChristianRRL
Honored Contributor

Wednesday

Hi there, I'm trying to run a pretty barebones test here with community connectors. I'm trying out the "vanilla" Microsoft Teams connector by:

  • (A) going to the Data Ingestion tab in our workspace,
  • (B) selecting the Microsoft Teams community connector,
  • (C) creating a new basic connection based on what the Microsoft Teams connection requires (connection name, tenant id, client id, client secret), and
  • (D) setting up the ingestion pipeline (pipeline name, event log location, root path).

Very basic stuff.

Additionally, due to the way our network is setup, we can't leverage Serverless, so a small modification needed for us is to go to the pipeline settings (gear icon) > compute > untoggle Serverless. For simplicity, I'm keeping all the default non-serverless settings.

However, even with this extremely basic setup I'm seeing two issues when attempting to Dry run the microsoft teams ingestion pipeline (I have a workaround for the 1st, but not the 2nd issue):

1. Failed to load notebook due to error UNSUPPORTED_LANGUAGE

Failed to load notebook '/Users/***/lakeflow_connect/microsoft_teams_pipeline_test1/src/ingest.py'. Only SQL and Python notebooks are supported currently.
UNSUPPORTED_LANGUAGE
Failed to load  notebook '/Users/***/lakeflow_connect/microsoft_teams_pipeline_test1/src/ingest.py'. Only SQL and Python notebooks are supported currently.
  • The default generated ingest.py file is.. well it's a python file of course, but it's not being recognized.
  • I thought this might be due to "Default file format for notebooks" being set incorrectly. I updated it from Jupyter to Source, but I'm still seeing the issue.
  • My workaround here, is to create a new ingest file (notebook), copy the contents of the originally generated ingest.py file into the notebook, go to the pipeline settings (gear icon) > repoint the source code to the new ingest notebook, and run it again.

2. ModuleNotFoundError: No module named 'databricks.labs'

ChristianRRL_1-1784735725394.png

  • This error is pretty self-explanatory
  • There are a few attempted workarounds but none have worked so far:
    • Approach 1: import sys + sys.path.insert(0, ...)
    • Result 1: No change, same error
    • Approach 2: pip install -e ... + dbutils.library.restartPython()
    • Result 2: No change, same error
  • Hunch: I can't personally test this on Serverless, but maybe this works differently between Serverless and non-serverless (job compute I assume)

Not sure if there's something else, I would need to do here with the non-serverless compute configuration to make sure that this runs successfully. If not that, then I have no clue what may be going on.

Would heavily appreciate some assistance with this!

0 Kudos