VS Code - ipynb vs py execution - spark issue

Sisi
New Contributor

Databricks Connect works inside VS Code notebook but the same code fails in a standalone script with
ValueError: default auth: cannot configure default credentials

I’m developing locally with **Databricks Connect 16.1.6** and VS Code.


Inside a Jupyter notebook(.venv) running one cell works fine:

import sys
sys.path.append("../src")
from test_dab import main
main.get_taxis(spark).show(10)
 
However, when I am trying to debug the main file (Python Debugger: Debug Python file) I am getting the error:
from pyspark.sql import SparkSession, DataFrame

def get_taxis(spark: SparkSession) -> DataFrame:
    return spark.read.table("samples.nyctaxi.trips")

def get_spark() -> SparkSession:
    try:
        from databricks.connect import DatabricksSession

        return DatabricksSession.builder.getOrCreate()
    except ImportError:
        return SparkSession.builder.getOrCreate()


def main():
    get_taxis(get_spark()).show(5)


if __name__ == "__main__":
    main()​

File "c:\[...]\main.py", line 15, in get_spark return DatabricksSession.builder.getOrCreate()

...

ValueError: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method.

When I run main.py by "Debug current file with Databricks Connect" then it works. 
Why is that?