Databricks Community

Yousry_Ibrahim

Let's assume we have a workspace folder containing two Python files.module1 with a simple addition function:def add_numbers(a, b): return a + bmodule2 with a dummy PySpark custom data source:from pyspark.sql.datasource import DataSource, DataSource...

Yousry_Ibrahim

Hi all,Thanks for the feedback and proposed ideas.@szymon_dybczak Your idea of relative imports work when the module is hosted in a child directory to the current running notebook. It does not work if we need to go up one or two directories and navi...

Yousry_Ibrahim

Thanks @szymon_dybczak , it is a great idea.I will give it a go but in my case the notebook and the modules live in completely different hierarchies so there will be some sort of relative path handling.The sys.path.append is declared by Databricks to...

Yousry_Ibrahim

Hi @-werners- The reason why I am keen to use shared mode is due to some imposed templates prepared by the DevOps team. I don't have full control on the clusters or access mode I can use.There is a solution proposed by @szymon_dybczak and I will see...

Databricks Community

User Stats

User Activity

Directories added to the Python sys.path do not always work fine on executors for shared access mod

Re: Directories added to the Python sys.path do not always work fine on executors for shared access

Re: Directories added to the Python sys.path do not always work fine on executors for shared access

Re: Directories added to the Python sys.path do not always work fine on executors for shared access