https://community.databricks.com/t5/data-engineering/can-you-share-variables-defined-in-a-python-based-cell-with/m-p/22099#M15105 Sun, 20 Jun 2021 03:59:35 GMT aladda 2021-06-20T03:59:35Z https://community.databricks.com/t5/data-engineering/can-you-share-variables-defined-in-a-python-based-cell-with/m-p/22099#M15105 Sun, 20 Jun 2021 03:59:35 GMT https://community.databricks.com/t5/data-engineering/can-you-share-variables-defined-in-a-python-based-cell-with/m-p/22099#M15105 aladda 2021-06-20T03:59:35Z https://community.databricks.com/t5/data-engineering/can-you-share-variables-defined-in-a-python-based-cell-with/m-p/22100#M15106 <P>Each databricks notebook has an indepedent Scala and Python shell/REPL. Thus variables defined in one can't be shared with the other. However Temp-Views are kept within the Notebook's own spark session and thus can be shared across different language cells and is a common technique to do so</P> Sun, 20 Jun 2021 04:01:15 GMT https://community.databricks.com/t5/data-engineering/can-you-share-variables-defined-in-a-python-based-cell-with/m-p/22100#M15106 aladda 2021-06-20T04:01:15Z https://community.databricks.com/t5/data-engineering/can-you-share-variables-defined-in-a-python-based-cell-with/m-p/22101#M15107 <P>The workaround is available <A href="https://community.databricks.com/s/question/0D53f00001JSnTACA1/can-we-access-the-variables-created-in-python-in-scalas-code-or-notebook-" alt="https://community.databricks.com/s/question/0D53f00001JSnTACA1/can-we-access-the-variables-created-in-python-in-scalas-code-or-notebook-" target="_blank"><B>here</B></A>.</P><P>Idea is to use spark.conf.set(), spark.conf.get() methods.</P> Wed, 05 Apr 2023 18:34:51 GMT https://community.databricks.com/t5/data-engineering/can-you-share-variables-defined-in-a-python-based-cell-with/m-p/22101#M15107 Imtiyaz_Shaikh 2023-04-05T18:34:51Z