https://community.databricks.com/t5/data-engineering/how-to-restart-snowflake-connector/m-p/9049#M4544 <P>Yes, that would work. However, it is a longish Snowflake query producing a number of tables that are all called by the Databricks notebook, so it requires quite a few changes. I'll use this alternative if I automate the process. </P><P>However, I think this is a serious issue that deserves a warning from Databricks when using snowflake connector. One implicitly trusts that the connection will work, and there is no reason programmers will limit their snowflake changes to the particular ongoing connection. </P><P>In any case, under the hood, I imagine a connection engine has been created that could be closed and reopened. Maybe one could access that engine with standard snowflake sqlalchemy commands from the notebook?</P> Sat, 25 Feb 2023 00:51:40 GMT DavidMayer-Foul 2023-02-25T00:51:40Z https://community.databricks.com/t5/data-engineering/how-to-restart-snowflake-connector/m-p/9047#M4542 <P>After using spark.read.format("snowflake").options(**options).option("dbtable", "table_name").load() to read a table from Snowflake, when I then change the table from Snowflake and read it again, it gives me the first version of the table. I have worked around the problem by restarting the cluster. Is there a better way? Maybe restarting the snowflake connector or configuring it differently?</P> Tue, 21 Feb 2023 02:51:16 GMT https://community.databricks.com/t5/data-engineering/how-to-restart-snowflake-connector/m-p/9047#M4542 DavidMayer-Foul 2023-02-21T02:51:16Z https://community.databricks.com/t5/data-engineering/how-to-restart-snowflake-connector/m-p/9049#M4544 <P>Yes, that would work. However, it is a longish Snowflake query producing a number of tables that are all called by the Databricks notebook, so it requires quite a few changes. I'll use this alternative if I automate the process. </P><P>However, I think this is a serious issue that deserves a warning from Databricks when using snowflake connector. One implicitly trusts that the connection will work, and there is no reason programmers will limit their snowflake changes to the particular ongoing connection. </P><P>In any case, under the hood, I imagine a connection engine has been created that could be closed and reopened. Maybe one could access that engine with standard snowflake sqlalchemy commands from the notebook?</P> Sat, 25 Feb 2023 00:51:40 GMT https://community.databricks.com/t5/data-engineering/how-to-restart-snowflake-connector/m-p/9049#M4544 DavidMayer-Foul 2023-02-25T00:51:40Z