https://community.databricks.com/t5/data-engineering/migration-scripts-for-distribution-embedded-in-library/m-p/37346#M26327 <P>I'm working on a python package that can be installed via pip. The package will manage a delta table for the user, and new versions of the package may need to run migrations on this table</P><P>Is this an okay format to use?</P><LI-CODE lang="python">def migrate(table_path): mm_path = f"{table_path}/_migration_marker" marker = 0 try: marker = int(dbutils.fs.head(mm_path, 10)) except Exception as e: if "java.io.FileNotFoundException" in str(e): print("Migration marker not found") else: raise e if marker &lt; 1: sql = f"""CREATE TABLE catalog.schema.tablename ... LOCATION {table_path}""" spark.sql(sql).count() marker = 1 dbutils.fs.put(mm_path, marker, True) if marker &lt; 2: sql = f"""ALTER TABLE catalog.schema.tablename ...""" spark.sql(sql).count() marker = 2 dbutils.fs.put(mm_path, marker, True)</LI-CODE><P>&nbsp;</P><P>&nbsp;</P> Tue, 11 Jul 2023 01:43:47 GMT matty_f 2023-07-11T01:43:47Z https://community.databricks.com/t5/data-engineering/migration-scripts-for-distribution-embedded-in-library/m-p/37346#M26327 <P>I'm working on a python package that can be installed via pip. The package will manage a delta table for the user, and new versions of the package may need to run migrations on this table</P><P>Is this an okay format to use?</P><LI-CODE lang="python">def migrate(table_path): mm_path = f"{table_path}/_migration_marker" marker = 0 try: marker = int(dbutils.fs.head(mm_path, 10)) except Exception as e: if "java.io.FileNotFoundException" in str(e): print("Migration marker not found") else: raise e if marker &lt; 1: sql = f"""CREATE TABLE catalog.schema.tablename ... LOCATION {table_path}""" spark.sql(sql).count() marker = 1 dbutils.fs.put(mm_path, marker, True) if marker &lt; 2: sql = f"""ALTER TABLE catalog.schema.tablename ...""" spark.sql(sql).count() marker = 2 dbutils.fs.put(mm_path, marker, True)</LI-CODE><P>&nbsp;</P><P>&nbsp;</P> Tue, 11 Jul 2023 01:43:47 GMT https://community.databricks.com/t5/data-engineering/migration-scripts-for-distribution-embedded-in-library/m-p/37346#M26327 matty_f 2023-07-11T01:43:47Z https://community.databricks.com/t5/data-engineering/migration-scripts-for-distribution-embedded-in-library/m-p/37652#M26418 <P>Not much community happening here&nbsp;</P> Sat, 15 Jul 2023 02:01:25 GMT https://community.databricks.com/t5/data-engineering/migration-scripts-for-distribution-embedded-in-library/m-p/37652#M26418 matty_f 2023-07-15T02:01:25Z