Programmatically create Databricks Notebook

wesg2 — Fri, 30 Aug 2024 14:23:56 GMT

I am creating a databricks notebook via string concats (sample below)

Notebook_Head = """# Databricks notebook source

# from pyspark.sql.types import StringType

# from pyspark.sql.functions import split

# COMMAND ----------

"""

Full_NB = Notebook_Head + Mid + Tail

I am using dbutils.fs.put() to write to a storage location.
When I then try to import that file as a NB in databricks it displays as one large text file, NOT as a Databricks Notebook.
Line 1 = # Databricks notebook source

The notebook does not break up into command cells even though I have the proper # COMMAND ----------

Re: Programmatically create Databricks Notebook

filipniziol — Fri, 30 Aug 2024 17:00:42 GMT

Hi @wesg2 ,

One needs to be very precise when building this.

The below code WORKS:

# Define the content of the .py file with cell separators (Works!) notebook_content = """# Databricks notebook source # This is the header of the notebook # You can add imports or setup code here # COMMAND ---------- # This is the first code cell print('Hello from the first cell!') # COMMAND ---------- # This is the second code cell print('This is another cell.') """ # Define the output path (update the path as needed) output_path = "<your path>/notebook_script.py" # Save the notebook content to the .py file dbutils.fs.put(output_path, notebook_content, overwrite=True) # Inform the user of the location of the saved file print(f"Notebook saved as .py file at: {output_path}")

The below code does NOT work (extra new line character):

# Define the content of the .py file with cell separators (New Line character - does not work!) notebook_content = """ # Databricks notebook source # This is the header of the notebook # You can add imports or setup code here # COMMAND ---------- # This is the first code cell print('Hello from the first cell!') # COMMAND ---------- # This is the second code cell print('This is another cell.') """ # Define the output path (update the path as needed) output_path = "<your path>/notebook_script.py" # Save the notebook content to the .py file dbutils.fs.put(output_path, notebook_content, overwrite=True) # Inform the user of the location of the saved file print(f"Notebook saved as .py file at: {output_path}")

However, I have copied your Notebook_Head, saved this in ADLS as .py file and then imported back to Databricks and it also works.

Have you tried with just Notebook_Head? If yes, do you followed the same steps as I or do you do it differently?

topic Re: Programmatically create Databricks Notebook in Data Engineering

Programmatically create Databricks Notebook

Re: Programmatically create Databricks Notebook