Programmatically create Databricks Notebook
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-30-2024 07:19 AM - edited 08-30-2024 07:23 AM
I am creating a databricks notebook via string concats (sample below)
Notebook_Head = """# Databricks notebook source
# from pyspark.sql.types import StringType
# from pyspark.sql.functions import split
# COMMAND ----------
"""
Full_NB = Notebook_Head + Mid + Tail
I am using dbutils.fs.put() to write to a storage location.
When I then try to import that file as a NB in databricks it displays as one large text file, NOT as a Databricks Notebook.
Line 1 = # Databricks notebook source
When I then try to import that file as a NB in databricks it displays as one large text file, NOT as a Databricks Notebook.
Line 1 = # Databricks notebook source
The notebook does not break up into command cells even though I have the proper # COMMAND ----------
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-30-2024 10:00 AM
Hi @wesg2 ,
One needs to be very precise when building this.
The below code WORKS:
# Define the content of the .py file with cell separators (Works!)
notebook_content = """# Databricks notebook source
# This is the header of the notebook
# You can add imports or setup code here
# COMMAND ----------
# This is the first code cell
print('Hello from the first cell!')
# COMMAND ----------
# This is the second code cell
print('This is another cell.')
"""
# Define the output path (update the path as needed)
output_path = "<your path>/notebook_script.py"
# Save the notebook content to the .py file
dbutils.fs.put(output_path, notebook_content, overwrite=True)
# Inform the user of the location of the saved file
print(f"Notebook saved as .py file at: {output_path}")The below code does NOT work (extra new line character):
# Define the content of the .py file with cell separators (New Line character - does not work!)
notebook_content = """
# Databricks notebook source
# This is the header of the notebook
# You can add imports or setup code here
# COMMAND ----------
# This is the first code cell
print('Hello from the first cell!')
# COMMAND ----------
# This is the second code cell
print('This is another cell.')
"""
# Define the output path (update the path as needed)
output_path = "<your path>/notebook_script.py"
# Save the notebook content to the .py file
dbutils.fs.put(output_path, notebook_content, overwrite=True)
# Inform the user of the location of the saved file
print(f"Notebook saved as .py file at: {output_path}")However, I have copied your Notebook_Head, saved this in ADLS as .py file and then imported back to Databricks and it also works.
Have you tried with just Notebook_Head? If yes, do you followed the same steps as I or do you do it differently?