@Deepak Bhatt​ :

Yes, you can write your PySpark code in modular Python files outside of Databricks and then call them from a Databricks notebook. Here are the steps you can follow:

1. Create a Python file in your local development environment (e.g., VS Code) and write your PySpark code in it. You can define a main function in this file which will be called from the Databricks notebook.
2. Save the Python file to a Git repository or a cloud storage service such as Azure Blob Storage or Amazon S3.
3. In the Databricks notebook, you can clone the Git repository or mount the cloud storage service to access the Python file.
4. Import the Python file in your notebook using the Python import statement. For example, if your Python file is named my_pyspark_code.py, you can import it like this:

import my_pyspark_code

Call the main function in your Python file from the Databricks notebook. For example, if your main function is named run_spark_job() you can call it like this:

my_pyspark_code.run_spark_job()

By following these steps, you can write your PySpark code in a modular and maintainable way outside of Databricks, and then easily call it from a Databricks notebook.