topic Re: Enable custom Ipython Extension in Data Engineering

Enable custom Ipython Extension

Soma — Fri, 18 Mar 2022 10:39:16 GMT

How to enable custom Ipython Extension on Databricks Notebook Start

Re: Enable custom Ipython Extension

Hubert-Dudek — Fri, 18 Mar 2022 12:12:52 GMT

Some extension can be just install via pip. In such a case it can be added permanently to cluster (in compute -> select your cluster -> library tab) or installed via notebooks (with magic %pip install your_library command or alternatively %conda).

Re: Enable custom Ipython Extension

Soma — Fri, 18 Mar 2022 12:24:01 GMT

I want to load custom extensions which I create

like custom call back events on cell run

https://ipython.readthedocs.io/en/stable/config/callbacks.html

Re: Enable custom Ipython Extension

Hubert-Dudek — Fri, 18 Mar 2022 12:31:56 GMT

For sure you need to try to change kernel to ipython as described here https://docs.microsoft.com/en-us/azure/databricks/notebooks/ipython-kernel by adding following spark config value in cluster settings:

spark.databricks.python.defaultPythonRepl ipykernel