How to NOT install or disable or uninstall Databricks Delta Live Tables dlt module on jobs cluster?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-02-2024 05:54 AM
I need to NOT have the Databricks Delta Live Tables (DLT) Python stub installed on job cluster b/c of naming conflict w/ pip library dlt (and I also don't need delta live tables).
There is no "simple" way of uninstalling. It's not installed via pip as databricks-dlt but as
<module 'dlt' from '/databricks/spark/python/dlt/__init__.py'>
Also this doesn't do the trick.
Thx in advance!
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-04-2024 06:31 AM
For anyone facing a similar problem: I've addressed the issue of the dlt module conflict on my job cluster, by using an init script to remove the dlt module from the cluster's Python environment.
Simply by doing:
%bash
#!/bin/bash
rm -rf /databricks/spark/python/dlt
hth

