Databricks Community

fermin_vicente · 11-11-2022

More concretely, should we fix the dependency version at MAJOR, MINOR or PATCH?For example, MLFlow 1.30.0 is available and latest DBR 11.3 LTS is compatible with 1.29.0 My question comes from the fact that installing our own libraries that use MLFlow...

fermin_vicente · 03-28-2022

Hi there, if I set any secret in an env var to be used by a cluster-scoped init script, it remains available for the users attaching any notebook to the cluster and easily extracted with a print.There's some hint in the documentation about the secret...

fermin_vicente · 10-24-2024

I personally went without type hint for the dbutils param to avoid extra dependencies just for that, but I think you can actually get that type from the Databricks SDK, according to the docs:https://docs.databricks.com/en/dev-tools/databricks-connect...

fermin_vicente · 11-21-2022

Hi,You can mock dbutils. An easy way to do it is to:Try to receive dbutils as a parameter in your functions (inject it) instead of using it globally. This way your code is more testable and you won't to do patching which is a bit more cumbersome.Use ...

fermin_vicente · 11-14-2022

Hi! thanks for the reply, although maybe you didn't notice that I linked to the same url, so we're aware of the matrix. The question is, is it compatible solely with 1.29.0? We want to know which dependency should we use in all our projects that migh...

fermin_vicente · 03-29-2022

Thanks a lot Pavan, that approach works like a charm!

fermin_vicente · 03-29-2022

Hi Pavan,if you doprint(os.environ['SPARKPASSWORD'])the output is [REDACTED]however, if you run the command I put in my previous reply (and it's just one of many ways to do it), you can perfectly see the contents of the secret.Is there a way to make ...

Databricks Community

User Stats

User Activity

Is it wise to use a more recent MLFlow Python package version or is the DB Runtime compatibility matrix strict about MLFlow versions?

Can secrets be retrieved only for the scope of an init script?

Re: Python mocking dbutils in unittests

Re: Python mocking dbutils in unittests

Re: Is it wise to use a more recent MLFlow Python package version or is the DB Runtime compatibility matrix strict about MLFlow versions?

Re: Can secrets be retrieved only for the scope of an init script?

Re: Can secrets be retrieved only for the scope of an init script?