topic Re: How to add pre-commit hook to the Git Client on Databricks Cluster? in Data Engineering

How to add pre-commit hook to the Git Client on Databricks Cluster?

de-hru — Wed, 03 May 2023 12:26:09 GMT

I'd like to add a Git pre-commit hook to the Databricks Cluster.

This pre-commit hook should be executed when pushing to GitHub.

Why would I need a pre-commit hook on a Databricks Cluster?

My goal is to run blackbricks and format all notebooks automatically, so that well formatted code is pushed to GutHub only.

Is it possible to add a pre-commit hook to a Databricks Cluster?

I've found /root/.gitconfig on my cluster with this content:

[credential]
        helper = cache

git config --global --list says:

credential.helper=cache

So, this seems to be the right (starting) place.

I've tried this:

git config -f /root/.gitconfig core.hooksPath /root/git-hooks

git config --global --list says now:

credential.helper=cache
core.hookspath=/root/git-hooks

Then I've added a pre-commit hook to the directory /root/git-hooks

The content of the pre-commit hook is:

touch /root/git-hooks/test.sample

This works, if I run it manually. The file test.sample is created in /root/git-hooks/.

File test.sample deleted.

In Databricks UI:

checkout branch
do some change
push

The file test.sample in /root/git-hooks/ has not been created -> The pre-commit hook was not executed.

What can I do to add a pre-commit hook to a Databricks Cluster, so that this pre-commit hook is executed before the commit (to the remote branch) is added?

Re: How to add pre-commit hook to the Git Client on Databricks Cluster?

-werners- — Wed, 03 May 2023 13:03:38 GMT

Very interesting question!

Git on Databricks is not bound to a cluster, the code is separated from the clusters. So to use git pre-commit hooks, we need a way to configure git on Databricks (aka Repos). The only place where you can do this is in the admin panel. I looked and there is no possibility to add pre-commit hooks.

But this is definitely an interesting feature to have.

Maybe someone pulled it off in some way? If not:

you should post it as a feature request IMO.

Re: How to add pre-commit hook to the Git Client on Databricks Cluster?

de-hru — Fri, 05 May 2023 14:00:16 GMT

Thanks a lot for your answer.

~~I'll create a FR~~

There is an existing FR already

Enable pre-commits on Databricks Repos

Re: How to add pre-commit hook to the Git Client on Databricks Cluster?

Anonymous — Fri, 19 May 2023 07:45:40 GMT

Hi @Dejan Hrubenja

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: How to add pre-commit hook to the Git Client on Databricks Cluster?

de-hru — Fri, 19 May 2023 08:35:12 GMT

There is no possibility to add/use pre-commit hooks with Databricks yet.

FR: Enable pre-commits on Databricks Repos