topic Re: DAB - All projects files deployed in Data Engineering

DAB - All projects files deployed

cmathieu — Tue, 25 Mar 2025 13:33:32 GMT

I have an issue with DAB where all the project files, starting from root ., get deployed to the /files folder in the bundle.

I would prefer being able to deploy certain util notebooks, but not all the files of the project.

I'm able to not deploy any files by adding a wildcard to the sync - exclude in the databricks.yml, but it stops me from selectively add files in the sync - include.

Re: DAB - All projects files deployed

ashraf1395 — Wed, 26 Mar 2025 05:24:07 GMT

Hi there @cmathieu ,

You can use sync -paths to specify the files you want to deploy in the files folders instead of all using the include or exclude folder. -paths works better for me I can use it to deploy any files in my local to workspace even outside my bundle dir just need to use correct path relative to my databricks.yml

Here are the docs: https://docs.databricks.com/aws/en/dev-tools/bundles/settings#paths

Re: DAB - All projects files deployed

cmathieu — Wed, 26 Mar 2025 16:01:28 GMT

Thanks for this hint. My goal is to deploy a whole directory, but not any others.

Re: DAB - All projects files deployed

ashraf1395 — Thu, 27 Mar 2025 05:14:33 GMT

@cmathieu , It will support deployment of whole directory and not others as well.

Re: DAB - All projects files deployed

cmathieu — Thu, 27 Mar 2025 14:02:10 GMT

I tried it and it's the same behaviour as sync-include. As long as all files are excluded, no files can be included in the sync-include or sync-files. Not sure who to approach this without having to list all folders and files to exclude.