Asset bundle doesn't sync files to workspace

[JonathanFlint]

I've created a completely fresh project with a completely empty workspace

Locally I have the databricks CLI version 0.230.0 installed

I run

databricks bundle init default-python

I have auth set up with a PAT generated by an account which has workspace admin. when I run bundle deploy it deploys the resources and jobs, and created the pipeline but does not sync the src folder and files to the workspace and it does not give an error.

The resources which do deploy when viewed in the workspace give an error saying the source code for them cant be found in the workspace.

If I add a sync block to the databricks.yml file at the top level

sync:

  include:

    - src/**

and run validate I get 

Warning: Pattern src\** does not match any files
at sync.include[0]
in databricks.yml:11:7

Warning: There are no files to sync, please check your .gitignore

I've tried all possible formatting of src I can think of and always get the same error, I've tried 

src, src/, src/*, .src, .src/, .src/*, src/*.ipynb

nothing works and nothing I do other than manually syncing my entire directory with a manual databricks bundle sync or vscode extension sync actually pushes the files to the remote workspace.

After I manually sync the files the errors on the resources in the workspace disappear

The only thing I changed was the catalogue being used by the dlt pipeline in my_project.pipline.yml file to use an existing catalogue because the workspace is enabled for UC instead of the hive_metastore

I've also tried adding the include to the top level include mapping

bundle:

  name: my_project

include:

  - resources/*.yml

Lost an entire day to this now and I can't figure out why the basic template generated by the cli itself doesn't deploy any notebooks or other files.

[-werners-]

have you looked into the .gitignore file?
chances are there is an entry with /resources/*
If so, you can remove everything from .gitignore which you think should be deployed.
For sure the resources folder.

[JonathanFlint]

Theres no entry for the resources folder in the git ignore, its not the resources I'm having trouble with, they are created successfully in the workspace but the notebooks they use are not copied to the workspace

I've tried deleting the gitignore file entirely, and theres no references to the src folder in the git ignore to begin with anyway

[saurabh18cs]

do you not see this in your databricks workspace users:

[JonathanFlint]

Yes I can see the folder structure but nothing in the src folder apart from a "my_project.egg-info" subfolder with some txt files in it:

[filipniziol]

Hi @JonathanFlint ,

1. Could you remove the below line. Let's try to make the deployment work without any filters:

sync:

  include:

    - src/**

2. Could you run "databricks bundle destroy" and then try "databricks bundle deploy". If you were "deploying manually" then I guess you were adding some notebooks manually, but the deployments keep the history of what was deployed and not and probably it is kind of messed up after those manual changes.

[JonathanFlint]

When I first encountered the issue I did not have the 

sync:

  include:

    - src/**

block in my databricks.yml file, i made no changes to the databrick.yml file before I first did the deployment

I've also tried destroying the bundle in the remote workspace and deploying it again, same result

[Mathias_Peters]

Hi, I had a similar problem today. I changed the way, that we deploy our main bundle using pull requests and in order to play around with it locally, I copied python and dbt code into the databricks src dir (that is normally done during a github workflow). To avoid accidental commits, I also added the two dirs to .gitignore on the top level of our repo  (i.e. one level above the databricks dir). After that, bundle deploy stopped copying files. 

Is that intended? That feels like a bug to me, tbh

[pherrera]

I've got the exact same issue with the default-python template (CLI v0.242.0).  I've made no changes to the generated output but no src files are uploaded.  If I upload them manually via the UI, the sample Job runs.

I even tried syncing directly via a few variants of:

databricks sync . /Workspace/Users/me/.bundle/backup_automation/dev/files/src/ --full --debug

and no files were uploaded and I saw no useful messages in the log.

[pherrera]

Ok, I feel silly.  Despite reading the other messages in this thread, I didn't twig to the fact that I had in fact added the subfolder I had created the DAB in to my top-level project .gitignore since I was just playing around and didn't want to commit anything accidentally.

Removing that entry fixed the problem instantly