Exhausted Server when deploying a Databricks Assets Bundle (DAB)

PabloCSD
Valued Contributor II

Hello, I'm currently with a colleague inspecting the code and when trying to deploy the DAB it gets stuck:

 

(.venv) my_user@my_pc my-dab-project % databricks bundle deploy -t=dev -p=my-dab-project-prod      
Building wheel...
Uploading my-dab-project-1.0.1-py3-none-any.whl...
Uploading bundle files to /Users/my_databricks_user/.bundle/my-dab-project/dev/files...

 

When doing it in debug mode it shows that it tries to erase some files, it doesn't find it and then the server got exhausted: RESOURCE_EXHAUSTED.

 

databricks bundle deploy -t=dev -p=my-dab-project-prod --debug

17:17:40 DEBUG POST /api/2.0/workspace/delete
> {
>   "path": "/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/pytho... (55 more bytes)"
> }
< HTTP/2.0 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "Path (/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib... (77 more bytes)"
< } pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true
17:17:40 DEBUG non-retriable error: Path (/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/python3.10/site-packages/pip/_vendor/platformdirs/windows.py) doesn't exist. pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true
17:17:40 DEBUG POST /api/2.0/workspace/delete
> {
>   "path": "/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/pytho... (58 more bytes)"
> }
< HTTP/2.0 404 Not Found
< {
<   "error_code": "RESOURCE_DOES_NOT_EXIST",
<   "message": "Path (/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib... (80 more bytes)"
< } pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true
17:17:40 DEBUG non-retriable error: Path (/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/python3.10/site-packages/setuptools/_distutils/command/bdist.py) doesn't exist. pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true
17:17:40 DEBUG POST /api/2.0/workspace/delete
> {
>   "path": "/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/pytho... (63 more bytes)"
> }
< HTTP/2.0 429 Too Many Requests
< {
<   "error_code": "RESOURCE_EXHAUSTED",
<   "message": "Too many requests. Please wait a moment and try again. If the issue persists, consider adjusting... (66 more bytes)"
< } pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true
17:17:40 DEBUG POST /api/2.0/workspace/delete
> {
>   "path": "/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/pytho... (51 more bytes)"
> }
< HTTP/2.0 429 Too Many Requests
< {
<   "error_code": "RESOURCE_EXHAUSTED",
<   "message": "Too many requests. Please wait a moment and try again. If the issue persists, consider adjusting... (66 more bytes)"
< } pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true
^C

 

How can I fix it without renaming the project (this way helps but is not the most adequate)? Is there a way to deploy it erasing some sort of cache that tries to erase a non existing element?

Thanks in advance

 

Walter_C
Databricks Employee
Databricks Employee

Can you provide the code you are running on this DAB to understand the reason why this is calling this directory?

TinSlim
New Contributor III

You are using a venv, the venv has too many files and is not needed to be included, try adding this on your databricks.yml

sync:
  exclude:
  - "venv"

 Hope it helps

View solution in original post