Exhausted Server when deploying a Databricks Assets Bundle (DAB)

PabloCSD — Thu, 12 Dec 2024 20:35:46 GMT

Hello, I'm currently with a colleague inspecting the code and when trying to deploy the DAB it gets stuck:

(.venv) my_user@my_pc my-dab-project % databricks bundle deploy -t=dev -p=my-dab-project-prod Building wheel... Uploading my-dab-project-1.0.1-py3-none-any.whl... Uploading bundle files to /Users/my_databricks_user/.bundle/my-dab-project/dev/files...

When doing it in debug mode it shows that it tries to erase some files, it doesn't find it and then the server got exhausted: RESOURCE_EXHAUSTED.

databricks bundle deploy -t=dev -p=my-dab-project-prod --debug 17:17:40 DEBUG POST /api/2.0/workspace/delete > { > "path": "/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/pytho... (55 more bytes)" > } < HTTP/2.0 404 Not Found < { < "error_code": "RESOURCE_DOES_NOT_EXIST", < "message": "Path (/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib... (77 more bytes)" < } pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true 17:17:40 DEBUG non-retriable error: Path (/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/python3.10/site-packages/pip/_vendor/platformdirs/windows.py) doesn't exist. pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true 17:17:40 DEBUG POST /api/2.0/workspace/delete > { > "path": "/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/pytho... (58 more bytes)" > } < HTTP/2.0 404 Not Found < { < "error_code": "RESOURCE_DOES_NOT_EXIST", < "message": "Path (/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib... (80 more bytes)" < } pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true 17:17:40 DEBUG non-retriable error: Path (/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/python3.10/site-packages/setuptools/_distutils/command/bdist.py) doesn't exist. pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true 17:17:40 DEBUG POST /api/2.0/workspace/delete > { > "path": "/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/pytho... (63 more bytes)" > } < HTTP/2.0 429 Too Many Requests < { < "error_code": "RESOURCE_EXHAUSTED", < "message": "Too many requests. Please wait a moment and try again. If the issue persists, consider adjusting... (66 more bytes)" < } pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true 17:17:40 DEBUG POST /api/2.0/workspace/delete > { > "path": "/Users/my_databricks_user/.bundle/my-dab-project/dev/files/venv/lib/pytho... (51 more bytes)" > } < HTTP/2.0 429 Too Many Requests < { < "error_code": "RESOURCE_EXHAUSTED", < "message": "Too many requests. Please wait a moment and try again. If the issue persists, consider adjusting... (66 more bytes)" < } pid=83938 mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred mutator=seq mutator=files.Upload sdk=true ^C

How can I fix it without renaming the project (this way helps but is not the most adequate)? Is there a way to deploy it erasing some sort of cache that tries to erase a non existing element?

Thanks in advance

Re: Exhausted Server when deploying a Databricks Assets Bundle (DAB)

Walter_C — Thu, 12 Dec 2024 22:48:54 GMT

Can you provide the code you are running on this DAB to understand the reason why this is calling this directory?

Re: Exhausted Server when deploying a Databricks Assets Bundle (DAB)

TinSlim — Fri, 13 Dec 2024 14:39:30 GMT

You are using a venv, the venv has too many files and is not needed to be included, try adding this on your databricks.yml

sync: exclude: - "venv"

Hope it helps

topic Exhausted Server when deploying a Databricks Assets Bundle (DAB) in Administration & Architecture

Exhausted Server when deploying a Databricks Assets Bundle (DAB)

Re: Exhausted Server when deploying a Databricks Assets Bundle (DAB)

Re: Exhausted Server when deploying a Databricks Assets Bundle (DAB)