cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks Devops Release Pipeline Abort

Jonas89
New Contributor

We've built a release pipeline to our Databricks Workspaces, using the VNET Template. It's working end-to-end but intermittent aborts are occurring when workspace is recreated.

For example, 4th of April (Monday) We recreated the workspaces and no abort happened.

The script is straightforward, check if it's required to recreate the workspace, if required, delete databricks workpace, vnet and network security groups. Once finished start to execute the workspace deploy, based in the VNET template.

1_PipelineError 

This first part always runs fine, without aborts, the problem is the next step, when we try to sync our Notebook Repository with Databricks Repo. On Monday, or Tuesday it worked perfectly, syncing it without problem. On Wednesday it failed:

2_PipelineError 

3_PipelineError 

Maybe it could be a sync problem, the error message suggest that invalid credentials were used, with an error code = "INVALID_STATE". The interesting is that the repo is created partially after this error, and in a 2nd attempt it works without problems:

4_PipelineError 

Any help is appreciate. Thank you

2 REPLIES 2

Debayan
Esteemed Contributor III

Anonymous
Not applicable

Hi @Jonas Oliveira de Souzaโ€‹ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group