cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks pipeline script is not creating schema before creation of table

suk
New Contributor II

Hello 

We are facing some issue while executing the databricks pipeline i.e it takes all scripts in random sequence, and if no schema was created before it scheduled a job to create table it'll fail

As a alternative we are executing  schema pipeline first then table creation ...actually it should create those in  through none notebo

Would appreciate if anyone could help me.

1 REPLY 1

lingareddy_Alva
Honored Contributor III

Hi @suk 

This is a common issue with Databricks pipelines where dependencies aren't properly managed, causing scripts to execute in random order. 

Use Databricks Workflows with Task Dependencies
Configure explicit task dependencies:

lingareddy_Alva_0-1750976850495.png

 

LR