topic Re: More than expected number of Jobs created in Databricks in Get Started Discussions

More than expected number of Jobs created in Databricks

DBEnthusiast — Sat, 02 Dec 2023 11:45:12 GMT

Hi Databricks Gurus !

I am trying to run a very simple snippet :

data_emp=[["1","sarvan","1"],["2","John","2"],["3","Jose","1"]]

emp_columns=["EmpId","Name","Dept"]

df=spark.createDataFrame(data=data_emp, schema=emp_columns)

df.show()

--------

Based on a general understanding data bricks should create at the most 2 Jobs

One to read the data(this works for files like that, don't know if it would apply here)

One for show()

But it somehow creating 3 jobs

Can someone explain why is the behavior ?

DBEnthusiast — Mon, 04 Dec 2023 00:00:18 GMT

Thank You @Retired_mod !!

I was also suspecting the same and your response helped in the conclusion

gauravjoshi20 — Fri, 10 Oct 2025 17:30:47 GMT

Hi @DBEnthusiast,
Please share or explain the solution as I am unable to see previous reply on this thread.