cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

andrew0117
by Contributor
  • 1527 Views
  • 4 replies
  • 2 kudos

Resolved! master notebook cannot find the udf registered in the child notebook

The master notebook is calling a child notebook using dbutils.notebook.run("PathToChildnotebook"). The child notebook defines a user-defined function (UDF) and registers it using spark.udf.register. However, when the child notebook finishes running a...

  • 1527 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@andrew li​ :The reason why the UDF cannot be found is that when the child notebook finishes running, the Spark context that was used to define and register the UDF is destroyed. Therefore, the UDF is no longer available in the Spark context used by ...

  • 2 kudos
3 More Replies
SailajaB
by Valued Contributor III
  • 2540 Views
  • 2 replies
  • 6 kudos

Resolved! How to pass a variable which holds a value to child notebook using run command

Hello,We have 3 notebooks as below. And trying to send a variable where we pass the value to it via job to child notebook with %run commandNotebooks:notebook_par,notebook_child1 and notebook_child2.In parent note book we are calling notebook_child1 u...

  • 2540 Views
  • 2 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hey there @Sailaja B​ How's it going?Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Chee...

  • 6 kudos
1 More Replies
Labels