cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
rbelidrv
by • New Contributor II
  • 3955 Views
  • 3 replies
  • 1 kudos

How to apply a UDF to a property in an array of structs

I have a column that contains an array of structs as follows:"column" : [ { "struct_field1": "struct_value", "struct_field2": "struct_value" }, { "struct_field1": "struct_value", "struct_field2": "struct_value" } ]I want to apply a udf to each f...

Data Engineering
Reply
  • 3955 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @Richard Belihomji​, It looks like you are trying to apply a UDF to each field of the structs in an array column in a Spark DataFrame. However, it seems you are encountering an issue with the UDF not receiving the context.To nest a UDF inside a tr...

  • 1 kudos
2 More Replies
Phani1
by • Valued Contributor
  • 915 Views
  • 3 replies
  • 4 kudos

Efficiently orchestrate data bricks jobs

Hi Team,How efficiently can orchestrate data bricks jobs which involve a lot of transformations, dependencies, and complexity?At source have a lot of SSIS packages that have complex dependencies and more transformation.     We have the following opti...

Data Engineering
Reply
  • 915 Views
  • 3 replies
  • 4 kudos
View Replies
Latest Reply
Phani1
Valued Contributor
  • 4 kudos

My question is, how do we reliably orchestrate multiple Databricks Jobs/Workflows that are running in a mixed latency and can write to the same silver and gold delta tables? Could you please suggest the best approach and practices for the same?

  • 4 kudos
2 More Replies
Labels