cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

weldermartins
by Honored Contributor
  • 18513 Views
  • 17 replies
  • 13 kudos

Resolved! Created nested struct schema SPARK - Schema Jira

Hello guys,I'm using Jira API to return "ISSUES". But to be able to use pyspark I need to create the Dataframe passing in the Schema. But I am not able to create the Schema based on the model below. Would you have any ideas?root |-- expand: string ...

  • 18513 Views
  • 17 replies
  • 13 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 13 kudos

if columns are missing, that particular data is not present in the json. I am not aware of spark skipping columns when reading json with inferschema. There is an option dropFieldIfAllNull but that is False by default.That makes me think: you might ...

  • 13 kudos
16 More Replies
Labels