cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

User16826994223
by Honored Contributor III
  • 1159 Views
  • 2 replies
  • 1 kudos

Resolved! Init script execution order

In which sequence init scripts are getting executed

  • 1159 Views
  • 2 replies
  • 1 kudos
Latest Reply
Mike2
New Contributor II
  • 1 kudos

If I have multiple cluster-scoped init scripts, can I guarantee the order that these scripts are run in?  I have to create a folder on the workers and other scripts copy files into that folder. 

  • 1 kudos
1 More Replies
rammy
by Contributor III
  • 3778 Views
  • 5 replies
  • 18 kudos

How would i upload file stream object to S3 bucket using pyspark?

I could able to save data using pyspark into S3 but not sure on how to save a file stream object into S3 bucket using pyspark. I could achieve this with help of python but when Unity catalog was enabled on Databrciks it always ends up with an access ...

  • 3778 Views
  • 5 replies
  • 18 kudos
Latest Reply
rammy
Contributor III
  • 18 kudos

I got to know that there is a change required at Unity-catalog to make it work with Python and got a recommendation to use pyspark to store file into S3.I do not see much information about storing a file stream object in an S3 bucket anywhere. Can an...

  • 18 kudos
4 More Replies
129876
by New Contributor III
  • 1316 Views
  • 2 replies
  • 2 kudos

Unable to run spark sql commands from ipywidget button click event

I'm unable to run any command that queries data from the unity catalog within a function that executes in the event of an ipywidget button click. Code block below. I cannot do queries such as spark.sql(f"SHOW SCHEMAS;") or spark.sql(f"select * from d...

  • 1316 Views
  • 2 replies
  • 2 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 2 kudos

can you try to println out? val databricksApiTokenKey = CredentialContext.INHERITED_PROPERTY_DATABRICKS_API_TOKEN val databricksApiCredentialOpt = CredentialContext.getCredential(databricksApiTokenKey) val rawUrlProp = spark.sparkContext.get...

  • 2 kudos
1 More Replies
Labels