cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

User16826994223
by Honored Contributor III
  • 1802 Views
  • 2 replies
  • 1 kudos

Resolved! Init script execution order

In which sequence init scripts are getting executed

  • 1802 Views
  • 2 replies
  • 1 kudos
Latest Reply
Mike2
New Contributor II
  • 1 kudos

If I have multiple cluster-scoped init scripts, can I guarantee the order that these scripts are run in?  I have to create a folder on the workers and other scripts copy files into that folder. 

  • 1 kudos
1 More Replies
rammy
by Contributor III
  • 5424 Views
  • 4 replies
  • 15 kudos

How would i upload file stream object to S3 bucket using pyspark?

I could able to save data using pyspark into S3 but not sure on how to save a file stream object into S3 bucket using pyspark. I could achieve this with help of python but when Unity catalog was enabled on Databrciks it always ends up with an access ...

  • 5424 Views
  • 4 replies
  • 15 kudos
Latest Reply
rammy
Contributor III
  • 15 kudos

I got to know that there is a change required at Unity-catalog to make it work with Python and got a recommendation to use pyspark to store file into S3.I do not see much information about storing a file stream object in an S3 bucket anywhere. Can an...

  • 15 kudos
3 More Replies
129876
by New Contributor III
  • 1984 Views
  • 2 replies
  • 2 kudos

Unable to run spark sql commands from ipywidget button click event

I'm unable to run any command that queries data from the unity catalog within a function that executes in the event of an ipywidget button click. Code block below. I cannot do queries such as spark.sql(f"SHOW SCHEMAS;") or spark.sql(f"select * from d...

  • 1984 Views
  • 2 replies
  • 2 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 2 kudos

can you try to println out? val databricksApiTokenKey = CredentialContext.INHERITED_PROPERTY_DATABRICKS_API_TOKEN val databricksApiCredentialOpt = CredentialContext.getCredential(databricksApiTokenKey) val rawUrlProp = spark.sparkContext.get...

  • 2 kudos
1 More Replies
Labels