- 2384 Views
- 2 replies
- 1 kudos
Resolved! Init script execution order
In which sequence init scripts are getting executed
- 2384 Views
- 2 replies
- 1 kudos
- 1 kudos
If I have multiple cluster-scoped init scripts, can I guarantee the order that these scripts are run in? I have to create a folder on the workers and other scripts copy files into that folder.
- 1 kudos
- 7070 Views
- 4 replies
- 15 kudos
How would i upload file stream object to S3 bucket using pyspark?
I could able to save data using pyspark into S3 but not sure on how to save a file stream object into S3 bucket using pyspark. I could achieve this with help of python but when Unity catalog was enabled on Databrciks it always ends up with an access ...
- 7070 Views
- 4 replies
- 15 kudos
- 15 kudos
I got to know that there is a change required at Unity-catalog to make it work with Python and got a recommendation to use pyspark to store file into S3.I do not see much information about storing a file stream object in an S3 bucket anywhere. Can an...
- 15 kudos
- 2428 Views
- 2 replies
- 2 kudos
Unable to run spark sql commands from ipywidget button click event
I'm unable to run any command that queries data from the unity catalog within a function that executes in the event of an ipywidget button click. Code block below. I cannot do queries such as spark.sql(f"SHOW SCHEMAS;") or spark.sql(f"select * from d...
- 2428 Views
- 2 replies
- 2 kudos
- 2 kudos
can you try to println out? val databricksApiTokenKey = CredentialContext.INHERITED_PROPERTY_DATABRICKS_API_TOKEN val databricksApiCredentialOpt = CredentialContext.getCredential(databricksApiTokenKey) val rawUrlProp = spark.sparkContext.get...
- 2 kudos