Databricks Community

naga_databricks · 06-14-2024

I have a table A that is used in a spark.sql and joins with multiple other tables to get data. this data will be overwritten to the same table A.When i tried this, i get an error consistently as below: ERROR: An error occurred while calling o382.save...

Shared access vs Single user access mode · 10-16-2023

I am running a notebook to get secret value from GCP Secret Manager. This is working well with Single user Access Mode, however it fail when i use a cluster with Shared Access mode. I have specified the same GCP service account on both of these clust...

naga_databricks · 10-09-2023

Hi All,I am using Databricks Asset Bundles to deploy my code on github to databricks workspace. I have written out the Github Action as provided on databricks documentation.I have setup the personal access token for the service principal I want to us...

naga_databricks · 08-23-2023

To read Bigquery data using spark.read, i'm using a query. This query executes and creates a table on the materializationDataset. df = spark.read.format("bigquery") \.option("query", query) \.option("materializationProject", materializationProject) \...

naga_databricks · 07-05-2023

I have a notebook where i want to use the workflow name and task name that it will be running under. How do i access these information?

naga_databricks · 06-14-2024

Found this to be a transient error. Once i restarted the cluster, the overwrite was successful.

naga_databricks · 10-17-2023

Thanks for your response.I am using a cloud service account (same account that was used to create the workspace) on the cluster properties in case of both the single user cluster and on the shared user cluster. This service account has all the necess...

naga_databricks · 10-10-2023

Finally, i was able to identify the missing piece. This was setting up the environment identifier for the runner. name: "Deploy bundle" runs-on: ubuntu-latest environment: ${{github.event.inputs.Environment}}With this, the action was able...

naga_databricks · 08-30-2023

I have seen this problem with Identity column causing concurrency issues. But you seem to be getting similar error when writing to files. I don't know completely know your use case completely here, but would advice retrying this operation by managing...

naga_databricks · 08-25-2023

Thanks Kaniz for the explanation. It sounds good to me that i'm working with tables here.For me the only cause of confusion was the underlined writing:

Databricks Community

User Stats

User Activity

Overwriting same table

Shared access vs Single user access mode

Databricks asset bundles deployment to development

Reading bigquery data using a query

From within a Databricks Notebook, how do you get the workflow and task name

Re: Overwriting same table

Re: Shared access vs Single user access mode

Re: Databricks asset bundles deployment to development

Re: MetadataChangedException Exception in databricks

Re: Reading bigquery data using a query