Databricks

Hemant · 08-18-2022

I have a doubt row_num with order by in spark-sql gives different result(non-deterministic output) every time i execute it?It's due to parallelism in spark ?Any approach how to takle it?I order by with a date column and a integer column and take...

Hemant · 04-27-2022

We are working for client in offline environment(VM is private) azure databricks, we store some credentials in azure key vault, since azure databricks is run on offline environment, i am unable to create secret scopes on azure databricks, it always t...

Hemant · 07-20-2023

But using autoloader for the conversion into csv.

Hemant · 07-19-2023

Where autoloader is being used in the above snippet 樂 ?

Hemant · 07-19-2023

Unfortunately, Databricks autoloader doesn't support Excel file types to incrementally load new files.Link:https://docs.databricks.com/ingestion/auto-loader/options.html If your excel file contains a single sheet then there is a workaround.

Hemant · 07-17-2023

HI@zyang Unity Catalog Enabled workspace, you can access your databases in a cross environment.

Hemant · 07-14-2023

HI @Ramana, if you create mount point of the S3 bucket in databricks it will help you to leverage the functionality of the glob and os python module.Suppose your mount point is"/mnt/s3", just change it into '/dbfs/mnt/s3' and use glob and os module.H...

Databricks

User Stats

User Activity

Row_Num function in spark-sql

Is there any way to use key vault on offline environment azure databricks?

Re: Autoloader Excel Files

Re: Autoloader Excel Files

Re: Autoloader Excel Files

Re: Sync the production data in environment into test environment

Re: Is there a way to use wild card expressions inside dbutils functions