- 490 Views
- 1 replies
- 0 kudos
Databricks Asset Bundle Behaviour for new workflows and existing workflows
Dear Community Members -I am trying to deploy a workflow using DAB. After deploying if I am updating the same workflow with different bundle name it is creating a new workflow instead of updating the existing workflow. Also when I am trying to use sa...
- 490 Views
- 1 replies
- 0 kudos
- 0 kudos
@nicole_lu_PM : Do you have any suggestions or feedback for the above question ? It will be really helpful if we can get some insights.
- 0 kudos
- 582 Views
- 1 replies
- 0 kudos
Databricks asset bundles dependencies
Is anyone aware of a way to include a requirements.txt within the job definition of a databricks asset bundle? Documentation mentions how to have dependencies in workspace files, or Unity Catalog volumes, but I wanted to ask if it is possible to decl...
- 582 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @pdemeulenaer, When working with Databricks Asset Bundles, you can specify library dependencies for local development in the requirements*.txt file at the root of the bundle project. However, job task library dependencies are declared in your bund...
- 0 kudos
- 680 Views
- 1 replies
- 0 kudos
Creating Zip file on Azure storage explorer
I need to create zip file which will contain csv files which I have created from the dataframe. But I am unable to create valid zip file which will be containing all csv files. Is it possible to create zip file from code in databricks on azure storag...
- 680 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @PragyaS, References: Creating Zip file on Azure storage explorer - Databricks CommunityHow to zip files (on Azure Blob Storage) with shutil in DatabricksWorking with Unity Catalog in Azure DatabricksAnnouncing the General Availability of Unity C...
- 0 kudos
- 345 Views
- 1 replies
- 0 kudos
Spark structured streaming
hi,could someone please help me with this code :-input parameter df is a spark structured streaming dataframe def apply_duplicacy_check(df, duplicate_check_columns): if len(duplicate_check_columns) == 0: return None, df valid_df = df.dr...
- 345 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @nileshtiwaari , The error message you’re encountering indicates that the .exceptAll() operation is not supported on streaming DataFrames. In structured streaming, certain operations have limitations due to the nature of streaming data. To addr...
- 0 kudos
- 334 Views
- 1 replies
- 0 kudos
- 334 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @bamvallar, Welcome to the Databricks Community! We're thrilled to have you here and thank you for attending the DAIS and visiting our community booth. Your participation means a lot to us, and we're excited to see the contributions you'll bring ...
- 0 kudos
- 363 Views
- 1 replies
- 0 kudos
Databricks User Group
Are there any Databricks User Group Meetups in UK?
- 363 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, feel free to join. https://community.databricks.com/t5/london/gh-p/London
- 0 kudos
- 347 Views
- 1 replies
- 0 kudos
Databricks User Group Meetups
Are there any Databricks User Group Meetups in Charlotte?
- 347 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, feel free to join it. https://community.databricks.com/t5/charlotte/gh-p/Charlotte-User-Group
- 0 kudos
- 296 Views
- 1 replies
- 0 kudos
Does databricks support spearman rank correlation
  while working on spark scala. I tried working on code using pearson correaltion it works but when I tried to work on spearman rank correlation it displayed error like Currently only the calculation of the Pearson Correlation coefficient is support...
- 296 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @chand20025, It seems that Databricks currently only supports the calculation of the Pearson Correlation coefficient and not the Spearman rank correlation. If you’re specifically looking to calculate Spearman rank correlation, you might need to e...
- 0 kudos
- 299 Views
- 1 replies
- 0 kudos
Hi all, I recently enabled SSO on my Databricks account. Now, when a user signs in they see "No work
Hi all, I recently enabled SSO on my Databricks account. Now, when a user signs in they see "No workspaces have been enabled for your account", which is the expected behavior as I haven't created any workspaces yet
- 299 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @269violetta, Thank you for sharing your concern with us! To expedite your request, please list your concerns on our ticketing portal. Our support staff would be able to act faster on the resolution (our standard resolution time is 24-48 hours)....
- 0 kudos
- 618 Views
- 1 replies
- 0 kudos
Databricks summit 2024 unity catalog migration!!
Databricks summit 2024 unity catalog migration!!
- 618 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @269violetta, At the Data + AI Summit 2024, Databricks introduced exciting advancements for Unity Catalog, their unified governance solution for data and AI. Here are the highlights: Open Sourcing Unity Catalog: Databricks has open-sourced Un...
- 0 kudos
- 1981 Views
- 3 replies
- 0 kudos
Error: Folder xxxx@xxx.com is protected
Hello, On Azure Databricks i'm trying to remove a folder on the Repos folder using the following command : databricks workspace delete "/Repos/xxx@xx.com"I got the following error message:databricks workspace delete "/Repos/xxxx@xx.com"Error: Folder ...
- 1981 Views
- 3 replies
- 0 kudos
- 0 kudos
The repo/username folder is your home folder and it is a system folder. Therefore, the folder cannot be deleted or renamed
- 0 kudos
- 540 Views
- 1 replies
- 0 kudos
Facing issue with Databricks Context Limit Exceeded
We have a use case , where there may be chance of 200 jobs executing at once. But Few notebooks are failing with an issue "run failed with error message Too many execution contexts are open right now.(Limit set currently to 150)." Can anyone help how...
- 540 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello, you can refer to the following KB article for information and best practices in regards the issue you are facing: https://kb.databricks.com/en_US/notebooks/too-many-execution-contexts-are-open-right-now Best practices Use a job cluster instea...
- 0 kudos
- 832 Views
- 3 replies
- 0 kudos
Issue with importing data into Power BI from a view containing 100 million records in ADB
Hello, everyone. I'm trying to import data into Power BI from a view in Azure Databricks that contains over 10 million records. Every time it's failing with this following error"Failed to save modifications to the server. Error returned: 'OLE DB or O...
- 832 Views
- 3 replies
- 0 kudos
- 0 kudos
Users have the option of either using OAuth or switching to Personal Access Token as a solution. I am sorry to say that this is the only method I am aware of at the present. hi
- 0 kudos
- 387 Views
- 1 replies
- 0 kudos
What is the vector database to generate in DataBricks?
What is the vector database to generate in DataBricks?
- 387 Views
- 1 replies
- 0 kudos
- 0 kudos
it is used for RAG models (generative AI) and contains embeddings used by those models.https://docs.databricks.com/en/generative-ai/vector-search.html
- 0 kudos
- 641 Views
- 1 replies
- 0 kudos
Certification exam
Hello Team, Today my exam was scheduled at 7:30 AM. No one contacted me before the exam for any validation/guideline. About one hour after, the proctor suspended my exam. Please check the video. I was constantly looking at the screen only. I have rai...
- 641 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Databricks community and @Cert-Team,It is more than 24 hours and I did not get any update after my Databricks Professional Exam got suspended.Please help me to sort this out so that I can complete the exam. It is very important for me to complete ...
- 0 kudos
