- 745 Views
- 1 replies
- 0 kudos
Creating Zip file on Azure storage explorer
I need to create zip file which will contain csv files which I have created from the dataframe. But I am unable to create valid zip file which will be containing all csv files. Is it possible to create zip file from code in databricks on azure storag...
- 745 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @PragyaS, References: Creating Zip file on Azure storage explorer - Databricks CommunityHow to zip files (on Azure Blob Storage) with shutil in DatabricksWorking with Unity Catalog in Azure DatabricksAnnouncing the General Availability of Unity C...
- 0 kudos
- 356 Views
- 1 replies
- 0 kudos
Spark structured streaming
hi,could someone please help me with this code :-input parameter df is a spark structured streaming dataframe def apply_duplicacy_check(df, duplicate_check_columns): if len(duplicate_check_columns) == 0: return None, df valid_df = df.dr...
- 356 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @nileshtiwaari , The error message you’re encountering indicates that the .exceptAll() operation is not supported on streaming DataFrames. In structured streaming, certain operations have limitations due to the nature of streaming data. To addr...
- 0 kudos
- 352 Views
- 1 replies
- 0 kudos
- 352 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @bamvallar, Welcome to the Databricks Community! We're thrilled to have you here and thank you for attending the DAIS and visiting our community booth. Your participation means a lot to us, and we're excited to see the contributions you'll bring ...
- 0 kudos
- 379 Views
- 1 replies
- 0 kudos
Databricks User Group
Are there any Databricks User Group Meetups in UK?
- 379 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, feel free to join. https://community.databricks.com/t5/london/gh-p/London
- 0 kudos
- 357 Views
- 1 replies
- 0 kudos
Databricks User Group Meetups
Are there any Databricks User Group Meetups in Charlotte?
- 357 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, feel free to join it. https://community.databricks.com/t5/charlotte/gh-p/Charlotte-User-Group
- 0 kudos
- 312 Views
- 1 replies
- 0 kudos
Does databricks support spearman rank correlation
  while working on spark scala. I tried working on code using pearson correaltion it works but when I tried to work on spearman rank correlation it displayed error like Currently only the calculation of the Pearson Correlation coefficient is support...
- 312 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @chand20025, It seems that Databricks currently only supports the calculation of the Pearson Correlation coefficient and not the Spearman rank correlation. If you’re specifically looking to calculate Spearman rank correlation, you might need to e...
- 0 kudos
- 316 Views
- 1 replies
- 0 kudos
Hi all, I recently enabled SSO on my Databricks account. Now, when a user signs in they see "No work
Hi all, I recently enabled SSO on my Databricks account. Now, when a user signs in they see "No workspaces have been enabled for your account", which is the expected behavior as I haven't created any workspaces yet
- 316 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @269violetta, Thank you for sharing your concern with us! To expedite your request, please list your concerns on our ticketing portal. Our support staff would be able to act faster on the resolution (our standard resolution time is 24-48 hours)....
- 0 kudos
- 642 Views
- 1 replies
- 0 kudos
Databricks summit 2024 unity catalog migration!!
Databricks summit 2024 unity catalog migration!!
- 642 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @269violetta, At the Data + AI Summit 2024, Databricks introduced exciting advancements for Unity Catalog, their unified governance solution for data and AI. Here are the highlights: Open Sourcing Unity Catalog: Databricks has open-sourced Un...
- 0 kudos
- 3216 Views
- 3 replies
- 0 kudos
Error: Folder xxxx@xxx.com is protected
Hello, On Azure Databricks i'm trying to remove a folder on the Repos folder using the following command : databricks workspace delete "/Repos/xxx@xx.com"I got the following error message:databricks workspace delete "/Repos/xxxx@xx.com"Error: Folder ...
- 3216 Views
- 3 replies
- 0 kudos
- 0 kudos
The repo/username folder is your home folder and it is a system folder. Therefore, the folder cannot be deleted or renamed
- 0 kudos
- 574 Views
- 1 replies
- 0 kudos
Facing issue with Databricks Context Limit Exceeded
We have a use case , where there may be chance of 200 jobs executing at once. But Few notebooks are failing with an issue "run failed with error message Too many execution contexts are open right now.(Limit set currently to 150)." Can anyone help how...
- 574 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello, you can refer to the following KB article for information and best practices in regards the issue you are facing: https://kb.databricks.com/en_US/notebooks/too-many-execution-contexts-are-open-right-now Best practices Use a job cluster instea...
- 0 kudos
- 907 Views
- 3 replies
- 0 kudos
Issue with importing data into Power BI from a view containing 100 million records in ADB
Hello, everyone. I'm trying to import data into Power BI from a view in Azure Databricks that contains over 10 million records. Every time it's failing with this following error"Failed to save modifications to the server. Error returned: 'OLE DB or O...
- 907 Views
- 3 replies
- 0 kudos
- 0 kudos
Users have the option of either using OAuth or switching to Personal Access Token as a solution. I am sorry to say that this is the only method I am aware of at the present. hi
- 0 kudos
- 408 Views
- 1 replies
- 0 kudos
What is the vector database to generate in DataBricks?
What is the vector database to generate in DataBricks?
- 408 Views
- 1 replies
- 0 kudos
- 0 kudos
it is used for RAG models (generative AI) and contains embeddings used by those models.https://docs.databricks.com/en/generative-ai/vector-search.html
- 0 kudos
- 659 Views
- 1 replies
- 0 kudos
Certification exam
Hello Team, Today my exam was scheduled at 7:30 AM. No one contacted me before the exam for any validation/guideline. About one hour after, the proctor suspended my exam. Please check the video. I was constantly looking at the screen only. I have rai...
- 659 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Databricks community and @Cert-Team,It is more than 24 hours and I did not get any update after my Databricks Professional Exam got suspended.Please help me to sort this out so that I can complete the exam. It is very important for me to complete ...
- 0 kudos
- 995 Views
- 4 replies
- 1 kudos
Databricks DataEngineering Learning
I am new to Databricks. I am trying to do the lab work given Databricks DataEngineering course, at workbook 4.1 I am getting below error. Please help to resolve.Expected one and only one cluster definition.Edit the config via the JSON interface to re...
- 995 Views
- 4 replies
- 1 kudos
- 944 Views
- 1 replies
- 0 kudos
Vector Database
What is the vector database to generate in DataBricks?
- 944 Views
- 1 replies
- 0 kudos
- 0 kudos
Not sure if i understood the question. If you want to use Databricks Vector Database, just go to your table > create vector search index. First you need to create a Vector Search Endpoint (compute > vector search), and you need to have an enabled ser...
- 0 kudos
