cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

ganemouni
by New Contributor
  • 2789 Views
  • 1 replies
  • 0 kudos

Facing issue with Databricks Context Limit Exceeded

We have a use case , where there may be chance of 200 jobs executing at once. But Few notebooks are failing with an issue "run failed with error message Too many execution contexts are open right now.(Limit set currently to 150)." Can anyone help how...

ganemouni_0-1718803036914.png
  • 2789 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Hello, you can refer to the following KB article for information and best practices in regards the issue you are facing: https://kb.databricks.com/en_US/notebooks/too-many-execution-contexts-are-open-right-now  Best practices Use a job cluster instea...

  • 0 kudos
FerArribas
by Contributor
  • 3299 Views
  • 1 replies
  • 1 kudos
  • 3299 Views
  • 1 replies
  • 1 kudos
Latest Reply
jacovangelder
Databricks MVP
  • 1 kudos

There is no distinction to make, it's VM's and you can't choose. Databricks SQL Serverless Warehouses uses K8s under the hood though. 

  • 1 kudos
Jagadish_Vela
by New Contributor II
  • 4953 Views
  • 3 replies
  • 0 kudos

Issue with importing data into Power BI from a view containing 100 million records in ADB

Hello, everyone. I'm trying to import data into Power BI from a view in Azure Databricks that contains over 10 million records. Every time it's failing with this following error"Failed to save modifications to the server. Error returned: 'OLE DB or O...

ADB Token Error.png
  • 4953 Views
  • 3 replies
  • 0 kudos
Latest Reply
riquedan
New Contributor II
  • 0 kudos

Users have the option of either using OAuth or switching to Personal Access Token as a solution. I am sorry to say that this is the only method I am aware of at the present. hi 

  • 0 kudos
2 More Replies
chand20025
by New Contributor
  • 921 Views
  • 0 replies
  • 0 kudos

Does databricks  support spearman rank correlation

  while working on spark scala. I tried working on code using pearson correaltion it works but when I  tried to work on spearman rank correlation it displayed error like Currently only the calculation of the Pearson Correlation coefficient is support...

Screenshot (21).png
  • 921 Views
  • 0 replies
  • 0 kudos
fperry
by New Contributor III
  • 1550 Views
  • 0 replies
  • 0 kudos

Concurrent State Update from Worker Nodes Possible?

For a data processing pipeline I use structured streaming and arbitrary stateful processing. I was wondering if the partitioning over several worker nodes and thus updating the state from different worker nodes has to be considered (e.g. using a lock...

  • 1550 Views
  • 0 replies
  • 0 kudos
karola61
by New Contributor II
  • 1582 Views
  • 1 replies
  • 0 kudos

What is the vector database to generate in DataBricks?

What is the vector database to generate in DataBricks?

  • 1582 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

it is used for RAG models (generative AI) and contains embeddings used by those models.https://docs.databricks.com/en/generative-ai/vector-search.html

  • 0 kudos
eimis_pacheco
by Contributor
  • 4758 Views
  • 3 replies
  • 1 kudos

Confused with databricks Tips and Tricks - Optimizations regarding partitining

Hello Community,Today I was in Tips and Tricks - Optimizations webinar and I started being confused, they said:"Don't partition tables <1TB in size and plan carefully when partitioning• Partitions should be >=1GB" Now my confusion is if this recommen...

Get Started Discussions
data engineering
performance
  • 4758 Views
  • 3 replies
  • 1 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

that is partitions on disk.Defining the correct amount of partitions is not that easy.  One would think that more partitions is better because you can process more data in parallel.And that is true if you only have to do local transformations (no shu...

  • 1 kudos
2 More Replies
sucharita-das
by New Contributor II
  • 1491 Views
  • 1 replies
  • 0 kudos

Certification exam

Hello Team, Today my exam was scheduled at 7:30 AM. No one contacted me before the exam for any validation/guideline. About one hour after, the proctor suspended my exam. Please check the video. I was constantly looking at the screen only. I have rai...

Get Started Discussions
Databricks Professional
exam suspended
  • 1491 Views
  • 1 replies
  • 0 kudos
Latest Reply
sucharita-das
New Contributor II
  • 0 kudos

Hi Databricks community and @Cert-Team,It is more than 24 hours and I did not get any update after my Databricks Professional Exam got suspended.Please help me to sort this out so that I can complete the exam. It is very important for me to complete ...

  • 0 kudos
subahan_dbricks
by New Contributor II
  • 2676 Views
  • 4 replies
  • 1 kudos

Databricks DataEngineering Learning

I am new to Databricks. I am trying to do the lab work given Databricks DataEngineering course, at workbook 4.1 I am getting below error. Please help to resolve.Expected one and only one cluster definition.Edit the config via the JSON interface to re...

  • 2676 Views
  • 4 replies
  • 1 kudos
Latest Reply
subahan_dbricks
New Contributor II
  • 1 kudos

I am unable to resolve this. Any help would be appreciated. Thanks

  • 1 kudos
3 More Replies
Nexalyn33
by New Contributor II
  • 1915 Views
  • 1 replies
  • 0 kudos

[offisiell] Nexalyn Norge Erfaringer anmeldelser – Nexalyn Ingredienser pris, kjøp

Nexalyn Norge Opplevelser Dose, inntak: I en verden hvor vitalitet og ytelse ofte er synonymt med suksess, er det viktig å opprettholde topp fysisk form. For menn strekker dette seg ofte utover bare kondisjon til områder med vitalitet, virilitet og g...

  • 1915 Views
  • 1 replies
  • 0 kudos
Latest Reply
Nexalyn33
New Contributor II
  • 0 kudos

Klikk her for å kjøpe nå fra den offisielle nettsiden til Nexalyn

  • 0 kudos
Uderson
by New Contributor
  • 3003 Views
  • 1 replies
  • 0 kudos

Vector Database

What is the vector database to generate in DataBricks?

  • 3003 Views
  • 1 replies
  • 0 kudos
Latest Reply
mhiltner
Databricks Employee
  • 0 kudos

Not sure if i understood the question. If you want to use Databricks Vector Database, just go to your table > create vector search index. First you need to create a Vector Search Endpoint (compute > vector search), and you need to have an enabled ser...

  • 0 kudos
Newbienewbster
by New Contributor II
  • 895 Views
  • 1 replies
  • 1 kudos

How do you analyze performance

Curious to hear how you guys optimize compute. As in how you dig into the details of the Spark execution and improve?

  • 895 Views
  • 1 replies
  • 1 kudos
Latest Reply
mhiltner
Databricks Employee
  • 1 kudos

That is it. Usually, people take the time it takes to run a job/query/process as their KPI.  Then you start to check which processes are taking more time, drilling down one by one. Sometimes it could be a misplaced .cache(), .collect() or display() t...

  • 1 kudos
PragyaS
by New Contributor
  • 2051 Views
  • 0 replies
  • 0 kudos

Creating Zip file on Azure storage explorer

I need to create zip file which will contain csv files which I have created from the dataframe. But I am unable to create valid zip file which will be containing all csv files. Is it possible to create zip file from code in databricks on azure storag...

  • 2051 Views
  • 0 replies
  • 0 kudos
Labels