- 539 Views
- 0 replies
- 0 kudos
Hi, how can one go about assessing the value created due to the implementation of DBRX?
Hi, how can one go about assessing the value created due to the implementation of DBRX?
- 539 Views
- 0 replies
- 0 kudos
Forum Posts
- 614 Views
- 0 replies
- 0 kudos
How does Datbricks differ from snowflake with respect to AI tooling
How does Datbricks differ from snowflake with respect to AI tooling
- 614 Views
- 0 replies
- 0 kudos
- 1531 Views
- 1 replies
- 0 kudos
org.apache.spark.SparkException: Job aborted due to stage failure:
org.apache.spark.SparkException: Job aborted due to stage failure:
- 1531 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Along with Job aborted due to stage failure: if you see slave lost... then it is due to less memory allocated for executors, more cores per executor more memory required or the other possibility is you have used max cpu available in cluster and the d...
- 0 kudos
- 855 Views
- 0 replies
- 0 kudos
UC_COMMAND_NOT_SUPPORTED.WITHOUT_RECOMMENDATION in shared access mode?
I'm using a shared access cluster and am getting this error while trying to upload to Qdrant. #embeddings_df = embeddings_df.limit(5) options = { "qdrant_url": QDRANT_GRPC_URL, "api_key": QDRANT_API_KEY, "collection_name": QDRANT_COLLEC...
- 855 Views
- 0 replies
- 0 kudos
- 2088 Views
- 1 replies
- 0 kudos
Facing issue with Databricks Context Limit Exceeded
We have a use case , where there may be chance of 200 jobs executing at once. But Few notebooks are failing with an issue "run failed with error message Too many execution contexts are open right now.(Limit set currently to 150)." Can anyone help how...
- 2088 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello, you can refer to the following KB article for information and best practices in regards the issue you are facing: https://kb.databricks.com/en_US/notebooks/too-many-execution-contexts-are-open-right-now Best practices Use a job cluster instea...
- 0 kudos
- 2493 Views
- 1 replies
- 1 kudos
Resolved! In Azure Databricks, for when to use Kubernetes instead of Virtual Machines as compute backend?
- 2493 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
There is no distinction to make, it's VM's and you can't choose. Databricks SQL Serverless Warehouses uses K8s under the hood though.
- 1 kudos
- 3421 Views
- 3 replies
- 0 kudos
Issue with importing data into Power BI from a view containing 100 million records in ADB
Hello, everyone. I'm trying to import data into Power BI from a view in Azure Databricks that contains over 10 million records. Every time it's failing with this following error"Failed to save modifications to the server. Error returned: 'OLE DB or O...
- 3421 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Users have the option of either using OAuth or switching to Personal Access Token as a solution. I am sorry to say that this is the only method I am aware of at the present. hi
- 0 kudos
- 2707 Views
- 0 replies
- 0 kudos
Databricks summit 2024 unity catalog migration!!
Databricks summit 2024 unity catalog migration!!
- 2707 Views
- 0 replies
- 0 kudos
- 654 Views
- 0 replies
- 0 kudos
Hi all, I recently enabled SSO on my Databricks account. Now, when a user signs in they see "No work
Hi all, I recently enabled SSO on my Databricks account. Now, when a user signs in they see "No workspaces have been enabled for your account", which is the expected behavior as I haven't created any workspaces yet
- 654 Views
- 0 replies
- 0 kudos
- 681 Views
- 0 replies
- 0 kudos
Does databricks support spearman rank correlation
  while working on spark scala. I tried working on code using pearson correaltion it works but when I tried to work on spearman rank correlation it displayed error like Currently only the calculation of the Pearson Correlation coefficient is support...
- 681 Views
- 0 replies
- 0 kudos
- 1352 Views
- 0 replies
- 0 kudos
Concurrent State Update from Worker Nodes Possible?
For a data processing pipeline I use structured streaming and arbitrary stateful processing. I was wondering if the partitioning over several worker nodes and thus updating the state from different worker nodes has to be considered (e.g. using a lock...
- 1352 Views
- 0 replies
- 0 kudos
- 1138 Views
- 1 replies
- 0 kudos
What is the vector database to generate in DataBricks?
What is the vector database to generate in DataBricks?
- 1138 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
it is used for RAG models (generative AI) and contains embeddings used by those models.https://docs.databricks.com/en/generative-ai/vector-search.html
- 0 kudos
- 4187 Views
- 3 replies
- 1 kudos
Confused with databricks Tips and Tricks - Optimizations regarding partitining
Hello Community,Today I was in Tips and Tricks - Optimizations webinar and I started being confused, they said:"Don't partition tables <1TB in size and plan carefully when partitioning• Partitions should be >=1GB" Now my confusion is if this recommen...
- 4187 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
that is partitions on disk.Defining the correct amount of partitions is not that easy. One would think that more partitions is better because you can process more data in parallel.And that is true if you only have to do local transformations (no shu...
- 1 kudos
- 1189 Views
- 1 replies
- 0 kudos
Certification exam
Hello Team, Today my exam was scheduled at 7:30 AM. No one contacted me before the exam for any validation/guideline. About one hour after, the proctor suspended my exam. Please check the video. I was constantly looking at the screen only. I have rai...
- 1189 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Databricks community and @Cert-Team,It is more than 24 hours and I did not get any update after my Databricks Professional Exam got suspended.Please help me to sort this out so that I can complete the exam. It is very important for me to complete ...
- 0 kudos
- 2020 Views
- 4 replies
- 1 kudos
Databricks DataEngineering Learning
I am new to Databricks. I am trying to do the lab work given Databricks DataEngineering course, at workbook 4.1 I am getting below error. Please help to resolve.Expected one and only one cluster definition.Edit the config via the JSON interface to re...
- 2020 Views
- 4 replies
- 1 kudos
Latest Reply
