Join the Databricks Learning Festival (Virtual)! Mark your calendars from 10 October - 31 October 2024! Upskill today across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career w...
- 23291 Views
- 75 replies
- 35 kudos
3 weeks ago
Summary Databricks Apps, a new way to build and deploy internal data and AI applications, is now available in Public Preview on AWS and Azure.Ideal use cases include data visualization, AI applications, self-service analytics, and data quality monito...
- 23 Views
- 0 replies
- 1 kudos
55m ago
Meet Szymon Dybczak, a valued member of our community! Szymon is a Senior Data Engineer at Nordcloud. He brings a wealth of knowledge and expertise to the group, and we're thrilled to have him here. We presented him with a range of questions, and be...
- 807 Views
- 7 replies
- 7 kudos
a week ago
Don't Miss Out on "Making AI-Powered Data Engineering Practical"! Join us for this exciting virtual event where we’ll cut through the hype and explore how AI-powered data intelligence is transforming data engineering. AMER: Nov 4 / 10 AM PT EMEA: ...
- 1546 Views
- 1 replies
- 2 kudos
2 weeks ago
Shifting to customized GenAI that deeply understands your data AMER October 8 / 10 AM PTEMEA October 9 / 9 AM BST / 10 AM CESTAPJ October 10 / 12 PM SGT Click here to check out the agenda and speakers and register now! Why are 9 out of 10 organiz...
- 739 Views
- 1 replies
- 2 kudos
3 weeks ago
Get practical guidance, notebooks, code snippets You know this better than anyone: the best GenAI models in the world will not succeed without good data. That’s why data engineers are even more critical today. The challenge is staying ahead of the ra...
- 1436 Views
- 0 replies
- 3 kudos
3 weeks ago
Community Activity
- 0 Views
- 0 replies
- 0 kudos
Unable to create a databricks workspace
I am unable to create a databricks workspace with vnet injection. I get this error:│ { │ "status": "Failed", │ "error": { │ "code": "InternalServerError", │ "message": "INTERNAL_ERROR: Unexpected error: Cannot call getCertifiedMetastoreFo...
- 0 Views
- 0 replies
- 0 kudos
- 20 Views
- 5 replies
- 0 kudos
Joining huge delta tables in Databricks
Hello,I am trying to join few delta tables as per the code below.SQLCopy select <applicable columns> FROM ReportTable G LEFT JOIN EKBETable EKBE ON EKBE.BELNR = G.ORDER_ID LEFT JOIN PurchaseOrder POL ON EKBE.EBELN = POL.PO_NOThe PurchaseOrder table c...
- 20 Views
- 5 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Alok, try to gather statistics for the important columns. Databricks gathers stats for the first 32 columns of the table by default. This helps in data skipping. Check all your important/frequently used columns are in first 32 positions of the del...
- 0 kudos
- 4 Views
- 0 replies
- 0 kudos
No shareable link found for the "Databricks Fundamentals Accreditation" badge
I have just passed the "Databricks Fundamentals Accreditation" in the Databricks Academy but now I can't find any shareable link to showcase the badge in my social media.I followed this credentials.databricks.com link and created a new account using ...
- 4 Views
- 0 replies
- 0 kudos
- 102 Views
- 2 replies
- 0 kudos
Resolved! Databricks Certified Associate Developer for Apache Spark 3 Got Suspended. Require Immediate support
Hello @Cert-Team @Certificate Team,Request Id# 00544837I encountered a pathetic experience while attempting my Databricks Certified Associate Developer for Apache Spark 3 certification exam. I had answered more than 44 questions I applied for Data...
- 102 Views
- 2 replies
- 0 kudos
- 50 Views
- 3 replies
- 0 kudos
how to use rest api to find long running query in databricks
how to use rest api to find long running query in databricks from sql/queries/all
- 50 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
@Ajay-Pandey to find out if my databricks is using rest api, how do i check it
- 0 kudos
- 372 Views
- 7 replies
- 4 kudos
Resolved! How to get rid of a pesky gen AI feature in the editor ?
Hi,The editor interface has that gen AI feature following empty lines with a cursor. I find that very distracting and irritating. More over, once a line is deleted that unsolicited thing is interfering with code (snapshots included).How to get rid of...
- 372 Views
- 7 replies
- 4 kudos
Latest Reply
- 4 kudos
Summary:I was not able to solve the UI/UX artifact on my own (on user side)The UI/UX issue was resolved somewhere on Databricks side. The UI/AX artifact is no longer interfering with work.
- 4 kudos
- 35 Views
- 1 replies
- 0 kudos
Resolved! Attach a databricks_instance_pool to databricks_cluster_policy via terraform
Hello Team,I am trying to create a databricks instance pool and attach it to a cluster policy in our terraform code. But I am having hard time finding a good documentation. Has any one done it? Below is my sample code and I am getting errorI keep get...
- 35 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Fixed it! "instance_pool_id" : { type = "fixed" values = "databricks_instance_pool.dev_test_cluster_pool.id"}
- 0 kudos
- 23 Views
- 0 replies
- 1 kudos
Introducing Databricks Apps
Summary Databricks Apps, a new way to build and deploy internal data and AI applications, is now available in Public Preview on AWS and Azure.Ideal use cases include data visualization, AI applications, self-service analytics, and data quality monito...
- 23 Views
- 0 replies
- 1 kudos
- 8 Views
- 0 replies
- 0 kudos
Is it possible to do Sparse checkouts in databricks workflow?
Hi Team,When I am trying to integrate repo with the databricks workflow, I am getting this error "Working branches are limited to 200 MB.". To avoid this error, if we have an option for sparse checkouts mode it will be helpful.#workflows #spark
- 8 Views
- 0 replies
- 0 kudos
- 525 Views
- 4 replies
- 2 kudos
Enabling vector search
I have a team member doing the GenAI certification but she gets the following errorResponse content b'{"error_code":"BAD_REQUEST","message":"This trial workspace no longer has access to Vector Search. Please upgrade the workspace to continue to use V...
- 525 Views
- 4 replies
- 2 kudos
Latest Reply
- 2 kudos
I am trying to enable mosiac vector search for my workspace but so far I can see the metastore in my catalog but no vector search.though I am on paid tier and have enabled unity catalog on the workspace. I am still not able to see vector search on my...
- 2 kudos
- 17 Views
- 2 replies
- 1 kudos
asset bundles and compute policies
Did anyone succeed in using already existing compute policies (created using the UI) in asset bundles for creating a job?I defined the policy_id in the resources/job yml for the job_cluster, but when deploying I get errors saying spark version is not...
- 17 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
It looks like it, but I also get errors on non singlenode clusters.But there might be an underlying issue with policy settings not being applied.Tnx for the link though.
- 1 kudos
- 5 Views
- 0 replies
- 0 kudos
Help to lead the Dallas usergroup
The Databricks community team needs your help to lead and shape this local community. Why You Should Step Up: Membership: You are already a member of the Databricks community in this region, which gives you a unique perspective on topics relevant to ...
- 5 Views
- 0 replies
- 0 kudos
- 7 Views
- 0 replies
- 0 kudos
Help to lead the Charlotte usergroup
The Databricks community team needs your help to lead and shape this local community. Why You Should Step Up: Membership: You are already a member of the Databricks community in this region, which gives you a unique perspective on topics relevant to ...
- 7 Views
- 0 replies
- 0 kudos
- 8 Views
- 0 replies
- 0 kudos
Help to lead the Atlanta usergroup
The Databricks community team needs your help to lead and shape this local community. Why You Should Step Up: Membership: You are already a member of the Databricks community in this region, which gives you a unique perspective on topics relevant to ...
- 8 Views
- 0 replies
- 0 kudos
- 52 Views
- 2 replies
- 1 kudos
Help to lead the Austin usergroup
The Databricks community team needs your help to lead and shape this local community. Why You Should Step Up: Membership: You are already a member of the Databricks community in this region, which gives you a unique perspective on topics relevant to ...
- 52 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi Rishabh,I'm interested. Please connect with me.Regards,Brahma
- 1 kudos
Latest from our Blog
Private and Dedicated Connectivity Patterns for Databricks Serverless Using Private Link (Part 2)
Intro In the first part of our series, we explored the architecture and some typical use cases for a new feature in Databricks that allows customers to connect their Serverless clusters to private res...
SAM2 on Databricks
In this post we’ll walk through getting started with Meta’s latest Segment-Anything-Model 2 (SAM2) on Databricks. We’ll cover experimentation with SAM2 in a Databricks Notebook, expand on the default ...
Dynamic Segmentation in Geospatial Analytics on Databricks - Part 2
This is the second part of a two-part series blog on geospatial data processing on Databricks. In the first part, we covered ingesting and processing Overture Maps data on Databricks. In this second ...
Real-time Vehicle Fleet Analytics with Databricks Delta Live Tables
Introduction Today’s modern vehicles generate large streams of data which can be harnessed in a data-driven manner to boost efficiency, optimize costs, and minimize our environmental impact in real-ti...
