Join the Databricks Learning Festival (Virtual)! Mark your calendars from 10 October - 31 October 2024! Upskill today across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career w...
- 61858 Views
- 266 replies
- 48 kudos
09-17-2024
Thursday, 24 October 2024, 16:00 - 18:30 Slalom Consulting - 545 NW 26th St #300, Miami, FL 33127 Join us as the South Florida Databricks User Group debuts with an in-person event hosted at Slalom Consulting in Miami. We've planned an explora...
- 365 Views
- 0 replies
- 0 kudos
Wednesday
Databricks Apps, a new way to build and deploy internal data and AI applications, is now available in Public Preview on AWS and Azure. Ideal use cases include data visualization, AI applications, self-service analytics, and data quality monitoring.It...
- 12733 Views
- 0 replies
- 5 kudos
2 weeks ago
What if anyone in your organization could get insights from their data using just natural language? What if you didn’t have to manage a BI system separate from your data platform?Databricks is working to make your job easier by delivering these and o...
- 11793 Views
- 0 replies
- 1 kudos
2 weeks ago
Meet Szymon Dybczak, a valued member of our community! Szymon is a Senior Data Engineer at Nordcloud. He brings a wealth of knowledge and expertise to the group, and we're thrilled to have him here. We presented him with a range of questions, and be...
- 13712 Views
- 11 replies
- 9 kudos
3 weeks ago
Don't Miss Out on "Making AI-Powered Data Engineering Practical"! Join us for this exciting virtual event where we’ll cut through the hype and explore how AI-powered data intelligence is transforming data engineering. AMER: Nov 4 / 10 AM PT EMEA: ...
- 5872 Views
- 2 replies
- 3 kudos
a month ago
Shifting to customized GenAI that deeply understands your data AMER October 8 / 10 AM PTEMEA October 9 / 9 AM BST / 10 AM CESTAPJ October 10 / 12 PM SGT Click here to check out the agenda and speakers and register now! Why are 9 out of 10 organiz...
- 5105 Views
- 2 replies
- 2 kudos
09-19-2024
Get practical guidance, notebooks, code snippets You know this better than anyone: the best GenAI models in the world will not succeed without good data. That’s why data engineers are even more critical today. The challenge is staying ahead of the ra...
- 13546 Views
- 0 replies
- 3 kudos
09-19-2024
Community Activity
- 0 Views
- 0 replies
- 0 kudos
Databricks Learning Festival
I completed the course Data Engineer Learning Plan - Data Engineering with Databricks today. How will receive the 50%-off Databricks certification voucher (worth US$100)? Regards,Saurabh
- 0 Views
- 0 replies
- 0 kudos
- 63 Views
- 2 replies
- 2 kudos
Databricks Data Engineer Associate exam passed but Certificate Credentials not yet received
Hi I have passed the exam Databricks Data Engineer Associate on 20th Oct , but Certificate Credentials not yet received.Appreciate any help on this.
- 63 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi Databricks Team @Nadia1 Could you please respond. I passed the exam Databricks Data Engineer Associate on 20 Oct , but Certificate Credentials not yet received.I raised support ticket #00554369 also, but no response yet.Regards,Sridevi.
- 2 kudos
- 39 Views
- 2 replies
- 0 kudos
What is the best practice for connecting Power BI to Azure Databricks?
I refer this document to connect Power BI Desktop and Power BI Service to Azure Databricks.Connect Power BI to Azure Databricks - Azure Databricks | Microsoft LearnHowever, I have a couple of quesitions and concerns. Can anyone kindly help?It seems l...
- 39 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Looks like this is the only option. I need to use multiple-node mode. Cannot use single-node mode because I also need the compute to support Unity Catalog.single-node mode does not support adding service principal as the user for single user access f...
- 0 kudos
- 102 Views
- 2 replies
- 0 kudos
Help to lead the Charlotte usergroup
The Databricks community team needs your help to lead and shape this local community. Why You Should Step Up: Membership: You are already a member of the Databricks community in this region, which gives you a unique perspective on topics relevant to ...
- 102 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Rishabh, I would love to know more about how I can contribute for Charlotte Group. I'm based in Fort Mill, SC but Charlotte is at driveable distance. I recently joined Databricks and made a career pivot to Data Science recently after taking up a D...
- 0 kudos
- 236 Views
- 14 replies
- 1 kudos
Sql Error on MultiNode cluster, but fine on SingleNode
If I run the following code on a cluster in SingleNode mode it works fine, but if I run the exact same cell on a MultiNode Cluster It throws:SparkConnectGrpcException: (java.sql.SQLTransientConnectionException) Could not connect to address=(host=HOST...
- 236 Views
- 14 replies
- 1 kudos
Latest Reply
- 1 kudos
data_security_mode": "NONE": This is a non-Unity Catalog Cluster. No Governance enforced. "data_security_mode": "USER_ISOLATION": This is a UC Shared Compute cluster that has certain limitations when accessing Low-Level APIs, RDDs, and dbfs/data bri...
- 1 kudos
- 11095 Views
- 2 replies
- 1 kudos
DeltaFileNotFoundException: No file found in the directory (sudden task failure)
Hi all,I am currently running a job that will upsert a table by reading from delta change data feed from my silver table. Here is the relevent snippet of code: rds_changes = spark.read.format("delta") \ .option("readChangeFeed", "true") \ .optio...
- 11095 Views
- 2 replies
- 1 kudos
- 111 Views
- 2 replies
- 0 kudos
Connect Purview to AWS Databricks Unity Catalog?
Hello! Newbie here, so apologies if this is a super basic question. Trying to figure out if we can connect Purview to an AWS instance of Databricks (vice Azure instance), but I have only seen articles on connecting Azure Databricks to Purview. I r...
- 111 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Interesting question. And frankly I have no idea if it would extend to AWS. My first guess is: no.It might be possible with some kind of custom connector, but chances are this will not deliver the same experience.So you might have to wait on the rep...
- 0 kudos
- 13328 Views
- 9 replies
- 1 kudos
Resolved! Free voucher for data engineer associate certification
Hi,I'm looking for a free voucher to take up the Data Engineer Associate certification. Could anyone guide please ?
- 13328 Views
- 9 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Venkata-Sairam I'm sorry, Databricks does not offer free certification, but here is a current promo you could join for 50% off: https://community.databricks.com/t5/events/databricks-learning-festival-virtual-10-october-31-october/ev-p/90710
- 1 kudos
- 27 Views
- 0 replies
- 0 kudos
Lakehouse federation support for Oracle DB
https://docs.databricks.com/en/query-federation/index.htmlAre there plans to provide Oracle support for Databricks on AWS lakehouse federation? Not sure if that's possible or not. Thanks!
- 27 Views
- 0 replies
- 0 kudos
- 183 Views
- 4 replies
- 1 kudos
Resolved! How to use Databricks CLI as a service principal?
Hi all,I have a question about how to use Databricks CLI on my local environment as a service principal?I have installed Databricks CLI and configured the file `.databrickscfg` as shown below. [DEFAULT] host = https://adb-123123123.1.azuredatabr...
- 183 Views
- 4 replies
- 1 kudos
Latest Reply
- 1 kudos
got you.I found a working solution. Try this one:[devsp] azure_workspace_resource_id = /subscriptions/bc0cd1..././.../Databricks/workspaces/my-workspace azure_tenant_id = bc0cd1... azure_client_id = fa0cd1... azure_client_secr...
- 1 kudos
- 82 Views
- 2 replies
- 0 kudos
Embed Dashboard - GraphQL Operation Not Authentic
I have added a domain to my list of approved domains for embedding dashboards from my Databricks instance. This domain hosts my Docusaurus site. When the page with the embedded dashboard loads, it makes some network requests to Databricks that are fa...
- 82 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @trevormccormick ,Are you authenticated in your site that has the embedded dashboard?One thing is to add an approved domain, the other to authenticate to Databricks workspace.https://www.databricks.com/blog/how-embed-aibi-dashboards-your-websites-...
- 0 kudos
- 71 Views
- 1 replies
- 0 kudos
Machine Learning with Databricks
Hi,I am going through the Machine Learning with Databricks course under the learning festival. However, I do not see any link to the repository/notebooks/material where we can download and configure it in my environment. Could you help us sharing the...
- 71 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
See whether this helps. https://github.com/databricks/databricks-ml-examples
- 0 kudos
- 5263 Views
- 2 replies
- 1 kudos
Databricks time travel - how to get ALL changes ever done to a table
Hi time travel gurus,I am investigating creating a reporting solution with an AsOf functionality. Users will be able to create a report based on the current data or on the data AsOf some time ago. Due to the nature of our data this AsOf feature is qu...
- 5263 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
@oishimbo @jasbro This can be tricky because, to achieve this functionality, you need to consider both VACUUM retention and time travel settings. You can use the following query to retrieve data based on your time travel needs, but ensure that the VA...
- 1 kudos
- 3972 Views
- 9 replies
- 2 kudos
Resolved! Certificate and Badge not received
I have cleared my certification exam, Databricks Certified Data Engineer Associate on 28 Oct 2022. I am yet to receive a certificate or badge. Any help is much appreciate
- 3972 Views
- 9 replies
- 2 kudos
- 18 Views
- 0 replies
- 0 kudos
Handling Audit Columns and SCD Type 1 in Databricks DLT Pipeline with Unity Catalog: Circular Depend
I am working on a Delta Live Tables (DLT) pipeline with Unity Catalog, where we are reading data from Azure Data Lake Storage (ADLS) and creating a table in the silver layer with Slowly Changing Dimensions (SCD) Type 1 enabled. In addition, we are ad...
- 18 Views
- 0 replies
- 0 kudos
Latest from our Blog
Table-level governance and Access Control for the enterprise
Enterprises have often federated teams with a central governance structure. Depending on the sensitivity of the data, the governance team controls who can have access to data. In this article we show ...
What is a ComposerModel? Why is it useful for training neural networks?
When I first started using the OSS libraries Composer and LLM Foundry for training Large Language Models, I was surprised to see an abstraction I had not seen before — the ComposerModel . I had heard ...
Navigating the SAP Data Ocean: Demystifying SAP Data Extraction for Databricks Integration using ODP
Introduction Data silos affect businesses across all industries, not just Manufacturing, where IT/OT convergence is a game-changer. To address this, Databricks has built a Unified Analytics Platform t...
Metadata-Driven ETL Framework in Databricks (Part-1)
In modern data-driven enterprises, data flows like lifeblood through complex systems and repositories to drive decision-making and innovation. Each dataset, whether structured or unstructured, holds t...
