cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 
Databricks Learning Festival (Virtual): 10 October - 31 October

 Join the Databricks Learning Festival (Virtual)!   Mark your calendars from 10 October - 31 October 2024!  Upskill today across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career w...

  • 27868 Views
  • 96 replies
  • 38 kudos
3 weeks ago
Introducing Databricks Apps

Databricks Apps, a new way to build and deploy internal data and AI applications, is now available in Public Preview on AWS and Azure. Ideal use cases include data visualization, AI applications, self-service analytics, and data quality monitoring.It...

  • 396 Views
  • 0 replies
  • 3 kudos
Tuesday
Databricks Community Champion - September 2024 - Szymon Dybczak

Meet Szymon Dybczak, a valued member of our community! Szymon is a Senior Data Engineer at Nordcloud. He brings a wealth of knowledge and expertise to the group, and we're thrilled to have him here. We presented him with a range of questions, and be...

  • 947 Views
  • 8 replies
  • 8 kudos
a week ago
Intelligent Data Engineering: Beyond the AI Hype

Don't Miss Out on "Making AI-Powered Data Engineering Practical"! Join us for this exciting virtual event where we’ll cut through the hype and explore how AI-powered data intelligence is transforming data engineering.  AMER: Nov 4 / 10 AM PT EMEA: ...

  • 1647 Views
  • 1 replies
  • 2 kudos
2 weeks ago
GenAI: The Shift to Data Intelligence

Shifting to customized GenAI that deeply understands your data AMER  October 8 / 10 AM PTEMEA  October 9 / 9 AM BST / 10 AM CESTAPJ  October 10 / 12 PM SGT Click here to check out the agenda and speakers and register now! Why are 9 out of 10 organiz...

  • 917 Views
  • 2 replies
  • 2 kudos
3 weeks ago
Big Book of Data Engineering — 3rd Edition

Get practical guidance, notebooks, code snippets You know this better than anyone: the best GenAI models in the world will not succeed without good data. That’s why data engineers are even more critical today. The challenge is staying ahead of the ra...

  • 1728 Views
  • 0 replies
  • 3 kudos
3 weeks ago

Community Activity

endaemon
by > Visitor
  • 31 Views
  • 2 replies
  • 1 kudos

Differences among python libraries

I am confused as to the differences between various python libraries for databricks: especially with regard to differences among [databricks-connect](https://pypi.org/project/databricks-connect/), [databricks-api](https://pypi.org/project/databricks-...

  • 31 Views
  • 2 replies
  • 1 kudos
Latest Reply
endaemon
Visitor
  • 1 kudos

@szymon_dybczak,Thank you for typing all that up. It is very clear and helpful.Two follow ups if I may:1. If one's primary goal is to execute SQL queries why prefer databricks sql connector over a generic jdbc or odbc package?2. Did I miss any other ...

  • 1 kudos
1 More Replies
kunex
by > Visitor
  • 14 Views
  • 2 replies
  • 0 kudos

Asset Bundles Job If Condition

HiI am trying to use Asset Bundles to deploy a job with email notifications active only on production environment.I tried to use such if statement but it does not seem to do anything.resources: jobs: Master_Load: name: Master Load e...

  • 14 Views
  • 2 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @kunex ,You don't have to use if condition. The recommended approach is to use define "shared" configuration at the top level resources mapping and then when you need environment specific configuration use targets top level-level mapping to overri...

  • 0 kudos
1 More Replies
antr
by > New Contributor
  • 59 Views
  • 3 replies
  • 0 kudos

What does DLT INITIALIZING phase do?

In Delta Live Tables, the INITIALIZING phase takes sometimes a minute, sometimes 5 minutes. I'd like to learn what is it doing in the background, and can it be optimized in any way.

  • 59 Views
  • 3 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @antr, In Delta Live Tables (DLT), a feature of Databricks, the "Initializing" phase refers to the first step in the lifecycle of a DLT pipeline run. During this phase, the platform sets up the necessary resources, configurations, and dependencies...

  • 0 kudos
2 More Replies
noorbasha534
by > New Contributor
  • 20 Views
  • 1 replies
  • 0 kudos

Databricks as a "pure" data streaming software like Confluent

DearsI was wondering if anyone has leveraged Databricks as a "pure" data streaming software in place of Confluent, Flink, Kafka etc.I see the reference architectures placing Databricks on the data processing side mostly once data is made available by...

  • 20 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @noorbasha534 ,It depends on what you're asking for. Kafka is primarily a messaging system, optimized for handling high-throughput, distributed message logs. Databricks can read from Kafka as a data source but doesn't replace Kafka's role in messa...

  • 0 kudos
dyusuf
by > New Contributor
  • 42 Views
  • 1 replies
  • 0 kudos

Unity Catalog

Can we set up unity catalog on databricks community edition?If yes, please share the process.Thanks

  • 42 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @dyusuf ,No, it's not possible. You can setup free azure trial account though and then you should be able to create UC.

  • 0 kudos
Rishabh_Tiwari
by Community Manager
  • 14 Views
  • 0 replies
  • 0 kudos

Join Us for the Databricks Pittsburgh User Group Meetup!

Date & Time:Tuesday, October 22, 20244:00 PM - 6:00 PM Location:The Foundry Table381 N Shore Drive, Pittsburgh, PA 15212 Event Details We're excited to invite you to the next Databricks Pittsburgh User Group! This is a community for local users, ...

  • 14 Views
  • 0 replies
  • 0 kudos
Chandru13
by > Visitor
  • 35 Views
  • 1 replies
  • 0 kudos

Databricks Exam got Suspended. Need Urgent help

Dear Support Team,I hope this message finds you well. I am writing to express my concern regarding the recent suspension of my Databricks Associate Data Engineer certification exam, which I took on 10 October 2024(16:45 Asia/Calcutta). During the exa...

  • 35 Views
  • 1 replies
  • 0 kudos
Latest Reply
Cert-TeamOPS
Contributor III
  • 0 kudos

Hello @Chandru13 , Thank you for reaching out to Databricks Community support! We are sorry to hear that your exam was suspended.. Thank you for filing a ticket with our support team. Please allow the support team 24-48 hours for a resolution. In the...

  • 0 kudos
Vishalkhode1206
by > New Contributor II
  • 296 Views
  • 2 replies
  • 0 kudos

Resolved! Request for Certification Discount Voucher

Hey I am requesting databricks to provide one certification voucher as i have already the taken the exam couple of weeks ago but unfortunately i failed in the exam however i was 2 correct answers away from passing the test.I need to crack this course...

  • 296 Views
  • 2 replies
  • 0 kudos
Latest Reply
Cert-Team
Esteemed Contributor
  • 0 kudos

Hi @Vishalkhode1206 there is this promotion going on now: https://community.databricks.com/t5/events/databricks-learning-festival-virtual-10-october-31-october/ev-p/90710

  • 0 kudos
1 More Replies
RyoAriyama
by > Visitor
  • 30 Views
  • 1 replies
  • 0 kudos

Can't create table in unity catalog.

Hi all.I have created a Databricks workspace on AWS. I can log into the workspace and successfully perform select operations on files in S3, but I am unable to create tables.The error when creating the table is as follows. "Your request failed with s...

  • 30 Views
  • 1 replies
  • 0 kudos
Latest Reply
RyoAriyama
Visitor
  • 0 kudos

I attach the screenshot when I encounter the error.  

  • 0 kudos
arne_c
by > Visitor
  • 31 Views
  • 0 replies
  • 0 kudos

Set up compute policy to allow installing python libraries from a private package index

In our organization, we maintain a bunch of libraries we share code with. They're hosted on a private python package index, which requires a token to allow downloads. My idea was to store the token as a secret which would then be loaded into a cluste...

  • 31 Views
  • 0 replies
  • 0 kudos
Kayla
by > Valued Contributor
  • 117 Views
  • 2 replies
  • 1 kudos

Silly question-Easy way to show full notebook path or owner in UI?

We have a few people working in Databricks right now in different clones of the same repository. Occasionally we'll have multiple people with the same branch open- one working, another just has it open to see what it looks like, sort of deal.This has...

  • 117 Views
  • 2 replies
  • 1 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 1 kudos

Hi @Kayla ,I don't know if that's what you looking for, but for sure full path and owner information are available: 

  • 1 kudos
1 More Replies
mickniz
by > Contributor
  • 5776 Views
  • 7 replies
  • 0 kudos

Connect to Databricks from PowerApps

Hi All,Currently I trying to connect databricks Unity Catalog from Powerapps Dataflow by using spark connector specifying http url and using databricks personal access token as specified in below screenshot: I am able to connect but the issue is when...

mickniz_0-1714487746554.png mickniz_1-1714487891958.png
  • 5776 Views
  • 7 replies
  • 0 kudos
Latest Reply
william_lenotis
New Contributor II
  • 0 kudos

Do you have any firewall restriction? Maybe if the public IP are not allowed for your databricks workspace it is not allowing thew connection, besides that I don't have much clue on how to help. Would suggest open a support ticket.

  • 0 kudos
6 More Replies
osamam
by New Contributor II
  • 47 Views
  • 0 replies
  • 0 kudos

Optimising Data Integration and Serving patterns with Cosmos DB and Azure Databricks

Introduction In today's data-driven world, organisations aconstantly seekways to optimise their data integration and serving patterns. This blog post delves into the powerful combination of Azure Cosmos DB and Azure Databricks, exploring advanced int...

Azure databricks integration with CosmosDB (1).png Screenshot 2024-09-27 at 9.47.40 AM.png Screenshot 2024-09-27 at 9.49.32 AM.png Screenshot 2024-09-27 at 9.50.37 AM.png
  • 47 Views
  • 0 replies
  • 0 kudos
Rishabh_Tiwari
by Community Manager
  • 29 Views
  • 0 replies
  • 0 kudos

Help to lead the Vancouver usergroup

The Databricks community team needs your help to lead and shape this local community. Why You Should Step Up: Membership: You are already a member of the Databricks community in this region, which gives you a unique perspective on topics relevant to ...

  • 29 Views
  • 0 replies
  • 0 kudos
Rishabh_Tiwari
by Community Manager
  • 105 Views
  • 4 replies
  • 0 kudos

Help to lead the bay Area usergroup

The Databricks community team needs your help to lead and shape this local community. Why You Should Step Up: Membership: You are already a member of the Databricks community in this region, which gives you a unique perspective on topics relevant to ...

  • 105 Views
  • 4 replies
  • 0 kudos
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @Mike_Gerardi  Thank you so much for offering your support and space for hosting meetups—this is fantastic news! We're excited to have both you and @DChoi_WestMonro  leading the Bay Area user group together as co-leads. Your collaboration will bri...

  • 0 kudos
3 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors

Latest from our Blog

SAM2 on Databricks

In this post we’ll walk through getting started with Meta’s latest Segment-Anything-Model 2 (SAM2) on Databricks. We’ll cover experimentation with SAM2 in a Databricks Notebook, expand on the default ...

655Views 1kudos