cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
Engage in vibrant discussions covering diverse learning topics within the Databricks Community. Explore learning resources, share insights, and collaborate with peers to enhance your skills in data engineering, machine learning, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Certifications

Join dynamic discussions on Databricks certifications within the Community. Exchange insights, tips,...

831 Posts

Training offerings

Explore discussions on Databricks training programs and offerings within the Community. Get insights...

185 Posts

Get Started Discussions

Start your journey with Databricks by joining discussions on getting started guides, tutorials, and ...

2653 Posts

Activity in Community Discussions

Rohit00548
by > Visitor
  • 15 Views
  • 0 replies
  • 0 kudos

How to make a song for peace is a love you so jao ab

How to make sure you will be sala Uddin Ahmed to deewane season of the same hai bhai send your number and the first to make it to you too baby 

  • 15 Views
  • 0 replies
  • 0 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 19 Views
  • 0 replies
  • 1 kudos

Migrate External Tables to Managed

With managed tables, you can reduce your storage and compute costs thanks to predictive optimization or file list caching. Now it is really time to migrate external tables to managed ones, thanks to ALTER SET MANAGED functionality. Read more: - h...

external_migration.png
  • 19 Views
  • 0 replies
  • 1 kudos
sai_sakhamuri
by > New Contributor II
  • 20 Views
  • 1 replies
  • 0 kudos

Databricks Certification: Booking Slot While Taking Partner Academy Course?

Hi team,I'm currently diving into the Partnered Academy learning plan (e.g., Data Engineer Associate, ML Practitioner, etc.) and I'm looking to schedule my certification exam.I'm wondering about the official policy: Is it mandatory to complete the en...

  • 20 Views
  • 1 replies
  • 0 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 0 kudos

From my experience everything is independent from each other  

  • 0 kudos
AndreLIUfr
by > New Contributor III
  • 34715 Views
  • 47 replies
  • 30 kudos

community edition : "User is not a member of this workspace"

Something very strange has happened. When trying to login to my databricks community edition account. I'm getting the email with my verification code. but after entring that code, I'm getting the error message : "User is not a member of this workspac...

  • 34715 Views
  • 47 replies
  • 30 kudos
Latest Reply
abhaykpa
Visitor
  • 30 kudos

I'm also facing the same error on registering to community edition. After providing correct verification code, I'm getting error "User is not a member of this workspace". The url -> https://community.cloud.databricks.com/verify-code?tuuid=2cd7f036-38...

  • 30 kudos
46 More Replies
chris0991
by > New Contributor III
  • 1831 Views
  • 3 replies
  • 1 kudos

Best practices for optimizing Spark jobs

What are some best practices for optimizing Spark jobs in Databricks, especially when dealing large datasets? Any tips or resources would be greatly appreciated! I’m trying to analyze data on restaurant menu prices so that insights would be especiall...

  • 1831 Views
  • 3 replies
  • 1 kudos
Latest Reply
mo4
New Contributor II
  • 1 kudos

Their menu is full of creative options — it’s hard to pick just one favorite because everything tastes amazing!”

  • 1 kudos
2 More Replies
Prashanthkumar
by > New Contributor III
  • 11389 Views
  • 16 replies
  • 3 kudos

Is it possible to view Databricks cluster metrics using REST API

I am looking for some help on getting databricks cluster metrics such as memory utilization, CPU utilization, memory swap utilization, free file system using REST API.I am trying it in postman using databricks token and with my Service Principal bear...

Prashanthkumar_0-1705104529507.png
  • 11389 Views
  • 16 replies
  • 3 kudos
Latest Reply
ajayIG
New Contributor II
  • 3 kudos

Is there any solution found to get cpu, memory metrics for Hive meta store backed workloads ? We are not using UC. So can't use system tables 

  • 3 kudos
15 More Replies
Dimitry
by > Contributor III
  • 576 Views
  • 1 replies
  • 0 kudos

Unable to query external GraphQL API

Hi allI'm switching to evaluating Databricks after being unhappy with Microsoft Fabric.I'm looking for a platform to aggregate external files and tables, do processing and push into 3rd party using GraphQL. That push is going to be quite sophisticate...

Dimitry_0-1747295757996.png
  • 576 Views
  • 1 replies
  • 0 kudos
Latest Reply
jack_zaldivar
Databricks Employee
  • 0 kudos

hi @Dimitry , I'm not sure about this specific library, but I was able to query a graphQL endpoint with requests.post. Have you tested that at all? Happy to dig in deeper if you need to use this specific library for any reason, but if not - you may w...

  • 0 kudos
sjujjuru
by > New Contributor
  • 124 Views
  • 4 replies
  • 3 kudos

Resolved! REGARD COPOUN CODE

After completing all the relevant courses for the certification, I haven’t received the coupon code yet.

  • 124 Views
  • 4 replies
  • 3 kudos
Latest Reply
sjujjuru
New Contributor
  • 3 kudos

Thank you for confirming and taking the time to clarify this. I really appreciate your help and guidance!

  • 3 kudos
3 More Replies
prasad_vaze
by > New Contributor III
  • 12116 Views
  • 4 replies
  • 0 kudos

Resolved! Unable to add column comment on a View. Any way to update comments on multiple columns in bulk?

I noticed that unlike "Alter Table" there is no "Alter View"  command to add comment on a column in the existing view. This is a regular view created on Tables (and not Materialized view).  If the underlying table column has comment then the View inh...

Get Started Discussions
Data Governance
Unity Catalog
  • 12116 Views
  • 4 replies
  • 0 kudos
Latest Reply
Pinei
New Contributor II
  • 0 kudos

Use COMMENT ONCOMMENT ON | Databricks on AWS

  • 0 kudos
3 More Replies
Rajeshwar_Reddy
by > New Contributor II
  • 2057 Views
  • 4 replies
  • 0 kudos

ODBC connection issue Simba 64 bit

Hello AllAm getting the below error when trying to create ODBC DSN Simba 64 in local system to connect Databricks Server using the token and enabled SSL System trust store & Thrift Transport: HTTP.any helping hand really appreciated . [Simba][ThriftE...

  • 2057 Views
  • 4 replies
  • 0 kudos
Latest Reply
DougJames
New Contributor II
  • 0 kudos

Solved for my case. Still not sure why/how it was working on one server but not the other.Final fix was to add HTTPPath value to the Connection String I listed above.

  • 0 kudos
3 More Replies
viniciuscini
by > New Contributor
  • 5237 Views
  • 2 replies
  • 0 kudos

Improve query performance of direct query with Databricks

I’m building a dashboard in Power BI’s Pro Workspace, connecting data via Direct Query from Databricks (around 60 million rows from 15 combined tables), using a SQL Serverless (small size and 4 clusters).The problem is that the dashboard is taking to...

  • 5237 Views
  • 2 replies
  • 0 kudos
Latest Reply
ArekKemp
New Contributor
  • 0 kudos

@viniciuscini have you managed to get it working well for you?

  • 0 kudos
1 More Replies
Rezakorehi
by > New Contributor II
  • 480 Views
  • 7 replies
  • 11 kudos

Unity catalogues - What would you do

If you were creating Unity Catalogs again, what would you do differently based on your past experience?

  • 480 Views
  • 7 replies
  • 11 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 11 kudos

@nayan_wylde no don't do that hehe. It was example of extreme approach. Usually use catalog to separate environment + in enterprises to separate divisions like customer tower, marketing tower, finance tower etc

  • 11 kudos
6 More Replies
YuriS
by > New Contributor II
  • 324 Views
  • 3 replies
  • 1 kudos

Resolved! How to reduce data loss for Delta Lake on Azure when failing from primary to secondary regions?

Let’s say we have big data application where data loss is not an option.Having GZRS (geo-zone-redundant storage) redundancy we would achieve zero data loss if primary region is alive – writer is waiting for acks from two or more Azure availability zo...

  • 324 Views
  • 3 replies
  • 1 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 1 kudos

Databricks is working on improvements and new functionality related to that. For now, the only solution is a DEEP CLONE. You can run it more frequently or implement your own replication based on a change data feed. You could use delta sharing for tha...

  • 1 kudos
2 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 85 Views
  • 1 replies
  • 2 kudos

Databricks Free Edition Performance Test

How much time does it take to ingest two billion rows using the free databricks edition?https://www.databricks.com/blog/learn-experiment-and-build-databricks-free-edition

blogpost.png
  • 85 Views
  • 1 replies
  • 2 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 2 kudos

Thanks for sharing @Hubert-Dudek . Really cool idea to test performance of free edition

  • 2 kudos
VamsiDatabricks
by > New Contributor
  • 110 Views
  • 2 replies
  • 1 kudos

Delta comparison architecture using flatMapGroupsWithState in Structured Streaming

 I am designing structured streaming job in Azure data bricks(using Scala) which will consume messages from two event hubs, lets call them source and target.I would like your feedback on below flow, whether it is will survive the production load and ...

  • 110 Views
  • 2 replies
  • 1 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 1 kudos

It is hard to understand what the source is and what the target is. Some charts could be useful. Also, information on how long the state is kept. My solution usually is:- Use declarative lakeflow pipelines if possible (dlt) - if not, consider handlin...

  • 1 kudos
1 More Replies