Databricks Community

Summit 2024

The keynote has been my favorite sesión so dar! Really excited to keep learning!

Hello from Summit!!!

Hi folks! Who’s heading to the party later!?

Udf pyspaek

How to optimize in soark 3.0?

admin portal is not visible in my databricks account

admin portal is not visible in my databricks account  Please assist me

S60 Eliminate SPN secrets - Connect Azure Databricks to ADLS Gen2 , Gen1 via custom AD token

Hi Team,In Azure Databricks, we currently use Service Principal when creating Mount Points to Azure storage ( ADLS Gen1, ADLS Gen 2 and Azure Blob Storage).As part of S360 action to eliminate SPN secrets, we were asked to move to SPN+certificate / MS...

Field mapping

What’s a good way to map ddiiferent datasets all to a standard set of variables. For example in table 1 there is an ‘user_number’ field. And table 2 has the same field but it’s labeled ‘user_id’. They are both the same, and I want to plug both into a...

Databricks AI summit 2024

Lot of new announcements. Love the world we are getting into.

Summit 2014

Lot of exciting announcements. Love the GPU’s being used for Photon

At the conference!

Data Analytics Training for Underserved Communities

Thoughts on setting up labs for IT training and Hackathons (coding, not infiltrating) in underserved communities? 

Wrote an article about implementing real-time fraud detection with Databricks!

Hey ya'll!I've just started to dabble with Databricks recently and decided a fraud-detection pipeline would be a cool project to implement. Let me know what ya'll think about the article. Also would love more smaller scale project ideas I could work ...

New to Databricks!

Enjoying the conference and learning a lot as a new user to Databricks!

Inconsistent behavior while loading pickle file

I have a pickle file "vectorizer.pkl" and I am currently facing an inconsistent behavior when trying to load that file. Sometimes it gets loaded successfully and sometimes I face an error. Here is how I am trying to load the file:from joblib import l...

python multiprocessing hangs at map on one cluster but works fine on another

I have a simple python script which have been running fine on my cluster but recently the same script gets stuck at map. So I tried creating a new cluster with less resources and tried to run the same script over that and it ran just fine.Here are th...