- 89 Views
- 0 replies
- 0 kudos
Udf pyspaek
How to optimize in soark 3.0?
- 89 Views
- 0 replies
- 0 kudos
Forum Posts
- 94 Views
- 0 replies
- 0 kudos
Is anyone using Databricks for Advanced HR Analytcs?
We are starting out with Databricks and I would like to use the tools to build out Advanced Analytics for HR.
- 94 Views
- 0 replies
- 0 kudos
- 100 Views
- 0 replies
- 0 kudos
admin portal is not visible in my databricks account
admin portal is not visible in my databricks account Please assist me
- 100 Views
- 0 replies
- 0 kudos
- 262 Views
- 0 replies
- 0 kudos
S60 Eliminate SPN secrets - Connect Azure Databricks to ADLS Gen2 , Gen1 via custom AD token
Hi Team,In Azure Databricks, we currently use Service Principal when creating Mount Points to Azure storage ( ADLS Gen1, ADLS Gen 2 and Azure Blob Storage).As part of S360 action to eliminate SPN secrets, we were asked to move to SPN+certificate / MS...
- 262 Views
- 0 replies
- 0 kudos
- 171 Views
- 0 replies
- 0 kudos
Field mapping
What’s a good way to map ddiiferent datasets all to a standard set of variables. For example in table 1 there is an ‘user_number’ field. And table 2 has the same field but it’s labeled ‘user_id’. They are both the same, and I want to plug both into a...
- 171 Views
- 0 replies
- 0 kudos
- 93 Views
- 0 replies
- 0 kudos
Databricks AI summit 2024
Lot of new announcements. Love the world we are getting into.
- 93 Views
- 0 replies
- 0 kudos
- 141 Views
- 0 replies
- 0 kudos
Summit 2014
Lot of exciting announcements. Love the GPU’s being used for Photon
- 141 Views
- 0 replies
- 0 kudos
- 121 Views
- 1 replies
- 0 kudos
- 121 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
enjoying the conference and learning about open source unity
- 0 kudos
- 81 Views
- 0 replies
- 0 kudos
Data Analytics Training for Underserved Communities
Thoughts on setting up labs for IT training and Hackathons (coding, not infiltrating) in underserved communities?
- 81 Views
- 0 replies
- 0 kudos
- 124 Views
- 0 replies
- 0 kudos
Wrote an article about implementing real-time fraud detection with Databricks!
Hey ya'll!I've just started to dabble with Databricks recently and decided a fraud-detection pipeline would be a cool project to implement. Let me know what ya'll think about the article. Also would love more smaller scale project ideas I could work ...
- 124 Views
- 0 replies
- 0 kudos
- 450 Views
- 2 replies
- 1 kudos
Resolved! Regarding Certification renewal process
hi team,Anyone can guide me for certification renewal process?
- 450 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
@Yogic24 It's in certification FAQ. https://www.databricks.com/learn/certification/faq#certificationsTo recertify, you will need to take the full current live exam.
- 1 kudos
- 639 Views
- 2 replies
- 2 kudos
New to Databricks!
Enjoying the conference and learning a lot as a new user to Databricks!
- 639 Views
- 2 replies
- 2 kudos
- 241 Views
- 0 replies
- 0 kudos
Inconsistent behavior while loading pickle file
I have a pickle file "vectorizer.pkl" and I am currently facing an inconsistent behavior when trying to load that file. Sometimes it gets loaded successfully and sometimes I face an error. Here is how I am trying to load the file:from joblib import l...
- 241 Views
- 0 replies
- 0 kudos
- 753 Views
- 5 replies
- 8 kudos
Resolved! python multiprocessing hangs at map on one cluster but works fine on another
I have a simple python script which have been running fine on my cluster but recently the same script gets stuck at map. So I tried creating a new cluster with less resources and tried to run the same script over that and it ran just fine.Here are th...
- 753 Views
- 5 replies
- 8 kudos
Latest Reply
- 8 kudos
I agree with @raphaelblg. Most likely you're running out of memory. Multiprocessing or threadpools unfortunately do not benefit from extra workers as they only run on your driver node. This is very annoying and not a very known fact. Spark driver als...
- 8 kudos
- 629 Views
- 1 replies
- 1 kudos
Streaming Query
How to remove duplicates in streaming query on the basis of some id?
- 629 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
@nileshtiwaari Are you refering to Strucutred Streaming or DLT?In case of Structured Streaming: https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#streaming-deduplicationAbout DLT, here's a thread from a couple of months...
- 1 kudos
