cancel
Showing results for 
Search instead for 
Did you mean: 
Community Articles
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Hp3
by New Contributor
  • 1302 Views
  • 0 replies
  • 0 kudos

Hiring Databricks Data Architect roles

Hi,I am a recruiter and I am looking for places to post some data bricks I have coming out. I have several fully remote, high-level data databricks, architect roles. Of course I will post to LinkedIn, but I was just curious if there are any other pla...

  • 1302 Views
  • 0 replies
  • 0 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 2199 Views
  • 1 replies
  • 2 kudos

Databricks Notebook Workflow

Exciting news for Databricks users! The ability to view job details within the notebook workflow section, particularly for multithreaded jobs, is available now. Instead of manually inspecting each job for failures, this feature enables us to swiftly ...

AjayPandey_1-1713513883239.jpeg
Community Articles
Databricks
databrickscommunity
Knowledge Sharing Hub
newfeatures
workflow
  • 2199 Views
  • 1 replies
  • 2 kudos
Latest Reply
Lakshay
Databricks Employee
  • 2 kudos

Thank you for sharing

  • 2 kudos
Sujitha
by Databricks Employee
  • 2628 Views
  • 3 replies
  • 5 kudos

🌟 Welcome to the Knowledge Sharing Hub! 🌟

Are you passionate about sharing your discoveries and insights with the world? Look no further! Our Knowledge Sharing Hub is the perfect space for you to showcase your research and connect with like-minded individuals across the globe. Here's why you...

  • 2628 Views
  • 3 replies
  • 5 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 5 kudos

Great place to knowledge sharing and collaboration. 

  • 5 kudos
2 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 1266 Views
  • 1 replies
  • 1 kudos

Spot databricks VMs - eviction rates

Before using Spot machines in #databricks, it's a good idea to check their eviction rates in your region. Azure Resource Graph Explorer and that simple query will help. SpotResources | where type =~ 'microsoft.compute/skuspotevictionrate/location' ...

spots.png
  • 1266 Views
  • 1 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Thank you for sharing this @Hubert-Dudek 

  • 1 kudos
mm_soc
by New Contributor II
  • 2156 Views
  • 0 replies
  • 1 kudos

Installing CrowdStrike Falcon Sensor on Databricks Workers

Greetings,Does anyone here have experience deploying the CrowdStrike Falcon sensor on Databricks worker instances? For context, the cluster is deployed in AWS and we use a Databricks Ubuntu 20.04 AMI. Databricks allows adding a bootstrap/startup scri...

  • 2156 Views
  • 0 replies
  • 1 kudos
data_turtle
by New Contributor
  • 1001 Views
  • 0 replies
  • 0 kudos

Feedback request for Gradient, a tool to help optimize and monitor jobs automatically

Hi Everyone,We built Gradient, a tool to automatically optimize and monitor Databricks jobs to hit your business objectives of cost or runtime.Gradient works by applying a reinforcement ML model to automatically learn and custom tune your jobs cluste...

  • 1001 Views
  • 0 replies
  • 0 kudos
Danny_Lee
by Valued Contributor
  • 3035 Views
  • 1 replies
  • 1 kudos

Jonathan Frankel at Sigma talk

Hi @Sujitha Just to follow up on your suggestion to share my takeaways from Jonathan Frankel's talk at Sigma in NYC.  The key ideas I came away with is:Building in-house custom models is more than just possible, there's advantages to itThere's danger...

  • 3035 Views
  • 1 replies
  • 1 kudos
Latest Reply
Sujitha
Databricks Employee
  • 1 kudos

@Danny_Lee This is super insightful! Really appreciate your time to share your key takeaways with us. 

  • 1 kudos
Danny_Lee
by Valued Contributor
  • 1655 Views
  • 0 replies
  • 1 kudos

Databricks AI Security Framework

Today Databricks announced the release of the Databricks AI Security Framework (LinkedIn Post)You can download the paper (PDF) from blog post. Anyone else download this and have thoughts?   My first thought is its a great start and has an excellent G...

  • 1655 Views
  • 0 replies
  • 1 kudos
avrm91
by Contributor
  • 1557 Views
  • 0 replies
  • 0 kudos

GCP - Initial External Location to GCP Bucket is wrong

When creating a new Workspace in GCP the default GCP External Location is wrong.Its easily fixed by Catalog (on the left) > External Data (on the bottom) > External Locations > choose the connection and edit the URL by deleting the second BucketId af...

avrm91_1-1711447524318.png avrm91_0-1711447511611.png
  • 1557 Views
  • 0 replies
  • 0 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 1790 Views
  • 0 replies
  • 0 kudos

Predictive optimization log

After you enable predictive optimization, it is good to look at the system table and see what is going on with your tables #databricks

pl.png
  • 1790 Views
  • 0 replies
  • 0 kudos
MichTalebzadeh
by Valued Contributor
  • 1760 Views
  • 0 replies
  • 0 kudos

Feature article: Leveraging Generative AI with Apache Spark: Transforming Data Engineering

I created this article in Linkedlin to allow both this community and Apache Spark user community to have access to it.It is particularly useful for data engineers who want to have a basic understanding of what  Generative AI with Spark can do.Leverag...

Community Articles
Generative AI
spark
  • 1760 Views
  • 0 replies
  • 0 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 3892 Views
  • 1 replies
  • 3 kudos

DBR 15.0 beta

databricks runtime 15 is out there!Some breaking changes. More info here https://docs.databricks.com/en/release-notes/runtime/15.0.html

15.png
  • 3892 Views
  • 1 replies
  • 3 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 3 kudos

Thanks for sharing this information @Hubert-Dudek!!!

  • 3 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 3143 Views
  • 1 replies
  • 1 kudos

Notebook IDE

This is an excellent step for #databricks notebooks. Integrated debugger and CLI in notebook terminal is a big step towards a fully functional cloud IDE.

ide.png
  • 3143 Views
  • 1 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Thank you for sharing this @Hubert-Dudek!!!

  • 1 kudos
MichTalebzadeh
by Valued Contributor
  • 5835 Views
  • 2 replies
  • 0 kudos

Build a machine learning model to detect fraudulent transactions using PySpark's MLlib library

IntroductionFinancial fraud is a significant concern for businesses and consumers alike. I have written about this concern a few times in Linkedlin articles. Machine learning offers powerful tools to combat this issue by automatically identifying sus...

Community Articles
Financial Fraud
PySpark MLlib
spark
  • 5835 Views
  • 2 replies
  • 0 kudos
Latest Reply
deborah621
New Contributor II
  • 0 kudos

Looking to build a machine learning model for detecting fraudulent transactions using PySpark’s MLlib. Generate synthetic transaction data. Provides a dataset for model training without using sensitive real-world data. Enables the creation of diverse...

  • 0 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels