cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Knowledge Sharing Hub
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
SumitSingh
by • Contributor
  • 3595 Views
  • 7 replies
  • 11 kudos

From Associate to Professional: My Learning Plan to ace all Databricks Data Engineer Certifications

In today’s data-driven world, the role of a data engineer is critical in designing and maintaining the infrastructure that allows for the efficient collection, storage, and analysis of large volumes of data. Databricks certifications holds significan...

SumitSingh_0-1721402402230.png SumitSingh_1-1721402448677.png SumitSingh_2-1721402469214.png
Knowledge Sharing Hub
Reply
  • 3595 Views
  • 7 replies
  • 11 kudos
View Replies
Latest Reply
sandeepmankikar
New Contributor III
  • 11 kudos

As an additional tip for those working towards both the Associate and Professional certifications, I recommend avoiding a long gap between the two exams to maintain your momentum. If possible, try to schedule them back-to-back with just a few days in...

  • 11 kudos
6 More Replies
Yassine_bens
by • New Contributor
  • 1275 Views
  • 1 replies
  • 0 kudos

How to convert txt files to delta tables

Hello members of Databricks's comunity,I am currently working on a project where we collect data from machines, that data is in .txt format. The data is currently in an Azure container, I need to clean the files and convert them to delta tables, how ...

Knowledge Sharing Hub
Reply
  • 1275 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
feiyun0112
Honored Contributor
  • 0 kudos

https://docs.databricks.com/en/ingestion/add-data/upload-data.html 

  • 0 kudos
Hubert-Dudek
by • Esteemed Contributor III
  • 635 Views
  • 0 replies
  • 0 kudos

RocksDB for storing state stream

Now, you can keep the state of stateful streaming in RocksDB. For example, retrieving keys from memory to check for duplicate records inside the watermark is now faster. #databricks

state2.png
Knowledge Sharing Hub
Reply
  • 635 Views
  • 0 replies
  • 0 kudos
legobricks
by • New Contributor II
  • 1690 Views
  • 4 replies
  • 0 kudos

Unable to mount GCS bucket with underscores in the name

I have two buckets with the same configurations and labels.One is named my-bucket and the other is my_bucket. I am able to mount my-bucket but get an opaque error message when trying to mount my_bucket. Is this known/expected behavior? Are underscore...

Knowledge Sharing Hub
Reply
  • 1690 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

Hi @legobricks , Curious on the error that you are getting. However, for GCS - https://cloud.google.com/storage/docs/buckets#naming I do see underscores are allowed but there is also a note below: You can use a bucket name in a DNS record as part of ...

  • 0 kudos
3 More Replies
MichTalebzadeh
by • Valued Contributor
  • 1289 Views
  • 0 replies
  • 0 kudos

Financial Crime detection with the help of Apache Spark, Data Mesh and Data Lake

For those interested in Data Mesh and Data Lakes for FinCrime detection:Data mesh is a relatively new architectural concept for data management that emphasizes domain-driven data ownership and self-service data availability. It promotes the decentral...

Knowledge Sharing Hub
data lakes
Data Mesh
financial crime
spark
Reply
  • 1289 Views
  • 0 replies
  • 0 kudos
Hp3
by • New Contributor
  • 1202 Views
  • 0 replies
  • 0 kudos

Hiring Databricks Data Architect roles

Hi,I am a recruiter and I am looking for places to post some data bricks I have coming out. I have several fully remote, high-level data databricks, architect roles. Of course I will post to LinkedIn, but I was just curious if there are any other pla...

Knowledge Sharing Hub
Reply
  • 1202 Views
  • 0 replies
  • 0 kudos
Ajay-Pandey
by • Esteemed Contributor III
  • 1889 Views
  • 1 replies
  • 2 kudos

Databricks Notebook Workflow

Exciting news for Databricks users! The ability to view job details within the notebook workflow section, particularly for multithreaded jobs, is available now. Instead of manually inspecting each job for failures, this feature enables us to swiftly ...

AjayPandey_1-1713513883239.jpeg
Knowledge Sharing Hub
Databricks
databrickscommunity
Knowledge Sharing Hub
newfeatures
workflow
Reply
  • 1889 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Lakshay
Databricks Employee
  • 2 kudos

Thank you for sharing

  • 2 kudos
Sujitha
by • Databricks Employee
  • 2321 Views
  • 3 replies
  • 5 kudos

🌟 Welcome to the Knowledge Sharing Hub! 🌟

Are you passionate about sharing your discoveries and insights with the world? Look no further! Our Knowledge Sharing Hub is the perfect space for you to showcase your research and connect with like-minded individuals across the globe. Here's why you...

Knowledge Sharing Hub
Reply
  • 2321 Views
  • 3 replies
  • 5 kudos
View Replies
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 5 kudos

Great place to knowledge sharing and collaboration. 

  • 5 kudos
2 More Replies
Hubert-Dudek
by • Esteemed Contributor III
  • 1009 Views
  • 1 replies
  • 1 kudos

Spot databricks VMs - eviction rates

Before using Spot machines in #databricks, it's a good idea to check their eviction rates in your region. Azure Resource Graph Explorer and that simple query will help. SpotResources | where type =~ 'microsoft.compute/skuspotevictionrate/location' ...

spots.png
Knowledge Sharing Hub
Reply
  • 1009 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Thank you for sharing this @Hubert-Dudek 

  • 1 kudos
mm_soc
by • New Contributor II
  • 1667 Views
  • 0 replies
  • 1 kudos

Installing CrowdStrike Falcon Sensor on Databricks Workers

Greetings,Does anyone here have experience deploying the CrowdStrike Falcon sensor on Databricks worker instances? For context, the cluster is deployed in AWS and we use a Databricks Ubuntu 20.04 AMI. Databricks allows adding a bootstrap/startup scri...

Knowledge Sharing Hub
Reply
  • 1667 Views
  • 0 replies
  • 1 kudos
ThomazRossito
by • Contributor
  • 2210 Views
  • 0 replies
  • 1 kudos

Post: Lakehouse Federation - Databricks

Lakehouse Federation - Databricks In the world of data, innovation is constant. And the most recent revolution comes with Lakehouse Federation, a fusion between data lakes and data warehouses, taking data manipulation to a new level. This advancement...

Knowledge Sharing Hub
data engineer
Lakehouse
SQL Analytics
Reply
  • 2210 Views
  • 0 replies
  • 1 kudos
data_turtle
by • New Contributor
  • 857 Views
  • 0 replies
  • 0 kudos

Feedback request for Gradient, a tool to help optimize and monitor jobs automatically

Hi Everyone,We built Gradient, a tool to automatically optimize and monitor Databricks jobs to hit your business objectives of cost or runtime.Gradient works by applying a reinforcement ML model to automatically learn and custom tune your jobs cluste...

Knowledge Sharing Hub
Reply
  • 857 Views
  • 0 replies
  • 0 kudos
Danny_Lee
by • Valued Contributor
  • 2613 Views
  • 1 replies
  • 1 kudos

Jonathan Frankel at Sigma talk

Hi @Sujitha Just to follow up on your suggestion to share my takeaways from Jonathan Frankel's talk at Sigma in NYC.  The key ideas I came away with is:Building in-house custom models is more than just possible, there's advantages to itThere's danger...

Knowledge Sharing Hub
AI
ML
Reply
  • 2613 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Sujitha
Databricks Employee
  • 1 kudos

@Danny_Lee This is super insightful! Really appreciate your time to share your key takeaways with us. 

  • 1 kudos

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels
Top