cancel
Showing results for 
Search instead for 
Did you mean: 
Resources
cancel
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Knowledge Sharing Hub

Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practi...

33 Posts

Databricks TV

DatabricksTV provides valuable insights, tutorials, and best practices for leveraging Databricks in ...

7 Posts

Activity in Resources

DatabricksGuide
by Community Manager
  • 78 Views
  • 0 replies
  • 0 kudos

Getting Started with Databricks - Building a Forecasting Model on Databricks

Building a Forecasting Model on Databricks: A Step-by-Step Guide This guide offers a detailed, step-by-step approach for building a forecasting model on Databricks. By leveraging the power of Databricks, you will unlock new potentials in your data w...

DatabricksGuide_0-1715968895219.gif DatabricksGuide_0-1715967500817.gif DatabricksGuide_1-1715967500776.gif DatabricksGuide_2-1715968440657.png
  • 78 Views
  • 0 replies
  • 0 kudos
Sujitha
by Community Manager
  • 26104 Views
  • 52 replies
  • 111 kudos

🔔 Attention Databricks Academy Users: SSO Implementation Incoming! Secure Your Account Today!

On May 18th a new SSO user experience will launch in Customer and Partner Academy. In order to secure your account, please see the instructions below on how to update and verify that your data is correct and complete. Please ensure that you have two ...

Sujitha_0-1715596765217.png Sujitha_1-1715596782895.png Screenshot 2024-05-06 at 11.56.58 AM.png
  • 26104 Views
  • 52 replies
  • 111 kudos
Latest Reply
DavidStruzik
New Contributor
  • 111 kudos

I'm a consultant. I work with multiple companies, but I don't have a main one. I'm not comfortable setting any of my emails in clients' domains on my profile. Is it OK for me to keep using my personal one?

  • 111 kudos
51 More Replies
Sujitha
by Community Manager
  • 491 Views
  • 0 replies
  • 0 kudos

Building DBRX-class Custom LLMs with Mosaic AI Training

We recently introduced DBRX: an open, state-of-the-art, general-purpose LLM. DBRX was trained, fine-tuned, and evaluated using Mosaic AI Training, scaling training to 3072 NVIDIA H100s and processing more than 12 trillion tokens in the process. Train...

Screenshot 2024-05-17 at 6.33.11 PM.png
  • 491 Views
  • 0 replies
  • 0 kudos
Sujitha
by Community Manager
  • 751 Views
  • 0 replies
  • 1 kudos

Accurate, Safe and Governed: How to Move GenAI from POC to Production

In the realm of AI, achieving accuracy is paramount. The publication delves into techniques for refining models to ensure they reliably deliver precise outcomes in real-world scenarios. It covers methodologies such as continuous monitoring, data augm...

Screenshot 2024-05-17 at 6.25.11 PM.png
  • 751 Views
  • 0 replies
  • 1 kudos
DatabricksGuide
by Community Manager
  • 596 Views
  • 0 replies
  • 0 kudos

Getting Started with Databricks - Ingest to Analytics & BI

Getting Started with Databricks - From Ingest to Analytics & BIIntroduction to DatabricksAnalytics & BI on DatabricksIngest Setup Steps [20 minutes]Step 0: Check your required prerequisitesStep 1: Access and start your warehouseStep 2: Connect your w...

DatabricksGuide_0-1714140949313.png DatabricksGuide_1-1714141186202.gif DatabricksGuide_4-1714141571707.gif DatabricksGuide_5-1714141715140.png
  • 596 Views
  • 0 replies
  • 0 kudos
Yassine_bens
by New Contributor
  • 187 Views
  • 1 replies
  • 0 kudos

How to convert txt files to delta tables

Hello members of Databricks's comunity,I am currently working on a project where we collect data from machines, that data is in .txt format. The data is currently in an Azure container, I need to clean the files and convert them to delta tables, how ...

  • 187 Views
  • 1 replies
  • 0 kudos
Latest Reply
feiyun0112
Contributor III
  • 0 kudos

https://docs.databricks.com/en/ingestion/add-data/upload-data.html 

  • 0 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 94 Views
  • 0 replies
  • 0 kudos

RocksDB for storing state stream

Now, you can keep the state of stateful streaming in RocksDB. For example, retrieving keys from memory to check for duplicate records inside the watermark is now faster. #databricks

state2.png
  • 94 Views
  • 0 replies
  • 0 kudos
data_turtle
by New Contributor
  • 289 Views
  • 1 replies
  • 0 kudos

Understand why your jobs' performances are changing over time

Hi Folks -We released a new metrics view for databricks jobs in Gradient, which helps track and plot the metrics below over time to help engineers understand what's going on with their jobs over time.Job cost (DBU + Cloud fees)Job RuntimeNumber of co...

  • 289 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @data_turtle, That sounds like a valuable addition to Gradient! The new metrics view for Databricks jobs will surely help engineers gain better insights into their job performance and resource usage over time. Being able to track metrics such as j...

  • 0 kudos
data_turtle
by New Contributor
  • 228 Views
  • 1 replies
  • 0 kudos

Feedback request for Gradient, a tool to help optimize and monitor jobs automatically

Hi Everyone,We built Gradient, a tool to automatically optimize and monitor Databricks jobs to hit your business objectives of cost or runtime.Gradient works by applying a reinforcement ML model to automatically learn and custom tune your jobs cluste...

  • 228 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @data_turtle, Gradient sounds like an interesting tool for optimizing and monitoring Databricks jobs. The application of reinforcement ML to automatically tune job clusters is a compelling feature, especially for reducing costs and meeting SLAs. T...

  • 0 kudos
Hp3
by New Contributor
  • 502 Views
  • 1 replies
  • 0 kudos

Hiring Databricks Data Architect roles

Hi,I am a recruiter and I am looking for places to post some data bricks I have coming out. I have several fully remote, high-level data databricks, architect roles. Of course I will post to LinkedIn, but I was just curious if there are any other pla...

  • 502 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Hp3, As of now, you can post your Databricks roles in the Databricks Community's Knowledge Sharing Hub. In the future, we are planning to have a special node for such posts, so keep an eye out for that. If you need any further assistance or have ...

  • 0 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 165 Views
  • 0 replies
  • 1 kudos

State of stateful streaming

For stateful streaming in #databricks, you can now easily read what is in the state.

state.png
  • 165 Views
  • 0 replies
  • 1 kudos
legobricks
by New Contributor II
  • 336 Views
  • 4 replies
  • 0 kudos

Unable to mount GCS bucket with underscores in the name

I have two buckets with the same configurations and labels.One is named my-bucket and the other is my_bucket. I am able to mount my-bucket but get an opaque error message when trying to mount my_bucket. Is this known/expected behavior? Are underscore...

  • 336 Views
  • 4 replies
  • 0 kudos
Latest Reply
NandiniN
Valued Contributor III
  • 0 kudos

Hi @legobricks , Curious on the error that you are getting. However, for GCS - https://cloud.google.com/storage/docs/buckets#naming I do see underscores are allowed but there is also a note below: You can use a bucket name in a DNS record as part of ...

  • 0 kudos
3 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 1021 Views
  • 2 replies
  • 1 kudos

Debug stream

One of the easiest ways to debug a stream is to stream it to memory and query it with SQL #databricks

debug.png
  • 1021 Views
  • 2 replies
  • 1 kudos
Latest Reply
DB_Paul
Community Manager
  • 1 kudos

Cool @Hubert-Dudek !

  • 1 kudos
1 More Replies
Sujitha
by Community Manager
  • 6330 Views
  • 1 replies
  • 3 kudos

Announcing the General Availability of Databricks Asset Bundles

Get started with Databricks Asset Bundles (DABs) today! Simplify project management, versioning, testing, and deployment for your data and AI projects on the Databricks Platform. Embrace software engineering best practices with DABs—source control, c...

Screenshot 2024-04-30 at 9.25.26 PM.png
  • 6330 Views
  • 1 replies
  • 3 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 3 kudos

This was most awaited features.I am already using Bundle for workflow CICD.

  • 3 kudos
MichTalebzadeh
by Contributor
  • 351 Views
  • 0 replies
  • 0 kudos

Financial Crime detection with the help of Apache Spark, Data Mesh and Data Lake

For those interested in Data Mesh and Data Lakes for FinCrime detection:Data mesh is a relatively new architectural concept for data management that emphasizes domain-driven data ownership and self-service data availability. It promotes the decentral...

Knowledge Sharing Hub
data lakes
Data Mesh
financial crime
spark
  • 351 Views
  • 0 replies
  • 0 kudos
Top Kudoed Authors