- 2201 Views
- 1 replies
- 2 kudos
Databricks Table Protection Features
This article provides an overview of key Databricks features and best practices that protect Gold tables from accidental deletion. It also covers the implications if both the Gold and Landing layers are deleted without active retention or backup. Cor...
- 2201 Views
- 1 replies
- 2 kudos
- 2 kudos
Thanks for sharing this. Time Travel is applicable all tables in Databricks NOT restricted to gold.
- 2 kudos
- 657 Views
- 0 replies
- 2 kudos
5 Reasons You Should Be Using LakeFlow Jobs as Your Default Orchestrator
I recently saw a business case in which an external orchestrator accounted for nearly 30% of their total Databricks job costs. That's when it hit me: we're often paying a premium for complexity we don't need. Besides FinOps, I tried to gather all the...
- 657 Views
- 0 replies
- 2 kudos
- 1127 Views
- 0 replies
- 2 kudos
Databricks Asset Bundles Direct Mode
There is a new direct mode in Databricks Asset Bundles: the main difference is that there is no Terraform anymore, and a simple state in JSON. It offers a few significant benefits: - No requirement to download Terraform and terraform-provider-databr...
- 1127 Views
- 0 replies
- 2 kudos
- 2216 Views
- 1 replies
- 1 kudos
Version 1.1: Data Isolation and Governance within PySpark DataFrame's
See the comments below for a runnable notebookThroughout my career I have worked at several companies that handle sensitive data; including PII, PHI, EMR, HIPPA, Class I/II/III FOMC - Internal (FR). One entity I worked at even required a Department O...
- 2216 Views
- 1 replies
- 1 kudos
- 1 kudos
We just failed a HIPAA audit, they asked why our pipelines had patients names in the if the pipeline didnt need that info, they recommended it to be encrypted. We thought S3 encryption was good enough.We implemented row level encryption by extending ...
- 1 kudos
- 3880 Views
- 1 replies
- 4 kudos
Designing Reliable Stream–Stream Joins with Watermarks in Databricks
Stream–stream joins are one of the most powerful features in Databricks Structured Streaming – and also one of the easiest to misconfigure. As soon as you move from simple append-only pipelines to real-time correlations across multiple streams (order...
- 3880 Views
- 1 replies
- 4 kudos
- 12263 Views
- 4 replies
- 2 kudos
Optimizing Costs in Databricks by Dynamically Choosing Cluster Sizes
Databricks is a popular unified data analytics platform known for its powerful data processing capabilities and seamless integration with Apache Spark. However, managing and optimizing costs in Databricks can be challenging, especially when it comes ...
- 12263 Views
- 4 replies
- 2 kudos
- 2 kudos
@Second Reply You’re right just printing out selected_pool isn’t enough to actually leverage dynamic cluster sizing at runtime. In practice, the value of selected_pool would feed directly into your Databricks cluster creation API or workflow automati...
- 2 kudos
- 2024 Views
- 2 replies
- 6 kudos
Meta Ads is now a native data source in Databricks Lakeflow Connect
Meta Ads is now a native data source in DatabricksDatabricks just announced a Meta Ads connector (Beta) powered by Lakeflow Connect, making it easy to ingest advertising data directly into Databricks—no custom APIs, no CSV exports, no brittle scripts...
- 2024 Views
- 2 replies
- 6 kudos
- 6 kudos
Have to test it. Confluence Lakeflow Connect is amazing, so I suspect that one also!
- 6 kudos
- 837 Views
- 1 replies
- 2 kudos
Flexible Node Types
Recently, it has not only become difficult to get a quota in some regions, but even if you have one, it doesn't mean that there are available VMs. Even if you have a quota, you may need to move your bundles to a different subscription when different ...
- 837 Views
- 1 replies
- 2 kudos
- 2 kudos
Thanks for sharing, @Hubert-Dudek. This is a common challenge users face, and flexible node types can significantly improve compute launch reliability.
- 2 kudos
- 646 Views
- 0 replies
- 1 kudos
DABs: Referencing Your Resources
From hardcoded IDs, through lookups, to finally referencing resources. I think almost everyone, including me, wants to go through such a journey with Databricks Asset Bundles. #databricks In the article below, I am looking at how to reference a resou...
- 646 Views
- 0 replies
- 1 kudos
- 689 Views
- 0 replies
- 1 kudos
Confluence Lakeflow Connector
Incrementally upload data from Confluence. I remember there were a few times in my life when I spent weeks on it. Now, it is incredible how simple it is to implement it with Lakeflow Connect. Additionally, I love DABS's first approach for connectors,...
- 689 Views
- 0 replies
- 1 kudos
- 2696 Views
- 4 replies
- 4 kudos
From Learning to Enablement: My 2025 Databricks Journey
Celebrating platform capabilities, community impact, and responsible adoptionIn 2025, my Databricks journey evolved from mastering features to empowering outcomes.What became clear this year is that Databricks isn’t just a powerful platform — it’s a ...
- 2696 Views
- 4 replies
- 4 kudos
- 4 kudos
Nice write-up @Lakshmipriya_Na , I really like this framing. The “Builder” vs “Strategist” distinction maps almost perfectly to how Databricks shows up in the real world. You can move fast and iterate in notebooks, but the same Lakehouse naturally n...
- 4 kudos
- 636 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #23
Our calendar is coming to an end. One of the most significant innovations of last year is Agent Bricks. We received a few ready-made solutions for deploying agents. As the Agents ecosystem becomes more complex, one of my favourites is the Multi-Agent...
- 636 Views
- 0 replies
- 1 kudos
- 590 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #22
During the last two weeks, five new Lakeflow Connect connectors were announced. It allows incremental ingestion of the data in an easy way. In the coming weeks, there will be more announcements about Lakeflow Connect, and we can expect Databricks to ...
- 590 Views
- 0 replies
- 1 kudos
- 667 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #21
Your stream can have a state, and now, with TransformWithStateInPandas, it’s easy to manage - you can handle things like initial state, deduplication, recovery, etc., with the 2025 improvements.
- 667 Views
- 0 replies
- 1 kudos
- 820 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #20
As Unity Catalog becomes an enterprise catalog, bring-your-own lineage is one of my favorite features.
- 820 Views
- 0 replies
- 1 kudos
-
Access Data
1 -
Access Delta Tables
1 -
ADB
1 -
ADF Linked Service
1 -
ADF Pipeline
1 -
Advanced Data Engineering
7 -
Agent Architecture
2 -
Agent Bricks
2 -
Agent Skills
2 -
Agentic AI
8 -
agents
1 -
AI
5 -
AI Agents
12 -
AI BI Dashboards
1 -
ai gateway
2 -
AI Governance
1 -
AI Readiness
1 -
ai_query
1 -
AIAgents
1 -
AIBI
3 -
AIBI Dashboards
1 -
AIGovernance
1 -
Alerts
1 -
Analytics
2 -
Analytics Engineering
1 -
Apache Iceberg
2 -
Apache spark
3 -
Apache Spark 3.0
2 -
ApacheSpark
1 -
Aqe
1 -
Architecture
7 -
asset bundle
1 -
Asset Bundles
4 -
Associate Certification
2 -
Audit
1 -
Audit Logs
1 -
AUTO CDC
1 -
Auto-loader
1 -
Autoloader
1 -
Automation
3 -
Automl
1 -
AutoML Experiment
1 -
AWS
1 -
AWSDatabricksCluster
2 -
Azure
5 -
Azure databricks
6 -
Azure Databricks Delta Table
3 -
Azure Databricks Job
2 -
Azure Delta Lake
4 -
Azure devops integration
2 -
Azure Unity Catalog
3 -
AzureDatabricks
2 -
Banking
1 -
best practices
3 -
BI Integrations
1 -
Big data
1 -
Billing and Cost Management
2 -
Blog
1 -
BroadcastJoin
1 -
Bronze Layer
1 -
Bronze Table
1 -
Build a Databricks App on Free Edition with a Genie Agent at its core.
1 -
Caching
2 -
Cash Flow
1 -
CDC
5 -
CDF
1 -
Certification
1 -
Certification Badge
1 -
Certification Exam
1 -
Change data capture
1 -
CICD
5 -
CICDForDatabricksWorkflows
2 -
Cluster
1 -
Cluster Policies
1 -
Cluster Pools
1 -
Collect
1 -
Community Champion
1 -
Community Contest
1 -
Community Event
1 -
CommunityArticle
3 -
Concurrency
1 -
Cost Optimization Effort
2 -
CostOptimization
3 -
Creative Thinking
1 -
Credential Vending
1 -
Credit Risk
1 -
custom compute policy
1 -
Customer Record
1 -
CustomLibrary
1 -
DABs
3 -
DAIS 0206
3 -
DAIS 2026
2 -
Dashboards
2 -
Data
1 -
Data Agents
1 -
Data Analysis with Databricks
1 -
Data Architecture
2 -
Data Driven AI Roadmap
1 -
Data Engineering
29 -
Data Governance
10 -
Data Ingestion
2 -
Data Ingestion & connectivity
1 -
data layout
1 -
Data Mesh
1 -
data optimization
1 -
Data Processing
1 -
Data Quality
7 -
Data Quality Checks
2 -
Data warehouse
1 -
Data Warehousing
1 -
Databricks
6 -
Databricks App
2 -
Databricks Apps
17 -
Databricks Assistant
2 -
Databricks Blog
1 -
Databricks Certified
1 -
Databricks Clusters
1 -
Databricks Community
1 -
Databricks Dashboard
2 -
Databricks Delta Table
3 -
Databricks Demo Center
1 -
Databricks Free Edition
2 -
Databricks genAI associate
1 -
databricks genie
4 -
Databricks Job
2 -
Databricks Lakeflow
4 -
Databricks Lakehouse
4 -
Databricks Migration
4 -
Databricks Mlflow
1 -
Databricks News
1 -
Databricks Notebooks
1 -
Databricks Partner
1 -
Databricks Platform
1 -
Databricks Pyspark
3 -
Databricks Serverless
4 -
Databricks SQL
3 -
Databricks Support
1 -
Databricks Training
1 -
Databricks Unity Catalog
4 -
Databricks Workflows
3 -
DatabricksAutomation
2 -
DatabricksML
1 -
DatabricksOptimization
2 -
DataEngineering
2 -
DataGovernance
1 -
DateChange
1 -
DBR Versions
1 -
Dbt
1 -
Declartive Pipelines
2 -
DeepLearning
1 -
Deletion Vectors
1 -
Delta Lake
25 -
Delta Lake Files
1 -
Delta Live Table
4 -
Delta Live Tables
1 -
Delta Tables
1 -
Delta Time Travel
1 -
Delta-lake
1 -
DeltaLake
2 -
Developer
1 -
Development
1 -
DevOps
2 -
DimensionTables
1 -
DLT
3 -
DLT Pipeline
2 -
DLT Pipelines
3 -
DLT-Meta
1 -
Dns
1 -
DQX
3 -
Drift
1 -
Dynamic
1 -
Dynamic Partition
1 -
Engineering With Databricks
1 -
ETL
1 -
ETL Pipelines
2 -
fastapi
1 -
Feature Store
1 -
Financial Services
1 -
Fine Grained Access
1 -
FinOps
1 -
Forecasting
1 -
Fraud Detection
1 -
Free Databricks
3 -
Free Edition
1 -
GenAI
3 -
GenAI agent
2 -
GenAI and LLMs
5 -
GenAIGeneration AI
2 -
Generation AI
1 -
Generative AI
12 -
Generative AI Engineer
1 -
genie
19 -
Genie Agent
2 -
Genie Agents
12 -
Genie Code
1 -
Genie Ontology
1 -
Genie-Powered App Challenge - 2026
3 -
GenieCode
1 -
GenieOntology
2 -
Git
1 -
Github
1 -
Github actions
1 -
Gold Layer
1 -
GoldLayer
1 -
Google Bigquery
1 -
Google cloud
1 -
Governance
5 -
Governed Tag
1 -
Graph
1 -
Graphframes
1 -
hackathon
2 -
Hive metastore
1 -
Hubert Dudek
42 -
Hybrid Lakehouse
1 -
Incremental Load
1 -
infrastructure as code
1 -
Ingestion
1 -
Insurance · Reinsurance · Banking · Telecom · Retail · Agriculture — Per-Object API Calls to Per-Cell Scoring with H3 and Native Spatial SQL
1 -
Interoperability
1 -
IoT
1 -
Kafka streaming
3 -
Keyvault
1 -
Lakebase
6 -
Lakeflow
4 -
Lakeflow Declarative Pipelines
2 -
Lakeflow Pipelines
3 -
LakeflowDesigner
2 -
LakeflowJobs
1 -
Lakehouse
5 -
Lakehouse federation
1 -
Lakehouse Migration
2 -
Langchain
1 -
LangGraph
2 -
Lazy Evaluation
1 -
Learning
2 -
Library Installation
1 -
LightGBM
1 -
Lineage
2 -
Liquid Clustering
1 -
LiquidClustering
2 -
Live Tables CDC
3 -
Llama
1 -
LLM
1 -
LLM Evaluation
1 -
LLM-as-a-Judge
1 -
LLMs
1 -
Machine Learning
3 -
Materialized Views
1 -
mcp
5 -
Medallion Architecture
9 -
MERGE
2 -
MERGE Performance
3 -
Metadata
2 -
Metric Views
4 -
Migration
1 -
Migrations
1 -
ML Model
1 -
MlFlow
5 -
MLOps
1 -
Model Governance
1 -
Model Serving
2 -
mosic ai search
1 -
MSExcel
3 -
Multi-Table Transactions
1 -
Multiagent
4 -
Networking
2 -
New Features
1 -
NotMvpArticle
1 -
Ontology
1 -
Operations Analytics
2 -
Optimization
6 -
Optimize Command
1 -
Partitioning
4 -
Partner
2 -
Performance
3 -
Performance Tuning
5 -
Photon Engine
1 -
PII
1 -
Power BI
1 -
Power BI XLMA EndPoint
1 -
Powerbi
1 -
PredictiveOptimization
2 -
Private Link
1 -
Prompt Tuning
1 -
Pyspark
8 -
Pyspark Code
1 -
Pyspark Databricks
1 -
Pytest
1 -
Python
1 -
Query Plan
1 -
QueryProfile
1 -
RBAC
1 -
Reading-excel
2 -
Reliability
1 -
RescuedData
1 -
Rest APIs
1 -
Retail Forecasting
1 -
Retries
1 -
Row Level Security
2 -
SAP
4 -
SAP BW
1 -
Sap Hana Driver
1 -
Scala Code
1 -
Scd Type 2
1 -
Scheduled Tasks
1 -
SchemaEvolution
1 -
Scripting
1 -
SDK
1 -
Secrets
1 -
Security
1 -
Semantic Layer
1 -
serverless
2 -
Service Principals
1 -
Small Business
1 -
SmallFiles
1 -
Smart Routing
1 -
Snowflake
1 -
Spanish
2 -
Spark
9 -
Spark Caching
1 -
Spark Internals
1 -
Spark Performance
1 -
SparkSQL
1 -
SQL
4 -
Sql Scripts
2 -
SQL Serverless
1 -
SQLPerformance
1 -
SQLWarehouse
1 -
Stream
1 -
streaming
2 -
Streaming Tables
1 -
streamlit
3 -
Structured streaming
2 -
Students
2 -
Supervisor Agent
1 -
Supply Chain
1 -
Support Ticket
1 -
Sync
1 -
System Tables
1 -
SystemTables
1 -
Sytem Tags
1 -
Table Formats
1 -
Tags
1 -
terraform
1 -
Timeouts
1 -
Training
1 -
Tutorial
4 -
UCSD
1 -
Unit Test
1 -
Unit testing
1 -
Unity Catalog
36 -
Unity Cataloge
1 -
Unity Catalogue
1 -
Unity Catlog
2 -
Unity Gateway
1 -
UnityCatalog
2 -
UnityGateway
1 -
University Alliance
1 -
VACUUM Command
1 -
Variant
1 -
VisualDataPrep
1 -
Vnet Injection
1 -
Vnet peering
1 -
volumes
1 -
Warehousing
1 -
Web
1 -
Workflow Jobs
1 -
Workflows
10 -
Zerobus
4 -
Zorder
1 -
Zordering
2
- « Previous
- Next »
| User | Count |
|---|---|
| 85 | |
| 77 | |
| 73 | |
| 60 | |
| 43 |