Databricks Community is the official forum for data engineers, data scientists, data enthusiasts and AI practitioners to ask questions, share knowledge, and collaborate on everything related to Databricks. Members can find answers, share best practices, and support each other in an environment that ensures respectful interactions.
MuleGraph Investigator
From One Alert to the Hidden Network
How a Databricks App turns a single flagged account into a fully-traced fraud network - with Genie doing the actual investigating, not ...
In many data platforms, the problem is not that we do not have enough tables. Usually, we have too many. Bronze tables. Silver tables. Gold tables. Reporting views. Power BI calculations. Tabl...
Interview Question: Many people start with the row count. I would start with the architecture. Billions of records are not new in enterprise data engineering. The real challenge is designing a pip...
If I call powerbi refresh from databricks workflow job it waits until the refresh is complete unlike ADF(Azure Data Factory) pbi refresh through rest api. In case of big pbi dataset which might take ...
Hi all. If you've ever manually promoted resources from dev to prod on Databricks — copying notebooks, updating configs, hoping nothing breaks — this post is for you. I've been building a CI/CD set...
Hi, Is there a way to custom name an assetbundle file name and pass that to databricks bundle deploy? I mean right now I must use databricks.yml, so my question is whether there is a way to pass a ...
The Hidden Cost of Scaling the Lakehouse Over the past few years, many organizations have successfully migrated to Databricks to modernize their data platforms. The Lakehouse architecture has enable...
Introduction “AI First” - But Data Always Comes First I have been working in the data space for close to two decades. My journey started as an ETL developer and gradually evolved into roles spannin...
...Using PySpark, I mounted the S3 location and read the data directly into a DataFrame: df = spark.read.option("header", True).csv("s3a://roh-databricks-v1/ecommerce-data/S3 Order Line Items.csv") T...
Hey everyone 👋 I recently started my Databricks learning journey about a month ago, and I wanted to share what I’ve learned so far from one beginner to another. Here are a few highlights: 1️⃣ Und...