- 1241 Views
- 1 replies
- 5 kudos
Python Data Source API
If you’ve ever hacked together a one-off script to pull data from some random API into Spark, you’re exactly who the new Python Data Source API is for. Databricks has made this API generally available on Apache Spark™ 4.0 with Databricks Runtime 1...
- 1241 Views
- 1 replies
- 5 kudos
- 5 kudos
I have seen multiple glue jobs pulling data from such systems. This is certainly a solution to simplify and bring governance to it. will look forward to implement it.#Apache-4
- 5 kudos
- 806 Views
- 0 replies
- 1 kudos
Databricks Serverless Compute: Zero Infrastructure Management
You spend hours sizing clusters, tuning autoscaling configurations, and optimizing instance types.Clusters sit idle burning money between jobs or take minutes to start when you need them immediately.Managing compute infrastructure becomes a full-time...
- 806 Views
- 0 replies
- 1 kudos
- 1084 Views
- 1 replies
- 3 kudos
Databricks + Power BI
Last night I delivered my final session on Databricks + Power BI integration and it felt like the right moment to close this chapter on a high note. Because it was the last time I was presenting this topic, I rebuilt the flow completely. Instead of ...
- 1084 Views
- 1 replies
- 3 kudos
- 3 kudos
Well done Bianca. Look forward to Fabric session.
- 3 kudos
- 1769 Views
- 1 replies
- 8 kudos
Recap: Data & AI World Tour - Chicago, Nov 19, 2025
Yesterday in Chicago, I attended one of the final stops of the Databricks DAIWT (Data & AI World Tour). This tour brings the excitement and passion of our annual Data + AI Summit to over a dozen cities worldwide, in a condensed format. Packed into ...
- 1769 Views
- 1 replies
- 8 kudos
- 8 kudos
These are great events. I had the opportunity to attend the Toronto event.
- 8 kudos
- 1657 Views
- 1 replies
- 2 kudos
Databricks Architecture Center
Databricks Architecture Center — Your Blueprint for Building Modern Data & AI PlatformsThe Databricks Architecture Center is a centralized knowledge hub that provides:End-to-end reference architecturesIndustry-specific patternsArchitecture decision ...
- 1657 Views
- 1 replies
- 2 kudos
- 2 kudos
Its very useful. I suppose you missed to provide the link. Here is the link for easy accesshttps://www.databricks.com/resources/architectures
- 2 kudos
- 2810 Views
- 1 replies
- 2 kudos
Resolved! Building an End-to-End ETL Pipeline with Data from S3 in Databricks
Hey everyone I’m excited to share the progress of my Databricks learning journey! Recently, I worked on building an end-to-end ETL pipeline in Databricks, starting from data extraction from AWS S3 to creating a dynamic dashboard for insights.Here’s h...
- 2810 Views
- 1 replies
- 2 kudos
- 2 kudos
@Rohan_Samariya this is fantastic work! I’m genuinely impressed with how you’ve taken the Databricks stack end-to-end: S3 ingestion → PySpark transformations → Delta optimisation → interactive SQL dashboards. This is exactly the type of hands-on, fu...
- 2 kudos
- 1234 Views
- 2 replies
- 5 kudos
Databricks Release Hub
I launched a new app this week to help keep track of Databricks releases.you can view and filter the latest releases in the timeline view, or go to the resources page and go to a product area and see the latest releases alongside useful links for blo...
- 1234 Views
- 2 replies
- 5 kudos
- 5 kudos
@alcole - thanks for sharing it. I already bookmarked it last week when saw it on social.
- 5 kudos
- 2162 Views
- 0 replies
- 1 kudos
Handling the Chaos: Data Quality Strategies with PySpark Ingestion
Tips and Techniques for Ingesting Large JSON files with PySparkIntroductionSuppose you’ve ever struggled or grappled with consuming massive JSON files with PySpark. In that case, you are aware that insufficient data can always creep in and silently d...
- 2162 Views
- 0 replies
- 1 kudos
- 1373 Views
- 1 replies
- 4 kudos
Hackathon Project: Recipe Recommendation Engine with Traditional ML + Genie on Databricks Free Edit
Hi everyone, For the Databricks Free Edition Hackathon, I wanted to show that traditional ML still has a big role today, and how it can work hand-in-hand with Databricks’ newer AI tooling. As a concrete use case, I created a recipe recommendation eng...
- 1373 Views
- 1 replies
- 4 kudos
- 4 kudos
This is amazing @hasnat_unifeye. Well done and good luck for the hackathon.
- 4 kudos
- 10545 Views
- 5 replies
- 8 kudos
API Consumption on Databricks
In this blog, I will be talking about the building the architecture to serve the API consumption on Databricks Platform. I will be using Lakebase approach for this. It will be useful for this kind of API requirement.API Requirement: Performance:Curre...
- 10545 Views
- 5 replies
- 8 kudos
- 1455 Views
- 2 replies
- 2 kudos
Resolved! My First Month Learning Databricks - Key Takeaways So Far.
Hey everyone I recently started my Databricks learning journey about a month ago, and I wanted to share what I’ve learned so far from one beginner to another.Here are a few highlights:1️⃣ Understanding the Lakehouse Concept - Realized how Databricks ...
- 1455 Views
- 2 replies
- 2 kudos
- 2 kudos
I was planning to build an ETL pipeline, but I hadn’t considered using MLflow to predict sales and ratings. Thanks for the suggestion, I’ll work on creating this demo soon to test and enhance my skills.
- 2 kudos
- 1164 Views
- 2 replies
- 5 kudos
I Tried Teaching Databricks About Itself — Here’s What Happened
Hi All, How are you doing today?I wanted to share something interesting from my recent Databricks work — I’ve been playing around with an idea I call “Real-Time Metadata Intelligence.” Most of us focus on optimizing data pipelines, query performance,...
- 1164 Views
- 2 replies
- 5 kudos
- 5 kudos
I like the core idea. You are mining signals the platform already emits.I would start rules first, track small files ratio and average file size trend, watch skew per partition and shuffle bytes per input gigabyte. Compare job time to input size to c...
- 5 kudos
- 252 Views
- 0 replies
- 1 kudos
Last chance to register for our LIVE Lakebase BrickTalks session!
Join us tomorrow, Thursday, Nov 13 at 9 am PT for the latest BrickTalks! We'll talk about bringing data intelligence from your Lakehouse into every app. Register now. What you’ll learn: Use Lakebase (PostgreSQL-compatible, serverless OLTP) to serve...
- 252 Views
- 0 replies
- 1 kudos
- 664 Views
- 0 replies
- 1 kudos
How Upgrading to Databricks Runtime 16.4 sped up our Python script by 10x
Wanted to share something that might save others time and money. We had a complex Databricks script that ran over 1.5 hours, when the target was under 20 minutes. Initially tried scaling up the cluster, but real progress came from simply upgrading th...
- 664 Views
- 0 replies
- 1 kudos
- 1550 Views
- 0 replies
- 1 kudos
Control Databricks Costs with AI & BI Dashboards - Video Summary
In this video, I try to showcase in a very simplified way how to enable and setup AI & BI dashboards to control costs and take actions. I hope this could be useful. I think it is a superb feature to get insights on costs while straightforward to setu...
- 1550 Views
- 0 replies
- 1 kudos
| User | Count |
|---|---|
| 85 | |
| 74 | |
| 55 | |
| 44 | |
| 42 |
