Databricks Notebooks aren’t just for running code — a few built-in features can make your work a lot smoother: Modularize with %run to reuse common functions across notebooks. Add parameters with widgets so notebooks become interactive and easy to run in different scenarios. Mix SQL and Python cells to query and transform data side by side, without switching tools. Use Repos integration to connect with GitHub, GitLab, or Azure DevOps for version control and collaboration. Manage notebook dependencies with the Environment panel when using serverless compute. These are some of my favorites — but everyone has their own little tricks.  What’s your go-to notebook hack in Databricks? ... View more

Greetings to all the fantastic members who joined us during the exhilarating DI Day event in Mumbai! 🌟 It's with immense joy that we extend our warmest greetings and heartfelt welcome to each one of you. We're absolutely thrilled to have you here with us on Community! Before we dive into the exciting discussions and activities ahead, let's take a moment to get to know each other better. As you've joined us from the vibrant tech scene of Mumbai, we're eager to learn about your backgrounds, interests, and what brought you to our community. Whether you're a seasoned professional, an enthusiastic newcomer, or somewhere in between, your unique perspective and experiences enrich our community immensely. So, let's kick things off by introducing ourselves! Share a bit about who you are, what you do, and what you're passionate about in the tech world. Don't be shy – this is your opportunity to connect with fellow members, spark interesting conversations, and forge new friendships within our Mumbai User Group. We can't wait to hear from each of you and embark on this exciting journey together. Let's make our community a place where every voice is heard, every story is valued, and every member feels at home. Looking forward to getting to know you all! ... View more

Hello Team, I wanted to check how we can get the confirmation for the AI tour in Mumbai. Also, if there is any special thing for the group leaders. Looking forward to hearing from you. ... View more

📅 Friday, April 25, 2025 🕗 5:30 PM Onwards 📍 Sofitel Mumbai BKC 🔗 Secure your spot today! Register Here! Join us for a half-day exploration into strategies that drive success with data and AI. Learn how to eliminate data silos, simplify governance, and make data and AI more accessible across your organization. Through an engaging keynote and deep-dive sessions with Databricks leaders, customers, and partners, you’ll discover how to: Use natural language to gain insights from data across your organization Develop high-quality AI applications without compromising data privacy or control Leverage data intelligence to simplify governance, engineering, streaming, warehousing, and user experiences Agenda Highlights 5:30 PM – 6:30 PM | Registrations 6:30 PM – 7:00 PM | Welcome + Keynote 7:00 PM – 7:15 PM | Customer Story 7:15 PM – 7:35 PM | Simplify Data and AI Governance at Scale With Unity Catalog 7:35 PM – 7:55 PM | Build and Deploy AI Apps With Databricks 7:55 PM – 8:15 PM | AI-Powered Data Warehousing With Databricks SQL 8:15 PM – 8:30 PM | Partner Session 8:30 PM – 9:00 PM | Closing Remarks ... View more

If you’re using Delta Live Tables to simplify ETL processes and manage data pipelines at scale, there’s great news: You can now access query history and query profiles to gain deep insights into your pipeline runs. This powerful feature enables you to: ✅ Debug queries effortlessly. ✅ Identify performance bottlenecks. ✅ Optimize your pipeline execution. Here’s how you can make the most of this feature: Prerequisites 1️⃣ Your pipeline must run in the preview channel (Delta Live Tables runtime channels). 2️⃣ It must be configured in triggered mode. Accessing Query History 1️⃣ From the Query History UI: Click the Query History icon in the Databricks sidebar. Use the Compute filter to select Delta Live Tables (DLT) compute. Click a query to view summary details, such as duration and aggregated metrics. Dive deeper by clicking See query profile, where you’ll find granular query performance insights. 2️⃣ From Notebooks: Open a notebook attached to your DLT pipeline. At the bottom of the notebook, switch to the DLT Query History tab. Click a query name for detailed insights, such as execution duration, query source, and metrics. 3️⃣ From the DLT Pipeline UI: Navigate to the Pipeline Details page for your pipeline. Select the Query History tab at the bottom of the screen. Click on any query to view its detailed profile. Why This Matters Understanding query histories helps data teams identify inefficiencies, debug faster, and improve data flow performance—all while working directly in the familiar Databricks ecosystem. Limitations ⚠️ Provisioning and queued times are currently unavailable. ⚠️ Metrics update live during execution, but the full query profile becomes available only after execution finishes. 💡 How to Enable: Workspace admins can activate this feature from the Previews page in the Databricks admin console. Are you ready to elevate your Delta Live Tables workflows? Share your experience in the comments 🌟 #Databricks #DeltaLiveTables #ETL #DataEngineering #DataOps ... View more