Join APJ's premier AI/BI virtual challenge to solve real-world business problems, sharpen your skills, and compete for prizes using the Databricks Data Intelligence Platform. This challenge provides a unique opportunity to work together, apply AI-dri...
- 3086 Views
- 0 replies
- 1 kudos
a week ago
We’re excited to invite you to our MONTHLY Databricks Get Started Days, a half-day virtual learning experience designed to jumpstart your journey with Databricks. This global event is tailored to equip you with essential data engineering and analysi...
- 3211 Views
- 0 replies
- 0 kudos
3 weeks ago
AI is transforming BI by changing the way organizations manage, analyze and get insights from their data. Everyone is rushing to democratize access to data through LLMs. But without the right foundation, you can’t get the most from AI. Our new appro...
- 3849 Views
- 0 replies
- 2 kudos
3 weeks ago
Join us for the return of the Virtual Learning Festival! Mark your calendars from 9 April - 30 April 2025! Upskill across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career with...
- 67599 Views
- 242 replies
- 42 kudos
03-03-2025
Be part of a global movement! Connect with 22,000 data enthusiasts at 700+ sessions, keynotes, and hands-on training at this year’s Data + AI Summit. Whether you’re into data intelligence, governance, AI, or data warehousing, this is your chance to l...
- 9618 Views
- 1 replies
- 3 kudos
02-19-2025
Community Activity
- 516 Views
- 15 replies
- 17 kudos
Exclusive Databricks Certified Jacket at Data & AI Summit
Our iconic #DatabricksCertified jackets are returning to #DataAISummit this June — and we want your input! Drop an or in the comments below to vote for your favorite! *Note that these jackets will only be available at Data + AI Summit and will n...
- 516 Views
- 15 replies
- 17 kudos
Latest Reply
- 17 kudos
B Looks good But I have a suggestion to Remove the badge and replace it with logo instead
- 17 kudos
- 5 Views
- 0 replies
- 0 kudos
Databricks App with DAB
Hi All,I am trying to deploy a DBX APP via DAB, however source_code_path seems not to be parsed correctly to the app configuration.- dbx_dash/-- resources/---- app.yml-- src/---- app.yaml---- app.py-- databricks.ymlresources/app.yml:resources:apps: m...
- 5 Views
- 0 replies
- 0 kudos
- 31 Views
- 2 replies
- 0 kudos
Unexpected SKU Names in Usage Table for Job Cost Calculation
I'm trying to calculate the cost of a job using the usage and list_prices system tables, but I'm encountering some unexpected behavior that I can't explain.When I run a job using a shared cluster, the sku_name in the usage table is PREMIUM_JOBS_SERVE...
- 31 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
This could happen if there was at least one run of the job when a task was set to serverless. Alas, the toggle to enable serverless is no more.
- 0 kudos
- 50 Views
- 2 replies
- 0 kudos
Impossible to access Terraform created external location?!
Hi all,There seems to be an external location created that nobody within the organization can actually see or manage, because it has been created with a Google service account in Terraform.Here is the problem:DESCRIBE EXTERNAL LOCATION `gcsbucketname...
- 50 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
I would agree that the metastore admin(s) should be able to see the external location. This issue can happen with terraform scripts if the script doesn't grant additional rights on the external location.
- 0 kudos
- 24 Views
- 1 replies
- 0 kudos
UCX Installation error
Error Message: databricks.sdk.errors.platform.ResourceDoesNotExist: Can't find a cluster policy with id: 00127F76E005AE12.
- 24 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Please provide more info on what you were doing when this error occurred? Are you upgrading a legacy workspace to Unity Catalog?
- 0 kudos
- 550 Views
- 1 replies
- 5 kudos
🚀 Explore Query History for Databricks Delta Live Tables (DLT) Pipelines: Now in Public Preview!
If you’re using Delta Live Tables to simplify ETL processes and manage data pipelines at scale, there’s great news: You can now access query history and query profiles to gain deep insights into your pipeline runs. This powerful feature enables you t...
- 550 Views
- 1 replies
- 5 kudos
Latest Reply
- 5 kudos
Do you have any recommendations for instruction on how to apply this information for performance optimization?
- 5 kudos
- 2734 Views
- 13 replies
- 4 kudos
Newbie learning DLT pipelines
Hello,I am learning to create DLT pipelines using different graphs using a 14 day trial version of the premium Databricks. I have currently one graph Mat view -> Streaming Table -> Mat view. When i ran this pipeline(serverless compute) 1st time, ran...
- 2734 Views
- 13 replies
- 4 kudos
Latest Reply
- 4 kudos
use this https://www.youtube.com/watch?v=iqf_QHC7tgQ&list=PL2IsFZBGM_IGpBGqxhkiNyEt4AuJXA0Gg it will help you a lot
- 4 kudos
- 50 Views
- 1 replies
- 0 kudos
Free training for Generative AI Engineering with Databricks 2025
Hi People, I am looking for a self paced training required to prepare and pass the Generative AI Engineering certification in 2025 April/May. The old training has been expired.Could you please help. Appreciate for your support! Ali
- 50 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi,The Databricks Certified Generative AI Engineer Associate exam information page mentions that there is a series of self-paced courses in Databricks Academy.You can login to Databricks Academy from this page: https://www.databricks.com/learn/traini...
- 0 kudos
- 202 Views
- 1 replies
- 0 kudos
Problem with ipywidgets and plotly on Databricks
Hi everyone, I am encountering a problem when using ipywidgets with plotly on Databricks. I am trying to pass interactive arguments to a function and then plot with plotly. When I do the followingdef f(m, b) : plt.figure(2) x = np.linspace(-10,...
- 202 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
It would help a lot if you attach a notebook or copy the full code that demonstrates the problem. This would make it easy for someone to copy/paste, run, and troubleshoot it and you are more likely to get effective help that way Also, since your Data...
- 0 kudos
- 62 Views
- 3 replies
- 0 kudos
dbutils.fs.mv inefficient with ADLS
dbutils.fs.mv with ADLS currently copies the file and then deletes the old one. This incurs costs and has a lot of overhead vs using the rename functionality in ADLS which is instant and doesn't incur extra costs involved with writing the 'new' data....
- 62 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
The tool is really meant for dbfs and is only accessible from within Databricks. If I had to guess the idea is that most folks will not be using dbfs for production or sensitive data (for a host of good reasons) and as such there has not been a big ...
- 0 kudos
- 80 Views
- 2 replies
- 1 kudos
LangChain Compatibility in Databricks – Painfully Inconsistent Experience
Hey all,Is anyone else experiencing major friction when using LangChain within Databricks?Even after carefully pinning versions like:%pip install \ loguru \ mlflow==2.10.1 \ langchain==0.1.5 \ langchain-google-vertexai \ databricks-vectorse...
- 80 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi irfanghat,I’ve been thinking about the issue of the silent crashing, and I have a couple of suggestions that might help us diagnose and stabilize things:Start Logging All Library Requests:It could be helpful to begin logging all library installati...
- 1 kudos
- 138 Views
- 4 replies
- 0 kudos
Resolved! FeatureEngineeringClient workspace id error
Hi, I am working from local notebook using vscode databricks extension.I am trying to use FeatureEngineeringClient, when I create data set training_set = fe.create_training_set( df=filtered_data_train, feature_lookups=payments_feat...
- 138 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
I’ve done some additional research and found that the FeatureStoreClient is not officially supported when accessing a managed Databricks environment from an external IDE, even when using Databricks Connect. The client library is designed to operate w...
- 0 kudos
- 22 Views
- 0 replies
- 0 kudos
Can we have a depend-on for jobs to run on two different dabs?
If there are two different DABs. Can we have a dependency for one job from one DAB to run after a job run from another DAB? Similar to how tasks can depend on each other to run one after the other in the same DAB. Can we have the same for two differ...
- 22 Views
- 0 replies
- 0 kudos
- 309 Views
- 1 replies
- 0 kudos
Configure verbose audit logs through terraform
Hi everyone,I was looking into the databricks_workspace_conf Terraform resource to configure Verbose Audit Logs (and avoid changing it through the UI). However, I attempted to apply this configuration and encountered the following error:Error: cannot...
- 309 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@fedemgp I was able to turn the desired setting on and off with Terraform with this code: GitHub Gist I'm using Databricks Terraform provider version 1.74.0 and my Databricks runs on Google Cloud.
- 0 kudos
- 28 Views
- 0 replies
- 0 kudos
Join us for Databricks DevConnect I London on May 1!
Hi all! We are excited to invite you to Databricks DevConnect London which will be next week on Thursday, May 1 from 5:00pm - 8:30pm BST at the Databricks London office! RSVP to attend the event HERE lu.ma/w3vsbwkb Databricks DevConnect I London ...
- 28 Views
- 0 replies
- 0 kudos
Latest from our Blog
Databricks for Identity Systems - Part 1
Howdy! Welcome to my first blog! As an Identity System Administrator, I would say that a good portion of my day was spent configuring Okta and other identity systems. As I have moved more into an Iden...
Mastering DBR Migrations at Scale
In an era where data drives innovation and competitive advantage, protecting it becomes a non-negotiable priority. Particularly when it involves sensitive information, even minor lapses can translate ...
Building Intelligent AI Agents: The Complete Guide From Blueprint to Enterprise Deployment
Introduction The world of Artificial Intelligence (AI) is evolving rapidly, and AI Agents are at the forefront of this transformation. These intelligent systems, powered by Large Language Models (LLMs...
Read Delta Tables with Snowflake via Unity Catalog
Data practitioners often struggle to choose between open data formats like Apache Iceberg and Delta Lake (Linux Foundation) due to inconsistent support across different data platforms. To address this...
