cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 
Meet the Databricks MVPs

The Databricks MVP Program is our way of thanking and recognizing the community members, data scientists, data engineers, developers and open source enthusiasts who go above and beyond to uplift the data and AI community. Whether they’re speaking at ...

  • 150 Views
  • 0 replies
  • 4 kudos
20 hours ago
Databricks training invests in closing the data + AI skills gap across enterprises

The Data + AI Skills Gap The “skills gap” has been a concern for CEOs and leaders for many years, and the gap is only widening. According to McKinsey, as many as 375 million workers globally might have to change occupations soon to meet company needs...

  • 219 Views
  • 0 replies
  • 1 kudos
Monday
Now Hiring: Databricks Community Technical Moderator

Apply Now! Are you passionate about data and want to make a difference for thousands of data practitioners? Databricks is looking for a dedicated and knowledgeable Community Technical Moderator to guide our thriving online community and empower users...

  • 727 Views
  • 1 replies
  • 3 kudos
2 weeks ago
Insights from a global survey of 1,100 technologists and interviews with 28 CIOs

How to unlock enterprise AI: “You can have all the AI in the world, but if it’s on a shaky data foundation, then it’s not going to bring you any value.” — Carol Clements, Chief Digital and Technology Officer, JetBlue Companies everywhere have been qu...

  • 216 Views
  • 1 replies
  • 1 kudos
Thursday
Data + AI Summit: Call for Presentations

Are you solving real-world problems with data and AI? Your peers want to be inspired by your work! At Data + AI Summit 2025, we’re looking for data engineers, ML engineers, data scientists and analysts who are pushing the limits of AI, analytics and ...

  • 327 Views
  • 0 replies
  • 2 kudos
a week ago
Season's Speedings: Databricks SQL Delivers 4x Performance Boost Over Two Years

Databricks Performance Index is derived statistically from repeating workloads, accounting for changes irrelevant to the engine, and computed against billions of production queries. Lower is better.  As the season of giving approaches, we at Databri...

  • 379 Views
  • 0 replies
  • 2 kudos
a week ago

Community Activity

ameet9257
by > New Contributor II
  • 9 Views
  • 1 replies
  • 0 kudos

Cloning of Workflow from One env to different env using Job API

Hi Team,One of my team members recently shared one requirement: he wants to migrate the 10 Workflows from the sandbox to the dev environment to run his model in dev env.I wanted to move all these workflows in an automated way and one of the solutions...

ameet9257_0-1732126423665.png ameet9257_1-1732126508104.png ameet9257_2-1732126684331.png
  • 9 Views
  • 1 replies
  • 0 kudos
Latest Reply
Stefan-Koch
Contributor II
  • 0 kudos

Hi AmetDatabricks Asset Bundles are designed precisely for the requirements you have.https://docs.databricks.com/en/dev-tools/bundles/index.htmlYou can also transfer existing jobs that were created manually into a bundle. This works like this, for ex...

  • 0 kudos
zach_goehring
by Databricks Employee
  • 846 Views
  • 2 replies
  • 7 kudos

Unity Catalog Functions: Tools for Users and Agents

Everyone wants to use Generative AI to solve business problems, but developers have realized that simply deploying a large language model isn’t going to be enough to cross the finish line. Business problems are complex and require a solution capable ...

blogimage.png chat1.png blogimage.png blogimage.png
  • 846 Views
  • 2 replies
  • 7 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 7 kudos

@zach_goehring  Thanks for sharing

  • 7 kudos
1 More Replies
rpatel
by > New Contributor III
  • 33294 Views
  • 31 replies
  • 14 kudos

Not received my certificate after passing Data Engineer Associate exam.

I've successfully passed Databricks Data Engineer Associate Certified exam but still have not received the certificate.-----------------------------------------------Hello Rupesh,You have completed the assessment, Databricks Certified Data Engineer A...

  • 33294 Views
  • 31 replies
  • 14 kudos
Latest Reply
kUnal_bHagat
  • 14 kudos

I've successfully passed Databricks Data Engineer Associate Certified exam on 20th Nov 2024 9am IST but still have not received the certificate.whom do I need to connect to receive the certificate ?

  • 14 kudos
30 More Replies
jorperort
by > New Contributor III
  • 4607 Views
  • 6 replies
  • 1 kudos

Resolved! [Databricks Assets Bundles] no deployment state

Good morning, I'm trying to run: databricks bundle run --debug -t dev integration_tests_job My bundle looks: bundle: name: x include: - ./resources/*.yml targets: dev: mode: development default: true workspace: host: x r...

Data Engineering
Databricks Assets Bundles
Deployment Error
pid=265687
  • 4607 Views
  • 6 replies
  • 1 kudos
Latest Reply
jtberman
New Contributor
  • 1 kudos

Hello, Reopening this ticket in hopes that either of you had some luck in resolving your bug.  I am currently facing the same issue where I can deploy an asset bundle via the local CLI without issue (by deploy I mean the bundle code is written to my ...

  • 1 kudos
5 More Replies
jonxu
by > New Contributor II
  • 20 Views
  • 0 replies
  • 0 kudos

PAT needed but not allowed in "Advanced Data Engineering - 6.5L Deploy pipeline with the CLI Lab"

It is stated in the lab notebook that:Run the setupRun the setup script for this lesson by running the cell below. This will ensure that:The Databricks CLI is installedAuthentication is configuredA pipeline is createdHowever, when I tried to run the ...

  • 20 Views
  • 0 replies
  • 0 kudos
Andrewcon
by > New Contributor II
  • 1841 Views
  • 1 replies
  • 1 kudos

Delta tables and YOLO computer vision tasks

 Hi all,I would really appreciate if someone could help me out. I feel it’s both a data engineering and ML question.One thing we use at wo is YOLO for object detection. I’ve managed to run YOLO by loading data from the blob storage, but I’ve seen tha...

Data Engineering
computer vision
Delta table
YOLO
  • 1841 Views
  • 1 replies
  • 1 kudos
Latest Reply
jnap
Visitor
  • 1 kudos

I am also looking for an answer to this question. Did you manage to find a solution @Andrewcon ?

  • 1 kudos
AlbertWang
by > Contributor III
  • 41 Views
  • 2 replies
  • 0 kudos

Networking configuration of Azure Databricks managed storage account

Hi all,I created an Azure Databricks Workspace, and the workspace creates an Azure Databricks managed storage account.The networking configuration of the storage account is "Enabled from all networks".Shall I change it to "Enabled from selected virtu...

  • 41 Views
  • 2 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Changing the networking configuration of your Azure Databricks managed storage account to "Enabled from selected virtual networks and IP addresses" is a good step for enhancing security. However, the "Insufficient permissions" status you are seeing f...

  • 0 kudos
1 More Replies
NehaR
by > New Contributor III
  • 13 Views
  • 2 replies
  • 1 kudos

Is there any option in databricks to estimate cost of a query before execution

Hi Team,I want to check if there is any option in data bricks which can help to estimate cost of a query before execution?I mean calculate DBU before actual query execution based on the logical plan? Regards 

  • 13 Views
  • 2 replies
  • 1 kudos
Latest Reply
NehaR
New Contributor III
  • 1 kudos

Is there any way to track the progress or ETA?Do we have access to ideas portal? Where can we search this reference number DB-I-5730? 

  • 1 kudos
1 More Replies
scharly3
by > New Contributor II
  • 8673 Views
  • 5 replies
  • 0 kudos

Error: Folder xxxx@xxx.com is protected

Hello, On Azure Databricks i'm trying to remove a folder on the Repos folder using the following command : databricks workspace delete "/Repos/xxx@xx.com"I got the following error message:databricks workspace delete "/Repos/xxxx@xx.com"Error: Folder ...

  • 8673 Views
  • 5 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

You have deleted the user from the workspace and the folder is still showing?

  • 0 kudos
4 More Replies
184754
by > New Contributor
  • 25 Views
  • 1 replies
  • 0 kudos

Table Trigger - Too many logfiles

Hi, we have implemented a job that runs on a trigger of a table update. The job worked perfectly, until the source table now have accumulated too many log files and the job isn't running anymore. Only the error message below:Storage location /abcd/_d...

  • 25 Views
  • 1 replies
  • 0 kudos
Latest Reply
radothede
Contributor II
  • 0 kudos

Hi @184754 Interesting topic, as the docs says:"Log files are deleted automatically and asynchronously after checkpoint operations and are not governed by VACUUM. While the default retention period of log files is 30 days, running VACUUM on a table r...

  • 0 kudos
DataGeek_JT
by > New Contributor II
  • 1032 Views
  • 2 replies
  • 0 kudos

Is it possible to use Liquid Clustering on Delta Live Tables / Materialised Views?

Is it possible to use Liquid Clustering on Delta Live Tables? If it is available what is the Python syntax for adding liquid clustering to a Delta Live Table / Materialised view please? 

  • 1032 Views
  • 2 replies
  • 0 kudos
Latest Reply
kerem
New Contributor II
  • 0 kudos

Hi @amr, materialised views are not tables, they are views. Liquid clustering is not supported on views so it will throw [EXPECT_TABLE_NOT_VIEW.NO_ALTERNATIVE] error. Unfortunately it will be the same case for the "optimize" command as well. 

  • 0 kudos
1 More Replies
Kulasekaran0107
by > Visitor
  • 23 Views
  • 0 replies
  • 0 kudos

Voucher for Data Engineer Associate certification

Hi, I am looking for a voucher to complete the Data Engineer Associate certification. Could anyone help me please?

  • 23 Views
  • 0 replies
  • 0 kudos
Brahmareddy
by > Valued Contributor III
  • 18 Views
  • 0 replies
  • 0 kudos

Mark Your Calendar: Data Day Texas + AI 2025 on January 25!

Hey Austin Databricks Group,If you're passionate about data and AI, here's an event you won't want to miss: Data Day Texas + AI 2025, happening on Saturday, January 25. This event brings together experts and enthusiasts to share insights, trends, and...

  • 18 Views
  • 0 replies
  • 0 kudos
ksenija
by > Contributor
  • 1280 Views
  • 2 replies
  • 0 kudos

Foreign table to delta streaming table

I want to copy a table from a foreign catalog as my streaming table. This is the code I used but I am getting error: Table table_name does not support either micro-batch or continuous scan.; spark.readStream                .table(table_name)         ...

  • 1280 Views
  • 2 replies
  • 0 kudos
Latest Reply
sbiales
Visitor
  • 0 kudos

I also want to bump this! This is my exact problem right now as well.

  • 0 kudos
1 More Replies
FabianGutierrez
by > New Contributor III
  • 143 Views
  • 9 replies
  • 1 kudos

My DABS CLI Deploy call not generating a .tfstate file

Hi Community,I'm running into an issue, when executing Databricks CLI Bundle Deploy I dont get the Terraform State file (.tfstate). I know that I should get one but even when defining the state_apth on my YAML (.yml) DABS file I still do not get it.D...

FabianGutierrez_0-1731932526298.png
  • 143 Views
  • 9 replies
  • 1 kudos
Latest Reply
FabianGutierrez
New Contributor III
  • 1 kudos

Forgot to also share this print screen of the last section in the logs. Somehow the State file keeps getting ignored (not found) so how can the deployment still take place I wonder.  

  • 1 kudos
8 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors
Read Databricks Data Intelligence Platform reviews on G2

Latest from our Blog

Understanding Unity Catalog

Throughout the dozens of engagements I’ve had since joining Databricks, I’ve found that customers often struggle to understand the scope and concept of Unity Catalog. Questions like “Does it store my ...

211Views 2kudos