Databricks Community

cmathieu

I'm trying to run the OPTIMIZE command on a table with less than 2000 rows, but it is causing an out of memory issue. The problem seems to come from the fact that it is a heavily nested table in staging between a json file and flattened table. The ta...

cmathieu

I have an issue with DAB where all the project files, starting from root ., get deployed to the /files folder in the bundle. I would prefer being able to deploy certain util notebooks, but not all the files of the project. I'm able to not deploy any ...

cmathieu · 01-16-2025

I'm currently helping a team migrating to DABs from dbx and they would like to be able to work on multiple features at the same time.What I was able to do is pass the current branch as a variable in the root_path and various names, so when the bundle...

cmathieu

Can you store it in a service principal's workspace? I read this somewhere in the documentation but can't find it again.

cmathieu

@ChrisChieu I've had the same issue happen to me today. A previously working workflow using serverless compute that is doing streaming foreachbatch operation. It sort of silently failed and ended up timing out.

cmathieu

I tried it and it's the same behaviour as sync-include. As long as all files are excluded, no files can be included in the sync-include or sync-files. Not sure who to approach this without having to list all folders and files to exclude.

cmathieu

Thanks for this hint. My goal is to deploy a whole directory, but not any others.

cmathieu · 01-16-2025

deleting the DABs in Databricks and also in my local version under .databricks solved this one for me.

Databricks Community

User Stats

User Activity

OPTIMIZE command on heavily nested table OOM error

DAB - All projects files deployed

Multiple feature branches per user using Databricks Asset Bundles

Re: How to copy notebooks from local to the tarrget folder via asset bundles

Re: Internal GRPC errors when using databricks connect

Re: DAB - All projects files deployed

Re: DAB - All projects files deployed

Re: Multiple feature branches per user using Databricks Asset Bundles