Databricks Community

madams · 11-19-2024

I'm using the Databricks CLI to deploy an asset bundle for a job. I'm attempting to setup the configuration such that the "dev" target does not have a trigger on it, and the "prod" target does. Essentially the dev job is not scheduled to run and th...

madams

Thanks for providing that whole example, it was really easy to fiddle with. I think I've found your solution. Update the original two tasks on the job (if you want to keep them) like this: tasks: - task_key: notebook_task job...

madams · 11-19-2024

Oh no! I swear I wrote a reply out, but I must not have submitted it. Setting that aside...I think in your case a materialized view makes the most sense, and it's what I'd go with. It's essentially precomputing the logic, but reusing some of the s...

madams · 11-08-2024

Hi Dharinip. I've had similar conversations internally with developers asking very similar things. This is my general advice for this situation, but I think there are a lot of considerations to how you create your gold layer.How large is this datas...

madams · 09-13-2024

I've had quite a bit of fun with UC and view permissions. I don't think this is specific to using the IDENTIFIER() function, but I suspect it's related to UC permissions. What you'll need to ensure:The user or group who owns the view on catalog_b h...

madams · 09-13-2024

Ah, I have had that same error before when cloning from git. I'm guessing you got the Repo URL by hitting the "clone" button in ADO and copy/pasting it in to Databricks. One thing I've done in the past is in the Repository URL, remove the "org@" pa...

Databricks Community

User Stats

User Activity

Databricks Asset Bundle - variables for job trigger

Re: databricks bundle deploy fails when job includes dbt task and git_source

Re: How to decide on creating views vs Tables in Gold layer?

Re: How to decide on creating views vs Tables in Gold layer?

Re: Permission Issue with IDENTIFIER clause

Re: Azure Databricks Workflows with Git Integration