Databricks Community

p4pratikjain · 08-19-2024

I use DAB for deploying Jobs, I want to add a specific Task in dev only but not in staging or prod. Is there any way to achieve this using DAB ?

p4pratikjain · 08-20-2023

I am using Following job trigger API -https://docs.databricks.com/api/workspace/jobs/runnowHere is what I want to achieve -I have complex workflow with multiple tasks and each task has python parameter named "--conf-file".While using runnow REST API ...

p4pratikjain · 10-30-2022

I completed databricks Machine learning associate Certification exam and its been 3-4 days I haven't received My certificate yet. I have raised support ticket but haven't heard anything from them either. can anyone tell me how can I get my certificat...

p4pratikjain · 07-28-2024

If you want to cache the dataframe without having to perform action, You can use SparSQL. Python API is always evaluated as lazy for both cache() and persist(). Instead SparkSQL gives option to specify if you want to evaluate it lazy or eager.Referen...

p4pratikjain · 07-16-2024

Interesting use case!! Ideally having seperate bundle for each customer seems like a clean solution. But if you dont want that then - You can just include all the yaml files in databricks.yml with include: - resources/*/*.yml Inside the yaml files...

p4pratikjain · 07-14-2024

You can do this using Delta Live Tables, For design you can follow medallion architecture (https://www.databricks.com/glossary/medallion-architecture).You can have your 1st table as bronze with everything just appended and second table as silver with...

p4pratikjain · 07-14-2024

I have faced similar problem while developing CI/CD using DABs for ML Jobs. There is no straight forward documentation listing all the options for DABs(As far as I know). As a workaround you can look at the REST API documentation which has all the op...

p4pratikjain · 07-14-2024

can you provide complete stacktrace ?

Databricks Community

User Stats

User Activity

DAB - Add/remove task depending on workspace.

How to send Task Specific Parameters using run-now REST API.

Haven't received certificate for Machine Learning Associate Exam even after Passing.

Re: Efficient caching/persisting

Re: Variables in databricks.yml "include:" - Asset Bundles

Re: Delta table with unique columns incremental refresh

Re: Where to find comprehensive docs on databricks.yaml / DAB settings options

Re: Py4JJavaError: An error occurred while calling o465.coun