Databricks Community

Poorva21

Databricks is positioning the platform as a full stack for LLM development — from data ingestion → feature/embedding pipelines → fine-tuning (Mosaic AI) → evaluation → deployment (Model Serving) → monitoring (Lakehouse Monitoring).I’m curious about r...

Poorva21

Hi everyone,I'm working on optimizing Databricks costs for a production-grade data pipeline (Spark + Delta Lake) on Azure. I’m looking for practical, field-tested strategies to reduce compute and storage spend without impacting performance.So far, I’...

Poorva21

This is an exciting step forward from Databricks! Looking forward to diving into the curriculum and exploring what's next in the world of data + AI! Thanks for sharing @Sujitha

Poorva21

Possible reasons:1. Setting spot_bid_max_price = -1 is not accepted by Azure poolsAzure Databricks only accepts:0 → on-demand onlypositive numbers → max spot price-1 is allowed in cluster policies, but not inside pools, so validation never completes....

Poorva21

Step 1 - Remove widget creation linesDelete:dbutils.widgets.text("target_catalog", "", "Target Catalog")dbutils.widgets.text("target_schema", "", "Target Schema")dbutils.widgets.text("processing_date", "", "Processing Date")Step 2 - Use only widget g...

Poorva21

This error occurs because instance pools require a concrete spot bid max price value, even if the cluster policy marks it as unlimited. Set an explicit value (e.g., 100) directly in the instance pool configuration, or switch the pool to on-demand nod...

Poorva21

1. Explicitly define spot_bid_max_price in the instance poolIf your pool uses Spot instances, check that the pool configuration includes a valid value.How to check:Go to Compute → Instance PoolsEdit the poolConfirm that spot_bid_max_price is visible ...

Databricks Community

User Stats

User Activity

How realistic is truly end-to-end LLMOps on Databricks?

Best Practices for Optimizing Databricks Costs in Production Workloads?

Re: Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certificat

Re: Troubleshooting Azure Databricks Cluster Pools & spot_bid_max_price Validation Error

Re: How can I pass parameters from DABs to something(like notebooks)?

Re: On Demand Pool Configuration & Policy definition

Re: Troubleshooting Azure Databricks Cluster Pools & spot_bid_max_price Validation Error