Databricks Community

Phani1

Hi All, I'm looking to implement an automated, scalable, and auditable purge mechanism on Azure Databricks to manage data retention, deletion and archival policies across our Unity Catalog-governed Delta tables.I've come across various approaches, s...

Phani1

Hi Databricks Team, Is there a standard Databricks cost estimation template(xl), sizing calculator, or TCO tool that allows us to provide the following inputs and derive an approximate monthly and annual platform cost:Source systems and their types (...

Phani1 · 03-09-2026

Hi All,We are planning a bulk migration of LUA Script / Exasol scripts to Databricks native PySpark and are evaluating the best approach for large-scale automated code conversion and testing in Databricks.So far, we have analyzed the following option...

Phani1 · 03-04-2026

Hi All,We are experiencing problem while we are querying the table which has collate unicode_ci attribute enabled, Kindly find the below findings and let me know if any alternative approach to fix this issue?Problem StatementThe LIKE operator in a q...

Phani1 · 09-24-2025

Hi Team,How can we migrate a Genie space to a higher environment while preserving its existing intelligence and learning, along with the catalog, to avoid recreating the same knowledge.Regards,Phani

Phani1

Thank you for sharing this — the plan is well structured and demonstrates a strong, governance‑first approach with clear execution and auditability.It provides a scalable, defensible roadmap aligned with Unity Catalog–native controls and enterprise c...

Phani1

Hi @LingaReddy,Thanks for taking the time to share such a comprehensive and well-structured breakdown — really appreciate the depth and clarity you’ve brought to this topic. The layer-wise retention model, clear separation between REORG vs. VACUUM, a...

Phani1

Thanks, Sumit, for the valuable insights. This makes complete sense — adopting the Delta purge lifecycle with metadata-driven automation and layer-wise retention will help keep the solution scalable, compliant, and audit-ready.

Phani1

Thank you!!

Phani1

Hi @Ashwin_DSA,Thanks for your response—this really helps validate our approach.We agree on the need to split SQL-heavy and LUA-heavy logic and are planning a “migration factory” with LLM-assisted conversion, followed by strong validation (parallel r...

Databricks Community

User Stats

User Activity

Best Practices for Implementing Automated, Scalable, and Auditable Purge Mechanism on Azure Databric

Databricks Cost Estimation Template

Seeking Best Approach for Bulk Migration of LUA/Exasol Scripts to Databricks PySpark

LIKE Operator Fails with COLLATE UNICODE_CI Columns

Preserving Intelligence and Learning during Genie Space Migration

Re: Best Practices for Implementing Automated, Scalable, and Auditable Purge Mechanism on Azure Data

Re: Best Practices for Implementing Automated, Scalable, and Auditable Purge Mechanism on Azure Data

Re: Best Practices for Implementing Automated, Scalable, and Auditable Purge Mechanism on Azure Data

Re: Databricks Cost Estimation Template

Re: Seeking Best Approach for Bulk Migration of LUA/Exasol Scripts to Databricks PySpark