I leverage Databricks across the full data lifecycle — from transform to advanced analytics. Here's a quick breakdown:
Data Transformation:
Perform ETL pipelines with Delta Lake and pyspark, ensuring scalable, ACID-compliant transformations.Currently working on RAG pipelines using Databricks Model Serving, Feature Store, and Unity Catalog for secured access to embeddings.
Created Gold Layer views in Unity Catalog, consumed by Power BI dashboards.
Used Unity Catalog for fine-grained access control across tables, views, and functions.
Enabled Photon runtimes, Z-Ordering, and Delta Caching for cost-effective performance.
Collaborate via Notebooks, Git integration, and Repos, improving team productivity in cross-functional projects.
