Databricks Community

krishgarikipati

Hi Databricks Community,I am working on a data lakehouse architecture on AWS, where HVR captures changes from OLTP databases such as Oracle and PostgreSQL, AWS S3 is used for storage, and Databricks handles data processing across Bronze, Silver, and ...

krishgarikipati

Hi @Khasim_1 Maintenance operations: OPTIMIZE and VACUUMThe frequency of maintenance operations should depend on the table’s ingestion pattern, update/delete volume, file sizes, query performance, and storage cost. Avoid running them on a fixed sch...

krishgarikipati

Thanks for sharing this article! I particularly like the point that separating transformation logic, persisting intermediate results, and exposing datasets to consumers are three different architectural decisions.In our environment, we use Databricks...

krishgarikipati · 09-24-2026

Hi @AdamLucas I had recent attempt, Which has more focused on SDP(Spark Declarative Pipelines) related questions and Data Governance. Pls have proper roadmap with all these topics and make the attempt.Wish you very all the best on your certification ...

krishgarikipati · 09-24-2026

Hi @yit337 This is already mentioned in the Databricks documentation. PFB snippet for the reference. https://docs.databricks.com/gcp/en/ldp/properties?utm_source=chatgpt.com

krishgarikipati · 08-24-2026

Thanks for the overview. Since you are using auto loader for files ingestion,Can we say is this pipeline brings real time data or near real time data feed to the ML model?

Databricks Community

User Stats

User Activity

Best practices for processing HVR CDC data in Databricks when handling out-of-order updates, delete

Re: Best practices and optimization tips for Delta Lake tables

Re: Beyond Bronze, Silver, and Gold: When Does a Separate Stage Make Sense?

Re: Which Topics Deserve More Attention Before the Databricks Data Engineer Associate Certification

Re: Is Advanced edition required for Serverless pipeline?

Re: End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and Databricks