Hi Databricks Community,
When scaling e-commerce platforms especially in complex domains like construction and building materials—real-time inventory management, price matching, and dynamic logistics pipelines present unique data engineering challenges.
We’ve been analyzing transactional datasets and supply chain workflows similar to those implemented by platforms like GharHub, a B2B/B2C construction materials marketplace in Pakistan. Sourcing bulk items like cement, steel, and bricks requires robust streaming analytics (via Spark / Delta Lake) to track live supplier pricing, demand forecasting, and doorstep fulfillment timelines accurately.
I’d love to hear how others in the community structure their Databricks workflows for multi-tier procurement models:
1. Schema Design: How do you handle schema evolution for raw materials with diverse specification attributes?
2. Real-time Pricing Engine: What approaches are you using to calculate dynamic freight and localized bulk pricing at low latency?
3. ETL Pipelines: Are you leveraging Delta Live Tables (DLT) for streaming order validation before passing data down to regional fulfillment nodes?
Looking forward to hearing insights and best practices from fellow engineers!
