Databricks Community

The most expensive assumption in a streaming architecture is often the assumption that streaming means always-on compute.Very good point. I think the distinction between streaming data and always-on compute is one of the most important architectural ...

 Totalmente de acuerdo. En muchos entornos, el costo de los SQL Warehouses puede crecer rápidamente, pero normalmente no se trata solamente de “usar un Warehouse más pequeño”. Yo revisaría el problema en varias capas:Optimizar las queries primeroAnte...

Hi @srikanthp24 Yes, the solution mentioned by @Ashwin_DSA is the mechanism currently available is either with DABs (preferred)/ CLI based as the UI doesn't yet allow configuring notifications at pipeline level.One caveat is that, even after configur...

I’m running into the same kind of issue with projects that already have their own environment setup. Being able to manually select the Python environment/kernel without the extension forcing uv changes would make Connect much more flexible.

@Marbricks Follow this approach: define the function’s interface first, then connect the real data. Use a temporary function for testing within the same session, and a persistent Unity Catalog function in a development schema for testing with the ext...

Exactly Similar situtation , i have crated my own using Pythopn , VetcorDB and Knowledge graph, you can simulate same or Hire me , i can able to do it for you LOL..See this Video on my Linked In

Hello @TyreseR19, Could you please ensure that both required courses for the Data Analyst learning path were fully completed during the Learning Festival event window? Please verify that every component of each course is marked as completed, includin...

Most modern enterprise data platforms favor managed ingestion (Lakeflow Connect) for standard relational databases like SQL Server. The operational complexity of monitoring 50 managed pipelines is significantly lower than maintaining custom CDC code,...

did you ever figure this out, looking for those specific notebooks myself? I'm not sure if this is helpful, but in the Databricks free edition home page, there is a Databricks fundamentals course, which covers similar things (it is not exactly the sa...

Try this link https://community.databricks.com/t5/data-engineering/how-to-calculate-cost-of-each-table-for-the-specific-databricks/td-p/167086 

@gowri_databrick Parquet's columnar storage organizes data by column rather than by row - all values for order_date are stored together, all order_amount values are stored together and so on. In your commerce scenario with 500 million records, when t...