Databricks Community

We have a VNET Injected workspace in Azure. There are multiple SQL Warehouse which are easily accessible from Private Network - both directly from VM and via VPN on client's machine. We deployed Lakebase. Inside the Workspace, connectivity is working...

I need to track codes of records that were ingested in foreachBatch function, and pass it as a task value, so downstream tasks can take actions based on this output. What would be the best approach to achieve that? Now, I have a following solution, b...

We have structured streaming that reads from external delta table defined in following way: try: df_silver = ( spark.readStream .format("delta") .option("skipChangeCommits", True) .table(src_location) ...