Databricks Community

Hello @i4ata , Following up on the answer above by @balajij8  with a test run of your script and one detail on where the lookup happens. Thank you for posting a complete script with its log output. I could rerun it almost as written. Short answer: I ...

@NinaThoni Thank you for your response.I believe the best approach would be to implement a custom primary key reconciliation mechanism and optimize its performance by limiting the comparison to the last two months of data. In our case, older data can...

Hi @dans-vander  By default Databricks Serverless generic compute cannot reach any of private Azure resources. To enable connectivity a private endpoint is needed. Unfortunately setup does not have unlimited private endpoints that can be used with th...

Last chance to lock in your spot! Don't miss out on tomorrow's BrickTalk! If you're looking to streamline your Change Data Capture (CDC) ingestion through Debezium to Zerobus Ingest and bypass intermediate streaming systems like Kafka, this is the s...

Hi @xavi-albert it is possible that users group at account level has been granted account admin entitlement switched on. This looks like a manual intervention. Otherwise grant this account admin entitlement to few chosen one users.below is example fr...

For the Postgres-to-Delta direction, Lakebase Change Data Feed captures every insert, update, and delete on a Lakebase Postgres table and appends each change to a Unity Catalog managed Delta table named lb_<table_name>_history, flushed about every 15...

If your Lakebase connection string uses a regional host (*.database.<region>.azuredatabricks.net), Postgres clients connecting from outside the workspace need inbound Private Link for performance-intensive services on port 5432 alongside the workspac...

Reading the PDFs with the binaryFile format returns a fixed schema (path, modificationTime, length, content) with content already typed as BINARY, so there's no column type to set (binary file data source, read_files). You can run this in the SQL edi...

Your tests show that DNS64/NAT64-addressed connections use the gateway, while a raw IPv4 connection can use Databricks’ default egress. That behavior is not described as an exception in the current docs. They say ALL_TRAFFIC routes all outbound traff...

Hi,Both checks timing out, the control plane (centralindia.azuredatabricks.net) and www.databricks.com, usually means the cluster subnets have no outbound path at all, not a problem with one specific endpoint. The error codes doc says the same: when ...