Hello,
Im trying to write to databricks managed icerberg table using pyiceberg inside a spark_python_task (Serverless compute).
Im facing an error when writing :
Error writing to Iceberg table: When reading information for key '' in bucket '' : AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 35, SSL connect error; Details: Recv failure: Connection reset by peer.
Im running this code :
catalog_config = {
"type": "rest",
"uri": (
f"https://{workspace_url}/api/2.1/unity-catalog/iceberg-rest"
),
"warehouse": catalog_name,
"token": databricks_pat,
}
catalog = load_catalog("catalog_name", **catalog_config)
table = catalog.load_table(table_identifier)
table.append(df)
I want to know if there is a special uri to use because im inside databricks compute, or if its just not possible.
This code run find outside databricks.
Thank you