Resolved! slow Fetching results by client in databricks SQL calling from Azure Compute Instance (AML)
I'm using `databricks-sql-connector` in python3.8 to connect to an Azure SQL Wharehouse inside an Azure Machine Learning Compute Instance.I have this large result query, looking at the `query history` I check the time spent on doing the query, and se...
- 5729 Views
- 4 replies
- 2 kudos
Latest Reply
So I made some few tests. Since you said that the Databricks SQL driver wasn't made to retrieve that amount of data. I went on Spark.I fired up a small spark cluster, the query was as fast as on SQL Warehouse, then I did a df.write.parquet("/my_path/...
- 2 kudos