โ03-17-2024 09:17 PM
Is anyone able to advise why I am getting the error not a delta table? The table was created in Unity Catalog. I've also tried DeltaTable.forName and also using 13.3 LTS and 14.3 LTS clusters. Any advice would be much appreciated
โ03-19-2024 07:51 AM - edited โ03-19-2024 07:52 AM
@StogponI believe if you are using DeltaTable.forPath then you have to pass the path where the table is. You can get this path from the Catalog. It is available in the details tab of the table.
Example:
delta_table_path = "dbfs:/user/hive/warehouse/xyz"
# Read the Delta table using forPath
deltaTable = DeltaTable.forPath(spark, delta_table_path)
The other way would be to use would be DeltaTable.forName(spark, 'tablename')
Hopefully this helps.
โ03-17-2024 10:31 PM
Hi @Stogpon, Could you please paste the error stack here?
โ03-18-2024 04:52 AM
@Kaniz_Fatma the attached screen was using 14.3 LTS cluster. Thanks
โ03-19-2024 07:51 AM - edited โ03-19-2024 07:52 AM
@StogponI believe if you are using DeltaTable.forPath then you have to pass the path where the table is. You can get this path from the Catalog. It is available in the details tab of the table.
Example:
delta_table_path = "dbfs:/user/hive/warehouse/xyz"
# Read the Delta table using forPath
deltaTable = DeltaTable.forPath(spark, delta_table_path)
The other way would be to use would be DeltaTable.forName(spark, 'tablename')
Hopefully this helps.
โ03-19-2024 08:22 AM
@addy thanks for your reply. I tried with forName and it's now working. I had tried this both forName and forPath previously with the same error so I'm not sure why it's working now. I can't recall what has changed since whilst I was experimenting/diagnosing. If I experience it again I'll have to take closer notes of the steps I took to reach that step. Anyhow thanks for your help, it's much appreciated.
Excited to expand your horizons with us? Click here to Register and begin your journey to success!
Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!