data is not written back to data lake
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-18-2023 12:14 AM
I have this strange case where data is not written back to data lake.
I have 3 container- . Bronze, Silver and Gold. I have done the mounting and have not problem to read the source data and write it Bronze layer ( using hive meta store catalog).
Then I pick the data from bronze for cleanisng and creating Silver tables and writing it to Silver. In data brick I can query this silver tables but when I check the silver container in azure portal it empty.
Below is the location i use and I have checked multiple times if I use correct storage name and its right one.
LOCATION 'dbfs:/mnt/silver@{{ table.STORAGE_ACCOUNT_NAME }}.dfs.core.windows.net/{{ table.TABLE_SOURCE }}/{{ table.TABLE_NAME }}';
- Labels:
-
Data
-
Datalake
-
Silver Tables
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-18-2023 08:12 AM
If I understood you description well, it sounds as if you are actually writing the data to dbfs and not to the external location when creating the table and writing to silver. Are you sure you are defining the external path on table creation? For instance have a look here:
https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-external-tables

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-23-2023 09:34 PM
Hi @Givi Salu
Hope everything is going great.
Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you.
Cheers!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-02-2023 03:42 AM
no I have not

