Read CSV file using SQL
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-20-2023 07:49 AM
Hi, I am trying to reverse engineer to get to the source file for a table. Looking at the query history, I came across SQL string which loads data from file to table, however the code looks little mystery to me. I haven't come across idbfs, Can somebody help me understanding.
COPY INTO 'tablename_temp' FROM (
SELECT
try_cast(`_type` AS string) `_type`,
try_cast(`ID` AS double) `ID`,
try_cast(`Number` AS string) `Number`,
try_cast(`Name` AS string) `Name`,
try_cast(`Type` AS string) `Type`
FROM
'idbfs:/2023-03-29/10/_fe93bfcf-4ad6-4e14-b2d7-9811ddbc0c7d'
) FILEFORMAT = CSV FORMAT_OPTIONS (
'inferSchema' = 'true',
'multiLine' = 'false',
'header' = 'true',
'delimiter' = ',',
'escape' = '"',
'mergeSchema' = 'true',
'recursiveFileLookup' = 'true'
) COPY_OPTIONS ('overwrite' = 'true', 'overwriteSchema' = 'true')
)
- Labels:
-
CSV File
-
Databricks SQL
-
SQL
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-21-2023 12:41 AM
Hi Kaniz,
Thanks for your helpful reply. I guessed the same. I have never come across idbfs file system. Is this something new with Databricks ? Any reference would be helpful to get to the source.

