Read CSV file using SQL

Rami2023
New Contributor II

Hi, I am trying to reverse engineer to get to the source file for a table. Looking at the query history, I came across SQL string which loads data from file to table, however the code looks little mystery to me. I haven't come across idbfs, Can somebody help me understanding.

COPY INTO 'tablename_temp' FROM (

SELECT

   try_cast(`_type` AS string) `_type`,

   try_cast(`ID` AS double) `ID`,

   try_cast(`Number` AS string) `Number`,

   try_cast(`Name` AS string) `Name`,

   try_cast(`Type` AS string) `Type`

FROM

   'idbfs:/2023-03-29/10/_fe93bfcf-4ad6-4e14-b2d7-9811ddbc0c7d'

 ) FILEFORMAT = CSV FORMAT_OPTIONS (

  'inferSchema' = 'true',

  'multiLine' = 'false',

  'header' = 'true',

  'delimiter' = ',',

  'escape' = '"',

  'mergeSchema' = 'true',

  'recursiveFileLookup' = 'true'

 ) COPY_OPTIONS ('overwrite' = 'true', 'overwriteSchema' = 'true')

)

Hi Kaniz,

Thanks for your helpful reply. I guessed the same. I have never come across idbfs file system. Is this something new with Databricks ? Any reference would be helpful to get to the source.