How to add a column to a new table containing the original source filenames in DataBricks.

joeyslaptop
New Contributor II

If this isn't the right spot to post this, please move it or refer me to the right area.

I recently learned about the "_metadata.file_name".  It's not quite what I need.

I'm creating a new table in DataBricks and want to add a USR_File_Name column containing the filenames of the uploaded/imported files.  For example, I'm loading a table with a bunch of files with a naming scheme like "HT00114_DXLS_PROD_20240102_00001.txt".  I need to keep the filename as a field value in my table. 

I also want to create a USR_File_Create_Date column where I add the '20240102' from the filename as a value.

Can anyone direct me to the right info for this?

“Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.” — Albert Einstein