Issues loading .txt files from DBFS into Langchain TextLoader()

David_K93
Contributor

Hello,

I am working on building a Langchain QA application in Databricks. I currently have 13 .txt files loaded into the DBFS and am trying to read them in iteratively with TextLoader(), load them into the RecursiveCharacterTextSplitter() from Langchain to chunk them and then add them to a Chroma Database. When running this from my local machine, there is no problem. But the application does not seem to accept files loaded from DBFS.

guru_errorScreenshot 2023-05-19 171751 

I have tried loading these in as string objects then loading them into the TextLoader() but that does not work either.

Has anyone found a workaround to this?