I have a sqlite database that I want to read into databricks to create delta tables/dataframes in Python that I can export to power BI and have a live connection. When there is new data added to my sqlite data base, the changes will need to reflect in my Power BI.

My sqlite file is uploaded to an s3 bucket. I was able to read the file in by creating a temporary local path and then using the sqlite3.connect(local_path) to connect. I then used pandas to read the sql query to create my pandas dataframe, then output my dataframe to my s3 bucket as a csv file. Once this was output to my s3 bucket, I could then manually create a table from that output file using UI in databricks. I do know how to connect to power BI through spark and was able to get my tables in there, so I don't need help on that part.

However, this does not give me a live connection. How can I read the sqlite file from my s3 bucket into databricks and create delta tables/dataframes? Thank you in advance.