Autoloader with SQLite db files
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-29-2023 11:45 PM
Hi Everyone,
Is it possible to ingest SQLite db files with Databricks Autoloader? Is it currently supported? If so, could you please share an example?
- Labels:
-
Delta Lake
-
Spark
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-05-2024 04:23 PM
Hello @eva_mcmf , I hope this message finds you well.
As per the documentation, Auto Loader incrementally and efficiently processes new data files as they arrive in cloud storage. Auto Loader can load data files from AWS S3, Azure Data Lake Storage Gen2, Google Cloud Storage, Azure Blob Storage, ADLS Gen1, and Databricks File System. Auto Loader can ingest JSON
, CSV
, XML
, PARQUET
, AVRO
, ORC
, TEXT
, and BINARYFILE
file formats.
Currently I couldn't find this possibility of directly ingesting from SQLite with Autoloader through the documentation.
How does Auto Loader work: https://docs.databricks.com/en/ingestion/auto-loader/index.html#how-does-auto-loader-work
If you believe that ingesting SQLite db files with Auto Loader would be beneficial, you can submit an idea in Databricks' Ideas Portal.
Best regards,
Lucas Rocha

