Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-16-2026 09:11 AM - edited 01-16-2026 09:12 AM
Hi @ChristianRRL ,
No, autoloader is propriety to Databricks. It's not open sourced. Open source version of SDP uses spark structured streaming for incremental processing.
Keep in mind that Auto Loader is basically just Spark streaming under the hood with additional features for event-driven ingestion (and some other things).
Schema evolution is property of Delta protocol which is open sourced. Also spark can natively infer schema for various sources, this is not something that is unique to auto loader.
InferSchema & Schema Enforcement in Spark | by Omkar Patil | Medium