vimeo.com

- - Certifications
- - Learning Paths

- - Community Discussions
- - GenAI Insight Hub
- - Get Started Discussions

- - Get Started Resources
- - Events
- - Product Platform Updates
- - Support FAQs
- - Technical Blog
- - What's New in Databricks
- - Get Started Guides
- - Knowledge Sharing Hub
- - Announcements
- - Databricks TV

- - Technical Councils
- - Private Groups
- - Skills@Scale

- - Databricks Community Champions
- - Khoros Community Forums Support (Not for Databricks Product Questions)
- - Databricks Community Code of Conduct
- - Community Newsletter

Data Engineering

Auto Loader provides Python and Scala methods to ingest new data from a folder location into a Delta Lake table by using directory listing or file notifications. Here's a quick video (7:00) on how to use Auto Loader for Databricks on AWS with Databricks Runtime (DBR) 8.3 or higher.

To follow along with the video, import this notebook into your workspace:

https://files.training.databricks.com/lms/aws-onboarding-incremental-etl/incremental-etl.dbc

For more on Auto Loader, visit the docs. Question and ideas? Leave a comment below!