Hi All,
I am preparing for the Data Engineering Professional Certification. I can't find the documentation for the following topics.
- Configure alerting and storage to monitor and log production jobs, including:
- Setting up notifications
- Configuring SparkListener
- Recording logged metrics
- Navigating and interpreting the Spark UI
- Debugging errors
- Follow best practices for managing, testing, and deploying code, including:
- Managing dependencies
- Creating unit tests
- Creating integration tests
- Scheduling Jobs
- Versioning code/notebooks
- Orchestration Jobs
Any help is appreciated.
I am reading the
Advanced-Data Engineering with Databricks from Databricks academy and the notebooks associated with that course.
Thank you!