Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a week ago
Hi,
You can use Databricks SQL Connector for the activities instead of Databricks Connect if you require simple setup & easier management (session etc) with features below
- SQL queries (SELECT, INSERT, UPDATE)
- Executing parameterized SQL statements
- Fetching data and doing transformations in Python after retrieval
More details here
Apache Spark generally requires you to explicitly declare that they are complete by using commands such as sys.exit() or sc.stop(). Databricks automatically terminates and cleans up jobs as they reach completion, so these commands are not necessary and should be removed. The automatic cleanup occurs when the request completes.