balajij8
Contributor III

Hi, 

You can use Databricks SQL Connector for the activities instead of Databricks Connect if you require simple setup & easier management (session etc) with features below

  • SQL queries (SELECT, INSERT, UPDATE)
  • Executing parameterized SQL statements
  • Fetching data and doing transformations in Python after retrieval

More details here

Apache Spark generally requires you to explicitly declare that they are complete by using commands such as sys.exit() or sc.stop(). Databricks automatically terminates and cleans up jobs as they reach completion, so these commands are not necessary and should be removed. The automatic cleanup occurs when the request completes.

View solution in original post