Hello, I'm having this issue with job-computes:

The snippet of the code is as follows:

     84 if self.conf["persist_to_sql"]:
     85     # persist to sql
     86     df_parsed.write.format(
     87         "com.microsoft.sqlserver.jdbc.spark"
     88     ).option("url", self.get_sql_connection_string()).option(
     89         "dbtable", self.conf["sql_tables"]["report"]
     90     ).option(
     91         "driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver"
     92     ).mode(
     93         "append"
---> 94     ).save()
     95     self.logger.info(
     96         f"Saved to {self.conf['sql_tables']['report']}"
     97     )

 How can I install the driver to this job-compute?

This is the driver I'm want to install: spark-mssql-connector_2.12-1.4.0-BETA.jar

I tried:

• Upload to the workspace shared folder an init script that installs the driver (didn't work)
• Indicate to the databricks.yml configuration to install the library at the cluster definition from maven

Thanks