I have a simple SparkSQL Select statement

offers_df = (spark.sql("""
SELECT *
FROM delta.`{}`
""".format(TABLE_LOCATION)))

It runs under DBR 9.0 and previous.

When changing the DBR to 9.1 ...10.1

I get an exception

org.apache.spark.SparkException: Unable to fetch tables of db delta at org.apache.spark.sql.errors.QueryExecutionErrors$.cannotFetchTablesOfDatabaseError(QueryExecutionErrors.scala:1353)

spark_conf of the cluster

    "spark_conf": {
        "spark.hadoop.fs.s3a.canned.acl": "BucketOwnerFullControl",
        "spark.databricks.hive.metastore.glueCatalog.enabled": "true",
        "spark.hadoop.fs.s3a.acl.default": "BucketOwnerFullControl"
    },

when removing the glueCatalog entry, the Select statement works.

Question:

How can the AWS Glue Catalogue be used in DBR 9.1 .. 10.1 ?

How to configure it in spark_conf?