After granting more permissions for users to the table I also see this in the log:

chown: invalid user: ‘null:spark-users’
chown: invalid user: ‘null:spark-users’
do_all_setup_for_username: iptables command took 0.0015606880187988281 seconds
do_all_setup_for_username: set_process_identity took 0.0005297660827636719 seconds
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/databricks/spark/python/pyspark/sql/connect/streaming/worker/foreach_batch_worker.py", line 154, in <module>
(sock_file, sock) = local_connect_and_auth(java_port, auth_secret)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/databricks/spark/python/pyspark/util.py", line 726, in local_connect_and_auth
raise PySparkRuntimeError(
pyspark.errors.exceptions.base.PySparkRuntimeError: [CANNOT_OPEN_SOCKET] Can not open socket: ["tried to connect to ('127.0.0.1', 45771), but an error occurred: [Errno 111] Connection refused"].
do_all_setup_for_username: iptables command took 0.0020797252655029297 seconds
do_all_setup_for_username: set_process_identity took 0.0008292198181152344 seconds
Streaming foreachBatch worker is starting with url unix:///databricks/sparkconnect/grpc.sock and root sessionId 8a167574-1d44-4777-9309-c455c7fceab2.
finished setting up the root session 8a167574-1d44-4777-9309-c455c7fceab2.