Without streaming everything works fine. When I try to set up some example structured streaming in my company's new Databricks workspace (we're trying it out) I can't seem to figure out the correct way to save it/write to storage.
Please see the picture below. I don't think my code is correct. I tried to adapt some code I found. If I include the .table line (in the writeStream) then I get the error message as shown. The error message is so unclear I have no idea what I'm supposed to do. If I comment the .table line, then it will initialize the stream, create the directory in adls gen2 storage but never complete, just stuck at "Initializing stream". I do want the .table line though so I can specify catalog, schema and table name of the table.
We are using Unity Catalog and it works fine for normal non-streaming workloads, reading from the same files as this stream does and writing to the same storage accounts+containers.
