I wrote simple code: from pyspark.sql import SparkSession
from pyspark.sql.window import Window
from pyspark.sql.functions import row_number, max
import pyspark.sql.functions as F
streaming_data = ...
Getting this error in dbt when trying to run a query. Not happening in the actual SQL warehouse in Databricks. Is this a bug? Can only find source code when I search 'DatabricksSQLCursorWrapper' but ...
Hello Databricks Community, I have a hard time understanding how is Databricks SQL different from microsoft SQL ? Also, why does databricks provide spark SQL ? If you direct me to a well-writ...
Hi Community! I have the requirement that business users shall be able to edit/update tables in Unity Catalog, e.g. master data records, mapping tables. I also want thes actions to be logged for aud...
Hi, I cannot see the query execution time in the response to the "api/2.0/sql/history/queries" request. Basically, I get only the following fields: { "next_page_token":..., "has_next_page":...
Hi, am sure am missing something as this should be something trivial but am struggling to find how to add a suffix with a date to a table name. Does anyone have a way to do this? Thanks
Hi, we have implemented a Databricks Workflow that saves an Excel Sheet to a Databricks Volume. Now we want to notify users with an Alert, when new data arrives in the volume. In the docs I found t...
Hello, I am running spark structured streaming, reading from one table table_1, do some aggregation and then write results to another table. table_1 is partitioned by ["datehour", "customerID"] My ...
Hi, Is it possible to convert existing delta table with partition having data to clustering? If so can you please suggest the steps required? I tried and searched but couldn't find any. Is it that l...