Databricks Community

Having issue trying to download a csv file from a website using FireFox Selenium.

Hi, When I clicked on the download button from a website thru Firefox selenium using element.click(), and the download destination is being set as Azure datalake storage. Then, after the download started, those .csv and .csv.part files never gotten m...

Your guide to data and AI governance talks at the Data + AI Summit 2023

The highly anticipated Data+AI summit is just around the corner. Are you curious about redefining your data governance strategy in the era of LLMs and generative AI? Look no further! This year's Data+AI summit offers a dynamic lineup of keynotes, dem...

Parquet column cannot be converted. Column: [Rainfall_Value], Expected: DoubleType, Found: INT64

df.printSchema()root |-- Device_ID: string (nullable = true) |-- Location: string (nullable = true) |-- Latitude: double (nullable = true) |-- Longitude: double (nullable = true) |-- DateTime: timestamp (nullable = true) |-- Rainfall_Value: double (n...

StreamingQueryException: Read timed out // Reading from delta share'd dataset

I have a workspace in GCP that's reading from a delta-shared dataset hosted in S3. When trying to run a very basic DLT pipeline, I'm getting the below error. Any help would be awesome!Code:import dlt     @dlt.table def fn(): return (spark.readStr...

If we do z-order on a particular column will delta log stats collection be affected?

Let's assume a table contains more than 40 columns, now we know it automatically collects stat for the first 32 columns. If we run a z-order on a particular column(let's say column 1), then will the log file collect stats for all the 32 columns or wi...

Will consecutive delete insert affect z-ordering?

Let's say there is a delta table with a date field as its partition. In a table where condition, we delete all the rows according to the division. The data is currently being inserted into the same date field. If we do a z-order after inserting the d...

 Presenting top 3 members who contributed to Community last week between 11th June-17th June- ​ @Tyler Heflin​ @Werner Stinckens​  and @Bharathan K​  ...

 Presenting top 3 members who contributed to Community last week between 11th June-17th June- ​ @Tyler Heflin​ @Werner Stinckens​  and @Bharathan K​ We would like to express our gratitude for your participation and dedication in the Databricks Commun...

I need a code where i can generate Dynamically Generating Tables with DLT using SQL

Model Serving error - Java gateway process exited before sending its port number

Hello, I am trying to serve a model endpoint (using Databricks GUI) for a model that was successfully logged to the Model Registry. However, the endpoint creation failed with the following errors: Endpoint logs with error messagesEndpoint events with...

delta table

can we connect delta table of databricks from one workspace to another in different subscription and run vacuum command?

DLT started by SERVICE_UPGRADE

HelloI'm developing a dlt pipeline, configured in continuous mode.I'm still in dev mode, so I stop my pipeline when i'm not working on it.My problem is that the pipeline is frequently started by SERVICE_UPGRADE.example of message:'Update xxxxx starte...