Databricks Community

where can I review the exam

"Detected schema change" error while reading from delta table in streaming after applying "ALTER COLUMN DROP NOT NULL" to more than one columns.

Hi!I have a delta table and a process that reading a stream from this table.I need to drop the NOT NULL constraint from some of the columns of this table.The first drop command does not affect the reading stream.But the second command results in erro...

Certificate not received -

Hi ThereI successfully passed the exam on 27th Oct 2022 - but I haven't received the certificate yet.

I've successfully passed Databricks Data Engineer Associate Certified exam but still have not received the certificate. Could you help on it pleas...

I've successfully passed Databricks Data Engineer Associate Certified exam but still have not received the certificate. Could you help on it please.

Configure DLT Pipeline to use existing running cluster

How can I configure a DLT pipeline to use an existing running cluster? I don't see where in the settings to set the pipeline to use an existing cluster. Instead it wants to always standup a new cluster.

Cluster not following the zone: auto configuration

Hi, I'm trying some new instance types, so to be sure I will get one I set the Availability zone to 'auto'.But this is happening, the image of the error and JSON are attached.So, the cluster is trying three times to upscale and the three times is usi...

Time Parameter Returns Incorrect Time

For some reason, the parameter keeps showing me my local time in UTC. When converting this parameter from EST, the offset is wrong. In the screenshot, you can see that my local session TZ is set to ETC/UTC (for whatever reason). It looks like the par...

databricks learning Full screen disabled in selfpaced course

databricks learning self paced course Databricks Certified Associate Developer for Apache Spark 3 has below issues 1) Full screen button is disabled, difficult to see small font and stress on eyes.PFA2) courses do not have captions. Sometimes it is d...

bug: add csv data UI: missing leading zero

use add data UI, add csv manually, even set data type as string, the leading zero will be missingexample csvval1,val20012345, abcafter load data, 123,abc is stored in table

Connection from on-premise R sparklyr session to Databricks, " invalid method toDF for object 17/org.apache.spark.sql.DataFrameReader fields 0 selected 0"

Hello,I'm working with an on-premise R session and would like to connect to Databricks using sparklyr. RStudio server in this case is not an option.I would like to use jdbc. I tested RJDBC + DBI and can conect locally and perform operations. However,...

10.4LTS has outdated snowflake spark connector, how to force latest snowflake spark connector

Hi,I am trying to run my code from a scala fatjar on azure-databricks, which connects to snowflake for the data.I usually run my jar on 9.1 LTS.However when I run on 10.4 LTS the performace was 4x degraded and in the log it says WARN SnowflakeConnect...

Databricks Lakehouse Fundamentals Accreditation V2 Badge/Certificate Date Not Updated

I had appeared for Databricks Lakehouse Fundamentals Accreditation for both V1 and V2. Recently I came to know that when you take V1 test, and you get a badge and certificate - once you take V2 test, it updates that same badge and certificate to the ...

GET_COLUMNS fails with Unexpected character (\\'t\\' (code 116)): was expecting comma to separate Object entries - how to fix?

I just run `cursor.columns()` via the python client and I'll get back a `org.apache.hive.service.cli.HiveSQLException` as response. There is also a long stack trace, I'll just paste the last bit because it might be illuminating: org.apache.spark.sql....