cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

amrim
by New Contributor III
  • 138 Views
  • 1 replies
  • 0 kudos

Dashboard "Export as HTML" fails for job-associated notebook dashboards (UI only)

Context:A notebook is scheduled to run as a Databricks Job. The notebook has an associated dashboard. When viewing the dashboard from the job run, exporting via "Export as HTML" fails.Error:When clicking "Export as HTML," the download fails and the f...

amrim_0-1782980409213.png amrim_1-1782980423193.png
  • 138 Views
  • 1 replies
  • 0 kudos
Latest Reply
Louis_Frolio
Databricks Employee
  • 0 kudos

Greetings @amrim , this is a well-scoped report, and your isolation table does most of the diagnostic work for us. Thanks for laying it out that way. The behavior you documented is consistent, and it points at the product, not your notebook or cluste...

  • 0 kudos
ChristianRRL
by Honored Contributor
  • 27 Views
  • 0 replies
  • 0 kudos

Lakeflow Connect SharePoint - 403 Error

Hi, I'd like some help clarifying how lakeflow connect pipelines work (or should work). I went through the steps to (1) set up a connection, and (2) setup the ingestion, but I keep running into the following error:Operation failed: "This request is n...

ChristianRRL_2-1784142904393.png ChristianRRL_1-1784142354143.png ChristianRRL_0-1784142213049.png
  • 27 Views
  • 0 replies
  • 0 kudos
shan-databricks
by Databricks Partner
  • 62 Views
  • 1 replies
  • 0 kudos

How to Deploy Lakehouse Federation Using DABs from the Dev Environment to the Stage Environment

We are using Lakehouse Federation to connect to a SQL Server database and are currently looking for a CI/CD deployment solution using DABs. Could you please advise on the available options for deploying from the Dev environment to the Stage environme...

  • 62 Views
  • 1 replies
  • 0 kudos
Latest Reply
iyashk-DB
Databricks Employee
  • 0 kudos

There's a bit of a split here between what DABs can and can't manage for Lakehouse Federation. The connection object itself, the one that holds the SQL Server host, port, and credentials, isn't a supported bundle resource type. DABs don't hav...

  • 0 kudos
Sainath368
by Contributor
  • 73 Views
  • 1 replies
  • 0 kudos

SCIM API shows only databricks-sql-access,but Workspace Access is still displayed (grayed out) in UI

Hi Team,I'm testing Databricks entitlement automation using the Workspace Groups SCIM API.I have an Entra ID group that is synchronized to the Databricks Account via SCIM. I'm using a Service Principal authentication to manage workspace-level entitle...

Sainath368_0-1784035197931.png Sainath368_1-1784035230093.png
  • 73 Views
  • 1 replies
  • 0 kudos
Latest Reply
iyashk-DB
Databricks Employee
  • 0 kudos

What you're seeing is expected, and it comes down to how "Workspace access" is granted right now versus how the other entitlements work. Right now, every workspace principal automatically inherits the "Workspace access" entitlement through im...

  • 0 kudos
Avinash_Turpu
by New Contributor
  • 228 Views
  • 1 replies
  • 0 kudos

Resolved! 50% discount voucher not received despite completing all 4 courses

Hi team, I completed all 4 required courses for the recent Learning Festival campaign (June 15 – July 6, 2026): - DevOps Essentials for Data Engineering - Build Data Pipelines with Lakeflow Spark Declarative Pipelines - Deploy Workloads with Lakeflow...

  • 228 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika
Community Manager
  • 0 kudos

Hello @Avinash_Turpu, Could you please send a DM to @Jim_Anderson with the email address associated with your Customer Academy account? That will help him verify your participation and assist you further.

  • 0 kudos
Taj2019
by New Contributor
  • 150 Views
  • 2 replies
  • 2 kudos

ALTER Not Working in Databricks for external Table.

Hi Team,i am getting error alter not suported . col1 DOUBLE NOT CONVERTING TO col1 DECIMAL(31,5) VIA ALTER STATEMENT.ERROR-liquibase.exception.DatabaseException: [Databricks][JDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL st...

  • 150 Views
  • 2 replies
  • 2 kudos
Latest Reply
ashukasma
New Contributor III
  • 2 kudos

Databricks Delta Lake doesn't support changing column data types directly via ALTER COLUMN, especially for type conversions like DOUBLE to DECIMAL.Based on the documentation, Databricks doesn't support arbitrary type changes via ALTER COLUMN (like DO...

  • 2 kudos
1 More Replies
emsmarcela
by New Contributor
  • 120 Views
  • 0 replies
  • 0 kudos

Genie Space and App Error

I built an App with a chatbot which generates queries and accesses data through a Genie Space. When making questions in the chatbot now, I keep getting the following error in the Genie monitoring page: An error happened. Please try again later.We're ...

  • 120 Views
  • 0 replies
  • 0 kudos
primopragma
by New Contributor II
  • 433 Views
  • 6 replies
  • 0 kudos

Getting 'Unauthorized Access' when Serverless compute is trying to read s3 bucket data

Getting 'Unauthorized Access' when Serverless compute is trying to read s3 bucket data, same works fine with normal compute storage and external location, credentials, IAM policy and serverless egress rules all seems to be correct because normal comp...

  • 433 Views
  • 6 replies
  • 0 kudos
Latest Reply
primopragma
New Contributor II
  • 0 kudos

Just for the record, this still does not work. I spent some more time trying to debug this with Genie as well, and we landed on the conclusion that we can't do anything else on the Premium plan at least. If I had the Enterprise Plan, I could have tri...

  • 0 kudos
5 More Replies
yit337
by Contributor
  • 96 Views
  • 1 replies
  • 0 kudos

What timezone is the “timestamp” value on a table's history?

Is it UTC, or the local zone of whoever accesses it?

  • 96 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 0 kudos

Hi  @yit337 , TIMESTAMP values are internally normalized and persisted in UTC, but they are displayed using the current session’s local timezone.Therefore, it is not based on the timezone of the person who originally performed the write. It depends o...

  • 0 kudos
APJESK
by Contributor
  • 130 Views
  • 1 replies
  • 0 kudos

Clarification on Auto Loader Managed File Events with Unity Catalog Managed Volumes

Hi Databricks Team,I'm trying to understand how Auto Loader Managed File Events work with Unity Catalog Managed Volumes, and I'm looking for some clarification.My understandingI create an External Location: CREATE EXTERNAL LOCATION app_ext_locURL 's3...

  • 130 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ashwin_DSA
Databricks Employee
  • 0 kudos

Hi @APJESK, I am wondering if the main source of confusion is the name cloudFiles.useManagedFileEvents. In the public docs, "managed" here refers to the Databricks-managed file-events service and cache layer, not to Unity Catalog managed volumes or t...

  • 0 kudos
None123
by New Contributor III
  • 13283 Views
  • 4 replies
  • 3 kudos

Open a Support Ticket

Anyone know how to submit a support ticket? I keep getting into a loop that takes me back to the community page, but I need to submit an urgent ticket. I'm told our company pays a ridiculous sum for this feature yet it is impossible to find.Thanks ...

  • 13283 Views
  • 4 replies
  • 3 kudos
Latest Reply
vermyas
New Contributor
  • 3 kudos

Hello Databricks Support Team,We are currently using:databricks-sql-connector==4.3.0thrift==0.22.0Our organization's security scanning tool (Prisma) is flagging Thrift 0.22.0 and requires us to upgrade to a newer version (0.23.0 or later).As part of ...

  • 3 kudos
3 More Replies
theanhdo
by New Contributor III
  • 135 Views
  • 0 replies
  • 0 kudos

Trunked-based development model for Databricks question

Hi there,My name is William Do (a senior engineer at Hub24, Australia). I am writing to seek for your advice on a Trunk-based development model that we are hoping to implement for our Databricks project.Problem statementTo give you some context, I am...

  • 135 Views
  • 0 replies
  • 0 kudos
animeshjain
by New Contributor II
  • 377 Views
  • 4 replies
  • 0 kudos

Bundle Validation Error: Volume lifecycle Field Rejected Despite Being in Schema

Environment:Bundle engine: directResource type: Volume (Unity Catalog)Problem:I'm trying to use the lifecycle field with prevent_destroy: true on a Unity Catalog volume resource in my Declarative Automation Bundle (DAB). the validation fails with:Err...

animeshjain_0-1783340968236.png animeshjain_1-1783340987403.png
  • 377 Views
  • 4 replies
  • 0 kudos
Latest Reply
balajij8
Contributor III
  • 0 kudos

Hi, The configuration is valid and the CLI version supports the lifecycle field. However, the error is most likely coming from outdated Databricks Python packages in your virtual environment at /home/animesh/datazone/.venv/. CLI uses the packages fro...

  • 0 kudos
3 More Replies
JD18
by New Contributor
  • 2286 Views
  • 3 replies
  • 2 kudos

SCD-2 backfilling with streaming tabels

Hi there,Im new to Databricks and trying to build a SCD2 type table using AUTO CDC approach. while it quite simple to create a scd2 table Im unable to do a backfill.Full context.I have raw data(order, customer info) from 2019 and creating a dimension...

  • 2286 Views
  • 3 replies
  • 2 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 2 kudos

Hi @JD18, Welcome to Databricks and Thanks for raising this. SCD Type 2 backfilling with streaming tables is a common need, and the good news is that the AUTO CDC framework (formerly APPLY CHANGES INTO) has built-in capabilities to handle this -- you...

  • 2 kudos
2 More Replies
stravi001
by New Contributor II
  • 461 Views
  • 5 replies
  • 1 kudos

Resolved! Permissions on job with SQL task

Hello,I am authenticated as GCP service account. I have to create query, run job as service principle and get result.I'm both cases, I set permissions CAN_MANAGE for group "users" for both query and job. When I create the query with RunAsMode.OWNER, ...

  • 461 Views
  • 5 replies
  • 1 kudos
Latest Reply
stravi001
New Contributor II
  • 1 kudos

Hello @balajij8,I tried to use SQL File Task and was able to achieve the result. Only issue was, that I was not able to get output using WorkspaceClient.jobs.get_run_output(). It worked will direct call of /api/2.2/jobs/runs/get-output endpoint.Thank...

  • 1 kudos
4 More Replies
Labels