Databricks Platform Discussions
Dive into comprehensive discussions covering various aspects of the Databricks platform. Join the co...
16275 Posts
Activity in Discussions
- 336 Views
- 4 replies
- 1 kudos
Resolved! Uploading file to volume and start ingestion job
Hello Community!I am writing to you with my idea about data ingestion job which we have to implement in our project.The data which we have are in CSV file format and depending on the case it differs a little bit. Before uploading we pivoting csv file...
- 336 Views
- 4 replies
- 1 kudos
- 11 Views
- 0 replies
- 0 kudos
Job tasks monitoring
Hello Community,We have a case in our project that we would like to solve in an elegant and scalable manner. As always, I would really appreciate your suggestions and experience.In short:We have a multi-step job consisting of 4 stages. In one of the ...
- 11 Views
- 0 replies
- 0 kudos
- 1096 Views
- 7 replies
- 1 kudos
Resolved! How should I correctly extract the full table name from request_params in audit logs?
’m trying to build a UC usage/refresh tracking table for every workspace. For each workspace, I want to know how many times a UC table was refreshed or accessed each month. To do this, I’m reading the Databricks audit logs and I need to extract only ...
- 1096 Views
- 7 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Danish11052000, You are on the right track with the COALESCE approach. The reason for the inconsistency is that different Unity Catalog action types populate different keys in request_params. Here is a breakdown of the key fields and which action...
- 1 kudos
- 20 Views
- 1 replies
- 0 kudos
Schema Evolution and Schema Enforcement without Delta live Tables & Unity catalog
In Delta Lake, schema evolution with mergeSchema handles column additions perfectly — new columns get added and old rows get NULL. But when there is a data type change in the incoming data (for example, a column that was INT now coming as STRING from...
- 20 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Here are the answers to your questions: Is Delta’s restriction a design decision?Yes. In Delta, mergeSchema is mainly for schema evolution by adding columns; type changes are still controlled by schema enforcement unless the change qualifies for typ...
- 0 kudos
- 20 Views
- 0 replies
- 0 kudos
Building a Spark Declarative Pipeline OSS with Apache Iceberg and AWS Glue Catalog
Hey everyone,I recently worked on building a modern financial data lakehouse using Spark Declarative Pipeline OSS (SDP OSS), Apache Iceberg, and AWS Glue Catalog.The blog covers:- Building declarative data pipelines with Spark- Using Apache Iceberg a...
- 20 Views
- 0 replies
- 0 kudos
- 83 Views
- 2 replies
- 2 kudos
certificação diferente entre países ?
Eu estava pesquisando certificações e entrei por um link em Inglês quando selecionei português observei que mudou o escudo, de 4 estrelas em inglês foi para 2 estrelas em português brasil.Isso tem algo a ver com a importância, dificuldade da certific...
- 83 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Thanks for looping me in @WiliamRosa. Olá@tabcs, Obrigado por referires isto! O crachá que é emitido tem 4 estrelas, independentemente da língua em que o exame seja realizado. Esta é uma versão anterior do crachá e vamos enviar um pedido para que sej...
- 2 kudos
- 114 Views
- 2 replies
- 0 kudos
Request for One Time Reschedule Missed Databricks Certification Exam
@data_help @data_help @helpdesk @Cert-Team @Cert-TeamOPS Hello Databricks Training Support Team,I hope you are doing well.I am writing regarding my Databricks Certified Data Engineer Associate exam that was scheduled for 09 May 2026 at 09:00 AM Austr...
- 114 Views
- 2 replies
- 0 kudos
- 26 Views
- 0 replies
- 0 kudos
solved
the problem disappeared but so did the button for deleting this post.
- 26 Views
- 0 replies
- 0 kudos
- 602 Views
- 3 replies
- 1 kudos
Resolved! Best practices for health monitoring
Status page: https://status.databricks.com/ REST API: https://docs.databricks.com/api/workspace/workspace/getstatus I'm trying to perform health monitoring using the above status page and API, but is this the best method? If the API returns an error ...
- 602 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
We are considering using batch processing for system monitoring, but can we obtain the same information as the status page using the following API?https://docs.databricks.com/aws/ja/resources/status#%E5%85%AC%E9%96%8B%E3%82%B9%E3%83%86%E3%83%BC%E3%82...
- 1 kudos
- 21 Views
- 0 replies
- 0 kudos
No Response to Request for Age Threshold Exception
Hello,I had submitted a request for an exception to the 18-year age threshold for Databricks certifications on these 2 pages:https://help.databricks.com/s/contact-ushttps://www.databricks.com/company/contactHowever, it’s now been around 10 days since...
- 21 Views
- 0 replies
- 0 kudos
- 62 Views
- 1 replies
- 0 kudos
Managing Unity Catalog Permissions for Databricks Apps via DABs
I’m currently developing a Databricks App, and the app’s service principal needs access to Unity Catalog tables. From what I can tell, it doesn’t seem possible to grant Unity Catalog permissions through DABs yet — only through the UI, based on the cu...
- 62 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @mnissen1337 ,But there is a way to do this in DABs. Look at following section in documentation:Manage Databricks apps using Declarative Automation Bundles | Databricks on AWSIf my answer was helpful, please consider marking it as accepted solutio...
- 0 kudos
- 213 Views
- 5 replies
- 1 kudos
Compute tab doesn't show and doesn't give the option to create a cluster
I've just created an Azure Databricks workspace, tier (Premium) and when trying to create a cluster, when I click on compute, the UI opens automatically the menu SQL Warehouse, not sure if it's a glitch as shown below. Someone said "Ask the admin to ...
- 213 Views
- 5 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @sminamioka This does not look like a UI glitch. In newer Azure Databricks workspaces, access to classic compute / clusters depends on workspace entitlements and compute policy permissions.If clicking Compute takes you directly to SQL Warehouses, ...
- 1 kudos
- 40 Views
- 0 replies
- 1 kudos
TruProxy - Live Cost Estimator - Clusters
Hi everyone, I'm continuing to build a live cost estimator for Databricks to get immediate cost estimates every second instead of having to wait for the system tables to update. (see Live Cost Estimator - Databricks Community - 156374)I've finished t...
- 40 Views
- 0 replies
- 1 kudos
- 81 Views
- 2 replies
- 1 kudos
Resolved! Is Databricks Genie android app not available in Databricks free edition?
Hi guys,Just wanted to ask if Databricks Genie android app is disabled or not for free edition accounts. I'm encountering the following error: {"error_description":"OAuth application with client_id: 'databricks-mobile' not available in Databricks acc...
- 81 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @PROAC, It looks like the Genie Android app isn’t enabled for Free Edition accounts at the moment. The error specifically says the mobile OAuth app databricks-mobile isn’t available for that Databricks account, which usually means the account isn’...
- 1 kudos
- 190 Views
- 6 replies
- 1 kudos
Is the databricks token expired?
Hi all,I set up a serverless endpoint for my agent via an S-account. This S-account is also granted access to another serverless endpoint. During the inference of my endpoint, I use the Databricks SDK to connect to the other endpoint without explicit...
- 190 Views
- 6 replies
- 1 kudos
Latest Reply
- 1 kudos
Hello !Yes ! it seems so WorkspaceClient() is not anonymous and your get_open_ai_client() uses the DBKS auth attached to that WorkspaceClient to query serving endpoints. OpenAI client must be preconfigured with DBKS auth for the workspace client.So ...
- 1 kudos
| User | Count |
|---|---|
| 1901 | |
| 1008 | |
| 930 | |
| 479 | |
| 355 |
