- 9542 Views
- 7 replies
- 3 kudos
How to Call Oracle Stored Procedures from Databricks?
How to Call Oracle Stored Procedures from Databricks?
- 9542 Views
- 7 replies
- 3 kudos
- 900 Views
- 1 replies
- 1 kudos
Resolved! Databricks App deployment failing due to mysqlclient build error (pkg-config not found)
Hi Team,I am trying to deploy a Python project as a Databricks App, but the deployment fails during dependency installation with the following error: ERROR: Failed to build 'mysqlclient' when getting requirements to build wheel pkg-config: not found ...
- 900 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @vamsi_simbus!This post looks similar to another post you shared, and a response has already been provided there. I recommend continuing the discussion in that thread to keep the conversation focused and organised.
- 1 kudos
- 1424 Views
- 6 replies
- 0 kudos
Calling exe from notebook
How to call exe (c-sharp code based) from data bricks notebook?#csharp exe
- 1424 Views
- 6 replies
- 0 kudos
- 0 kudos
Hello @nkrish! Did the suggestions shared above help resolve your concern?If yes, please consider marking the response that helped most as the accepted solution.If you ended up using a different approach, sharing it here would be helpful for others i...
- 0 kudos
- 2889 Views
- 2 replies
- 1 kudos
Resolved! Partner Lab session of 900 min ended, how to restart Lab
Hello, I am new to this Databricks Learning journey. Recently I enrolled for Partner academy Lab course - Data Ingestion with Lakeflow Connect.However, my lab session limit of 900 min has ended. How can I restart the Lab only? I tried it but it gives...
- 2889 Views
- 2 replies
- 1 kudos
- 922 Views
- 2 replies
- 3 kudos
Resolved! How to pass a string as as parameter for EXECUTE IMMEDIATE?
I'm taking the SQL programming and procedural logic course on the academy.One of the lessons concerns EXECUTE IMMEDIATE. It's a bit confusing: the slides use a syntax with (?) as parameter marker, while the video demo uses a syntax with double pipe.S...
- 922 Views
- 2 replies
- 3 kudos
- 3 kudos
This is expected behaviour.? parameter markers (with USING) are only for values, not for SQL identifiers.When you pass a column name via ?, Databricks correctly treats it as a string literal, not as a column reference.That’s why:SELECT (?) FROM order...
- 3 kudos
- 753 Views
- 3 replies
- 1 kudos
Resolved! Do LDPs require job for scheduled trigger?
What are the best practices on implementing scheduled LDP (Lakeflow Declarative Pipeline) in DAB?1. Implement a job with a single pipeline task2. Use just a pipeline in the DAB, without a job
- 753 Views
- 3 replies
- 1 kudos
- 804 Views
- 1 replies
- 1 kudos
Resolved! Best Practice in Reading Delta Files in Storage Bucket Created through Fivetran MDLS
Hello, Databricks Community!I'd like to ask for advice on how you approach in reading and maintaining bucket files in Delta file format created through Fivetran Managed Data Lake (MDLS) destination?I've set-up a Fivetran Destination to a GCP Bucket w...
- 804 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @wanglee , 1. While your solution is valid, it might turn out to be not the most cost-efficient. While the tables are small (< 50k rows) and assuming they will not grow over time, more cost-effective solution is to define tables (and not materiali...
- 1 kudos
- 2186 Views
- 8 replies
- 0 kudos
Resolved! Databricks API - Get Dashboard Owner?
Hi all!I'm trying to identify the owner of a dashboard using the API.Here's a code snippet as an example:import json dashboard_id = "XXXXXXXXXXXXXXXXXXXXXXXXXX" url = f"{workspace_url}/api/2.0/lakeview/dashboards/{dashboard_id}" headers = {"Authoriz...
- 2186 Views
- 8 replies
- 0 kudos
- 0 kudos
Hello, meanwhile, I've found a workaround to get the owner, or more likely the creator of the dashboard.SELECT request_params.dashboard_id AS dashboard_id, FIRST(user_identity.email) AS owner_email FROM system.access....
- 0 kudos
- 1240 Views
- 3 replies
- 2 kudos
Resolved! The best way to write and debug python notebooks
Hello Community,do you have a suggestion, what is the best way to develop python notebooks in databricks? I know that it can be done directly in databricks UI but for more complex projects where I would like to split notebook and import functions fro...
- 1240 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @maikel for us databricks extension on vs code to deploy workflow using asset bundles works like a charm. if plugin is not helping out then i suggest you to install databricks CLI separately or point to the one installed together with databricks e...
- 2 kudos
- 1439 Views
- 2 replies
- 1 kudos
Resolved! SDP: Your table schema requires manually enablement of the following table feature(s): timestampNtz.
Trying to create a Spark Declarative Pipeline with a Materialized View based on a Streaming Table. The Streaming Table works fine, but when trying to add a Materialized View the pipeline fails to initialize with:Your table schema requires manually en...
- 1439 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @liquibricks , I see you said that the materialized view does not exist, so you can't run the ALTER TABLE command. Try to add TBLPROPERTIES ('delta.feature.timestampNtz'='supported') to the MV CREATE statement (or pass it via table_properties in y...
- 1 kudos
- 3496 Views
- 4 replies
- 3 kudos
Resolved! How to handle load of 300 tables to delta lake
My task is to sync 300 tables from on prem sql server to delta lake. I will load CDC from Raw. First step is to move CDC data to bronze with autoloader. Then using delta stream get changes from bronze, make simple datatype changes and merge this data...
- 3496 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi Alesventus,The approach suggested by syzmon_dybczak is definitely the same direction I would go in. To supplement their answer, have a look at this Databricks Labs repository that you may find useful as inspiration (DLT-Meta): https://github.com/d...
- 3 kudos
- 1607 Views
- 1 replies
- 0 kudos
Resolved! Implementing Ruff linter in Gitlab CI/CD for Databrick notebooks
Hi all, I am trying to implement ruff linting in Gitlab CI/CD where I am only trying to catch undefined name errors ["F821"]. However it is picking up my magic commands like %sql and %pip as syntax errors and I am unable to exclude it in the pyprojec...
- 1607 Views
- 1 replies
- 0 kudos
- 0 kudos
you're using ruff for linting. The issue with Databricks magic commands (%sql, %pip, %run) is that they're Databricks-specific syntax, not standard Python, so ruff treats them as errors. you should exclude databricks notebook from ruff.
- 0 kudos
- 2744 Views
- 11 replies
- 7 kudos
Resolved! Declarative Pipeline error: Name 'kdf' is not defined. Did you mean: 'sdf'
We have a Lakeflow Spark Declarative Pipeline using the new PySpark Pipelines API. This was working fine until about 7am (Central European) this morning when the pipeline started failing with a PYTHON.NAME_ERROR: name 'kdf' is not defined. Did you me...
- 2744 Views
- 11 replies
- 7 kudos
- 7 kudos
Hi, I ran into the same error starting late last week.The issue was caused by dependency version conflicts between my custom package and the libraries preinstalled on the Databricks serverless environment.I fixed it by pinning all my package depende...
- 7 kudos
- 667 Views
- 1 replies
- 1 kudos
Resolved! Soumitra Dutta : How do I debug exceptions in Databricks notebooks?
Hello,My name is Soumitra Dutta,I’m currently in the USA and running into some exceptions in my Databricks notebook. For anyone who has dealt with similar issues, what steps do you usually take to debug them? I’d really appreciate your suggestions.”
- 667 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Soumitra, this is quite a broad question as it depends on what kind of error you are seeing. If there is something specific, please reply here for something more tailored. In general, when debugging I usually start with the Databricks Assistant as...
- 1 kudos
- 7881 Views
- 10 replies
- 0 kudos
Databricks Certification voucher not received
Hello team,I have attended the webinar Databricks Certification Overview Series- Data Engineer on Jan 17Completed the Databricks Lakehouse fundamentals accreditation and Completed the survey.As per communication it is expected that I will receive Dat...
- 7881 Views
- 10 replies
- 0 kudos
- 0 kudos
Hi @Indika Debnath Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 0 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
