cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

kulasangar
by New Contributor II
  • 2779 Views
  • 0 replies
  • 0 kudos

Permission Denied while trying to update a yaml file within a python project in Databricks

I have a python project and within that I do have a yaml file. Currently i'm building the project using poetry and creating an asset bundle to deploy it in Databricks as a workflow job.So when the workflow runs, I do have an __init__.py within my ent...

kulasangar_0-1742321907764.png
  • 2779 Views
  • 0 replies
  • 0 kudos
saraM
by New Contributor III
  • 584 Views
  • 1 replies
  • 0 kudos

Resolved! My databricks Exam still suspended request: 00635015

My exam has been suspended, and I received an email to schedule it. But I see that there is a delay of 10 to 15 days. Is it possible to schedule it for 19 March  at 11 a.m.?because I have a deadline that I must not exceed to obtain this certificate.o...

  • 584 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika
Databricks Employee
  • 0 kudos

Hello @saraM! Thank you for reaching out to us again and confirming.As you requested, the support team has rescheduled your exam for March 19, 2025, at 11:00 AM in Europe/Paris.Please log into your account and confirm. The rescheduled exam will be re...

  • 0 kudos
kulasangar
by New Contributor II
  • 980 Views
  • 2 replies
  • 1 kudos

ERROR: Could not find a version that satisfies the requirement

I'm using Databricks workflows along with asset bundle to run my pipeline and recently we moved our pipeline from development to acceptance. We are also using JFrog as our package artifactory.Even though I do see the release candidate version under t...

kulasangar_0-1741255645400.png
  • 980 Views
  • 2 replies
  • 1 kudos
Latest Reply
kulasangar
New Contributor II
  • 1 kudos

Thank you for the response. Apparently one of our internal library failed to install, since an init script was missing on the cluster. After adding that, this got resolved

  • 1 kudos
1 More Replies
nikhil_2212
by New Contributor II
  • 1063 Views
  • 1 replies
  • 0 kudos

Lakehouse monitoring metrices tables not created automatically

Hello,I have an external table created in databricks unity catalog workspace and trying to "Create a monitor" for the same from quality tab.While creating the same the dashboard is getting created however the two metrices tables "profile" & "drift" a...

  • 1063 Views
  • 1 replies
  • 0 kudos
Latest Reply
Brahmareddy
Esteemed Contributor
  • 0 kudos

Hi Nikhil,How are you doing today?, As per my understanding, It looks like the monitoring setup isn't fully completing, likely due to permissions, external table limitations, or system settings. First, check if your Unity Catalog schema has the right...

  • 0 kudos
lottiejanedev
by New Contributor
  • 440 Views
  • 1 replies
  • 0 kudos

Liquid Clustering with Derived / Generated Fields

I'm having issues with liquid clustering not working when creating derived/generated fields in an asset bundle. Are there any known limitations here? 

  • 440 Views
  • 1 replies
  • 0 kudos
Latest Reply
Brahmareddy
Esteemed Contributor
  • 0 kudos

Hi Lottiejanedev,How are you doing today?, As per my understanding, Yes, Liquid Clustering works best with directly stored columns, and it might not behave as expected when clustering on derived or generated fields. One possible issue is that generat...

  • 0 kudos
labromb
by Contributor
  • 2082 Views
  • 2 replies
  • 2 kudos

Capturing notebook return codes in databricks jobs

Hi, I currently am running a number of notebook jobs from Azure Data Factory. A new requirement has come up where I need to capture a return code in ADF that has been generated from the note. I tried using  dbutils.notebook.exit(json.dumps({"return_v...

  • 2082 Views
  • 2 replies
  • 2 kudos
Latest Reply
JaviRuiz
New Contributor III
  • 2 kudos

I tried dbutils.notebook.exit(<value>) where value is 0 and 1. I also tried sys.exit(<value>). None of this produces a difference in the output of the databricks CLI commanddatabricks jobs run-now --json '{"job_id": <JOB_ID>, "job_parameters": {"will...

  • 2 kudos
1 More Replies
jim12321
by New Contributor II
  • 5327 Views
  • 4 replies
  • 1 kudos

Databricks CLI how to start a job and pass the parameters?

I try to start a job ID 85218616788189 and pass one parameters 'demo' in Windows Shell.This works:databricks jobs run-now 85218616788189  If I try this one,databricks jobs run-now --json '{"job_id":85218616788189,"notebook_params": {"demo":"parameter...

jim12321_0-1710267172994.png
  • 5327 Views
  • 4 replies
  • 1 kudos
Latest Reply
JaviRuiz
New Contributor III
  • 1 kudos

If the job itself has parameters you have to use 'job_parameters' instead. No need for escaping as of version 0.243.0databricks jobs run-now --json '{"job_id": <JOB_ID>, "job_parameters": {"will_exit": "yes"}}' 

  • 1 kudos
3 More Replies
saraM
by New Contributor III
  • 371 Views
  • 1 replies
  • 0 kudos

My databricks exam got suspened on 18th march and it is still in the suspened state.

My databricks exam got suspened on 18th march and it is still in the suspened state. I have raised a support request using the below link.https://help.databricks.com/s/contact-us?ReqType=trainingand my request id : #00635015 I would really like to ge...

  • 371 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika
Databricks Employee
  • 0 kudos

Hello @saraM! It looks like this post duplicates the one you recently posted. The Original post has already been answered. I recommend continuing the discussion there to keep the conversation focused and organised.

  • 0 kudos
ewe
by New Contributor III
  • 2681 Views
  • 3 replies
  • 2 kudos

Resolved! Databricks apps (streamlit) not able to install python libs

So, I have a databricks streamlit app that is not able to install any python lib defined on the requirements.txt.Issue is not specific to one lib, tried other ones but no python lib can be installed. Anyone with similar issue to help ? [2025-02-19 10...

  • 2681 Views
  • 3 replies
  • 2 kudos
Latest Reply
ewe
New Contributor III
  • 2 kudos

Hi, returning to inform how the issue was solved. A bit of context, looks like the databricks apps runs on the databricks/microsoft backend, and not on our clusters. The control of the network ingress rules is done via SEG (https://www.databricks.com...

  • 2 kudos
2 More Replies
Upendra_Dwivedi
by Contributor
  • 1194 Views
  • 3 replies
  • 0 kudos

ON-Prem SQL Server Direct Connection with Azure Databricks

Hi All,I have SSMS installed and have some data there which i want to export to Databricks and process it there. Databricks is hosted on Azure Cloud and I am wondering if it is possible. I have tested it using JDBC connection but i am getting error: ...

  • 1194 Views
  • 3 replies
  • 0 kudos
Latest Reply
MariuszK
Valued Contributor III
  • 0 kudos

If you want to play with SQL Server the easy way is to use Azure SQL that will be visuable for Databricks. Alternativly you can use ADF with slef-hosted run time to extract data to Azure from SQL Server.Network configuration requires many steps and i...

  • 0 kudos
2 More Replies
Mumrel
by Contributor
  • 7695 Views
  • 5 replies
  • 1 kudos

Resolved! Usage of forum: How to find the threads where I commented on and my bookmarks

Hi quick questions, 1) the other day I commented on a thread and cannot find it. Is there a feature to find all my posts? I cannot find it under my profile.2) I find interesting threads and want to watch (in the sense: email me when updates occure) t...

  • 7695 Views
  • 5 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Jan St.​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your ...

  • 1 kudos
4 More Replies
Prasanna_N
by New Contributor
  • 2867 Views
  • 0 replies
  • 0 kudos

Inference table Monitoring

i have data from march1 to march 14 in the final inference table and i have given 1 week granularity. after that profile and drift table is generated and i see the window start time as like this objectstart: "2025-02-24T00:00:00.000Z"end: "2025-03-03...

  • 2867 Views
  • 0 replies
  • 0 kudos
Danny_Lee
by Valued Contributor
  • 3105 Views
  • 0 replies
  • 1 kudos

re: Welcoming Bladebridge to Databricks!

Hi @Sujitha and Databricks team,Congrats on the acquisition of Bladebridge.  We used this tool a couple years back to migrate an important ETL process from Informatica.  I'm glad to see its part of the Data Intelligence Platform and have already take...

  • 3105 Views
  • 0 replies
  • 1 kudos
databicky
by Contributor II
  • 1025 Views
  • 1 replies
  • 0 kudos

How to Copy the notebooks from one environment to another environment

I have one requirement to copy the notebooks from one environment to another environment by one notebook automatically , how can I achieve it.  

  • 1025 Views
  • 1 replies
  • 0 kudos
Latest Reply
Isi
Honored Contributor III
  • 0 kudos

Hey @databicky ,You can automate the process of copying notebooks from one Databricks environment to another using the Databricks REST API within a notebook. I show you the easiest way I found to do itimport json import requests import base64 # ====...

  • 0 kudos
Kabil
by New Contributor
  • 2738 Views
  • 0 replies
  • 0 kudos

useing dlt metadata as runtime parameter

i have started using DLT pipeline, and i have common code which is used by multiple DLT pipeline. now i need to read metadata information like name of the pipeline and start time of the pipeline during run time, but since im using common code and pip...

  • 2738 Views
  • 0 replies
  • 0 kudos

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels