Following best practice, we want to avoid reusing code by putting commonly used transformations into function libraries and then importing and calling those functions where required.We also want to follow Databricks recommendations to use serverless ...
I am trying to configure a policy in a job cluster using Databricks Linked Service in Azure Data Factory (ADF). The goal is to restrict access to a specific group or individuals who can use this cluster.I want to know:How to define and enforce a clus...
In the Linked Services panel in the Azure Data Factory UI, find your Databricks Linked Service and click the squiggly braces {} to edit the service. From here, you'll see a definition in JSON. Create a new key: properties.typeProperties.policyId and ...
When using partner connect to connect Serverless Databricks to my BI tool Preset, you must manually define the schema that Preset has access to. In my case, I individually selected all databases currently in my hive_metastore:The problem is, once cre...
Connecting to Databricks using OpenJDK 17 I got the exception below. Are there any plans to fix the driver for OpenJDK17?java.sql.SQLException: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 44. Ba...
I still see the above error with databricks jdbc driver 2.6.33. Anyone aware of fix available either in driver or java?
I have a python project and within that I do have a yaml file. Currently i'm building the project using poetry and creating an asset bundle to deploy it in Databricks as a workflow job.So when the workflow runs, I do have an __init__.py within my ent...
My exam has been suspended, and I received an email to schedule it. But I see that there is a delay of 10 to 15 days. Is it possible to schedule it for 19 March at 11 a.m.?because I have a deadline that I must not exceed to obtain this certificate.o...
Hello @saraM! Thank you for reaching out to us again and confirming.As you requested, the support team has rescheduled your exam for March 19, 2025, at 11:00 AM in Europe/Paris.Please log into your account and confirm. The rescheduled exam will be re...
I'm using Databricks workflows along with asset bundle to run my pipeline and recently we moved our pipeline from development to acceptance. We are also using JFrog as our package artifactory.Even though I do see the release candidate version under t...
Thank you for the response. Apparently one of our internal library failed to install, since an init script was missing on the cluster. After adding that, this got resolved
Hello,I have an external table created in databricks unity catalog workspace and trying to "Create a monitor" for the same from quality tab.While creating the same the dashboard is getting created however the two metrices tables "profile" & "drift" a...
Hi Nikhil,How are you doing today?, As per my understanding, It looks like the monitoring setup isn't fully completing, likely due to permissions, external table limitations, or system settings. First, check if your Unity Catalog schema has the right...
I'm having issues with liquid clustering not working when creating derived/generated fields in an asset bundle. Are there any known limitations here?
Hi Lottiejanedev,How are you doing today?, As per my understanding, Yes, Liquid Clustering works best with directly stored columns, and it might not behave as expected when clustering on derived or generated fields. One possible issue is that generat...
Hi, I currently am running a number of notebook jobs from Azure Data Factory. A new requirement has come up where I need to capture a return code in ADF that has been generated from the note. I tried using dbutils.notebook.exit(json.dumps({"return_v...
I tried dbutils.notebook.exit(<value>) where value is 0 and 1. I also tried sys.exit(<value>). None of this produces a difference in the output of the databricks CLI commanddatabricks jobs run-now --json '{"job_id": <JOB_ID>, "job_parameters": {"will...
I try to start a job ID 85218616788189 and pass one parameters 'demo' in Windows Shell.This works:databricks jobs run-now 85218616788189 If I try this one,databricks jobs run-now --json '{"job_id":85218616788189,"notebook_params": {"demo":"parameter...
If the job itself has parameters you have to use 'job_parameters' instead. No need for escaping as of version 0.243.0databricks jobs run-now --json '{"job_id": <JOB_ID>, "job_parameters": {"will_exit": "yes"}}'
My databricks exam got suspened on 18th march and it is still in the suspened state. I have raised a support request using the below link.https://help.databricks.com/s/contact-us?ReqType=trainingand my request id : #00635015 I would really like to ge...
Hello @saraM! It looks like this post duplicates the one you recently posted. The Original post has already been answered. I recommend continuing the discussion there to keep the conversation focused and organised.
So, I have a databricks streamlit app that is not able to install any python lib defined on the requirements.txt.Issue is not specific to one lib, tried other ones but no python lib can be installed. Anyone with similar issue to help ? [2025-02-19 10...
Hi, returning to inform how the issue was solved. A bit of context, looks like the databricks apps runs on the databricks/microsoft backend, and not on our clusters. The control of the network ingress rules is done via SEG (https://www.databricks.com...
Hi All,I have SSMS installed and have some data there which i want to export to Databricks and process it there. Databricks is hosted on Azure Cloud and I am wondering if it is possible. I have tested it using JDBC connection but i am getting error: ...
If you want to play with SQL Server the easy way is to use Azure SQL that will be visuable for Databricks. Alternativly you can use ADF with slef-hosted run time to extract data to Azure from SQL Server.Network configuration requires many steps and i...
Hi quick questions, 1) the other day I commented on a thread and cannot find it. Is there a feature to find all my posts? I cannot find it under my profile.2) I find interesting threads and want to watch (in the sense: email me when updates occure) t...
Hi @Jan St.​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your ...
| User | Count |
|---|---|
| 1616 | |
| 790 | |
| 466 | |
| 349 | |
| 287 |
