The current workflow function in Databricks gives a series of options such as DLT, Dbt, python scripts, python files, JAR, etc. It would be good to add a docker file to that and simplify the development process a lot, especially on the unit and integ...
Unity Catalog securable objects can be tagged with key value pairs: https://learn.microsoft.com/en-us/azure/databricks/database-objects/tagsIs it possible tag objects via REST API calls?I initially thought any Unity Catalog resource in the Databricks...
I recently saw a new feature in databricks that allows to set an upper limit of run time for jobs, so it fails when the limit is breached. See image below:Are we able to terraform code this new feature?
Hi @Rjdudley, Thanks for reaching out! Please review the response and let us know which best addresses your question. Your feedback is valuable to us and the community. If the response resolves your issue, kindly mark it as the accepted solution. T...
I am using the Databricks SQL Driver for Node.js to create an endpoint that queries a Databricks database following the guide here Databricks SQL Driver for Node.js | Databricks on AWS . This code was working previously but now I am getting a TypeErr...
Hi - I have Azure Databricks with Unity Catalog, and the SCIM Connector provisioning up and working.Due to a misunderstanding on my part, I deleted some individual users (using the Databricks admin console) thinking that I was deleting individual acc...
This was resolved...but why it was resolved remains a mystery. We simply re-provisioned by turning off and then on the SCIM provisioning in the Azure portal for the Databricks SCIM connector (which is supposed to cause a full re-provisioning, but di...
Hello everyoneI'm trying to setup my admin account in Databricks Admin Console and am following these steps:https://learn.microsoft.com/en-us/azure/databricks/administration-guide/#--what-are-account-adminsHowever, when I access accounts.azuredatabri...
Hi everyoneI asked that question a year and somehow it'd been solved.Not sure however if it's related to generic email providers, but I have multiple registered accounts, gmail and corporate one. For databricks accademy, I'm using gmail account, whic...
Can you please help me with how to authenticate account level rest apis?curl --request GET https://accounts.cloud.databricks.com/api/2.0/accounts/{account_id}/usageI use Permanent Access Token to authenticate workspace APIs but it's not working for w...
Hi @shubhdhanuka ,To get access to account API you cannot use personal access token. Instead, you must use the Microsoft Entra ID tokens of Azure Databricks account admins. Azure Databricks account admins can be users or service principals.Below you ...
I receive this respone when trying to download a csv file: adb-737684297741988.8.azuredatabricks.net/ajax-api/2.0/workspace/export?path=/Repos/jhu_admin%40siemensgamesa.com/sgre-dfp-lifestrategy-python/cpo/output/Rattlesnake Flats/Rdata/AssetCheck_Ra...
@szymon_dybczak I am using the GUI (see the screenshot in the original post): File marked -> Three dots -> Export
Connecting to Azure Databricks using latest JDBC driver and OpenJDK 21 I got this error. I already set:--add-opens=java.base/java.nio=ALL-UNNAMED-Dio.netty.tryReflectionSetAccessible=true java.sql.SQLException: [Databricks][JDBCDriver](500618) Error ...
Version 02.06.38.1068Using SQLSquirrel with ibm-semeru-certified-17-jdk (17.0.11+9)With JRE options --add-opens=jdk.unsupported/sun.misc=ALL-UNNAMED failsShould be --add-opens=java.base/java.nio=ALL-UNNAMEDcaused by: java.lang.reflect.InaccessibleOb...
Hi I'm a Data engineer and recently developed a Notebook analytics template for general purposes that I would like to be the standard on my company. Continuing, I created another notebook with a text widget that uses the user input to map the folder ...
Hi all,hope you are doing well. We have a metastore in tenant A, and workspaces in tenant B. Tenant B has Contributor Role to tenant A, to access the metastore. For some reasons, we can not assign the metastore from tenant A to the workspace in tenan...
Hi @data_bricklayer, To address this, ensure the Contributor Role in tenant B has the right permissions for tenant A's metastore. Verify that the workspace in tenant B is properly registered and visible in tenant A and that both are in the same regio...
Using a Databricks cluster running Databricks Runtime version 15.4 LTS Beta, and with no init-scripts, the following code works fine:import numpy as npimport pandas as pddf = pd.DataFrame(np.random.randn(2000, 10))df.plot(subplots=True, figsize=(10, ...
OK, I understand the error now:This specific secret is a small alphanumeric string. The probability of it being part of the base64 of a big PNG is quite large.Is there any workaround for this? I know I can't turn off the redaction of secrets. Can I d...
I recently set up an Azure Databricks workspace with an automatically created metastore and metastore-level root storage within the metastore blob storage account. All the catalogs, schemas, and tables/volumes have been created without a specified or...
I have a python application which I need to connect with databricks (Databricks SQL Connector). The process for that is quite simple -1. connect with dbr2. execute your query3. get the result4. close the connectionExample from - https://docs.databric...
Hi @Retired_mod, thanks for your reply. I have 2 follow-up questions for you - When you say "consider managing connection lifetimes by closing and recreating them periodically" - the problem I face with this is when my API request comes in and I crea...
I'm getting the following error when trying to install UCX on a specific workspace.I have already installed ucx from my machine on other workspace and works, now on this workspace i'm getting this error ERROR [d.l.blueprint.parallel][installing_das...
Issue was resolved in lsql v0.7.4, a ucx dependency: https://github.com/databrickslabs/lsql/releases/tag/v0.7.41. Upgrade the Databricks cli: https://github.com/databricks/cli/releases/2. Upgrade ucx: `databricks labs install ucx`
